在 Odroid-C2 上 安裝 docker 程式 (ARM aarch64 版本)
前言
手上有一片Odroid-C2 開發板, 並且安裝了armbian Debian-jessie 作業系統. 想說可否在這板子上執行 docker , 但是發現, 目前docker 官方並未釋出ARM aarch64 架構的編譯版本, 若要使用docker的話, 只能自己手動編譯了.
linux kernel 版本
由於 docker 必須依賴 linux kernel 3.10 以上. 並且需要一些指定的 kernel feature 有被啟用才可以. docker source code 中有一個程式check-config.sh, 可以檢查你的linux kernel 是否能夠執行docker.
# uname -a
Linux odroidc2 3.14.73-odroidc2 #2 SMP PREEMPT Mon Aug 1 14:01:38 CST 2016 aarch64 GNU/Linux
下載check-config.sh
# wget https://github.com/docker/docker/raw/master/contrib/check-config.sh
# chmod +x check-config.sh
執行check-confg.sh, 會出現類似如下的訊息:
# ./check-config.sh
info: reading kernel config from /proc/config.gz ...
Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- apparmor: enabled and tools installed
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_DEVPTS_MULTIPLE_INSTANCES: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_KEYS: enabled
- CONFIG_VETH: enabled (as module)
- CONFIG_BRIDGE: enabled
- CONFIG_BRIDGE_NETFILTER: enabled
- CONFIG_NF_NAT_IPV4: enabled (as module)
- CONFIG_IP_NF_FILTER: enabled (as module)
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_NF_NAT_NEEDED: enabled
- CONFIG_POSIX_MQUEUE: enabled
Optional Features:
- CONFIG_USER_NS: enabled
- CONFIG_SECCOMP: enabled
- CONFIG_CGROUP_PIDS: missing
- CONFIG_MEMCG_SWAP: enabled
- CONFIG_MEMCG_SWAP_ENABLED: enabled
- CONFIG_MEMCG_KMEM: enabled
- CONFIG_RESOURCE_COUNTERS: enabled
- CONFIG_BLK_CGROUP: enabled
- CONFIG_BLK_DEV_THROTTLING: enabled
- CONFIG_IOSCHED_CFQ: enabled
- CONFIG_CFQ_GROUP_IOSCHED: missing
- CONFIG_CGROUP_PERF: enabled
- CONFIG_CGROUP_HUGETLB: enabled
- CONFIG_NET_CLS_CGROUP: enabled (as module)
- CONFIG_CGROUP_NET_PRIO: enabled
- CONFIG_CFS_BANDWIDTH: enabled
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: enabled
- CONFIG_IP_VS: enabled (as module)
- CONFIG_EXT3_FS: missing
- CONFIG_EXT3_FS_XATTR: missing
- CONFIG_EXT3_FS_POSIX_ACL: missing
- CONFIG_EXT3_FS_SECURITY: missing
(enable these ext3 configs if you are using ext3 as backing filesystem)
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: enabled
- CONFIG_EXT4_FS_SECURITY: enabled
- Network Drivers:
- "overlay":
- CONFIG_VXLAN: missing
Optional (for secure networks):
- CONFIG_XFRM_ALGO: enabled
- CONFIG_XFRM_USER: enabled (as module)
- "ipvlan":
- CONFIG_IPVLAN: missing
- "macvlan":
- CONFIG_MACVLAN: enabled (as module)
- CONFIG_DUMMY: enabled (as module)
- Storage Drivers:
- "aufs":
- CONFIG_AUFS_FS: missing
- "btrfs":
- CONFIG_BTRFS_FS: enabled (as module)
- CONFIG_BTRFS_FS_POSIX_ACL: enabled
- "devicemapper":
- CONFIG_BLK_DEV_DM: enabled (as module)
- CONFIG_DM_THIN_PROVISIONING: enabled (as module)
- "overlay":
- CONFIG_OVERLAY_FS: enabled (as module)
- "zfs":
- /dev/zfs: missing
- zfs command: missing
- zpool command: missing
Limits:
- /proc/sys/kernel/keys/root_maxkeys: 1000000
其中Generally Necessary 的features 必須全部都是enabled 才能夠執行docker, 若有任何一項沒有enabled, 那麼你可能就必須重新編譯你的linux kernel, 或者是依照此程式裡面的建議方式來啟用這些features.
編譯 docker
查了一下docker官方文件, 要編譯docker, 需要有docker 的環境才能編譯. 這個就很奇怪了, 我就是因為沒有aarch64 版本的docker, 才想要編譯docker for aarch64的啊. 這不就變成雞生蛋,蛋生雞的問題了嗎? 查了許久, 才知道docker 有提供無中生有的編譯方式啦. 只是官方文件沒有說明. 編譯方式我是依照最後面的參考資料 1 來操作的. 步驟如下:
首先 安裝 golang, 版本必須是1.6以上
# apt-get update; apt-get install golang-go
# go version
go version go1.6.2 linux/arm64
安裝編譯時必要的套件
# apt-get install btrfs-tools libsqlite3-dev libdevmapper-dev
下載docker source code,首先 編譯docker v1.10.2版. 因為v1.10.2 開始有支援ARM aarch64架構
# mkdir -p ~/code
# cd ~/code
# git clone https://github.com/docker/docker
# cd docker
# git checkout v1.10.2
開始編譯
# AUTO_GOPATH=1 ./hack/make.sh dynbinary
編譯完成後訊息如下:
# WARNING! I don't seem to be running in the Docker container.
# The result of this command might be an incorrect build, and will not be
# officially supported.
#
# Try this instead: make all
#
---> Making bundle: dynbinary (in bundles/1.10.2/dynbinary)
Created binary: bundles/1.10.2/dynbinary/dockerinit-1.10.2
Building: bundles/1.10.2/dynbinary/docker-1.10.2
Created binary: bundles/1.10.2/dynbinary/docker-1.10.2
編譯完後的程式被放在~/code/docker/bundles/1.10.2/dynbinary/
# ls -al ~/code/docker/bundles/1.10.2/dynbinary
total 41788
drwxr-xr-x 2 root root 4096 Sep 22 08:18 .
drwxr-xr-x 3 root root 4096 Sep 22 08:11 ..
lrwxrwxrwx 1 root root 13 Sep 22 08:18 docker -> docker-1.10.2
-rwxr-xr-x 1 root root 32942232 Sep 22 08:18 docker-1.10.2
-rw-r--r-- 1 root root 48 Sep 22 08:18 docker-1.10.2.md5
-rw-r--r-- 1 root root 80 Sep 22 08:18 docker-1.10.2.sha256
lrwxrwxrwx 1 root root 17 Sep 22 08:12 dockerinit -> dockerinit-1.10.2
-rwxr-xr-x 1 root root 9819424 Sep 22 08:12 dockerinit-1.10.2
-rw-r--r-- 1 root root 52 Sep 22 08:12 dockerinit-1.10.2.md5
-rw-r--r-- 1 root root 84 Sep 22 08:12 dockerinit-1.10.2.sha256
我們只需將docker-1.10.2 及 dockerinit-1.10.2 拷貝到適當的目錄下即可:
# cd ~/code/docker/
# mkdir -p /usr/lib/docker/
# cp bundles/1.10.2/dynbinary/docker-1.10.2 /usr/bin/docker
# cp bundles/1.10.2/dynbinary/dockerinit-1.10.2 /usr/lib/docker/dockerinit
測試docker
在使用docker之前, 需要先啟動docker daemon, 啟動方式如下:
# docker daemon &
開始測試docker
# docker run -ti ebspace/aarch64-debian
ERRO[0140] Handler for POST /v1.22/containers/create returned error: No such image: ebspace/aarch64-debian:latest
Unable to find image 'ebspace/aarch64-debian:latest' locally
latest: Pulling from ebspace/aarch64-debian
59d7421c5877: Pull complete
Digest: sha256:b4993a58d12d822a62ebf4b683c74716a77078f6310fd0bf97451c9d5465dbd4
Status: Downloaded newer image for ebspace/aarch64-debian:latest
root@e39ac0f7cbcc:/# ls -al
total 68
drwxr-xr-x 20 root root 4096 Sep 22 14:56 .
drwxr-xr-x 20 root root 4096 Sep 22 14:56 ..
-rwxr-xr-x 1 root root 0 Sep 22 14:56 .dockerenv
-rwxr-xr-x 1 root root 0 Sep 22 14:56 .dockerinit
drwxr-xr-x 2 root root 4096 Sep 21 20:56 bin
drwxr-xr-x 2 root root 4096 Sep 12 04:11 boot
drwxr-xr-x 5 root root 380 Sep 22 14:56 dev
drwxr-xr-x 42 root root 4096 Sep 22 14:56 etc
drwxr-xr-x 2 root root 4096 Sep 12 04:11 home
drwxr-xr-x 9 root root 4096 Sep 21 20:49 lib
drwxr-xr-x 2 root root 4096 Sep 21 20:46 media
drwxr-xr-x 2 root root 4096 Sep 21 20:46 mnt
drwxr-xr-x 2 root root 4096 Sep 21 20:46 opt
dr-xr-xr-x 123 root root 0 Sep 22 14:56 proc
drwx------ 2 root root 4096 Sep 21 20:46 root
drwxr-xr-x 3 root root 4096 Sep 21 20:46 run
drwxr-xr-x 2 root root 4096 Sep 21 20:56 sbin
drwxr-xr-x 2 root root 4096 Sep 21 20:46 srv
dr-xr-xr-x 12 root root 0 Sep 22 00:39 sys
drwxrwxrwt 2 root root 4096 Sep 21 20:58 tmp
drwxr-xr-x 10 root root 4096 Sep 21 20:46 usr
drwxr-xr-x 11 root root 4096 Sep 21 20:46 var
root@e39ac0f7cbcc:/# exit
exit
# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e39ac0f7cbcc ebspace/aarch64-debian "/bin/bash" 5 minutes ago Exited (0) 2 minutes ago backstabbing_kalam
Yes!! docker 成功產生了一個 container.
編譯最新版本的docker
我們已經編譯出第一個版本的docker v1.10.2 了 (也就是第一隻母雞). 若以後想要試試最新版的docker, 就可以採用官方建議的編譯方式. 用 docker 來 產生新版的 docker. (用這隻母雞來孵雞蛋吧).
依照官方文件, 編譯指令很簡單, 只要兩個指令make build, make binary
更新至最新版本的 docker source code
目前最新release 的版本為 v1.12.1
# cd ~/code/docker
# git pull
# git checkout v1.12.1
編譯 docker
# make build
# make binary
編譯完後, 執行檔會放在 ~/code/docker/bundles/1.12.1/ 目錄裡.
裡面有兩個目錄, 分別為binary-client 以及 binary-daemon
移除舊版版本docker
- 首先停止正在執行的 docker deamon.
- 然後移除舊的docker 執行檔.
- 最後移除 /var/lib/docker 目錄. !!注意!! 這個動作也會遺失舊的docker images, 以及相關資料.
在第三項的目的, 是為了防止docker 新舊版資料相衝突而增加的. 因為v1.10.2版在/var/lib/docker建立的資料, 會造成新版啟動失敗. 所以最好的方式是全部移除乾淨, 再讓新版docker 建立新資料.
安裝新版本 docker
將新編譯後的執行檔拷貝到/usr/bin 目錄即可
# cp ~/code/docker/bundles/1.12.1/binary-client/docker* /usr/bin
# cp ~/code/docker/bundles/1.12.1/binary-daemon/docker* /usr/bin
注意事項
我在編譯v1.12.1 過程中, 有出現編譯失敗, 經查詢後, 原來是記憶體不足而造成編譯失敗. 後來將swap memory 加大之後, 就解決問題了.
Odroid-C2 板子上有2G RAM, 看來拿來編譯docker, 仍然稍嫌不足. 也只能靠swap memory 來解決了.