在 Odroid-C2 上 安裝 docker 程式 (ARM aarch64 版本)

在 Odroid-C2 上 安裝 docker 程式 (ARM aarch64 版本)

前言

手上有一片Odroid-C2 開發板, 並且安裝了armbian Debian-jessie 作業系統. 想說可否在這板子上執行 docker , 但是發現, 目前docker 官方並未釋出ARM aarch64 架構的編譯版本, 若要使用docker的話, 只能自己手動編譯了.

linux kernel 版本

由於 docker 必須依賴 linux kernel 3.10 以上. 並且需要一些指定的 kernel feature 有被啟用才可以. docker source code 中有一個程式check-config.sh, 可以檢查你的linux kernel 是否能夠執行docker.

# uname -a
Linux odroidc2 3.14.73-odroidc2 #2 SMP PREEMPT Mon Aug 1 14:01:38 CST 2016 aarch64 GNU/Linux

下載check-config.sh

# wget https://github.com/docker/docker/raw/master/contrib/check-config.sh
# chmod +x check-config.sh

執行check-confg.sh, 會出現類似如下的訊息:

# ./check-config.sh      
info: reading kernel config from /proc/config.gz ...

Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- apparmor: enabled and tools installed
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_DEVPTS_MULTIPLE_INSTANCES: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_KEYS: enabled
- CONFIG_VETH: enabled (as module)
- CONFIG_BRIDGE: enabled
- CONFIG_BRIDGE_NETFILTER: enabled
- CONFIG_NF_NAT_IPV4: enabled (as module)
- CONFIG_IP_NF_FILTER: enabled (as module)
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_NF_NAT_NEEDED: enabled
- CONFIG_POSIX_MQUEUE: enabled

Optional Features:
- CONFIG_USER_NS: enabled
- CONFIG_SECCOMP: enabled
- CONFIG_CGROUP_PIDS: missing
- CONFIG_MEMCG_SWAP: enabled
- CONFIG_MEMCG_SWAP_ENABLED: enabled
- CONFIG_MEMCG_KMEM: enabled
- CONFIG_RESOURCE_COUNTERS: enabled
- CONFIG_BLK_CGROUP: enabled
- CONFIG_BLK_DEV_THROTTLING: enabled
- CONFIG_IOSCHED_CFQ: enabled
- CONFIG_CFQ_GROUP_IOSCHED: missing
- CONFIG_CGROUP_PERF: enabled
- CONFIG_CGROUP_HUGETLB: enabled
- CONFIG_NET_CLS_CGROUP: enabled (as module)
- CONFIG_CGROUP_NET_PRIO: enabled
- CONFIG_CFS_BANDWIDTH: enabled
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: enabled
- CONFIG_IP_VS: enabled (as module)
- CONFIG_EXT3_FS: missing
- CONFIG_EXT3_FS_XATTR: missing
- CONFIG_EXT3_FS_POSIX_ACL: missing
- CONFIG_EXT3_FS_SECURITY: missing
    (enable these ext3 configs if you are using ext3 as backing filesystem)
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: enabled
- CONFIG_EXT4_FS_SECURITY: enabled
- Network Drivers:
  - "overlay":
    - CONFIG_VXLAN: missing
    Optional (for secure networks):
    - CONFIG_XFRM_ALGO: enabled
    - CONFIG_XFRM_USER: enabled (as module)
  - "ipvlan":
    - CONFIG_IPVLAN: missing
  - "macvlan":
    - CONFIG_MACVLAN: enabled (as module)
    - CONFIG_DUMMY: enabled (as module)
- Storage Drivers:
  - "aufs":
    - CONFIG_AUFS_FS: missing
  - "btrfs":
    - CONFIG_BTRFS_FS: enabled (as module)
    - CONFIG_BTRFS_FS_POSIX_ACL: enabled
  - "devicemapper":
    - CONFIG_BLK_DEV_DM: enabled (as module)
    - CONFIG_DM_THIN_PROVISIONING: enabled (as module)
  - "overlay":
    - CONFIG_OVERLAY_FS: enabled (as module)
  - "zfs":
    - /dev/zfs: missing
    - zfs command: missing
    - zpool command: missing

Limits:
- /proc/sys/kernel/keys/root_maxkeys: 1000000

其中Generally Necessary 的features 必須全部都是enabled 才能夠執行docker, 若有任何一項沒有enabled, 那麼你可能就必須重新編譯你的linux kernel, 或者是依照此程式裡面的建議方式來啟用這些features.

編譯 docker

查了一下docker官方文件, 要編譯docker, 需要有docker 的環境才能編譯. 這個就很奇怪了, 我就是因為沒有aarch64 版本的docker, 才想要編譯docker for aarch64的啊. 這不就變成雞生蛋,蛋生雞的問題了嗎? 查了許久, 才知道docker 有提供無中生有的編譯方式啦. 只是官方文件沒有說明. 編譯方式我是依照最後面的參考資料 1 來操作的. 步驟如下:

首先 安裝 golang, 版本必須是1.6以上

# apt-get update; apt-get install golang-go

# go version
go version go1.6.2 linux/arm64

安裝編譯時必要的套件

# apt-get install  btrfs-tools libsqlite3-dev libdevmapper-dev

下載docker source code,首先 編譯docker v1.10.2版. 因為v1.10.2 開始有支援ARM aarch64架構

# mkdir -p ~/code
# cd ~/code
# git clone https://github.com/docker/docker
# cd docker
# git checkout v1.10.2

開始編譯

# AUTO_GOPATH=1 ./hack/make.sh dynbinary

編譯完成後訊息如下:

# WARNING! I don't seem to be running in the Docker container.
# The result of this command might be an incorrect build, and will not be
#   officially supported.
#
# Try this instead: make all
#

---> Making bundle: dynbinary (in bundles/1.10.2/dynbinary)
Created binary: bundles/1.10.2/dynbinary/dockerinit-1.10.2
Building: bundles/1.10.2/dynbinary/docker-1.10.2
Created binary: bundles/1.10.2/dynbinary/docker-1.10.2

編譯完後的程式被放在~/code/docker/bundles/1.10.2/dynbinary/

# ls -al ~/code/docker/bundles/1.10.2/dynbinary
total 41788
drwxr-xr-x 2 root root     4096 Sep 22 08:18 .
drwxr-xr-x 3 root root     4096 Sep 22 08:11 ..
lrwxrwxrwx 1 root root       13 Sep 22 08:18 docker -> docker-1.10.2
-rwxr-xr-x 1 root root 32942232 Sep 22 08:18 docker-1.10.2
-rw-r--r-- 1 root root       48 Sep 22 08:18 docker-1.10.2.md5
-rw-r--r-- 1 root root       80 Sep 22 08:18 docker-1.10.2.sha256
lrwxrwxrwx 1 root root       17 Sep 22 08:12 dockerinit -> dockerinit-1.10.2
-rwxr-xr-x 1 root root  9819424 Sep 22 08:12 dockerinit-1.10.2
-rw-r--r-- 1 root root       52 Sep 22 08:12 dockerinit-1.10.2.md5
-rw-r--r-- 1 root root       84 Sep 22 08:12 dockerinit-1.10.2.sha256

我們只需將docker-1.10.2 及 dockerinit-1.10.2 拷貝到適當的目錄下即可:

# cd ~/code/docker/
# mkdir -p /usr/lib/docker/
# cp bundles/1.10.2/dynbinary/docker-1.10.2 /usr/bin/docker
# cp bundles/1.10.2/dynbinary/dockerinit-1.10.2 /usr/lib/docker/dockerinit

測試docker

在使用docker之前, 需要先啟動docker daemon, 啟動方式如下:

# docker daemon &

開始測試docker

# docker run -ti ebspace/aarch64-debian
ERRO[0140] Handler for POST /v1.22/containers/create returned error: No such image: ebspace/aarch64-debian:latest 
Unable to find image 'ebspace/aarch64-debian:latest' locally
latest: Pulling from ebspace/aarch64-debian
59d7421c5877: Pull complete 
Digest: sha256:b4993a58d12d822a62ebf4b683c74716a77078f6310fd0bf97451c9d5465dbd4
Status: Downloaded newer image for ebspace/aarch64-debian:latest

root@e39ac0f7cbcc:/# ls -al
total 68
drwxr-xr-x  20 root root 4096 Sep 22 14:56 .
drwxr-xr-x  20 root root 4096 Sep 22 14:56 ..
-rwxr-xr-x   1 root root    0 Sep 22 14:56 .dockerenv
-rwxr-xr-x   1 root root    0 Sep 22 14:56 .dockerinit
drwxr-xr-x   2 root root 4096 Sep 21 20:56 bin
drwxr-xr-x   2 root root 4096 Sep 12 04:11 boot
drwxr-xr-x   5 root root  380 Sep 22 14:56 dev
drwxr-xr-x  42 root root 4096 Sep 22 14:56 etc
drwxr-xr-x   2 root root 4096 Sep 12 04:11 home
drwxr-xr-x   9 root root 4096 Sep 21 20:49 lib
drwxr-xr-x   2 root root 4096 Sep 21 20:46 media
drwxr-xr-x   2 root root 4096 Sep 21 20:46 mnt
drwxr-xr-x   2 root root 4096 Sep 21 20:46 opt
dr-xr-xr-x 123 root root    0 Sep 22 14:56 proc
drwx------   2 root root 4096 Sep 21 20:46 root
drwxr-xr-x   3 root root 4096 Sep 21 20:46 run
drwxr-xr-x   2 root root 4096 Sep 21 20:56 sbin
drwxr-xr-x   2 root root 4096 Sep 21 20:46 srv
dr-xr-xr-x  12 root root    0 Sep 22 00:39 sys
drwxrwxrwt   2 root root 4096 Sep 21 20:58 tmp
drwxr-xr-x  10 root root 4096 Sep 21 20:46 usr
drwxr-xr-x  11 root root 4096 Sep 21 20:46 var
root@e39ac0f7cbcc:/# exit
exit

# docker ps -a
CONTAINER ID        IMAGE                    COMMAND             CREATED             STATUS                     PORTS               NAMES
e39ac0f7cbcc        ebspace/aarch64-debian   "/bin/bash"         5 minutes ago       Exited (0) 2 minutes ago                       backstabbing_kalam

Yes!! docker 成功產生了一個 container.

編譯最新版本的docker

我們已經編譯出第一個版本的docker v1.10.2 了 (也就是第一隻母雞). 若以後想要試試最新版的docker, 就可以採用官方建議的編譯方式. 用 docker 來 產生新版的 docker. (用這隻母雞來孵雞蛋吧).

依照官方文件, 編譯指令很簡單, 只要兩個指令make build, make binary

更新至最新版本的 docker source code
目前最新release 的版本為 v1.12.1
# cd ~/code/docker
# git pull
# git checkout v1.12.1

編譯 docker
# make build
# make binary

編譯完後, 執行檔會放在 ~/code/docker/bundles/1.12.1/ 目錄裡.
裡面有兩個目錄, 分別為binary-client 以及 binary-daemon

移除舊版版本docker

  1. 首先停止正在執行的 docker deamon.
  2. 然後移除舊的docker 執行檔.
  3. 最後移除 /var/lib/docker 目錄. !!注意!! 這個動作也會遺失舊的docker images, 以及相關資料.

在第三項的目的, 是為了防止docker 新舊版資料相衝突而增加的. 因為v1.10.2版在/var/lib/docker建立的資料, 會造成新版啟動失敗. 所以最好的方式是全部移除乾淨, 再讓新版docker 建立新資料.

安裝新版本 docker

將新編譯後的執行檔拷貝到/usr/bin 目錄即可

# cp ~/code/docker/bundles/1.12.1/binary-client/docker* /usr/bin
# cp ~/code/docker/bundles/1.12.1/binary-daemon/docker* /usr/bin

注意事項

我在編譯v1.12.1 過程中, 有出現編譯失敗, 經查詢後, 原來是記憶體不足而造成編譯失敗. 後來將swap memory 加大之後, 就解決問題了.
Odroid-C2 板子上有2G RAM, 看來拿來編譯docker, 仍然稍嫌不足. 也只能靠swap memory 來解決了.


參考資料:

  1. Getting Docker running on a high-density ARMv8 development board from HiSilicon
This entry was posted in ARM, docker, embedded system, virtual machine and tagged . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *