DIB流程-2 root阶段-谢先斌的博客

DIB流程-2 root阶段

发布时间： 2017-05-08 更新时间： 2024-04-14 总字数：2000 阅读时间：4m 作者：谢先斌 IP上海

diskimage-builder 是openstack社区用于制作镜像的工具.为了深入了解dib制作镜像的全过程,对一个简单的例子进行贯通的分析。

怎么让猪跑

这里先由一个简单的例子,来看看"猪是怎么跑的",顺便"吃点猪肉":

$ disk-image-create vm ubuntu-minimal

该例子中的root阶段的脚本

之前了解了run_d方法的流程,但是实际上完成root阶段工作的还是root阶段的脚本,所以接下来看看在这里例子中每个脚本做了什么.

这里对应了日志中的757:1154line

root阶段的脚本从日志757line开始,之前导入创建了钩子文件夹,导入了一些环境变量.

运行如下命令,就可以看到生产的钩子文件夹中的内容:

$ break=before-root disk-image-create  vm ubuntu
$ cd /tmp/dib_build.xxxx/hooks

查看所有脚本:

-rwxrwxr-x  1 xion xion  611 10月  1 11:22 01-ccache
-rwxrwxr-x  1 xion xion 2523 10月  1 11:22 10-cache-ubuntu-tarball
-rwxrwxr-x  1 xion xion  308 10月  1 11:22 50-build-with-http-cache
-rwxrwxr-x  1 xion xion  486 10月  1 11:22 60-block-apt-translations
-rwxrwxr-x  1 xion xion  365 10月  1 11:22 90-base-dib-run-parts
-rwxrwxr-x  1 xion xion  983 10月  1 11:22 99-block-daemons
-rwxrwxr-x  1 xion xion  376 10月  1 11:22 99-shared_apt_cache
-rwxrwxr-x  1 xion xion  403 10月  1 11:22 99-trim-dpkg

下面按照顺序说明每个脚本的作用

01-ccache

  1 #!/bin/bash
  2
  3 if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
  4     set -x
  5 fi
  6 set -eu
  7 set -o pipefail
  8
  9 # Don't do anything if already mounted (if disk-image-create is invoked with
 10 # no elements specified, this hook actually fires twice, once during
 11 # `run_d root` for the base element, then again when `run_d root` is called
 12 # after automatically pulling in the Ubuntu element)
 13 grep " $TMP_MOUNT_PATH/tmp/ccache" /proc/mounts && exit
 14
 15 DIB_CCACHE_DIR=${DIB_CCACHE_DIR:-$DIB_IMAGE_CACHE/ccache}
 16 mkdir -p $DIB_CCACHE_DIR
 17
 18 sudo mkdir -p $TMP_MOUNT_PATH/tmp/ccache
 19 sudo mount --bind $DIB_CCACHE_DIR $TMP_MOUNT_PATH/tmp/ccache

这个脚本属于 base element. 此处该做脚本做了下面事情

检查cache文件夹(一个用于存放临时文件的文件夹)是否已经挂载如果没有挂载,创建cache文件夹和挂载点,并且将其挂载到镜像build的目录下(使用bind的mount方式)

10-cache-ubuntu-tarball

  1 #!/bin/bash
  2 # These are useful, or at worst not harmful, for all images we build.
  3
  4 if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then
  5     set -x
  6 fi
  7 set -eu
  8 set -o pipefail
  9
 10 [ -n "$ARCH" ]
 11 [ -n "$TARGET_ROOT" ]
 12
 13 shopt -s extglob
 14
 15 DIB_CLOUD_IMAGES=${DIB_CLOUD_IMAGES:-http://cloud-images.ubuntu.com}
 16 DIB_RELEASE=${DIB_RELEASE:-trusty}
 17 BASE_IMAGE_FILE=${BASE_IMAGE_FILE:-$DIB_RELEASE-server-cloudimg-$ARCH-root.tar.gz}
 18 SHA256SUMS=${SHA256SUMS:-https://${DIB_CLOUD_IMAGES##http?(s)://}/$DIB_RELEASE/current/SHA256SU    MS}
 19 CACHED_FILE=$DIB_IMAGE_CACHE/$BASE_IMAGE_FILE
 20 CACHED_FILE_LOCK=$DIB_IMAGE_CACHE/$BASE_IMAGE_FILE.lock
 21 CACHED_SUMS=$DIB_IMAGE_CACHE/SHA256SUMS.ubuntu.$DIB_RELEASE.$ARCH
 22
 23 function get_ubuntu_tarball() {
 24     if [ -n "$DIB_OFFLINE" -a -f "$CACHED_FILE" ] ; then
 25         echo "Not checking freshness of cached $CACHED_FILE."
 26     else
 27         echo "Fetching Base Image"
 28         $TMP_HOOKS_PATH/bin/cache-url $SHA256SUMS $CACHED_SUMS
 29         $TMP_HOOKS_PATH/bin/cache-url \
 30             $DIB_CLOUD_IMAGES/$DIB_RELEASE/current/$BASE_IMAGE_FILE $CACHED_FILE
 31         pushd $DIB_IMAGE_CACHE
 32         if ! grep "$BASE_IMAGE_FILE" $CACHED_SUMS | sha256sum --check - ; then
 33             # It is likely that an upstream http(s) proxy has given us a skewed
 34             # result - either a cached SHA file or a cached image. Use cache-busting
 35             # to get (as long as caches are compliant...) fresh files.
 36             # Try the sha256sum first, just in case that is the stale one (avoiding
 37             # downloading the larger image), and then if the sums still fail retry
 38             # the image.
 39             $TMP_HOOKS_PATH/bin/cache-url -f $SHA256SUMS $CACHED_SUMS
 40             if ! grep "$BASE_IMAGE_FILE" $CACHED_SUMS | sha256sum --check - ; then
 41                 $TMP_HOOKS_PATH/bin/cache-url -f \
 42                     $DIB_CLOUD_IMAGES/$DIB_RELEASE/current/$BASE_IMAGE_FILE $CACHED_FILE
 43                 grep "$BASE_IMAGE_FILE" $CACHED_SUMS | sha256sum --check -
 44             fi
 45         fi
 46         popd
 47     fi
 48     # Extract the base image (use --numeric-owner to avoid UID/GID mismatch between
 49     # image tarball and host OS e.g. when building Ubuntu image on an openSUSE host)
 50     sudo tar -C $TARGET_ROOT --numeric-owner -xzf $DIB_IMAGE_CACHE/$BASE_IMAGE_FILE
 51 }
 52
 53 (
 54     echo "Getting $CACHED_FILE_LOCK: $(date)"
 55     # Wait up to 20 minutes for another process to download
 56     if ! flock -w 1200 9 ; then
 57         echo "Did not get $CACHED_FILE_LOCK: $(date)"
 58         exit 1
 59     fi
 60     get_ubuntu_tarball
 61 ) 9> $CACHED_FILE_LOCK

这个脚本属于ubuntu element 它做了以下事情:

等待锁下载镜像:

NOTE

shopt -s extglob

shopt -s extglob 用于开启支持更多的通配符
?(pattern)        匹配0次或1次；
*(pattern)         匹配0次以上包括0次；
+(pattern)        匹配1次以上包括1次；
@(pattern)       匹配1次；
!(pattern)          不匹配。

BASH Locking

(
    # Wait for lock on /var/lock/.myscript.exclusivelock (fd 200) for 10 seconds
    flock -n 200

    # Do stuff

) 200>/var/lock/.myscript.exclusivelock
200>/var/lock/.myscript.exclusivelock 的意思是将该程序的文件描述符号200指向"/var/lock/.myscript.exclusivelock"文件.
flock -w 1200 9的意思是等待1200s,等待文件描述符为9的文件.

pushd popd

pushd的全称是push dir
栈顶的目录就会作为当前的工作目录
popd的全称是pop dir
popd会弹出目前栈顶的目录

50-build-with-http-cache

1 #!/bin/bash
2
3 if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
4     set -x
5 fi
6 set -eu
7 set -o pipefail
8
9 [ -n "$TARGET_ROOT" ]
10
11 # If we have a network proxy, use it.
12 if [ -n "${http_proxy:-}" ] ; then
13     sudo dd of=$TARGET_ROOT/etc/apt/apt.conf.d/60img-build-proxy << _EOF_
14 Acquire::http::Proxy "$http_proxy";
15 _EOF_
16 fi

这个脚本属于dpkg 它的作用就是如果配置了代理,就将Acquire::http::Proxy “$http_proxy”;写入配置文件中

60-block-apt-translations

1 #!/bin/bash
2
3 if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
4     set -x
5 fi
6 set -eu
7 set -o pipefail
8
9 [ -n "$TARGET_ROOT" ]
10
11 # Configure APT not to fetch translations files
12 sudo dd of=$TARGET_ROOT/etc/apt/apt.conf.d/95no-translations <<EOF
13 Acquire::Languages "none";
14 EOF
15
16 # And now make sure that we don't fall foul of Debian bug 641967
17 find $TARGET_ROOT/var/lib/apt/lists/ -path $TARGET_ROOT/var/lib/apt/lists/partial -prune -o -ty    pe f -name '*_i18n_Translation-*' -print -exec sudo rm -f {} +

这个脚本属于dpkg元素这个脚本做了下面这些事:

配置apt不获取 translations files

90-base-dib-run-parts

1 #!/bin/bash
2
3 if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
4     set -x
5 fi
6 set -eu
7 set -o pipefail
8
9 # Abort early if dib-run-parts is not found to prevent a meaningless
10 # error message from the subsequent install command
11 DIB_RUN_PARTS=$(which dib-run-parts)
12
13 exec sudo install -m 0755 -o root -g root -D \
14     $DIB_RUN_PARTS \
15     $TARGET_ROOT/usr/local/bin/dib-run-parts

这个脚本的作用是将dib-run-parts放到镜像的bin目录下

99-block-daemons

1 #!/bin/bash
2
3 if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
4     set -x
5 fi
6 set -eu
7 set -o pipefail
8
9 [ -n "$TARGET_ROOT" ]
10
11 # Prevent package installs from starting daemons
12 sudo mv $TARGET_ROOT/sbin/start-stop-daemon $TARGET_ROOT/sbin/start-stop-daemon.REAL
13 sudo dd of=$TARGET_ROOT/sbin/start-stop-daemon <<EOF
14 #!/bin/sh
15 echo
16 echo "Warning: Fake start-stop-daemon called, doing nothing"
17 EOF
18 sudo chmod 755 $TARGET_ROOT/sbin/start-stop-daemon
19
20 if [ -f $TARGET_ROOT/sbin/initctl ]; then
21     sudo mv $TARGET_ROOT/sbin/initctl $TARGET_ROOT/sbin/initctl.REAL
22     sudo dd of=$TARGET_ROOT/sbin/initctl <<EOF
23 #!/bin/sh
24 echo "initctl (tripleo 1.0)"
25 echo "Warning: Fake initctl called, doing nothing"
26 EOF
27     sudo chmod 755 $TARGET_ROOT/sbin/initctl
28 fi
29
30 sudo dd of=$TARGET_ROOT/usr/sbin/policy-rc.d <<EOF
31 #!/bin/sh
32 # 101 Action not allowed. The requested action will not be performed because
33 #     of runlevel or local policy constraints.
34 exit 101
35 EOF
36 sudo chmod 755 $TARGET_ROOT/usr/sbin/policy-rc.d

这个脚本的作用是防止很多进程和服务自动启动

###NOTE policy-rc.d

99-shared_apt_cache

1 #!/bin/bash
2
3 if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
4     set -x
5 fi
6 set -eu
7 set -o pipefail
8
9 DIB_APT_LOCAL_CACHE=${DIB_APT_LOCAL_CACHE:-1}
10
11 if [ $DIB_APT_LOCAL_CACHE = "0" ]; then
12     exit 0
13 fi
14
15 apt_cache_dir=$DIB_IMAGE_CACHE/apt/$DISTRO_NAME
16 if [ ! -d $apt_cache_dir ]; then
17     mkdir -p $apt_cache_dir
18 fi
19 sudo mount --bind $apt_cache_dir $TARGET_ROOT/var/cache/apt/archives
~

这个脚本的作用就是创建了一个apt_cache_dir目录用于cacheapt的包,然后把这个目录挂载到了镜像下的目录

99-trim-dpkg

1 #!/bin/bash
2
3 if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
4     set -x
5 fi
6 set -eu
7 set -o pipefail
8
9 [ -n "$TARGET_ROOT" ]
10
11 # During image build, sync calls are expensive overhead
12 echo 'force-unsafe-io' | sudo tee $TARGET_ROOT/etc/dpkg/dpkg.cfg.d/02apt-speedup > /dev/null
13
14 # and remove the translations, too
15 echo 'Acquire::Languages "none";' | sudo tee $TARGET_ROOT/etc/apt/apt.conf.d/no-languages > /de    v/null

这个脚本配为dpkg配置了force-unsafe-io和无语言,主要是dpkg的配置.

下一篇介绍extra-data.d