ubuntu base のAMIを作ってみた

fedora core 4 は古い

awsでamazonが提供しているamiはfedora core 4 をbaseにしている．いくら何でも古いと思ったので，新しいversionのOSを作る方法を探していた．そうすると，Amazon Web Services Developer ConnectionのUbuntu 7.10 Gusty Base Install 64-Bitという記事に，OS作成用のスクリプトとその使い方が書いてあった．試してみたらさっくりとできあがったので，スクリプトを読んで何をやっているのかを調べてみた．

中身の解説

#!/bin/bash
#
# build-ubuntu-7.10-gutsy-ami
#
# This script builds, bundles, and uploads an Ubuntu 7.10 Gutsy base
# install AMI for Amazon EC2.  This can be run on Fedora Core 4/6 AMIs
# such as ami-20b65349 (32-bit) or ami-36ff1a5f (64-bit)

このスクリプトが動くことを確認したのはami-20b65349(32-bit)の方．64-bitの方ではまだfuseを動かす方法が見つかっていないので，32-bitの方が楽しそう．

#
# For an Amazon EC2 AMI built using this script see:
#
#   http://ec2gutsy.notlong.com
#
# Expects to be run in the following environment:
#
#   Certificate file: /mnt/cert-*.pem
#   Private Key file: /mnt/pk-*.pem
#
#   export AWS_USER_ID=
#   export AWS_ACCESS_KEY_ID=
#   export AWS_SECRET_ACCESS_KEY=
#   export bucket=
#   export prefix=ubuntu-7.10-gutsy-base-$(date +%Y%m%d)

AMIを起動した後，cert-*.pem, pk-*.pemをローカル環境から起動したインスタンスに転送し，上でexportしている環境変数を設定すれば準備完了．後はこのスクリプトを動かすだけ．

#
# History:
#
#   2007-12-25 Eric Hammond <ehammond@thinksome.com>
#   - Install fuse kernel module (32-bit)
#   - Upgrade to debootstrap 1.0.7
#
#   2007-12-02 Eric Hammond <ehammond@thinksome.com>
#   - Use architecture "amd64" instead of "i386" for debootstrap on 64-bit
#   - Add ia32-libs compatability package for 64-bit
#
#   2007-12-01 Eric Hammond <ehammond@thinksome.com>
#   - Add support for building on 64-bit kernel (large, extra large instances)
#
#   2007-11-23 Eric Hammond <ehammond@thinksome.com>
#   - ssh credentials retrieved from instance parameters or ephemeral storage.
#   - Patch ec2-unbundle to work on Ubuntu
#   - Also add locale to /etc/default/locale
#
#   2007-11-22 Eric Hammond <ehammond@thinksome.com>
#   - Upgrade Ubuntu AMI tools patch to match new AMI tools source.
#   - Install ca-certificates to better support ec2-upload-bundle per:
#     http://developer.amazonwebservices.com/connect/thread.jspa?threadID=16543&tstart=0
#   - ec2-bundle-vol excludes /etc/udev/rules.d/70-persistent-net.rules
#     so that the network works on a rebundled instance, per:
#   http://developer.amazonwebservices.com/connect/message.jspa?messageID=70873
#
#   2007-11-18 Eric Hammond <ehammond@thinksome.com>
#   - Original put together based on code, tricks, and fixes from many
#     others.
#

set -e
set -x

"set -e"は"Exit immediately if a simple command exits with a none-zero status."で，"set -x"は"After expanding each simple command, display the expanded value of PS4, followed by the command and its expantded arguments"だから，スクリプト内の各コマンドの終了コードが0（正常）でなければスクリプト停止．スクリプト内の各コマンドとその結果を標準出力する．こんなオプションを知らなかった．

mkdir -p /mnt/build
cd /mnt/build
mkdir -p ubuntu

# For Fedora
yum install -y binutils
# For Ubuntu
# apt-get install -y binutils

binutilsをインストールするのは，この後でarを使っているから．

if [ $(uname -i) = 'x86_64' ]
then
  modules=http://s3.amazonaws.com/ec2-downloads/ec2-modules-2.6.16.33-xenU-x86_64.tgz
  bundlearch="x86_64"
  bsarch="amd64"
#  bsarch="i386"
else
  modules=http://s3.amazonaws.com/ec2-downloads/modules-2.6.16-ec2.tgz
  bundlearch="i386"
  bsarch="i386"
fi

AMIが起動すると，EC2標準のXen 2.6.16 kernelが動く．つまり自分用にkernelを作ってもそれを使うことはできない．ただし，kernel moduleは使うことができるのでそれが回避策．上のtgzはAmazonが用意したコンパイル済みのモジュール．

# Bootstrap Ubuntu
BSURL=http://archive.ubuntu.com/ubuntu/pool/main/d/debootstrap
BSVER=debootstrap_1.0.7
BSDIR=debootstrap
curl -s ${BSURL}/${BSVER}.tar.gz |
  tar xz
curl -s $BSURL/${BSVER}_all.deb > /tmp/${BSVER}_all.deb
ar p /tmp/${BSVER}_all.deb data.tar.gz |
  tar xvzOf - ./usr/share/debootstrap/devices.tar.gz > $BSDIR/devices.tar.gz
ln -s ../scripts/ubuntu/gutsy ${BSDIR}/scripts/gutsy
export DEBOOTSTRAP_DIR=${BSDIR}
${BSDIR}/debootstrap --arch $bsarch gutsy ubuntu http://mirrors.kernel.org/ubuntu
mount -t proc none ubuntu/proc

debootstrapについての資料が見つからなくて，data.tar.gzをdebootstrap_1.0.7_all.debから引っ張りだしてdebootstrap/devices.tar.gzとして置いている理由がよくわからない．が，debootstrapスクリプトの中で

if [ "$DEBOOTSTRAP_DIR" = "" ]; then
        if [ -x /debootstrap/debootstrap ]; then
                DEBOOTSTRAP_DIR=/debootstrap
        else
                DEBOOTSTRAP_DIR=/usr/share/debootstrap
        fi
fi

if [ -x "/usr/bin/gettext" ]; then
        USE_GETTEXT_INTERACTION=yes
fi

DEVICES_TARGZ=$DEBOOTSTRAP_DIR/devices.tar.gz

となっているので，それに併せているっぽい．DEBOOTSTRAP_DIRをexportしているのも同様．元々debootstrapはDVDからインストールするためのツールらしい．

# Change these to your locale and timezone
chroot ubuntu localedef -i en_US -c -f UTF-8 en_US.UTF-8
echo 'America/Los_Angeles' >ubuntu/etc/timezone
echo 'LANG="en_US.UTF-8"'  >ubuntu/etc/default/locale
cp -f ubuntu/usr/share/zoneinfo/US/Pacific ubuntu/etc/localtime

ここいらが日本語環境に併せて変更すべき点．試してないけど，たぶんこんな感じ．

chroot ubuntu localedef -i ja_JP ja_JP.UTF-8 -c -f UTF-8
echo 'Asia/Tokyo' >ubuntu/etc/timezone
echo 'LANG="ja_JP.UTF-8"'  >ubuntu/etc/default/locale
cp -f ubuntu/usr/share/zoneinfo/Japan ubuntu/etc/localtime

スクリプトはさらに続く

# Basic sources.list
cat <<'EOF' >ubuntu/etc/apt/sources.list
deb http://mirrors.kernel.org/ubuntu gutsy main restricted universe multiverse
deb-src http://mirrors.kernel.org/ubuntu gutsy main restricted universe multiverse

deb http://mirrors.kernel.org/ubuntu gutsy-updates main restricted universe multiverse
deb-src http://mirrors.kernel.org/ubuntu gutsy-updates main restricted universe multiverse

deb http://security.ubuntu.com/ubuntu gutsy-security main restricted universe multiverse
deb-src http://security.ubuntu.com/ubuntu gutsy-security main restricted universe multiverse
EOF

# Update packages
chroot ubuntu apt-get update

if [ "$bundlearch" = "i386" ]
then
  # Performance under Xen. Doesn't seem to be needed for 64-bit?
  chroot ubuntu apt-get install -y libc6-xen
else
  chroot ubuntu apt-get install -y ia32-libs
fi

# Upgrade packages
chroot ubuntu apt-get -y upgrade

基本パッケージをインストールしたので，パッケージ情報の更新，Xen用ライブラリのインストール，パッケージの最新化を実施．

# Stuff we need
chroot ubuntu apt-get install -y openssh-server ruby libopenssl-ruby1.8 rsync openssl curl

# Needed for curl SSL in ec2-upload-bundle
chroot ubuntu apt-get install ca-certificates

# EC2 kernel modules
curl -s $modules | tar xzC ubuntu
chroot ubuntu depmod -a

# Base packages in Ubuntu server install
chroot ubuntu apt-get install -y ubuntu-standard

ubuntu本体と，S3を操作するためのAMI Tools用パッケージをインストール．この箇所でEC2用のkernel module($modules)もインストールしている．

# Xen epects a single tty1
/bin/rm -f ubuntu/etc/event.d/tty[2-6]

# Security
chroot ubuntu shadowconfig on
chroot ubuntu passwd -l root

# Basic networking
cat <<'EOF' >ubuntu/etc/network/interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
EOF

cat <<'EOF' >ubuntu/etc/hosts
127.0.0.1 localhost.localdomain localhost

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
EOF

cat <<'EOF' >>ubuntu/etc/ssh/sshd_config
UseDNS no
EOF

# get-credentials.sh
cat <<'EOF' >ubuntu/usr/local/sbin/get-credentials.sh
#!/bin/bash
# Retrieve the ssh credentials and add to authorized_keys file.
# Based on /usr/local/sbin/get-credentials.sh from ami-20b65349
public_key_url=http://169.254.169.254/1.0/meta-data/public-keys/0/openssh-key
public_key_file=/tmp/openssh_id.pub
public_key_ephemeral=/mnt/openssh_id.pub
authorized_keys=/root/.ssh/authorized_keys
test -d /root/.ssh || mkdir -p -m 700 /root/.ssh
curl --retry 3 --retry-delay 2 --silent --fail -o $public_key_file $public_key_url
if [ $? -eq 0 -a -e $public_key_file ] ; then
  if ! grep -q -f $public_key_file $authorized_keys
  then
    cat $public_key_file >> $authorized_keys
    echo "New ssh key added to $authorized_keys from $public_key_url" |
      logger -t "ec2"
  fi
  chmod 600 $authorized_keys
  rm -f $public_key_file
elif [ -e $public_key_ephemeral ] ; then
  if ! grep -q -f $public_key_ephemeral $authorized_keys
  then 
    cat $public_key_ephemeral >> $authorized_keys
    echo "New ssh key added to $authorized_keys from $public_key_ephemeral" |
      logger -t "ec2"
  fi
  chmod 600 $authorized_keys
  chmod 600 $public_key_ephemeral
fi
if [ -e /mnt/openssh_id.pub ] ; then
 if ! grep -q -f /mnt/openssh_id.pub /root/.ssh/authorized_keys
 then
   cat /mnt/openssh_id.pub >> /root/.ssh/authorized_keys
   echo "New key added to authorized keys file from ephemeral store"|logger -t "ec2"
  fi
  chmod 600 /root/.ssh/authorized_keys
fi
EOF
chmod 755 ubuntu/usr/local/sbin/get-credentials.sh

# motd
cat <<'EOF' >ubuntu/etc/rc.local
#!/bin/sh -e
# Create /tmp if missing (as it's nice to bundle without it).
test -d /tmp || mkdir -m 01777 /tmp

# Get ssh credentials from instance parameters or ephemeral storage.
/usr/local/sbin/get-credentials.sh

cat <<EOMOTD >>/etc/motd

Amazon EC2 Ubuntu 7.10 gutsy AMI built using code customized by
Eric Hammond <ehammond@thinksome.com>

For more information:

  http://ec2gutsy.notlong.com

EOMOTD

exit 0
EOF

# Install and patch Amazon AMI tools
curl -s http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.noarch.rpm \
  > ubuntu/tmp/ec2-ami-tools.noarch.rpm
chroot ubuntu apt-get install -y alien patch
chroot ubuntu alien -i /tmp/ec2-ami-tools.noarch.rpm

ネットワーク周りの設定とか，sshでアクセスするためのサーバ側のスクリプトとか，基本的なセキュリティ設定とかを行ってる．

ln -s /usr/lib/site_ruby/aes ubuntu/usr/local/lib/site_ruby/1.8/aes

cat > /tmp/amiutil.patch <<'EOF'
--- /usr/lib/site_ruby/aes/amiutil.orig/unbundle.rb.orig	2007-11-22 21:17:07.000000000 -0800
+++ /usr/lib/site_ruby/aes/amiutil/unbundle.rb	2007-11-22 21:17:13.000000000 -0800
@@ -46,13 +46,13 @@
     
     # Join, decrypt, decompress and untar.
     cmd = 
-      "openssl sha1 < #{digest_pipe} & \
+      "/bin/bash -c 'openssl sha1 < #{digest_pipe} & \
        cat #{part_files} | \
        openssl enc -d -aes-128-cbc -K #{key} -iv #{iv} | \
        gunzip | \
        tee #{digest_pipe} | \
        tar -x -C #{dst_dir}; \
-       for i in ${PIPESTATUS[@]}; do [ $i == 0 ] || exit $i; done"
+       for i in ${PIPESTATUS[@]}; do [ $i == 0 ] || exit $i; done'"
     output = `#{cmd}`.chomp.strip()
     unless $?.exitstatus == 0
       STDERR.puts "Unbundle failed."
diff -ru /usr/lib/site_ruby/aes/amiutil.orig/bundle.rb /usr/lib/site_ruby/aes/amiutil/bundle.rb
--- /usr/lib/site_ruby/aes/amiutil.orig/bundle.rb	2007-11-20 06:06:07.000000000 -0500
+++ /usr/lib/site_ruby/aes/amiutil/bundle.rb	2007-11-22 16:44:55.000000000 -0500
@@ -60,11 +60,11 @@
         # Cat the file and tee it to the digest process and to the 
         # tar, gzip and encryption process pipeline.
         pipe_cmd = 
-          "tar -chS -C #{File::dirname( image_file )} #{File::basename( image_file )} | \
+          "/bin/bash -c 'tar -chS -C #{File::dirname( image_file )} #{File::basename( image_file )} | \
            tee #{PIPE1} | gzip | \
            openssl enc -e -aes-128-cbc -K #{key} -iv #{iv} > \
            #{bundled_file_path}; \
-           for i in ${PIPESTATUS[@]}; do [ $i == 0 ] || exit $i; done"
+           for i in ${PIPESTATUS[@]}; do [ $i == 0 ] || exit $i; done'"
         unless system( pipe_cmd ) and $?.exitstatus == 0
           raise "error executing #{pipe_cmd}, exit status code #{$?.exitstatus}"
         end

diff -ru /usr/lib/site_ruby/aes/amiutil.orig/bundlevol.rb /usr/lib/site_ruby/aes/amiutil/bundlevol.rb
--- /usr/lib/site_ruby/aes/amiutil.orig/bundlevol.rb	2007-11-08 03:59:32.000000000 -0800
+++ /usr/lib/site_ruby/aes/amiutil/bundlevol.rb	2007-11-18 20:01:32.000000000 -0800
@@ -78,7 +78,7 @@
 TEXT
 
 ALWAYS_EXCLUDED = ['/dev', '/media', '/mnt', '/proc', '/sys']
-LOCAL_FS_TYPES = ['ext2', 'ext3', 'xfs', 'jfs', 'reiserfs']
+LOCAL_FS_TYPES = ['ext2', 'ext3', 'xfs', 'jfs', 'reiserfs', 'tmpfs']
 MAX_SIZE_MB = 10 * 1024  # 10 GB in MB
 MTAB_PATH = '/etc/mtab'
 DEBUGON = 'on'
diff -ru /usr/lib/site_ruby/aes/amiutil.orig/image.rb /usr/lib/site_ruby/aes/amiutil/image.rb
--- /usr/lib/site_ruby/aes/amiutil.orig/image.rb	2007-11-08 03:59:32.000000000 -0800
+++ /usr/lib/site_ruby/aes/amiutil/image.rb	2007-11-18 19:59:39.000000000 -0800
@@ -158,7 +158,7 @@
     #
     exclude = ''
     @exclude.each {|x| exclude += "--exclude '" + x + "' "}
-    exec( 'rsync -arlS ' + exclude + File::join( src, '*' ) + ' ' + dst )
+    exec( 'rsync -rlpgoDS ' + exclude + '--exclude /etc/udev/rules.d/70-persistent-net.rules ' + File::join( src, '*' ) + ' ' + dst )
   end
 
   #----------------------------------------------------------------------------#
--- /usr/lib/ruby/aes/amiutil.orig/http.rb	2007-11-08 03:59:32.000000000 -0800
+++ /usr/lib/ruby/aes/amiutil/http.rb	2007-11-19 01:41:06.000000000 -0800
@@ -58,7 +58,7 @@
     tf.close(false)
     
     begin
-      cmd_line = "curl -f  #{curl_arguments} #{url} 2> #{tf.path} | tee #{path} | openssl sha1; exit ${PIPESTATUS[0]}"
+      cmd_line = "/bin/bash -c 'curl -f  #{curl_arguments} #{url} 2> #{tf.path} | tee #{path} | openssl sha1; exit ${PIPESTATUS[0]}'"
       calculated_digest = IO.popen( cmd_line ) { |io| io.readline.chomp }
       
       unless $?.exitstatus == 0
EOF

chroot ubuntu patch -d /usr/lib/site_ruby/aes/amiutil < /tmp/amiutil.patch

どのバージョンからなのか資料を見つけられなかったが，このバージョンのubuntuではAMI Toolsが正しく動かないようで，その問題を回避するためのパッチを充てている．

# Install a copy of the fuse kernel module (copied from ami-26b6534f)
if [ "$bundlearch" = "i386" ]
then
  mkdir -p ubuntu/lib/modules/2.6.16-xenU/kernel/fs/fuse
  curl -s -o ubuntu/lib/modules/2.6.16-xenU/kernel/fs/fuse/fuse.ko \
    http://level22.s3.amazonaws.com/20071209-ami-26b6534f/fuse.ko
  chroot ubuntu depmod
  #TBD: Need to support 64-bit also
fi

fuse用のkernel moduleを追加．

# Might as well have a current slocate database
chroot ubuntu updatedb

# cleanup
chroot ubuntu apt-get -y autoremove --purge alien patch
chroot ubuntu apt-get clean
rm -rf ubuntu/tmp/* ubuntu/root/.bash_history

OSイメージをS3にアップロードする前にディスクの中身を整頓．

# Bundle & upload to S3
cp /mnt/{pk,cert}-*.pem ubuntu/tmp/

chroot ubuntu ec2-bundle-vol            \
  -r $bundlearch                        \
  -d /tmp                               \
  -p $prefix                            \
  -u $AWS_USER_ID                       \
  -k /tmp/pk-*.pem                      \
  -c /tmp/cert-*.pem                    \
  -s 4096                               \
  -e /tmp

ec2-upload-bundle                       \
    -b $bucket                          \
    -m ubuntu/tmp/$prefix.manifest.xml  \
    -a $AWS_ACCESS_KEY_ID               \
    -s $AWS_SECRET_ACCESS_KEY

OSイメージをS3にアップロード

umount ubuntu/proc

set +x

cat <<EOF

Now you might want to run this command:
  ec2-register $bucket/$prefix.manifest.xml

EOF

これで終了．アップロードしたイメージをamiとして登録するためには，AMI CLI Toolの一つである，ec2-registerを実行する必要がある．ちなみに，fuseを使ってS3をファイルシステムのように使う方法もこのスクリプトを作った人が見つけている．これは，また後で試してみるつもり．