Add diskimage-creating script, elements for mirrors

* Automatic building Fedora and Ubuntu cloud images with all elements from savanna-extra repository
* Using mirrors for updating images

Change-Id: I12632b5cee42b1dbfd79b7b7c3a7b26962ace625

README.rst

@@ -2,3 +2,11 @@ Savanna image elements project
 ==============================
 
 This repo is a place for Savanna-related for diskimage-builder elements.
+
+Script for creating Fedora and Ubuntu cloud images with our elements and default parameters. You should only need to run this command:
+
+.. sourcecode:: bash
+
+    sudo bash diskimage-create.sh
+
+Note: More information about script `diskimage-create <https://github.com/stackforge/savanna-image-elements/blob/master/diskimage-create/README.rst>`_

diskimage-create/README.rst

@@ -0,0 +1,18 @@
+Diskimage-builder script for creation cloud images
+=================================================
+
+This scrtips builds Ubuntu and Fedora cloud images with default parameters.
+
+For users:
+
+1. Use your environment (export / setenv) to alter the scripts behavior. Environment variables the script accepts are DIB_HADOOP_VERSION, JAVA_DOWNLOAD_URL, OOZIE_DOWNLOAD_URL, HIVE_VERSION, ubuntu_image_name, fedora_image_name.
+
+2. If you want to use your local mirrors, you should specify http urls for Fedora and Ubuntu mirrors using parameters 'FEDORA_MIRROR' and 'UBUNTU_MIRROR' like this:
+
+.. sourcecode:: bash
+
+  sudo USE_MIRRORS=true FEDORA_MIRROR="url_for_fedora_mirror" UBUNTU_MIRROR="url_for_ubuntu_mirror" bash diskimage-create.sh
+
+For developers:
+
+1. If you want to add your element to this repository, you should edit this script in your commit (you should export variables for your element and add name of element to variables 'element_sequence').

diskimage-create/diskimage-create.sh

@@ -0,0 +1,79 @@
+#!/bin/bash
+
+set -e
+
+# Export variables for elements
+
+export DIB_HADOOP_VERSION="1.1.2"
+export JAVA_DOWNLOAD_URL="http://download.oracle.com/otn-pub/java/jdk/7u25-b15/jdk-7u25-linux-x64.tar.gz"
+export ubuntu_image_name="ubuntu_savanna_latest"
+export fedora_image_name="fedora_savanna_latest"
+export OOZIE_DOWNLOAD_URL="http://a8e0dce84b3f00ed7910-a5806ff0396addabb148d230fde09b7b.r31.cf1.rackcdn.com/oozie-3.3.2.tar.gz"
+export HIVE_VERSION="0.11.0"
+
+str=$(head -1 /etc/os-release)
+if [ $str = 'NAME="Ubuntu"' ]; then
+  apt-get update -y
+  apt-get install qemu kpartx git -y
+elif [ $str = 'NAME=Fedora' ]; then
+  yum update -y
+  yum install qemu kpartx git -y
+fi
+
+if [ -d /home/$USER/.cache/image-create ]; then
+  rm -rf /home/$USER/.cache/image-create/*
+fi
+
+cur_dir=$(pwd)
+if [ ! -d "DIB_work" ]; then
+   mkdir DIB_work
+fi
+pushd DIB_work
+
+# Cloning repostiroies
+
+rm -rf diskimage-builder
+git clone https://github.com/openstack/diskimage-builder
+rm -rf savanna-extra
+git clone https://github.com/stackforge/savanna-extra
+
+pushd diskimage-builder
+export DIB_COMMIT_ID=`git show --format=%H | head -1`
+popd
+
+export PATH=$PATH:$cur_dir/DIB_work/diskimage-builder/bin
+export ELEMENTS_PATH=$cur_dir/DIB_work/diskimage-builder/elements
+
+pushd savanna-extra
+export SAVANNA_ELEMENTS_COMMIT_ID=`git show --format=%H | head -1`
+popd
+
+if [ -e $cur_dir/DIB_work/diskimage-builder/sudoers.d/img-build-sudoers ]; then
+  cp $cur_dir/DIB_work/diskimage-builder/sudoers.d/img-build-sudoers /etc/sudoers.d/
+  chown root:root /etc/sudoers.d/img-build-sudoers
+  chmod 0440 /etc/sudoers.d/img-build-sudoers
+fi
+cp -r $cur_dir/DIB_work/savanna-extra/elements/* $cur_dir/DIB_work/diskimage-builder/elements/
+
+ubuntu_elements_sequence="base vm ubuntu hadoop swift_hadoop oozie mysql hive"
+fedora_elements_sequence="base vm fedora hadoop swift_hadoop oozie mysql hive selinux-permissive"
+
+if [ -n "$USE_MIRRORS" ]; then
+  mirror_element=" apt-mirror"
+  ubuntu_elements_sequence=$ubuntu_elements_sequence$mirror_element
+  mirror_element=" yum-mirror"
+  fedora_elements_sequence=$fedora_elements_sequence$mirror_element
+fi
+
+# Creating Ubuntu cloud image
+disk-image-create $ubuntu_elements_sequence -o $ubuntu_image_name
+
+# Creating Fedora cloud image
+# Patameter 'DIB_IMAGE_SIZE' should be specified for Fedora only
+export DIB_IMAGE_SIZE="10"
+disk-image-create $fedora_elements_sequence -o $fedora_image_name
+
+mv $fedora_image_name.qcow2 ../
+mv $ubuntu_image_name.qcow2 ../
+popd
+rm -rf DIB_work

elements/apt-mirror/README.md

@@ -0,0 +1,2 @@
+This element setups mirror for updating Ubuntu cloud image. Using mirror increases speed of building image.
+You should specify http url for Ubuntu mirror using parameter 'UBUNTU_MIRROR'.

elements/apt-mirror/finalise.d/99-setup

@@ -0,0 +1,3 @@
+#!/bin/bash
+
+rm /etc/apt/apt.conf.d/01proxy

elements/apt-mirror/pre-install.d/00-apt

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+mkdir -p /etc/apt/apt.conf.d/
+touch /etc/apt/apt.conf.d/01proxy
+echo -e "Acquire::http { Proxy \"$UBUNTU_MIRROR\"; };" > /etc/apt/apt.conf.d/01proxy

elements/apt-mirror/root.d/0-check

@@ -0,0 +1,5 @@
+#!/bin/bash
+if [ -z "$UBUNTU_MIRROR" ]; then
+  echo "You should specify parameter 'UBUNTU_MIRROR'"
+  exit 2
+fi

elements/yum-mirror/README.md

@@ -0,0 +1,2 @@
+This element setups mirror for updating Fedora cloud image. Using mirror increases speed of building image.
+You should specify http url for Fedora mirror using parameter 'FEDORA_MIRROR'.

elements/yum-mirror/finalise.d/99-setup

@@ -0,0 +1,3 @@
+#!/bin/bash
+
+mv /etc/yum.repos.d/tempfile /etc/yum.repos.d/fedora.repo

elements/yum-mirror/pre-install.d/00-yum

@@ -0,0 +1,12 @@
+#!/bin/bash
+
+mv /etc/yum.repos.d/fedora.repo /etc/yum.repos.d/tempfile
+cat >> /etc/yum.repos.d/fedora.repo <<EOF
+[Local-Repository]
+name=Fedora \$releasever - \$basearch - Local
+baseurl=$FEDORA_MIRROR
+enabled=1
+gpgcheck=0
+priority=1
+EOF
+yum makecache

elements/yum-mirror/root.d/0-check

@@ -0,0 +1,5 @@
+#!/bin/bash
+if [ -z "$FEDORA_MIRROR" ]; then
+  echo "You should specify parameter 'FEDORA_MIRROR'"
+  exit 2
+fi