commit 9e1d71c8fe242b6a95ebd2663798bc0792a37267 Author: Dean Troyer Date: Fri Aug 3 17:07:12 2012 -0500 Initial repo setup diff --git a/HACKING.rst b/HACKING.rst new file mode 100644 index 00000000..1a80eb73 --- /dev/null +++ b/HACKING.rst @@ -0,0 +1,151 @@ +Contributing to Grenade +======================= + + +General +------- + +Grenade is written in POSIX shell script. It is specifies BASH and is +compatible with Bash 3. + +Grenade's official repository is located on GitHub at ??. + + +*** old devstack template needs to be upgraded below *** + + +Scripts +------- + +DevStack scripts should generally begin by calling ``env(1)`` in the shebang line:: + + #!/usr/bin/env bash + +Sometimes the script needs to know the location of the DevStack install directory. +``TOP_DIR`` should always point there, even if the script itself is located in +a subdirectory:: + + # Keep track of the current devstack directory. + TOP_DIR=$(cd $(dirname "$0") && pwd) + +Many scripts will utilize shared functions from the ``functions`` file. There are +also rc files (``stackrc`` and ``openrc``) that are often included to set the primary +configuration of the user environment:: + + # Keep track of the current devstack directory. + TOP_DIR=$(cd $(dirname "$0") && pwd) + + # Import common functions + source $TOP_DIR/functions + + # Import configuration + source $TOP_DIR/openrc + +``stack.sh`` is a rather large monolithic script that flows through from beginning +to end. There is a proposal to segment it to put the OpenStack projects +into their own sub-scripts to better document the projects as a unit rather than +have it scattered throughout ``stack.sh``. Someday. + + +Documentation +------------- + +The official DevStack repo on GitHub does not include a gh-pages branch that +GitHub uses to create static web sites. That branch is maintained in the +`CloudBuilders DevStack repo`__ mirror that supports the +http://devstack.org site. This is the primary DevStack +documentation along with the DevStack scripts themselves. + +__ repo_ +.. _repo: https://github.com/cloudbuilders/devstack + +All of the scripts are processed with shocco_ to render them with the comments +as text describing the script below. For this reason we tend to be a little +verbose in the comments _ABOVE_ the code they pertain to. Shocco also supports +Markdown formatting in the comments; use it sparingly. Specifically, ``stack.sh`` +uses Markdown headers to divide the script into logical sections. + +.. _shocco: http://rtomayko.github.com/shocco/ + + +Exercises +--------- + +The scripts in the exercises directory are meant to 1) perform basic operational +checks on certain aspects of OpenStack; and b) document the use of the +OpenStack command-line clients. + +In addition to the guidelines above, exercise scripts MUST follow the structure +outlined here. ``swift.sh`` is perhaps the clearest example of these guidelines. +These scripts are executed serially by ``exercise.sh`` in testing situations. + +* Begin and end with a banner that stands out in a sea of script logs to aid + in debugging failures, particularly in automated testing situations. If the + end banner is not displayed, the script ended prematurely and can be assumed + to have failed. + + :: + + echo "**************************************************" + echo "Begin DevStack Exercise: $0" + echo "**************************************************" + ... + set +o xtrace + echo "**************************************************" + echo "End DevStack Exercise: $0" + echo "**************************************************" + +* The scripts will generally have the shell ``xtrace`` attribute set to display + the actual commands being executed, and the ``errexit`` attribute set to exit + the script on non-zero exit codes:: + + # This script exits on an error so that errors don't compound and you see + # only the first error that occured. + set -o errexit + + # Print the commands being run so that we can see the command that triggers + # an error. It is also useful for following allowing as the install occurs. + set -o xtrace + +* Settings and configuration are stored in ``exerciserc``, which must be + sourced after ``openrc`` or ``stackrc``:: + + # Import exercise configuration + source $TOP_DIR/exerciserc + +* There are a couple of helper functions in the common ``functions`` sub-script + that will check for non-zero exit codes and unset environment variables and + print a message and exit the script. These should be called after most client + commands that are not otherwise checked to short-circuit long timeouts + (instance boot failure, for example):: + + swift post $CONTAINER + die_if_error "Failure creating container $CONTAINER" + + FLOATING_IP=`euca-allocate-address | cut -f2` + die_if_not_set FLOATING_IP "Failure allocating floating IP" + +* If you want an exercise to be skipped when for example a service wasn't + enabled for the exercise to be run, you can exit your exercise with the + special exitcode 55 and it will be detected as skipped. + +* The exercise scripts should only use the various OpenStack client binaries to + interact with OpenStack. This specifically excludes any ``*-manage`` tools + as those assume direct access to configuration and databases, as well as direct + database access from the exercise itself. + +* If specific configuration needs to be present for the exercise to complete, + it should be staged in ``stack.sh``, or called from ``stack.sh`` (see + ``files/keystone_data.sh`` for an example of this). + +* The ``OS_*`` environment variables should be the only ones used for all + authentication to OpenStack clients as documented in the CLIAuth_ wiki page. + +.. _CLIAuth: http://wiki.openstack.org/CLIAuth + +* The exercise MUST clean up after itself if successful. If it is not successful, + it is assumed that state will be left behind; this allows a chance for developers + to look around and attempt to debug the problem. The exercise SHOULD clean up + or graciously handle possible artifacts left over from previous runs if executed + again. It is acceptable to require a reboot or even a re-install of DevStack + to restore a clean test environment. diff --git a/LICENSE b/LICENSE new file mode 100644 index 00000000..68c771a0 --- /dev/null +++ b/LICENSE @@ -0,0 +1,176 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + diff --git a/README.md b/README.md new file mode 100644 index 00000000..db14b0a5 --- /dev/null +++ b/README.md @@ -0,0 +1,11 @@ +Grenade is an OpenStack upgrade test harness to exercise the +upgrade process from Essex to Folsom. It uses DevStack to perform +the initial # Openstack install + +# Goals + +* To quickly test OpenStack upgrades + +# Start An Upgrade Test + + ./grenade.sh diff --git a/functions b/functions new file mode 100644 index 00000000..b66dc159 --- /dev/null +++ b/functions @@ -0,0 +1,582 @@ +# -*- mode: Shell-script -*- +# functions - Common functions used by DevStack components +# +# ENABLED_SERVICES is used by is_service_enabled() + + +# Save trace setting +XTRACE=$(set +o | grep xtrace) +set +o xtrace + + +# apt-get wrapper to set arguments correctly +# apt_get operation package [package ...] +function apt_get() { + [[ "$OFFLINE" = "True" || -z "$@" ]] && return + local sudo="sudo" + [[ "$(id -u)" = "0" ]] && sudo="env" + $sudo DEBIAN_FRONTEND=noninteractive \ + http_proxy=$http_proxy https_proxy=$https_proxy \ + apt-get --option "Dpkg::Options::=--force-confold" --assume-yes "$@" +} + + +# Gracefully cp only if source file/dir exists +# cp_it source destination +function cp_it { + if [ -e $1 ] || [ -d $1 ]; then + cp -pRL $1 $2 + fi +} + + +# Prints "message" and exits +# die "message" +function die() { + local exitcode=$? + set +o xtrace + echo $@ + exit $exitcode +} + + +# Checks an environment variable is not set or has length 0 OR if the +# exit code is non-zero and prints "message" and exits +# NOTE: env-var is the variable name without a '$' +# die_if_not_set env-var "message" +function die_if_not_set() { + ( + local exitcode=$? + set +o xtrace + local evar=$1; shift + if ! is_set $evar || [ $exitcode != 0 ]; then + set +o xtrace + echo $@ + exit -1 + fi + ) +} + + +# Grab a numbered field from python prettytable output +# Fields are numbered starting with 1 +# Reverse syntax is supported: -1 is the last field, -2 is second to last, etc. +# get_field field-number +function get_field() { + while read data; do + if [ "$1" -lt 0 ]; then + field="(\$(NF$1))" + else + field="\$$(($1 + 1))" + fi + echo "$data" | awk -F'[ \t]*\\|[ \t]*' "{print $field}" + done +} + + +# get_packages() collects a list of package names of any type from the +# prerequisite files in ``files/{apts|pips}``. The list is intended +# to be passed to a package installer such as apt or pip. +# +# Only packages required for the services in ENABLED_SERVICES will be +# included. Two bits of metadata are recognized in the prerequisite files: +# - ``# NOPRIME`` defers installation to be performed later in stack.sh +# - ``# dist:DISTRO`` or ``dist:DISTRO1,DISTRO2`` limits the selection +# of the package to the distros listed. The distro names are case insensitive. +# +# get_packages dir +function get_packages() { + local package_dir=$1 + local file_to_parse + local service + + if [[ -z "$package_dir" ]]; then + echo "No package directory supplied" + return 1 + fi + if [[ -z "$DISTRO" ]]; then + echo "No distro set in DISTRO" + return 1 + fi + for service in general ${ENABLED_SERVICES//,/ }; do + # Allow individual services to specify dependencies + if [[ -e ${package_dir}/${service} ]]; then + file_to_parse="${file_to_parse} $service" + fi + # NOTE(sdague) n-api needs glance for now because that's where + # glance client is + if [[ $service == n-api ]]; then + if [[ ! $file_to_parse =~ nova ]]; then + file_to_parse="${file_to_parse} nova" + fi + if [[ ! $file_to_parse =~ glance ]]; then + file_to_parse="${file_to_parse} glance" + fi + elif [[ $service == c-* ]]; then + if [[ ! $file_to_parse =~ cinder ]]; then + file_to_parse="${file_to_parse} cinder" + fi + elif [[ $service == n-* ]]; then + if [[ ! $file_to_parse =~ nova ]]; then + file_to_parse="${file_to_parse} nova" + fi + elif [[ $service == g-* ]]; then + if [[ ! $file_to_parse =~ glance ]]; then + file_to_parse="${file_to_parse} glance" + fi + elif [[ $service == key* ]]; then + if [[ ! $file_to_parse =~ keystone ]]; then + file_to_parse="${file_to_parse} keystone" + fi + fi + done + + for file in ${file_to_parse}; do + local fname=${package_dir}/${file} + local OIFS line package distros distro + [[ -e $fname ]] || continue + + OIFS=$IFS + IFS=$'\n' + for line in $(<${fname}); do + if [[ $line =~ "NOPRIME" ]]; then + continue + fi + + if [[ $line =~ (.*)#.*dist:([^ ]*) ]]; then + # We are using BASH regexp matching feature. + package=${BASH_REMATCH[1]} + distros=${BASH_REMATCH[2]} + # In bash ${VAR,,} will lowecase VAR + [[ ${distros,,} =~ ${DISTRO,,} ]] && echo $package + continue + fi + + echo ${line%#*} + done + IFS=$OIFS + done +} + + +# Determine OS Vendor, Release and Update +# Tested with OS/X, Ubuntu, RedHat, CentOS, Fedora +# Returns results in global variables: +# os_VENDOR - vendor name +# os_RELEASE - release +# os_UPDATE - update +# os_PACKAGE - package type +# os_CODENAME - vendor's codename for release +# GetOSVersion +GetOSVersion() { + # Figure out which vendor we are + if [[ -n "`which sw_vers 2>/dev/null`" ]]; then + # OS/X + os_VENDOR=`sw_vers -productName` + os_RELEASE=`sw_vers -productVersion` + os_UPDATE=${os_RELEASE##*.} + os_RELEASE=${os_RELEASE%.*} + os_PACKAGE="" + if [[ "$os_RELEASE" =~ "10.7" ]]; then + os_CODENAME="lion" + elif [[ "$os_RELEASE" =~ "10.6" ]]; then + os_CODENAME="snow leopard" + elif [[ "$os_RELEASE" =~ "10.5" ]]; then + os_CODENAME="leopard" + elif [[ "$os_RELEASE" =~ "10.4" ]]; then + os_CODENAME="tiger" + elif [[ "$os_RELEASE" =~ "10.3" ]]; then + os_CODENAME="panther" + else + os_CODENAME="" + fi + elif [[ -x $(which lsb_release 2>/dev/null) ]]; then + os_VENDOR=$(lsb_release -i -s) + os_RELEASE=$(lsb_release -r -s) + os_UPDATE="" + if [[ "Debian,Ubuntu" =~ $os_VENDOR ]]; then + os_PACKAGE="deb" + else + os_PACKAGE="rpm" + fi + os_CODENAME=$(lsb_release -c -s) + elif [[ -r /etc/redhat-release ]]; then + # Red Hat Enterprise Linux Server release 5.5 (Tikanga) + # CentOS release 5.5 (Final) + # CentOS Linux release 6.0 (Final) + # Fedora release 16 (Verne) + os_CODENAME="" + for r in "Red Hat" CentOS Fedora; do + os_VENDOR=$r + if [[ -n "`grep \"$r\" /etc/redhat-release`" ]]; then + ver=`sed -e 's/^.* \(.*\) (\(.*\)).*$/\1\|\2/' /etc/redhat-release` + os_CODENAME=${ver#*|} + os_RELEASE=${ver%|*} + os_UPDATE=${os_RELEASE##*.} + os_RELEASE=${os_RELEASE%.*} + break + fi + os_VENDOR="" + done + os_PACKAGE="rpm" + fi + export os_VENDOR os_RELEASE os_UPDATE os_PACKAGE os_CODENAME +} + + +# Translate the OS version values into common nomenclature +# Sets ``DISTRO`` from the ``os_*`` values +function GetDistro() { + GetOSVersion + if [[ "$os_VENDOR" =~ (Ubuntu) ]]; then + # 'Everyone' refers to Ubuntu releases by the code name adjective + DISTRO=$os_CODENAME + elif [[ "$os_VENDOR" =~ (Fedora) ]]; then + # For Fedora, just use 'f' and the release + DISTRO="f$os_RELEASE" + else + # Catch-all for now is Vendor + Release + Update + DISTRO="$os_VENDOR-$os_RELEASE.$os_UPDATE" + fi + export DISTRO +} + + +# git clone only if directory doesn't exist already. Since ``DEST`` might not +# be owned by the installation user, we create the directory and change the +# ownership to the proper user. +# Set global RECLONE=yes to simulate a clone when dest-dir exists +# Set global ERROR_ON_CLONE=True to abort execution with an error if the git repo +# does not exist (default is False, meaning the repo will be cloned). +# git_clone remote dest-dir branch +function git_clone { + [[ "$OFFLINE" = "True" ]] && return + + GIT_REMOTE=$1 + GIT_DEST=$2 + GIT_BRANCH=$3 + + if echo $GIT_BRANCH | egrep -q "^refs"; then + # If our branch name is a gerrit style refs/changes/... + if [[ ! -d $GIT_DEST ]]; then + [[ "$ERROR_ON_CLONE" = "True" ]] && exit 1 + git clone $GIT_REMOTE $GIT_DEST + fi + cd $GIT_DEST + git fetch $GIT_REMOTE $GIT_BRANCH && git checkout FETCH_HEAD + else + # do a full clone only if the directory doesn't exist + if [[ ! -d $GIT_DEST ]]; then + [[ "$ERROR_ON_CLONE" = "True" ]] && exit 1 + git clone $GIT_REMOTE $GIT_DEST + cd $GIT_DEST + # This checkout syntax works for both branches and tags + git checkout $GIT_BRANCH + elif [[ "$RECLONE" == "yes" ]]; then + # if it does exist then simulate what clone does if asked to RECLONE + cd $GIT_DEST + # set the url to pull from and fetch + git remote set-url origin $GIT_REMOTE + git fetch origin + # remove the existing ignored files (like pyc) as they cause breakage + # (due to the py files having older timestamps than our pyc, so python + # thinks the pyc files are correct using them) + find $GIT_DEST -name '*.pyc' -delete + git checkout -f origin/$GIT_BRANCH + # a local branch might not exist + git branch -D $GIT_BRANCH || true + git checkout -b $GIT_BRANCH + fi + fi +} + + +# Comment an option in an INI file +# inicomment config-file section option +function inicomment() { + local file=$1 + local section=$2 + local option=$3 + sed -i -e "/^\[$section\]/,/^\[.*\]/ s|^\($option[ \t]*=.*$\)|#\1|" $file +} + +# Uncomment an option in an INI file +# iniuncomment config-file section option +function iniuncomment() { + local file=$1 + local section=$2 + local option=$3 + sed -i -e "/^\[$section\]/,/^\[.*\]/ s|[^ \t]*#[ \t]*\($option[ \t]*=.*$\)|\1|" $file +} + + +# Get an option from an INI file +# iniget config-file section option +function iniget() { + local file=$1 + local section=$2 + local option=$3 + local line + line=$(sed -ne "/^\[$section\]/,/^\[.*\]/ { /^$option[ \t]*=/ p; }" $file) + echo ${line#*=} +} + + +# Set an option in an INI file +# iniset config-file section option value +function iniset() { + local file=$1 + local section=$2 + local option=$3 + local value=$4 + if ! grep -q "^\[$section\]" $file; then + # Add section at the end + echo -e "\n[$section]" >>$file + fi + if [[ -z "$(iniget $file $section $option)" ]]; then + # Add it + sed -i -e "/^\[$section\]/ a\\ +$option = $value +" $file + else + # Replace it + sed -i -e "/^\[$section\]/,/^\[.*\]/ s|^\($option[ \t]*=[ \t]*\).*$|\1$value|" $file + fi +} + + +# is_service_enabled() checks if the service(s) specified as arguments are +# enabled by the user in **ENABLED_SERVICES**. +# +# If there are multiple services specified as arguments the test performs a +# boolean OR or if any of the services specified on the command line +# return true. +# +# There is a special cases for some 'catch-all' services:: +# **nova** returns true if any service enabled start with **n-** +# **glance** returns true if any service enabled start with **g-** +# **quantum** returns true if any service enabled start with **q-** +function is_service_enabled() { + services=$@ + for service in ${services}; do + [[ ,${ENABLED_SERVICES}, =~ ,${service}, ]] && return 0 + [[ ${service} == "nova" && ${ENABLED_SERVICES} =~ "n-" ]] && return 0 + [[ ${service} == "cinder" && ${ENABLED_SERVICES} =~ "c-" ]] && return 0 + [[ ${service} == "glance" && ${ENABLED_SERVICES} =~ "g-" ]] && return 0 + [[ ${service} == "quantum" && ${ENABLED_SERVICES} =~ "q-" ]] && return 0 + done + return 1 +} + +# remove extra commas from the input string (ENABLED_SERVICES) +function _cleanup_service_list () { + echo "$1" | sed -e ' + s/,,/,/g; + s/^,//; + s/,$// + ' +} + +# enable_service() adds the services passed as argument to the +# **ENABLED_SERVICES** list, if they are not already present. +# +# For example: +# +# enable_service n-vol +# +# This function does not know about the special cases +# for nova, glance, and quantum built into is_service_enabled(). +function enable_service() { + local tmpsvcs="${ENABLED_SERVICES}" + for service in $@; do + if ! is_service_enabled $service; then + tmpsvcs+=",$service" + fi + done + ENABLED_SERVICES=$(_cleanup_service_list "$tmpsvcs") + disable_negated_services +} + +# disable_service() removes the services passed as argument to the +# **ENABLED_SERVICES** list, if they are present. +# +# For example: +# +# disable_service n-vol +# +# This function does not know about the special cases +# for nova, glance, and quantum built into is_service_enabled(). +function disable_service() { + local tmpsvcs=",${ENABLED_SERVICES}," + local service + for service in $@; do + if is_service_enabled $service; then + tmpsvcs=${tmpsvcs//,$service,/,} + fi + done + ENABLED_SERVICES=$(_cleanup_service_list "$tmpsvcs") +} + +# disable_all_services() removes all current services +# from **ENABLED_SERVICES** to reset the configuration +# before a minimal installation +function disable_all_services() { + ENABLED_SERVICES="" +} + +# We are looking for services with a - at the beginning to force +# excluding those services. For example if you want to install all the default +# services but not nova-volume (n-vol) you can have this set in your localrc : +# ENABLED_SERVICES+=",-n-vol" +function disable_negated_services() { + local tmpsvcs="${ENABLED_SERVICES}" + local service + for service in ${tmpsvcs//,/ }; do + if [[ ${service} == -* ]]; then + tmpsvcs=$(echo ${tmpsvcs}|sed -r "s/(,)?(-)?${service#-}(,)?/,/g") + fi + done + ENABLED_SERVICES=$(_cleanup_service_list "$tmpsvcs") +} + +# Distro-agnostic package installer +# install_package package [package ...] +function install_package() { + if [[ -z "$os_PACKAGE" ]]; then + GetOSVersion + fi + if [[ "$os_PACKAGE" = "deb" ]]; then + apt_get install "$@" + else + yum_install "$@" + fi +} + + +# Test if the named environment variable is set and not zero length +# is_set env-var +function is_set() { + local var=\$"$1" + if eval "[ -z $var ]"; then + return 1 + fi + return 0 +} + + +# pip install wrapper to set cache and proxy environment variables +# pip_install package [package ...] +function pip_install { + [[ "$OFFLINE" = "True" || -z "$@" ]] && return + if [[ -z "$os_PACKAGE" ]]; then + GetOSVersion + fi + if [[ $TRACK_DEPENDS = True ]] ; then + source $DEST/.venv/bin/activate + CMD_PIP=$DEST/.venv/bin/pip + SUDO_PIP="env" + else + SUDO_PIP="sudo" + if [[ "$os_PACKAGE" = "deb" ]]; then + CMD_PIP=/usr/bin/pip + else + CMD_PIP=/usr/bin/pip-python + fi + fi + $SUDO_PIP PIP_DOWNLOAD_CACHE=${PIP_DOWNLOAD_CACHE:-/var/cache/pip} \ + HTTP_PROXY=$http_proxy \ + HTTPS_PROXY=$https_proxy \ + $CMD_PIP install --use-mirrors $@ +} + + +# Service wrapper to restart services +# restart_service service-name +function restart_service() { + if [[ -z "$os_PACKAGE" ]]; then + GetOSVersion + fi + if [[ "$os_PACKAGE" = "deb" ]]; then + sudo /usr/sbin/service $1 restart + else + sudo /sbin/service $1 restart + fi +} + + +# pip install the dependencies of the package before we do the setup.py +# develop, so that pip and not distutils process the dependency chain +# setup_develop directory +function setup_develop() { + if [[ $TRACK_DEPENDS = True ]] ; then + SUDO_CMD="env" + else + SUDO_CMD="sudo" + fi + (cd $1; \ + python setup.py egg_info; \ + raw_links=$(awk '/^.+/ {print "-f " $1}' *.egg-info/dependency_links.txt); \ + depend_links=$(echo $raw_links | xargs); \ + pip_install -r *-info/requires.txt $depend_links; \ + $SUDO_CMD \ + HTTP_PROXY=$http_proxy \ + HTTPS_PROXY=$https_proxy \ + python setup.py develop \ + ) +} + + +# Service wrapper to start services +# start_service service-name +function start_service() { + if [[ -z "$os_PACKAGE" ]]; then + GetOSVersion + fi + if [[ "$os_PACKAGE" = "deb" ]]; then + sudo /usr/sbin/service $1 start + else + sudo /sbin/service $1 start + fi +} + + +# Service wrapper to stop services +# stop_service service-name +function stop_service() { + if [[ -z "$os_PACKAGE" ]]; then + GetOSVersion + fi + if [[ "$os_PACKAGE" = "deb" ]]; then + sudo /usr/sbin/service $1 stop + else + sudo /sbin/service $1 stop + fi +} + + +# Normalize config values to True or False +# VAR=`trueorfalse default-value test-value` +function trueorfalse() { + local default=$1 + local testval=$2 + + [[ -z "$testval" ]] && { echo "$default"; return; } + [[ "0 no false False FALSE" =~ "$testval" ]] && { echo "False"; return; } + [[ "1 yes true True TRUE" =~ "$testval" ]] && { echo "True"; return; } + echo "$default" +} + + +# yum wrapper to set arguments correctly +# yum_install package [package ...] +function yum_install() { + [[ "$OFFLINE" = "True" ]] && return + local sudo="sudo" + [[ "$(id -u)" = "0" ]] && sudo="env" + $sudo http_proxy=$http_proxy https_proxy=$https_proxy \ + yum install -y "$@" +} + + +# Restore xtrace +$XTRACE diff --git a/grenade.sh b/grenade.sh new file mode 100644 index 00000000..577d6cbd --- /dev/null +++ b/grenade.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash + +# ``grenade.sh`` is an OpenStack upgrade test harness to exercise the +# upgrade process from Essex to Folsom. It uses DevStack to perform +# the initial # Openstack install + +# Grenade assumes it is running on the system that will be hosting the upgrade processes + + +# Keep track of the devstack directory +TOP_DIR=$(cd $(dirname "$0") && pwd) + +# Import common functions +source $TOP_DIR/functions + +# Determine what system we are running on. This provides ``os_VENDOR``, +# ``os_RELEASE``, ``os_UPDATE``, ``os_PACKAGE``, ``os_CODENAME`` +# and ``DISTRO`` +GetDistro + +#source $TOP_DIR/grenaderc + +# Possible grenaderc values + +# Destination path for installation ``DEST`` +DEST=${DEST:-/opt/stack} + +# Release info +START_RELEASE=essex +END_RELEASE=folsom + +DEVSTACK_BEFORE_REPO=https://github.com/openstack-dev/devstack.git +DEVSTACK_BEFORE_BRANCH=stable/$START_RELEASE +DEVSTACK_BEFORE_DIR=$DEST/devstack + +DEVSTACK_AFTER_REPO=$DEVSTACK_BEFORE_REPO +DEVSTACK_AFTER_BRANCH=master +DEVSTACK_AFTER_DIR=devstack-master + + +# System Preparation +# ================== + +# perform cleanup to ensure a clean starting environment + +# check out devstack +git_clone $DEVSTACK_BEFORE_REPO $DEVSTACK_BEFORE_DIR $DEVSTACK_BEFORE_BRANCH + + +# Essex Install +# ============= + +# Exercises +# ========= + diff --git a/setup-grenade b/setup-grenade new file mode 100755 index 00000000..e442c2d2 --- /dev/null +++ b/setup-grenade @@ -0,0 +1,27 @@ +#!/bin/bash +# setup-grenade hostname + +GRENADE_REPO=git@github.com:dtroyer/grenade.git + +HOST=$1 + +if [[ -z "$HOST" ]]; then + echo "Must supply hostname" + exit 1 +fi + +ssh $HOST " \ + rm -rf devstack; \ + git clone $DEVSTACK_REPO; \ + cp -p devstack/samples/local.sh devstack; \ +" + +# Extract IP address +HOST_IP=$(ssh $HOST " \ + ip addr show dev eth0 | awk '/inet / { split(\$2,a,\"\/\"); print a[1] }' +") + +sed "/^HOST_IP=/s/^HOST_IP=.*$/HOST_IP=$HOST_IP/" $HOST.localrc | + ssh $HOST "cat >devstack/localrc" +#scp -p $HOST.localrc $HOST:devstack/localrc +scp -p redt $HOST:devstack