Start using tripleo-quickstart-extras

Replace the extra role requirements from
quickstart-extras-requirements.txt with a single repository upstream
(tripleo-quickstart-extras) consisting of the merged history of all
previously used roles.

Further changes to these roles are to be submitted to this
tripleo-quickstart-extras repo through review.openstack.org instead of
review.gerrithub.io.

Several role name changes occured during this transition, which this
change also corrects in playbooks hosted within this repository.

Change-Id: I9b532b8f63d68d9791f7403c9d66558f8221d980

ci-scripts/full-deploy.sh

@@ -15,7 +15,6 @@ JOB_TYPE=$4
 
 if [ "$JOB_TYPE" = "gate" ] || \
    [ "$JOB_TYPE" = "periodic" ] || \
-   [ "$JOB_TYPE" = "roles-gate" ] || \
    [ "$JOB_TYPE" = "dlrn-gate" ]; then
     unset REL_TYPE
     if [ "$RELEASE" = "master-tripleo-ci" ]; then
@@ -26,10 +25,9 @@ elif [ "$JOB_TYPE" = "promote" ]; then
     REL_TYPE=$LOCATION
 else
     echo "Job type must be one of the following:"
-    echo " * gate - for gating changes on tripleo-quickstart"
+    echo " * gate - for gating changes on tripleo-quickstart or -extras"
     echo " * promote - for running promotion jobs"
     echo " * periodic - for running periodic jobs"
-    echo " * roles-gate - for gating changes to the extra roles"
     echo " * dlrn-gate - for gating upstream changes"
     exit 1
 fi
@@ -41,24 +39,13 @@ socketdir=$(mktemp -d /tmp/sockXXXXXX)
 export ANSIBLE_SSH_CONTROL_PATH=$socketdir/%%h-%%r
 
 # preparation steps to run with the gated roles
-if [ "$JOB_TYPE" = "roles-gate" ] || [ "$JOB_TYPE" = "gate" ]; then
+if [ "$JOB_TYPE" = "gate" ]; then
-    # set up the gated repos and modify the requirements file to use them
     bash quickstart.sh \
         --working-dir $WORKSPACE/ \
         --no-clone \
         --bootstrap \
         --requirements quickstart-extras-requirements.txt \
-        --playbook gate-roles.yml \
+        --playbook gate-quickstart.yml \
-        --release ${CI_ENV:+$CI_ENV/}$RELEASE${REL_TYPE:+-$REL_TYPE} \
-        $OPT_ADDITIONAL_PARAMETERS \
-        $VIRTHOST
-    # once more to let the gating role be gated as well
-    bash quickstart.sh \
-        --working-dir $WORKSPACE/ \
-        --no-clone \
-        --bootstrap \
-        --requirements quickstart-extras-requirements.txt \
-        --playbook gate-roles.yml \
         --release ${CI_ENV:+$CI_ENV/}$RELEASE${REL_TYPE:+-$REL_TYPE} \
         $OPT_ADDITIONAL_PARAMETERS \
         $VIRTHOST

doc/source/working-with-extras.rst

@@ -1,19 +1,56 @@
 .. _working-with-extras:
 
 Working With Quickstart Extras
-========================
+==============================
 
-Add-on extras for quickstart are available from the
+There are several additional roles compatible with tripleo-quickstart that
-`Redhat Openstack Github <https://github.com/redhat-openstack/>`_ with the
+extend the capability of TripleO Quickstart beyond setting up the basic
-format ansible-role-tripleo-task-to-perform.
+deployment structure on a virthost. There are roles for doing end-to-end
+deployments and deployment validation both for virthost and baremetal machines
+and a few auxiliary roles for doing CI related tasks.
 
-Extras can be installed manually using Python setuptools. TripleO Quickstart
+These roles are hosted at the tripleo-quickstart-extras_ repository.
+
+.. _tripleo-quickstart-extras: http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/
+
+Extras can be installed manually using Python setuptools, but ``quickstart.sh``
 provides an automated system for building the Python virtual environment and
 pulling in additional dependencies using ``pip install`` and the
-``quickstart-extras-requirements.txt`` file, so we suggest you use that.
+``quickstart-extras-requirements.txt`` file.
 
-If your role is in a git repository use the following syntax and append it to
+To run a full end-to-end deployment including verification, add these command
-the end of the ``quickstart-extras-requirements.txt`` file::
+line options when running ``quickstart.sh``::
+
+    --requirements quickstart-extras-requirements.txt --playbook quickstart-extras.yml
+
+See ``quickstart.sh --help`` for a full list of options, but here is a full
+example using some common developer settings::
+
+    ./quickstart.sh --requirements quickstart-extras-requirements.txt \
+                    --playbook quickstart-extras.yml \
+                    --tags all --teardown all \
+                    --release master --no-clone --clean \
+                    --config config/general_config/minimal_pacemaker.yml \
+                    virthost.example.com
+
+This uses the currently cloned tripleo-quickstart repository instead of
+re-cloning it in the working directory, doing the following:
+
+* deletes the working directory at `~/.quickstart`
+* this forces quickstart to create the virtual environment and redownload all
+  the requirements fresh, including the extras
+* it does a thorough cleanup of the `virthost.example.com` machine,
+  reinstalling libvirt, deleting networks and VMs
+* downloads the master image and creates new VMs based that
+* deploys both the underclound and the overcloud
+* verifies the deployed environment in a quick and simple way
+
+Developing new roles
+--------------------
+
+Developing new roles is possible by submitting new reviews for this repo, or
+creating it anywhere and adding a reference to the end of the
+``quickstart-extras-requirements.txt`` file::
 
     git+https://github.com/organization/ansible-role-example.git/#egg=ansible-role-example
 
@@ -22,16 +59,16 @@ To import a role that you are developing locally use the following syntax::
     file:///home/user/folder/ansible-role-example/#egg=ansible-role-example
 
 Once added to the role requirements file ``quickstart.sh`` will automatically
-install the specified extras into ``$WORKSPACE``, which is a Python virtual
+install the specified extras into the local working directory, which is a
-environment managed by quickstart. By default this environment will be placed
+Python virtual environment managed by quickstart. By default this environment
-in ``~/.quickstart`` but you can specify its location using the
+will be placed in ``~/.quickstart`` but you can specify its location using the
 ``--working-dir`` argument.
 
 To invoke quickstart with a playbook of your own or from a preexisting role run
 the following command::
 
-    ./quickstart.sh --requirements $REQUIREMENTS --playbook playbook.yml --working-dir \
+    ./quickstart.sh --requirements $REQUIREMENTS --playbook playbook.yml \
-$WORKSPACE $VIRTHOST
+                    --working-dir $WORKSPACE $VIRTHOST
 
 If the virtual environment in ``$WORKSPACE`` has not
 already been setup then ``quickstart.sh`` will create it and install all the extras.
@@ -46,21 +83,7 @@ and then run the ``setup.py`` for the role::
 Deleting the environment and allowing quickstart to regenerate it entirely also works.
 Both ``$REQUIREMENTS`` and ``$WORKSPACE`` should be absolute paths.
 
-Some extras are meant only to be added to an existing playbook. For example, if
+The playbook ``quickstart-extras.yml`` is the most complete playbook offered by default, it
-you wanted to perform a pingtest to validate the overcloud you would clone the
-ansible-role-tripleo-overcloud-validate repository and then edit
-``$WORKSPACE/playbooks/quickstart.yml`` like so::
-
-    - name:  Install undercloud and deploy overcloud
-      hosts: undercloud
-      gather_facts: no
-      roles:
-        - tripleo/undercloud
-        - tripleo/overcloud
-        - tripleo-overcloud-validate
-
-This will perform the pingtest validation after the deployment of the overcloud is finished.
-The playbook `quickstart-extras.yml` is the most complete playbook offered by default, it
 will perform all tasks up to deployment and testing the overcloud using this same method.
 
 While editing existing playbooks is a good way to become familiar with quickstart for actual usage

playbooks/gate-roles.yml

@@ -1,7 +0,0 @@
----
-# This playbooks is responsible for gating all the auxiliary
-# roles that quickstart uses
-- name: Clone the gated repos and modify the requirements
-  hosts: localhost
-  roles:
-    - { role: tripleo-gate, artg_mode: 'roles' }

playbooks/quickstart-extras.yml

@@ -34,14 +34,14 @@
   hosts: undercloud
   gather_facts: no
   roles:
-    - { role: tripleo-ssl, when: ssl_overcloud|bool }
+    - { role: overcloud-ssl, when: ssl_overcloud|bool }
 
 # Deploy the overcloud
 - name:  Deploy the overcloud
   hosts: undercloud
   gather_facts: yes
   roles:
-    - tripleo-overcloud
+    - overcloud-deploy
 
 - name: Add the overcloud nodes to the generated inventory
   hosts: undercloud
@@ -74,7 +74,7 @@
     - overcloud-validate
   gather_facts: no
   roles:
-    - { role: tripleo-overcloud-validate, when: test_ping|bool }
+    - { role: validate-simple, when: test_ping|bool }
 
 # Execute tempest against the overcloud deployment
 - name:  Execute tempest against the overcloud
@@ -83,4 +83,4 @@
     - overcloud-validate
   gather_facts: no
   roles:
-    - { role: tripleo-tempest, when: run_tempest|bool }
+    - { role: validate-tempest, when: run_tempest|bool }

quickstart-extras-requirements.txt

@@ -1,24 +1 @@
-# Roles
+git+https://git.openstack.org/openstack/tripleo-quickstart-extras/#egg=tripleo-quickstart-extras
-
-git+https://github.com/redhat-openstack/ansible-role-tripleo-overcloud-prep-config.git/#egg=ansible-role-tripleo-overcloud-prep-config
-git+https://github.com/redhat-openstack/ansible-role-tripleo-overcloud-prep-flavors.git/#egg=ansible-role-tripleo-overcloud-prep-flavors
-git+https://github.com/redhat-openstack/ansible-role-tripleo-overcloud-prep-images.git/#egg=ansible-role-tripleo-overcloud-prep-images
-git+https://github.com/redhat-openstack/ansible-role-tripleo-overcloud-prep-network.git/#egg=ansible-role-tripleo-overcloud-prep-network
-git+https://github.com/redhat-openstack/ansible-role-tripleo-overcloud.git/#egg=ansible-role-tripleo-overcloud
-git+https://github.com/redhat-openstack/ansible-role-tripleo-overcloud-validate.git/#egg=ansible-role-tripleo-overcloud-validate
-git+https://github.com/redhat-openstack/ansible-role-tripleo-cleanup-nfo.git/#egg=ansible-role-tripleo-cleanup-nfo
-git+https://github.com/redhat-openstack/ansible-role-tripleo-collect-logs.git/#egg=ansible-role-tripleo-collect-logs
-git+https://github.com/redhat-openstack/ansible-role-tripleo-gate.git#egg=ansible-role-tripleo-gate
-git+https://github.com/redhat-openstack/ansible-role-tripleo-overcloud-upgrade.git#egg=ansible-role-tripleo-overcloud-upgrade
-git+https://github.com/redhat-openstack/ansible-role-tripleo-overcloud-scale-nodes.git#egg=ansible-role-tripleo-overcloud-scale-nodes
-git+https://github.com/redhat-openstack/ansible-role-tripleo-tempest.git#egg=ansible-role-tripleo-tempest
-git+https://github.com/redhat-openstack/ansible-role-tripleo-baremetal-prep-virthost.git#egg=ansible-role-tripleo-baremetal-prep-virthost
-git+https://github.com/redhat-openstack/ansible-role-tripleo-overcloud-prep-baremetal.git#egg=ansible-role-tripleo-overcloud-prep-baremetal
-git+https://github.com/redhat-openstack/ansible-role-tripleo-provision-heat.git#egg=ansible-role-tripleo-provision-heat
-git+https://github.com/redhat-openstack/ansible-role-tripleo-validate-ipmi.git#egg=ansible-role-tripleo-validate-ipmi
-git+https://github.com/redhat-openstack/ansible-role-tripleo-ssl.git#egg=ansible-role-tripleo-ssl
-
-
-# To pull in local development changes
-#file:///<dir>/ansible-role-foo/#egg=ansible-role-foo
-