Commit Graph

99 Commits

Author SHA1 Message Date
Sagi Shnaidman 8c6db50589 Remove unnecessary code from toci_* scripts
Detect and remove unnecessary code from toci_* scripts
Simplify tripleo-upgrades roles installation with ZUUL_BRANCH

Change-Id: Ie230aa0640e41a33e10e6656b69e1db4a6819127
2018-06-25 14:56:07 +03:00
Ronelle Landy 1edb87d068 Streamline variables passed in different environments
- OVB and multinode/singlemode jobs are passed different settings
files based on the location where the jobs run. This review
changes toci-gate-test to pass both the general and rdocloud
multinode setting files as is the case with OVB.

 - Removes the period job registry definitions.
These should be passed in release files only.

- Sets update_containers to run with
containerized_overcloud or containerized_undercloud

Closes-Bug: #1775874
Change-Id: I8f498073f1df9de70372f09b8c43e0d4adb5b42a
2018-06-22 03:07:51 +00:00
Ronelle Landy 3dab6f0df0 Add dry run option to toci_quickstart - playbook order in tact
With dry run, playbooks are not executed.
The env variables, releases script output and playbooks
commands are written out to files for debug.

The review also moves the collect logs lines to a
function for easier organization

Documentation for the dry run option and variable
files is added.

Change-Id: I0bad5ee3150d94115bed018de9488590305a2b80
2018-06-15 14:27:38 +00:00
Emilien Macchi 86eb713c27 Introduce tripleo-ci-centos-7-standalone non-voting job
First iteration of testing TripleO All-In-One blueprint.

Depends-On: https://review.openstack.org/#/c/572220
Depends-On: https://review.openstack.org/#/c/572221

Related-Blueprint: all-in-one

Change-Id: I0163b466fc0a85869b7e951a1c32b267261d9c80
2018-06-11 13:55:31 +00:00
Ronelle Landy 0876f88029 Revert "Add dry run option to toci_quickstart"
This reverts commit 418f5a639f.

Closes-Bug: 1773381
Change-Id: If3086bfc96d792edcd5ea8139fd21702ad65d279
2018-05-25 14:56:25 +00:00
Ronelle Landy 418f5a639f Add dry run option to toci_quickstart
With dry run, playbooks are not executed.
The env variables, releases script output
and playbooks commands are written out to
files for debug.

The review also moves the collect logs lines
to a function for easier organization.

Documentation for the dry run option and
variable files is added.

Depends-On: I4c8f389978717848e755f12856dd454d605c9137
Change-Id: Ib6ae184a07ba291d719cabb48b51d890b0485a49
2018-05-24 23:16:34 +00:00
John Trowbridge 48e2b054e7 Add ability to use a different release per playbook
We are changing the way we are passing releases to the playbooks.
This change adds the ability to use the new release handling script. It
enables it only for 4 jobs. Two should be moved to the new way of
handling releases, two are there to test the backwards compatibility
offered by the script.
If the script is uses and produces an output, the new method will be
activated, by filling a dictionary with playbook as key and release as
value.
If there is no output from the script, the dictionary will be empty and
the default QUICKSTART_RELEASE file will be used instead.

Co-Authored-By: Gabriele Cerami <gcerami@redhat.com>
Co-Authored-By: Ronelle Landy <rlandy@redhat.com>
Change-Id: I6514ba15ff4300ac4bd4fe543d3a4954aeb6f175
2018-05-22 12:37:15 +00:00
Emilien Macchi 7b843c32bb undercloud-upgrade: deploy n-1 first then upgrade
The current undercloud upgrade job deploys an undercloud from master so
upgrade was never actually tested. This patch finds out which release we
want to test and provide the right featureset file.

Co-authored-by: Jose Luis Franco <jfrancoa@redhat.com>
Change-Id: I3c9d2396988bd3125740f0054c66e8243e85f437
2018-04-30 11:43:32 +02:00
Jose Luis Franco Arza 7ce16351bc Correct branch used in tripleo-upgrade installation.
STABLE_BRANCH variable omits the 'stable/'
section of the branch name. However, the
tripleo-upgrade role is branched out using
that structure. So when installing the
oooq requirements it will try to download
queens or pike branch, which won't be found.
It should be stable/pike or stable/queens
instead.

Change-Id: I93fe1d3a285a2d9ec40c4245ea012a66e6584a30
2018-04-23 15:28:55 +02:00
Jose Luis Franco Arza 995271ccc8 Use tripleo-upgrade role in undercloud upgrades job.
Featureset047 includes all needed parameters
to run an undercloud upgrade using the
tripleo-upgrade role.

Change toci_gate_test.sh logic to include the
mixed_upgrade environment file *-undercloud-*-overcloud.yml
only when overcloud-deploy is being performed. Otherwise,
it will try to load it for undercloud upgrade too.

Closes-Bug: #1735792
Depends-On: Ib204c89ad88ae4581e556710976325c2b2723039
Change-Id: Iaed38c620f39b66b0b560ae561580dce21ea199b
2018-04-16 16:20:22 +02:00
Zuul 642fc85da1 Merge "Import gated tripleo-upgrade role." 2018-04-13 02:28:38 +00:00
Jose Luis Franco Arza 7c476bf4cd Import gated tripleo-upgrade role.
To leveraget that tripleo-upgrade is now being
gated, let's import it into oooq requirements
from the location where zuul is downloading it.

Change-Id: Icc84645a16aa53ca2cb974dbeb52e6474cd98aab
Closes-Bug: #1749740
2018-04-11 11:28:24 +02:00
Juan Antonio Osorio Robles 3a1115f58f Add overcloud-deploy-post tag to CI deployment
This asures we have the overcloud DNS entries in /etc/hosts

Change-Id: I21bb02e8817048e7dec6bdd9521b45f0611f4616
2018-04-10 13:30:15 +03:00
Zuul 50637f9152 Merge "Add multinode-overcloud-update playbook to run list" 2018-04-06 16:19:08 +00:00
Enrique Llorente 06cc329f48 Use external RHCLOUD variable if set
To use  cloud environment different from RDO cloud it's needed to set this value to something that will
match to the used cloud environment quickstart configuration. It also need to be stored at toci-quickstart config files.

For example, for the RHCLOUD foobar if testing with OVB, the following file have to be installed before running the gate test:

/opt/stack/tripleo-ci/toci-quickstart/config/testenv/ovb-foobar.yml

Change-Id: Ifc88a76df8de32b0a66bc507e337a9b407f17e29
2018-04-02 11:32:09 +02:00
Jiri Stransky bc6cc706db Add multinode-overcloud-update playbook to run list
Co-Authored-By: Jose Luis Franco <jfrancoa@redhat.com>
Change-Id: I2699fd4c18e07e451af242194ea2593970912014
Depends-On: I75a358dea33c8befef37aceddeb02e30599d3f7a
Depends-On: I26301b0bf9fe8b4a988575656fcd32bfaca1b3b1
2018-03-28 14:42:17 +02:00
Zuul 3ade40abf2 Merge "Allow custom sequence of playbooks" 2018-03-21 18:38:27 +00:00
Zuul 16809be329 Merge "Run specific jobs with quickstart.sh" 2018-03-20 18:08:08 +00:00
Sofer Athlan-Guyot d3defcfd29 Take mixed upgrade into account for boostrap_subnode_minimal.
In mixed upgrade, we have to check the UPGRADE_RELEASE to know the
version of the overcloud.

Change-Id: Ib34fc0220883c2dc02884e0f09a3c87980089438
2018-03-14 18:20:51 +00:00
Sofer Athlan-Guyot 33afffa972 Find the right release file for ffu mixed version upgrade.
By setting ffu_overcloud_upgrade in the featureset this change find
the right previous version for ffu and thus the correct release file.

This also refactor all the upgrade/update related function.

This whole section of the ci code is related to this tech-debt.
Related-Bug: #1749978

Change-Id: I962c2e4ad4eb522839deb904e0c67fb6a5d0ead9
2018-02-21 15:48:30 +01:00
Sagi Shnaidman c8f05c5a35 Allow custom sequence of playbooks
In reproducing jobs sometimes there is need to specify which
exactly playbook to run, this patch allows customization of
playbooks sequence to run.
PLAYBOOKS variable can be exported in reproducer script.

Change-Id: Iefb4f6c62969efd4179544b453dd1e119d6073d8
2018-02-21 11:41:02 +02:00
Sagi Shnaidman 0c2a144a72 Run specific jobs with quickstart.sh
To continue to gate quickstart.sh in CI we need to run
a few jobs with quickstart.sh.
Set containers multinode and fs001 jobs to run with quickstart.sh

Change-Id: I6a1f4e13468a4bab1589d2b9e2904c0a21599b8c
2018-02-21 11:39:36 +02:00
yatin 5bf07a20c2 Add bits to support stable/queens
Add missing bits to support stable/queens branch in TripleO;
Some adjustments might be done later but this is a first iteration.

Change-Id: I4a369389fbecf6890a75ffaaf695f391cd6b7e2d
Closes-Bug: #1750311
2018-02-20 19:55:12 +05:30
John Trowbridge 3c93795b9c Allow for multiple playbooks to be run
This will allow us to run multiple playbooks instead of just a
single playbook. This depends on the two changes that split
the ovb and multinode playbooks.

Depends-On: I6cc171641c8390e458eb474be3479e732eb2c985

Change-Id: I461a83a3d1b162457a2f607736ca4feb7fdd3e14
2018-02-19 14:15:37 +02:00
Zuul 349736c83e Merge "Add check to enable additional ansible tag for updates" 2018-02-15 15:21:40 +00:00
Jiri Stransky d5ba8c5a34 Add check to enable additional ansible tag for updates
This will make sure we only go through the update tasks in jobs which
need it.

Change-Id: Ia6b4daeeb1924af5de7598c4a42b5ccf1d13d1de
2018-02-15 11:45:06 +01:00
Sagi Shnaidman 0ec3a9c25d Use CENTOS_MIRROR from provided mirror_info.sh file
Don't construct CENTOS_MIRROR env var because we have it in the
file mirror_info.sh which is provided from nodepool.
Remove EPEL_MIRROR because it's not used anywhere.

Change-Id: I40fe8f6b2be1864b27bbe369e7da988fa7f1a0bb
2018-02-12 16:48:22 +02:00
Sagi Shnaidman b96b0b3d8f Include extra nodes in te-broker config
Add support for starting extra node when creating OVB environment.

Change-Id: I0be05d336203d81d59040ebcce03a29b2758ade0
2018-01-23 18:15:01 +02:00
Zuul a7771be792 Merge "Run overcloud-upgrade role only in upgrades jobs." 2018-01-18 13:20:29 +00:00
Sagi Shnaidman f9380ada4c Export env variables required for statistics
When calculation job statistics we need a few environemnt vars
be available in post playbook. Write them to file and export
when running logs collection role in post playbook.

Change-Id: I2719999a7f3abda43ab322a23bbe4750c7233a76
2018-01-16 11:55:16 +02:00
Jose Luis Franco Arza a5db97e451 Run overcloud-upgrade role only in upgrades jobs.
Defined tags are the same for all multinode
jobs, independently on if they perform an
upgrade or not. Each featureset env file then
defines if an upgrade is to be performed or not,
being skipped all the tasks of the overcloud-upgrade
role if it's not an upgrades job.

This task skipping takes around 3 seconds,
what's not a big deal, but it makes the logs
more difficult to read and understand.

This patch appends the overcloud-upgrade role
into the tags only if the job to be run performs
an upgrade.

Change-Id: Ia037dd74470499d302cb098136fbe71608b7824a
2018-01-09 14:16:17 -05:00
Zuul ef0dd1b1b1 Merge "Collect logs in post playbook for multinode" 2018-01-08 19:13:45 +00:00
Sagi Shnaidman 39409e3754 Send logs size to grahite site
Send logs size to track it in graphite server

Change-Id: Iaee3b4cdaed9f04fc7ff56f79c306856caf02c4e
2018-01-07 21:53:24 +00:00
Sagi Shnaidman 2d96353c28 Collect logs in post playbook for multinode
Collect logs after main job is finished, so that we'll have logs
in any case of job result.
Do it for multinode jobs only, because ovb nodes are wiped right
after job finishes and collecting logs there requires a bit
different effort.
Depends-On: I7b7582469b01116bbe754af07c81cc698355d8c4

Change-Id: I600ade65052d28978c9d395323c8b86ed213fd38
2018-01-06 22:24:08 +02:00
Zuul 64a3c979da Merge "Upgrades job not finding featureset file." 2018-01-06 17:29:50 +00:00
Sagi Shnaidman 09f5854b34 Remove hardcoding of overcloud deploy timeout
Now overcloud deploy timeout is derived from DEVSTACK_GATE_TIMEOUT
which is always 170 and equal to 80. It's not appropriate for all
jobs. It will be calculated dynamically depending on how much time
remains in the job for deployment.
Pass end time of CI job to ansible playbook for calculating it.

Partial-Bug: #1738038
Change-Id: I89b743fc16ea4c100ba21ddbe29081fa3e5479e9
2017-12-28 05:29:45 +00:00
Jose Luis Franco Arza d8e5d0c8f2 Upgrades job not finding featureset file.
The content of the  variable $FEATURESET_FILE
has changed, containing now the whole path
to the featureset env file.

Change-Id: Ie35b7055259866e0044c31502c47711e3c7d8d4a
2017-12-26 11:35:45 +00:00
John Trowbridge 46c5a0b62c Use ansible-playbook in place of quickstart.sh
This will change CI to only use quicktart.sh to bootstrap the
virtualenv. The ansible-playbook command will then be run
directly for the deploy and collect logs.

This is a first step in breaking the deploy into multiple
ansible-playbook calls so that we CI the ability to run them in
stages for development purposes.

Change-Id: Ie0a3729277fb608c653e7bc2ab85781d9b815880
2017-12-12 10:45:01 -05:00
John Trowbridge 473afcc755 Use playbook from tripleo-quickstart-extras for OVB
This is the final step to cut over to the combined
playbook in tripleo-quickstart-extras, so that we can
use a common playbook whether there is a testenv broker
involved or not.

Change-Id: I3ed261e3660426e62ed608bc1bc7923f3912a508
2017-12-11 17:22:47 +00:00
Zuul a331a6ca82 Merge "Revert "Install ssh keys of tripleo ci admins on nodes"" 2017-12-09 06:08:49 +00:00
Zuul f649369a32 Merge "Allow running toci_gate_test on pre-provisioned ovb env" 2017-12-06 22:53:35 +00:00
Sagi Shnaidman decd72868e Remove workaround for periodic jobs
We run now periodic jobs as promotion jobs only, no need to use
different release files for them.

Change-Id: I47d25b062347f55d180f5e53781c5a29b51e1381
2017-12-06 08:21:40 +00:00
Zuul 2f6a036f2b Merge "Remove debug config of unbound since it's enabled" 2017-12-05 21:49:46 +00:00
Zuul 3cd744dfb3 Merge "Default $NODEPOOL_PROVIDER" 2017-12-05 03:31:26 +00:00
Sagi Shnaidman 1f7de38e47 Remove debug config of unbound since it's enabled
Infra already sets unbound to debug, so we can remove it from our
code.

Related-Bug: #1736157
Change-Id: Id5a50218d6b47354f12cc50f88d84c0c86196e3b
2017-12-04 15:00:59 +00:00
Sagi Shnaidman 2dc251f965 Create pre-generated fake subunit file
Return subnit file for tracking all failed jobs in openstack health
system, even if they didn't reach step of running tempest tests.

Change-Id: If6709390aa5f10d94e7be86e5cc3a5ab4236f472
2017-11-26 15:22:24 +02:00
Emilien Macchi e88105afa4 Remove fake pingtest subunit
python-os-testr package isn't available anymore and we're switching ovb
jobs to use tempest anyway, so we don't need this workaround anymore

Change-Id: Ie3b37b17bf41b5f36bc60bfd0e4607ed4a3aaf6d
2017-11-24 04:29:04 +00:00
Sagi Shnaidman de4a74ea6c Set debug in unbound only if config file exists
In CI reproducers unbound config files could not exist, so
check before changing them.

Change-Id: Iae7d1e23e819de3dfc37b5588eb73d5cb8ba8f33
2017-11-21 10:44:42 +02:00
Ben Nemec c72ef60ea4 Revert "Install ssh keys of tripleo ci admins on nodes"
It was highly inappropriate to add new ci admins in this change,
especially without mentioning it in the commit message.
Note that strictly speaking we require a quorum of the
existing admins to approve adding new ones, although with the
number of admins growing we may want to revisit that policy.
As of this time it is still in effect, however.[1]

And even if it weren't, it shouldn't have been
piggy-backed on this change.  Please resubmit as two separate
changes so the admin change can be properly reviewed.

1: Unfortunately we lost the documentation of this when the
admin list was moved from tripleo-incubator to tripleo-ci:
4de13b35d9/tripleo-cloud/README.md
That should be fixed too.

This reverts commit a4d1f8c960.

Change-Id: I6a5484dfb1c00431ae89ec20d9dd66b623b3e622
2017-11-20 22:54:27 +00:00
Sagi Shnaidman 962205603d Set ubound to debug level and dump cache
Debug unbound DNS queries in job for investigation of
DNS failures.
Dump unbound DNS cache server cache..

Depends-On: Ia76ac9b20d6b8402060b71e11e00e515bc74077d
Related-Bug: #1730931

Change-Id: Ic8b9e33067141366655fad1b99dee1adb40dfba3
2017-11-20 14:11:16 +02:00