Commit Graph

36 Commits

Author SHA1 Message Date
Monty Taylor 355f7090a6 Remove zuul v2 and jjb content
It's migrated, it can be removed now.

Depends-On: I6a89752d74ed7424267c3af3937ad01fb4bb8f86
Change-Id: Id67d0149b645bd23e460cef65719541f76f0967b
2017-10-24 08:50:05 +02:00
Andreas Jaeger e369275eee Convert jobs to use local_conf
Convert some more job templates to use local_conf.

This handles files [s-z]* and converts all templates where a conversion
was directly possible.

Change-Id: Idf3143ea38709d69e8de01667bc129fe9cacdbca
2017-05-27 16:48:40 +02:00
Andreas Jaeger 43c71fa427 Cleanup PROJECTS and DEVSTACK_PROJECT_FROM_GIT
Change Id2caf9561b361c1d4950856423282535d067e015 for devstack adds all
repositories in DEVSTACK_PROJECT_FROM_GIT to PROJECTS, so no need to add
repos in jobs to PROJECTS if there are in DEVSTACK_PROJECT_FROM_GIT.

This change went through all of jenkins/jobs and removed PROJECTS where
there is direct 1:1 correspondence. For a few files, especially ironic
and freezer, this was not doable.

Change-Id: I4b51e33b9bc0f1cb7948b69abf104e0b57b9ff52
2017-05-01 19:07:06 +02:00
Amrith Kumar 1fa3d6c717 Hot Fix: enable DEVSTACK_GATE_TROVE for trove's jobs
The devstack default (gate) doesn't clone trove anymore. This is
necessitated by changes in [1].

[1] https://review.openstack.org/#/c/446623/

Change-Id: I0e960c4bb5546a923b486e22179dfe3c351779ed
Related: Id2caf9561b361c1d4950856423282535d067e015
Related: Icb3c18ab955b53c028008d6c374301c1e6ca0ccd
Related: I787944217b8cfa856aae3af4a9fad0deb4d4cac5
2017-03-18 07:05:20 -04:00
Andreas Jaeger 4a61bc67f5 Change job-suffix to suffix
We have now some rather long job-template names, shorten them with
replacing job-suffix with suffix.

Before this change we had 1500 usages of job-suffix and 700 of just
suffix. This change therefore unifies to one setting.

Note that this does not change any job names at all, they are unchanged,
it changes and shortens only the names of job-templates.

Change-Id: I514df039a848d9f6619bca3adc05dfb84a761fc9
2016-12-08 22:05:01 +01:00
Andreas Jaeger 83b994fc71 Switch to xenial: lib-forward-testing/tempest-dsvm-neutron-src
Switch master and newton jobs to run on xenial, older branches test on
trusty.

Note that we have
{pipeline}-tempest-dsvm-neutron-src-{name}-{node}{job-suffix}' as a
clone of {pipeline}-tempest-dsvm-neutron-src-{name}-{job-suffix}', we
can merge these now.

This changes also the python-troveclient specific job since it uses
lib-forward-testing in zuul.

Use lib-forward-testing template in zuul when possible.

Change-Id: Icd4454d02c89c9912a0952b073b62d515300165a
2016-12-05 21:21:29 +01:00
Amrith Kumar 0cf6085415 update timeouts for some trove jobs
recently with the increased number of tests being conducted in the
functional and each of the scenario tests, we've been seeing a large
number of timeouts. further, the timeouts seem to only occur in the
rax cloud, things go fine in the osc cloud.

A review of the failures indicate that we're often within 10 minutes
of finishing when the job is terminated. However, in some cases we
would be approximately 15 or so minutes away.

The intent of this commit is to push the timeouts up with some
possible padding and I'll back them off once we know how the system
works with the xenial jobs.

We have done other things to reduce the amount of time to run tests
like splitting some jobs in two (see the single and multi jobs in
scenario as an example).

I know that with larger machines the tests run much faster, we
(tesora) operate a CI in rax and we don't take nearly as long in those
tests. So, given the limitation of the infrastructure in the upstream
CI, and given that we will be adding more tests given the number of
new features going into the release(s), this appears to be the only
solution in the short term.

Change-Id: I7fb49b4e95d06c58cf5bb7efb4a8961dcc6eb629
2016-12-05 08:31:21 -05:00
Jenkins c702a39406 Merge "revert trove testing change" 2016-11-28 16:44:34 +00:00
Amrith Kumar 1d56b1cf1b Have scenario tests run in two steps
The scenario tests currently run in a single stream and as we add more
tests, this takes a long time to complete. Changes in the way the jobs
are defined in [1] make it possible for us to split tests into
subgroups and run them independently and this is important as we
continue to add more scenario tests.

This change makes the CI jobs leverage that capability. The changes
that make this possible (in trove) were committed in [1] and a
dependency is therefore added here.

As we are midway in the trusty->xenial transition, this commit is
stacked on that pile; when the transition to xenial is complete, the
trusty jobs here will be removed and the xenial jobs made voting.

[1] https://review.openstack.org/#/c/399773/

Change-Id: Iba32f34b2ad70c38951668372a2e04eefb193720
Depends-On: Iea2a996043b4c2d1889995e160891af33ffab5b6
Co-Authored-By: Petr Malik <pmalik@tesora.com>
Co-Authored-By: Peter Stachowski <peter@tesora.com>
Co-Authored-By: Amrith Kumar <amrith@tesora.com>
2016-11-24 19:19:27 -05:00
Amrith Kumar f77e2911e8 specify node type in gate-trove-scenario-dsvm- jobs
This change updates the trove CI
gate-trove-scenario-dsvm-{datastore}{job-suffix} jobs and adds a
-{node} between the datastore and the job suffix.

All existing jobs are carried forward as is with a node type of
ubuntu-trusty. New non-voting jobs are defined for each such job with
a node type of xenial.

The changes to get trove's gate working with xenial are now almost
complete in [1] and the job created for that
(gate-trove-functional-{datastore}-{node}{job-suffix}) now passes
reliably. ie. we are able to run the functional test suite on xenial
and that implies that the image building is working on the xenial
node. This change gives us scenario tests for node type of
xenial. Once these jobs all pass, a subsequent change will flip the
xenial jobs to voting and remove the trusty jobs completing the
migration to xenial.

[1] https://review.openstack.org/#/c/393221/

Change-Id: I18c74486b4e130e99eddfa25bb43835634bc1b24
2016-11-24 18:05:35 -05:00
Amrith Kumar 8fe8400ca0 rename gate-trove-scenario-functional jobs
scenario jobs are by definition functional and the names are getting
long and messy. simplifying gate-trove-scenario-functional- and making
them gate-trove-scenario- instead so when I add the node prefix change
(coming to a theatre near you very soon) the names are not too long.

ditto for the legacy jobs.

Change-Id: I149434240fdf4c0c4110186e7bc86d38fcac3895
2016-11-24 13:17:21 -05:00
Amrith Kumar c42b4ff5e2 eliminate the trove and trove-integration build-image jobs
These jobs harken back to a time when trove used cached images for
testing. We now build the image on the fly and therefore these images
are no longer required.

Change-Id: Ib598aa9bbf8cddde434e0685f546bad0ab9a6e82
2016-11-24 13:01:09 -05:00
Amrith Kumar 09d41349cc make a non-voting gate-trove-functional-mysql-ubuntu-xenial job
The goal of this commit is to create a single non-voting
gate-trove-functional-mysql-ubuntu-xenial job that will only run in
the master branch of the trove project. It will be used to verify a
change that will make it possible to run the trove CI jobs for master
on xenial nodes.

Change-Id: Ibc0ac0ae205a748ffaa7fb4c8e4305d75515aae1
2016-11-21 14:04:20 -05:00
Amrith Kumar 1545ecf45f revert trove testing change
This reverts the trove testing change to experiment with neutron.

Change-Id: Ic0ddb3e5a3740b1d33f97533c23d0394e4a921f8
2016-11-14 15:52:00 -05:00
Amrith Kumar 67c4b18b02 test with neutron disabled
to see if trove CI failures are related to the nova networking to
neutron switch.

Change-Id: Ic8d7f696ceb99a7199a8de4b83ded377bba841e6
2016-11-14 15:42:38 -05:00
Amrith Kumar 508a1abd73 fix path for build-image job
Had one 'trove' too many in the path for the build-image job which
caused them all to fail, and missed the integration in the KEY_DIR
definition.

Change-Id: Ib16e02b58b31b56019a9641a2b4726d8834d49e7
2016-10-18 08:32:42 -04:00
Amrith Kumar dbd81032f3 Create new trove CI jobs
Part of the project [1] to get rid of the trove-integration repository
(in the Queens release), and to get rid of the tool redstack is to
fold into the trove project, all the elements and image building
components that were part of trove-integration.

To facilitate an easy transition, a multi-step approach was
chosen. Now that the "stuff" from trove-integration is in the trove
branch, it is time to turn on the CI on these things, and begin to
switch to using these as the voting CI for the trove project in Ocata
and forward, leaving the old trove-integration based approach for the
stable branches.

The trove-integration based approach will continue to be used for
Newton, and that will be history when the Queen release comes out. If
we can turn on the CI on trove that doesn't use trove-integration (for
master) we will be on the way to that.

This change accomplishes the following:

- rename gate-trove-buildimage-{datastore} to
  gate-trove-integration-buildimage-{datastore}{job-suffix}
- create new gate-trove-buildimage-{datastore}{job-suffix} jobs
- rename gate-trove-functional-* to gate-trove-legacy-functional-*
- create new gate-trove-functional-* job
- rename gate-trove-scenario-functional-dsvm-* to
  gate-trove-legacy-scenario-functional-dsvm-*
- create new gate-trove-scenario-functional-dsvm-* job
- fix branch override and datastores for gate-trove-functional (now
  gate-trove-legacy-functional)

The newly created jobs are experimental, and non-voting and as soon as
they are shown to be working OK, will be flipped into the check (and
gate if their legacy predecessors were voting) to replace the legacy
jobs for master.

[1] https://review.openstack.org/#/c/373153/

Change-Id: I5ff0f00052db18931946fc3b9618d6b5e52c5720
2016-10-18 07:43:45 -04:00
Andreas Jaeger 37870449b2 Jenkins/jobs: Kilo cleanup
Since kilo is EOL, remove now obsolete special handling for kilo.

Change-Id: I25fc335be91331b4ff283ceae6e16b9d79fbc1db
2016-06-24 17:58:56 +00:00
Peter Stachowski 99ee809175 [trove] Add trove-dashboard to PROJECTS
Add trove-dashboard to projects to allow the Trove plugin to install
the dashboard in the gate.

Partial-Bug: 1593806
Change-Id: I0fb2dc3d099f96cfd49d71fa7ed4f42d56026060
2016-06-17 20:48:45 +00:00
Clark Boylan ceda9719f4 Switch rest of devstack-trusty to ubuntu-trusty
Switch all remaining devstack-trusty jobs to ubuntu-trusty to complete
the migration.

Change-Id: I7f94b59f318cab581aa4f194f33acf8ab597fc36
2016-03-14 12:58:43 -07:00
Jeremy Stanley 724dad6b5f Remove revoke-sudo from storlets, trove functional
The I670641b676231233f1dd583ea36421f8ffef8f8e change incorrectly
added a revoke-sudo macro to the gate-storlets-functional and
gate-trove-buildimage-{datastore} jobs which did not previously have
it, so remove it to get them working correctly again. It also added
one to dev-zuul-coverage, but I expect that was missing as an
oversight so not undoing it here.

Change-Id: Ic427c053602017da3d77b01193b4215f57de6fdd
2016-03-08 03:31:46 +00:00
Andreas Jaeger edf30c9694 Convert storlets, trove, zuul jobs to ubuntu-trusty
Use ubuntu-trusty instead of bare-trusty.

Change-Id: I670641b676231233f1dd583ea36421f8ffef8f8e
2016-03-05 09:34:54 +01:00
Jenkins fbf985de46 Merge "Enable trove in devstack on stable/kilo" 2016-02-17 09:28:36 +00:00
Craig Vyvial ec595c5634 run trove experimental run scenario tests
Trove experimental jobs are running old tests that have been changed.
This change is to run the correct group of scenario tests for redis
and mysql. The tests are concidered experimental still because they
are still a work in progress to replace the existing int-tests that
trove uses for testing mysql. This change also helps us setup the
scenario tests so that we can verify other datastores supported in
trove in the future.

Change-Id: If6582ad2497e85b66ff5c90d8317004f059fbea7
2016-02-16 23:07:34 -06:00
Tony Breeds b46df5f60e Enable trove in devstack on stable/kilo
Currently the project-config for trove enables the devstack plugin
unconditionally.  The plugin is only supported in branches newer than
kilo.

Without trove enabled devstack jobs fail as no jobs are run see:

http://logs.openstack.org/34/276934/2/check/gate-tempest-dsvm-trove/b48d429/console.html.gz#_2016-02-09_09_13_18_056

This change enables trove services in devstack jobs for stable/kilo.

Change-Id: I0ded96609d915e3d7b594c4a74e845236a1332e4
2016-02-16 11:56:21 +11:00
Andreas Jaeger 065a80d9c2 Remove DEVSTACK_GATE_TIMEOUT [part 4]
With change Iff5e96bd80b8f165ccc7d6be09b53208613571eb, we do not need
to pass DEVSTACK_GATE_TIMEOUT anymore to devstack-gate, instead the
value from the build-timeout wrapper is passed.

Remove DEVSTACK_GATE_TIMEOUT for jenkins/jobs/[n-z]*.yaml

Increase timeout for all non-grenade jobs by 5 mins since
Iea866e4cc40e149e598f35e010ec855937bdf9ae will use a 10 minute buffer
and the internal timeout should stay the same. The grenade jobs already
have 10 mins of buffer, so no need adjust the timeout.

Adjust timeout for gate-rally-dsvm-cue-{broker},
the removed variable in builder devstack-cue-rally-gate was 100,
so set timeout to 110.

Change-Id: I623fff029fb2e0acdbe2bf93a13af02547062a71
2016-02-07 10:45:25 +01:00
Jeremy Stanley 0af9b2c178 Get rid of precise||trusty job node declarations
Between the Icehouse and Juno releases, OpenStack changed its
supported Ubuntu LTS platform from Precise to Trusty. In support of
continuing testing stable/icehouse and prior branches on Precise
while starting to test newer branches on Trusty, a branch-based
decision tree was introduced into our Zuul configuration by way of
complex parameter functions with an ever growing list of regular
expression matches on job names.

Now that Icehouse is EOL (in fact, Juno is too) this simply adds
unnecessary complexity and an attractive nuisance for cargo-cult
copying into new job and job-template definitions which don't even
need it. In preparation to remove the associated parameter
functions, get rid of multiple labels in all jobs and set them to
whatever label Zuul would ultimately select.

This change touches a vast number, possibly a majority, of our jobs
and so manual jenkins-jobs updates on all Jenkins masters will be
needed after this merges, before it's safe to approve the
corresponding Zuul configuration cleanup.

Change-Id: Ic952ee02da2c77fe2ace81c4e2fa87531be6119c
2015-12-21 20:52:17 +00:00
Andreas Jaeger bd882cdd7a s/branch-designator/job-suffix/g
As clarkb suggested on #openstack-infra at 17:46 2015-10-22:
"maybe we should s/branch-designator/job-suffix/g in all those files
 as job-suffix makes more sense considering how branch-designator has been
 used".

The rename should make it easier for newcomers to understand
these jobs.

This renames branch-designator everywhere in jenkins/jobs/*.

Change-Id: Icb11271b958d92957fb9907ff03c0b898d7b76fc
2015-10-28 07:50:13 +01:00
Clark Boylan 515d2adaf9 Be explicit when installing clients from src
Most jobs installing client libs from pypi now, this means that we don't
need set up the client lib git repos by default in devstack-gate. But
before we can clean up devstack-gate the jobs that do need the clients
need to explicitly state that.

Note that this also setups up ironic and related things because of how
the ironic jobs work they all share a common macro.

Change-Id: I97cc3758d5c97494704406d779a7326420940427
2015-10-13 18:27:23 -07:00
Craig Vyvial 09cce5376e Fix troveclient gate
This issue looks similar in nature to the saharaclient gate issues.

*-tempest-dsvm-neutron-src-python-troveclient job broken.
Need to enable trove devstack plugin for this job because need to test
trove in troveclient jobs

Change-Id: Icd34f4ba09528e691919bfc643c0e2a696f0ce76
2015-09-24 00:37:24 -05:00
Nikhil Manchanda d56b2e46d6 Use the latest trove client for trove-functional-dsvm job
The current trove-functional-dsvm job uses the trove client from
pypi which may be outdated. Update the job definitions so that it
uses the latest trove client from git instead.

Change-Id: I84d204f50cb18f6e96c48b4340219d8b69ba681c
2015-08-25 18:17:25 -07:00
Nikhil Manchanda 56ea12cc80 Enable trove plugin for trove-functional-dsvm-mysql gate job
After the merge of Ie7f5a817ff1bbeccf06cca372fd0d2e49d0e4ccb,
devstack-gate no longer enables the trove plugin implicitly based
on the "trove" service being enabled, which breaks the
trove-functional-dsvm-mysql gate job.

This fixes this gate job to explicitly enable the plugin using
DEVSTACK_LOCAL_CONFIG which should be the way to do it going forward.

Change-Id: I667e6099c73764d545338b32368af0228023bed1
2015-08-14 00:12:17 -07:00
Matthew Treinish 5c8588b760 Add stable compat jobs to trove-integration repo
The trove-integration repo is branchless and is used in runs against
kilo, juno, and master. However commits are only checked against
master, this leaves a huge target on the stable branches' back.

Change-Id: I37ea58a4a8242a24632e0f539f9bd204d7530019
2015-07-30 21:54:15 +02:00
Amrith Kumar 146a8adc4d Experiment with an increase time for gate-trove-functional job
Recently there have been many failures of this job at the current 105m
timeout. However, in some cases, it has been observed that the job is
being failed at 105m.

There appears to be considerable variability in the performance of the
hypervisors used for the CI and I would like to increase the time
(considerably) as proposed in this change and then measure the actual
time it takes to pass consistenly and then resubmit this change with a
more reasonable increase over 105m.

I could increase this by 5minutes and see if it worked or not, and
then come back if it didn't but it is generally the case that a binary
search will converge more quickly on a solution.

Hence, I'm proposing 145 minutes as a starting point in this search.

Change-Id: I7478dde226f5ea2d6b0777e234b53e66ebcb8e57
2015-07-07 00:51:25 -04:00
Amrith Kumar dc2bd188fc Increase trove gate job timeout
Increase timeout for trove gate job, gate-trove-functional-dsvm from
95 to 105 minutes and the corresponding devstack timeout to 100
minutes.

New code added for Kilo has increased the time required for some of
the tests and we are falling just under the threshold.

Change-Id: Ic63bb635785719a19deae0d43cae78e39731e3a3
2015-03-17 13:41:51 -04:00
Anita Kuno 1d21cd635f Reorganizes project-config
This repo was created from filter branching the openstack-infra/
config repo. This process brought a lot of cruft with it in the
form of directories that we no longer need. This patch removes
that cruft so we begin with a tidier repo.

Change-Id: Ibffad1b11c0c5f84eedfb0365369f60c4961a0f3
2014-09-25 11:41:04 -04:00