Commit Graph

271 Commits

Author SHA1 Message Date
Monty Taylor 355f7090a6 Remove zuul v2 and jjb content
It's migrated, it can be removed now.

Depends-On: I6a89752d74ed7424267c3af3937ad01fb4bb8f86
Change-Id: Id67d0149b645bd23e460cef65719541f76f0967b
2017-10-24 08:50:05 +02:00
Ian Wienand 2df98eeca1 Revert "Pin pip to <8 for openstackci-beaker jobs"
This reverts commit bf25cc9e7b.

The pin of pip/setuptools to such a low version is now causing the
gate jobs to fail to install packages, since it doesn't understand new
environment markers we use.  This probably opens Pandora's box of
issues that we never quite fixed?

Change-Id: Ie3f967d27b03a9594d54e8dd413f61daa65a73f3
2017-10-12 11:29:39 +11:00
Clark Boylan d0c95bb642 Don't remove puppet in puppet prep macro.
We are running into errors when attempting to uninstal hiera on trusty
jobs because the hiera package does not exist. We no longer install
puppet as part of our image build process so we shouldn't need to
uninstall puppet, hiera, and facter at all. Simply drop the the removals
to address this as there is nothing to remove any longer.

Change-Id: I1ce79f3f7dd164e0635a4ae45210194f50ab8ad9
2017-09-20 10:23:50 -07:00
Monty Taylor b4fd783219
Remove unmatched single quotes from jenkins jobs
The v3 migration script can handle these just fine - but if we go ahead
and remove them the output can be predominantely shell rather than
script.

Change-Id: I440851a0149e1cc7f8c5c6e8a4e3e0b94088ee82
2017-09-20 09:27:26 -05:00
James E. Blair cdc9673be1 Revert "Use https for logs.openstack.org"
This reverts commit 1f57c9c327.

The logs.openstack.org site is not configured correctly to serve
over https.  I think it's actually serving from docs-draft.
We should fix that, but in the interim, switch back to http.

Change-Id: I170b24f01bdd79e4728d509d2a0f654a479b2674
2017-09-14 14:04:50 -06:00
Andreas Jaeger 1f57c9c327 Use https for logs.openstack.org
We have a certificate, so let's use https.

Change-Id: I613abef49d2e3234e9994953dc8075bdbb3685bf
2017-09-12 16:07:17 +02:00
Ian Wienand 5fe3ba9561 Add Python 3.6 tox jobs
Add a sample python 3.6 job running on a Fedora 26 node.

Create the job macros and definitions for a Python 3.6 job running on
Fedora 26.  Add to the low-turnover bashate project so we can run some
tests on a real project.

At this point, I have not templated it out futher.  We could add it to
templates like python-jobs and create groups in zuul/layout.yaml to
make it eaiser if required.

Change-Id: I04fab592032621325f8638680fc5c842f316684a
2017-08-21 15:13:21 +10:00
Mohammed Naser dd6b03b65c
Fail early if any Puppet preparation commands fail
At the moment, all YUM failures are masked because the shell script
is not setup to exit on failures.  Once the failed stage is passed,
the script will always return true to handle requests removal.

This patch forces the script to exit if any commands return invalid
exit commands.

Change-Id: Ia9f4e5f8cdb9fa0735d609dd075cdc7d73591acf
2017-08-16 11:45:26 -04:00
Paul Belanger 5b61971f20 Revert "Temporarily start using the public registry again"
In 0b00bd6abd we added back our npmrc
files pointing to our reverse proxy cache. Manual testing confirmed
it to be working. Lets start having our jobs do the same.

This reverts commit 26edd71461.

Change-Id: Id308190ff0e938a4b010455275344aff3af3b2a6
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-08-14 13:52:33 -04:00
Alex Schultz dde067b268 Fix puppet install and execution issues
When adding a repository in Ubuntu, one must run apt-get update prior
to trying to install a package from the new repository. Additionally
when this new package installs binaries in an alternative path, you
need to ensure the PATH is properly updated for it to work.

Change-Id: I285baa486221e496baa8e205bd7a92c33d07dd2f
2017-05-30 13:51:18 -06:00
Alex Schultz acaf2582c5 Fix puppet-agent-install on Ubuntu
The wget parameter to download the file was wrong. It should be -O not
-o. This is currently blocking puppet module releases.

Change-Id: Ief269df165a8a46d794f21a6e7dcb6423556ebc8
2017-05-30 09:39:09 -06:00
Andreas Jaeger 9b8c1eb261 Handle traceroute in /usr/sbin
openSUSE installs traceroute in /usr/sbin - but it can run as normal
user. Add /usr/sbin to PATH so that the traceroute and traceroute6 calls
work for net-info.

Change-Id: I49874ca395d2fae173c6054aa9e1d90863460ecb
2017-05-25 10:09:32 +02:00
Alex Schultz 5328118f3c Create a puppet-agent-install builder
The puppet-module-build job requires puppet be available. Rather than
include it in bindep.txt which would conflict with the other jobs, this
change creates a builder to pull in the latest version of puppet from
puppetlabs.

Change-Id: I2c56d16ee3e0be6982e51e4457c9beb25f175c3e
2017-05-16 13:21:34 -06:00
Colleen Murphy e0b69c0225 Use gem-based puppet syntax jobs for infra
Since we removed puppet from the nodepool nodes, the puppet parser
validate command in the puppet-syntax job fails since puppet is not
installed. Rather than modifying that job to install puppet, we can
re-use the existing 'rake syntax' job that the puppet-openstack group
has defined for their modules, which installs puppet via gems and is
therefore simpler. We can also now remove the plain puppet-syntax job
since nothing is using it.

Change-Id: I082ea72743634b8ce0106786ab4ec35add874c8d
2017-05-01 20:06:23 +02:00
Alex Schultz 08180fc389 Remove centos openstack and ceph repos
I0c73034a85db5664e979b9b71cc340860265bf34 switched from using the
rdo-release to using the centos-release-openstack-* packages. Since our
jobs manage the repos, we need to clean out these rpms as part of our
node preparation. Also removing ceph repos, we already manage the repos
in our manifests.

Change-Id: I7a82f9804e2fe52bb1da8bcfa682c0685771af48
2017-04-26 19:01:28 +00:00
Andreas Jaeger 9dbe3ed765 Remove unused python33 builder
This is unused, remove it.

Change-Id: Ic1bb5844e1b7ec60213e9de739371cb0cf84d022
2017-04-12 19:34:17 +02:00
Jenkins ff5d7fc597 Merge "Sync dependencies list for puppet-prepare-node macros." 2017-04-11 19:58:45 +00:00
Anton Chevychalov d0be896e60 Sync dependencies list for puppet-prepare-node macros.
There is ruby-devel package installed here for CentOS but there is no
ruby-dev for Ubuntu. That commit fix that problem.
CentOS change-id: Ib504f3fea4713f14bf3dc2e9659fad0e999f8146

Change-Id: I29e50121f7e5fa48adb47aeec82c68e83282ded5
Closes-Bug: #1681851
2017-04-11 15:59:21 +00:00
Andreas Jaeger 84b763f07e Fix comments about bindep.txt
We now use bindep.txt instead of other-requirements.txt. Fix two
comments.

Change-Id: I6792b918b5202555745ae9afb6e99b559bd0b6e2
2017-04-11 17:28:16 +02:00
Andreas Jaeger 183aabbeaf Remove zuul_swift_upload
Remove the unused builders and publishers for zuul-swift, remove unused
script to upload, and remove unused swift-logs nodepool element.

Change-Id: Idfa83d435a951dcfe6f6545a2515aec281ed43cc
2017-04-10 22:46:44 +02:00
Andreas Jaeger bb75639300 Remove branch-git-prep
Remove branch-git-prep, we can use zuul-cloner everywhere.

Rework zuul-git-branch-prep to follow the logic of
zuul-git-branch-prep-upper-constraints. Note that zuul-git-branch-prep was not
used prior to this patch.

Change occurences of branch-git-prep to zuul-git-branch-prep.

Change-Id: Id938a858a0ff1967b97293405ef41b9b281f407f
2017-04-07 07:45:11 +02:00
Andreas Jaeger acf7572d10 s/gerrit-git-prep/zuul-git-prep/g
We do not need gerrit-git-prep anymore, zuul-git-prep works on all our
slaves and has the advantage of using zuul-cloner so that the local
cache is used and Depends-On works.

Remove gerrit-git-prep and use zuul-git-prep instead everywhere.

Change-Id: I2f106af58f78fceaf3c497fe43e6f5a002695645
2017-04-03 19:36:51 +02:00
Andrey Volkov 28249ca96e Add placement api ref jenkins job
Nova has subproject called Placement API. It will be separated
openstack project in the future but until some moment it's in Nova.
Placement API has its own api-ref run by `tox -e placement-api-ref`.

This change adds new jenkins job gate-placement-api-ref-nv run for Nova
project only. It uploads built api-ref to separate folder on docs-draft.

When Placement API has separate code base this changes should be reverted.

Change-Id: I891a2512efe067f3f45a24cc4ff19460a568046e
2017-03-28 19:34:52 +03:00
Samuel Cassiba 9c35299af8 Chef: adds basic multinode integration tests
- renames chef-test-jobs to chef-rake-jobs for consistency
- adds multinode integration jobs
- changes gerrit-git-prep to zuul-git-prep as zuul-git-prep is preferred
- removes aio periodic gates

Change-Id: I0c316804a2eea2574681778678a8599203007c0d
2017-03-22 19:19:47 -07:00
Mohammed Naser 203754fd02 Print out network configuration data in net-info
There has been a few scenarios where eth1 doesn't get an IP
address which results in subnodes being unaccessible and
builds failing.

This patch prints out the information consumed by Glean in
order to help troubleshoot the issue and figure out the
root cause of the issue if it occurs again.

Change-Id: Ifd50e83fe3d73064b9141ed62380ec57277241d7
2017-03-15 13:47:43 -04:00
Paul Belanger 0f111dc654 Reload sshd on centos for allow-local-ssh-root
We are seeing some issues restarting SSHD for centos.  Because we are
doing config file changes, and SSHD is running, we can use the reload
command.

Reload is a little different then restart, specifically reload does a
SIGUP on the process. In some local testing, this seems to be more
reliable to restart, which causes some bind address errors.

Change-Id: Ic5e00e1e1eed1de87c430180a6f00d30410a95b4
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-03-10 16:11:18 -05:00
Andreas Jaeger fcd4fde168 Remove mysql setup
bifrost sets up the mysql database itself, we can remove the setup from
project-config now.

Note that change I8485751eda745841c1436ecd766c9bb95446f407 has been
merged to all branches excpet mitaka since that branch is broken.

Proposal is to move forward without change in mitaka - and add it there
if critical changes need to go in.

Change-Id: I339d2a301a76ad34ea4bcbfb26c13a21ecd86b6f
2017-02-28 09:09:47 +01:00
Andreas Jaeger 9fc2f0eda8 Remove final db setup
The trove repo now uses tools/test-setup.sh we do not need the db jobs
anymore, change to the standard python unit jobs.

Also use standard pylint job-template which does the same as the tox
template.

Remove all templates and job-groups for the db-jobs since it's last user
has been removed.

Only leave mysql setup jobs for bifrost in.

Change-Id: I92e6e6502c2c516babf2bf66f3514875f77c460e
Depends-On: I3463e75057d0d4544f6a0212da888759ab5e171b
2017-02-28 09:09:08 +01:00
Thiago da Silva 5b9ddf927a update the mkfs.xfs path
While trying to add centos-7 gate job to swift tests
this script failed calling mkfs.xfs. Updating to use the full path
so that it can be used both on centos and ubuntu

Change-Id: I18adb1c1daa11d4f84518e449662ffe2b4e9dbf7
2017-02-27 22:06:31 -05:00
Honza Pokorny 058087d860 Be verbose when running npm commands
Change-Id: I77cf24466dd076cd88908704b4c8dca271c43fc3
2017-02-16 14:45:23 -04:00
Isaac Beckman 1ea492ab0b cleanup: remove gate-openstack-chef-repo builders
gate-openstack-chef-repo job was removed in:
https://review.openstack.org/#/c/276106

With the removal of this job, the following builders are not
needed anymore:
ruby-191-prep
chef-bundler-prep
chef-berkshelf-prep

Change-Id: I8ea78c43b91977a38d35512e66b5bf4ccc5ed455
2017-02-13 14:03:30 +02:00
Jenkins 2e2121ba6f Merge "cleanup: remove publisher upload-swagger-draft" 2017-02-13 02:49:36 +00:00
Isaac Beckman 328c90c07c cleanup: Remove builders of old chef-jobs tests
old chef-jobs have been removed in:
https://review.openstack.org/#/c/264130

The following builders are no longer needed:
chef-cookbook-chefspec
chef-cookbook-foodcritic
chef-cookbook-rubocop

Change-Id: Ie8cda6d175fe5307537e1187393bcf2bcd440736
2017-02-12 15:55:17 +02:00
Isaac Beckman 19da5f7df4 cleanup: remove publisher upload-swagger-draft
swagger job, is not used anymore. it was removed in:
https://review.openstack.org/#/c/364076
The publisher is not used any more as well.

Change-Id: I354af582c69aac08a6e4666788001beecbb9c47a
2017-02-12 13:17:30 +02:00
Sean Dague e1f1444ae5 Add macro for setting local.conf for devstack-gate
Change-Id: Idb4b2b25633dff6aaa816bc57a57ed59e2743cc8
2017-02-09 17:52:15 +00:00
Jenkins 3acca562e5 Merge "Fix comment typo error" 2017-02-01 08:44:22 +00:00
Jenkins 168b63f214 Merge "Revert "Fix translation proposal jobs"" 2017-01-31 20:14:42 +00:00
jeckxie 9ba63a757e Fix comment typo error
Fix a simple typo:
Likelyhood -> Likelihood

Change-Id: I19e5aa092ba05e6a2530fa0a3367769e681a088d
2017-01-31 08:30:51 +00:00
Andreas Jaeger d50a7d452f Revert "Fix translation proposal jobs"
Zuul-cloner / zuul-merger have been fixed and can be used again.

This reverts commit 759da2b1cf.

Change-Id: I99bf00b046951ed6ea544572c52138f671e78be0
Depends-On: If22d8cfbe591afe4f5516da1eb3b0be98b2de874
2017-01-28 18:10:59 +00:00
Andreas Jaeger 1ddc01e341 Cleanup salt remains
Remove 2 now unused builders since salt was retired.

Change-Id: Ie3dbbde96751ed52edb4b397d606d550e9d71f28
2017-01-26 21:04:22 +01:00
James E. Blair 1a258f41d5 Clone zuul into workspace in gate-zuul-nodepool
In order to use the run-tox macro, we need zuul to be in the
job workspace.

Change-Id: I59881626625b36d5f93a38277e8aa07cf2e27800
2017-01-24 16:37:10 -08:00
Andreas Jaeger 58b73f1f72 Stop publishing to previous Cloud sites docs.o.o
The AFS based publishing and serving via files.openstack.org works fine,
we can stop publishing to docs.openstack.org

Change-Id: I4d6b51c7f8065e5cd21942acc64ae56639bb185b
2017-01-13 19:58:02 +01:00
Jeremy Stanley da67129d34 Remove pointless tarball-site parameter
Across the entirety of our job config corpus, we only ever set the
tarball-site parameter to tarballs.openstack.org. Instead just set
it directly in the publishers where it's needed and reduce
unwarranted confusion for people writing job configs. This also
allows us to get rid of the parameterized sites in the tarball,
wheel and war publishers.

Change-Id: I31109f41177b8c07177d7ed8ef16cbfd135131ef
2016-12-19 14:48:08 +00:00
Andreas Jaeger 793e260567 Remove special searchlight unit test setup
With the introduction of tools/test-setup.sh, we can move the setup for
searchlight to the repositories and remove the special setup for these
repos and use the normal python jobs instead.

Also, remove gate-searchlight-python27-searchlight-es1, this is broken
since August and PTL agreed that it should be removed.

Note that this is needed to fix liberty branch of searchlight - liberty
needs elasticsearch 1 while newer branches need elasticsearch 2.

Change-Id: Ie038e9c9a38727c12532bded48a76fe2cecb7fd8
2016-11-25 10:23:16 +01:00
Jenkins 9781f4b2d0 Merge "Run tools/test-setup.sh for unittests" 2016-11-24 09:46:16 +00:00
Andreas Jaeger f0754c93b0 Fix tox-with-tmpdir-passed-in
This is a shell, we cannot exit script. Instead exit with return value
of the script.

Change-Id: I646c89582dd029667a45c4f436b5de65586f9881
2016-11-19 14:03:43 +00:00
Jeremy Stanley b53dd96245 Fix syntax error for tox-with-tmpdir-passed-in
Remove a stray double-quote from the end of the
tox-with-tmpdir-passed-in builder.

Change-Id: I5f729cde54b689300bb023a0a01470f00245569a
2016-11-18 20:41:39 +00:00
John Dickinson f19f2fc381 enable an xfs tmpdir for swift tests
Builder added:
 - mount-xfs-tmp-space: takes a path, creates a 1G file there,
   and loopback-mounts it to a passed in path
 - tox-with-tmpdir-passed-in: a copy of the tox job, but with an
   extra {new_tmpdir_loc} value passed in and TMPDIR set

projects.yaml changes (for swift):
 - replaced 'gate-{name}-tox-{envlist}-{node}' for
   'gate-{name}-tox-xfs-tmp-{envlist}-{node}' and placed
   'py27' and 'py34' envlist entries under it

scripts updated:
 - run-tox.sh: now only sets TMPDIR if it's currently unset

job-template updates:
 - create 'gate-{name}-tox-xfs-tmp-{envlist}-{node}' that is a copy
   of 'gate-{name}-tox-{envlist}' but calls 'mount-xfs-tmp-space'
   before dropping sudo. Then it calls 'tox-with-tmpdir-passed-in'
   instead of 'tox'

zuul/layouy.yaml updates:
 - added the "with xfs" tox jobs to swift

Change-Id: I3339de0d57726b339dfc6638d225e49d097f7b63
2016-11-17 15:45:27 -08:00
Andreas Jaeger ac6f82fb9d Run tools/test-setup.sh for unittests
Instead of setting up databases for jobs, allow projects to do this in
tools/test-setup.sh instead - and run this script whenever it exists
in our tox based unit tests.

Note that currently no repo has such a file, change
I9af7c7d09b40c103af09ae8dd0e50afe3762ecba introduces the first one.

Change-Id: Ic6269e0a03c8540569f242ebc78c98a5922c8026
2016-11-17 18:13:32 +01:00
Khai Do d679cf6e01 rename 'tox' macro to 'run-tox'
JJB already has a tox module[1].  Defining a macro with the same name is
confusing and may cause problems when attempting to use this macro
elsewhere. Renaming it to something else should clear things up.

This is the warning from JJB:

$ jenkins-jobs test -o out ../project-config/jenkins/jobs
WARNING:jenkins_jobs.registry:You have a macro ('tox') defined for
'builder' component type that is masking an inbuilt definition

[1] http://docs.openstack.org/infra/jenkins-job-builder/builders.html#builders.tox

Change-Id: I6946229f6e94a5f4c10d6a5fc3b7049adeacacb1
2016-11-15 11:54:16 +01:00