Commit Graph

11875 Commits

Author SHA1 Message Date
Andreas Jaeger 91400af8ba Good bye jenkins/scripts
We now only have afs-slug.sh in jenkins/scripts/ and it is only
used by nodepool/scripts/configure_mirror.sh. Move the file into the
nodepool/scripts directory and adjust paths.

Remove copying of jenkins/scripts from nodepool, the directory does not
exist anymore.

Remove 89-jenkins-scripts, we don't need it anymore.

Depends-On: https://review.openstack.org/548393
Change-Id: I13cac791d70a2e5bcb0de1ba0adcd79e1b0aee9f
2018-03-05 14:28:42 +01:00
Zuul 4fffe3de5b Merge "Move release-tools out of jenkins/scripts" 2018-03-05 12:39:46 +00:00
Dirk Mueller da2fc3bd30 Set Mirroring Slug properly for the Tumbleweed case
Change-Id: Ice3820690d0b28a9e71dc655183e6a046e8192eb
2018-03-02 02:10:54 +01:00
Andreas Jaeger fbc1f58c30 Move release-tools out of jenkins/scripts
To not install these on the images, create a new scripts directory to
hold them.

Create new copy-release-tools-scripts role to copy the files over,
we do not need using legacy-copy-project-config-scripts anymore since
everything is copied elsewhere.

Add the role to all playbooks that use release-tools scripts.

Change-Id: I36c458e297fec58023799f625739b341c4d403cf
2018-02-27 19:27:35 +01:00
Zuul cb2662adba Merge "have tag-releases skip closed series" 2018-02-24 20:54:20 +00:00
Doug Hellmann d76313f901 have tag-releases skip closed series
We don't want the tag-releases job to re-process very old branches if
we need to update the data files in openstack/releases. For now we
hard-code a list of closed series.

Change-Id: Ife79d13de4e2abcf9db34b2d15859e1d048e592e
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-02-23 14:09:54 -05:00
Andreas Jaeger 3cbae461ad Install proposal scripts only when needed
Move proposal scripts from jenkins/scripts to the proposal directory
so that we can install them only for proposal scripts and do not need
to bake them into nodes.

Change-Id: I356906e4a43597cc913bfa539cac7c2c6dfce117
2018-02-15 08:04:24 +01:00
Andreas Jaeger affdbab218 Consolidate common.sh script copying
Create new role for copying common scripts, this includes for now only
common.sh. Use this for proposal and translation jobs.

In the past we duplicated the file, this allows to have one definition
only.

Change-Id: I72c14341f511fdaa2c67a3758387afc9445869cc
2018-02-15 08:03:40 +01:00
Zuul 726be81b89 Merge "Don't skip branching for projects with no releases" 2018-02-14 17:36:20 +00:00
Sean McGinnis 237f4b2b91 Don't skip branching for projects with no releases
We have things like devstack and grenade that do not do releases
but still create stable branches. This removes a check that would
skip the branch creation if there are no releases present in the
deliverable.

Change-Id: I6c17dd55ca628b9ee7c31d30374550c3be053348
2018-02-14 10:02:17 -06:00
Andreas Jaeger 3a806df6c6 Move wheel-build scripts out of jenkins/scripts
We only need the three wheel-X scripts from jenkins/scripts and need
them only for the playbook and its roles. Move the scripts in the
roles directory and copy them only if needed - using the scripts module.

Remove copying of complete jenkins/script for this playbook using the
role legacy-copy-project-config-scripts, it's not needed anymore.

Change-Id: I2e481fcf7ca148aac8a36ae99d8598ba26078a25
2018-02-13 21:12:45 +01:00
Andreas Jaeger 8fb489c4ad Remove some moved slave scripts
These scripts are only used by legacy jobs that life in
openstack-zuul-jobs, move them over to the repo and remove
them from getting installed in our nodes.

Remove unused script package-gerrit.sh, it's not used for some time
(pre-Zuul v3 conversion).

Depends-On: https://review.openstack.org/542649
Change-Id: I9f3d69cff5bc88acd9d2252c0b563ec6f5084f65
2018-02-09 08:14:08 +01:00
Zuul 05ace22976 Merge "Move jenkins/data/bindep-fallback.txt (2/2)" 2018-02-22 21:14:31 +00:00
Andreas Jaeger 2c8541a21c Move jenkins/data/bindep-fallback.txt (2/2)
Remove using old location, we can switch now to the new location
everywhere.

Note: We can only merge this after new images are build with the new
location build in.

This needs a recheck once change 1 is in since project-config is
not self-testing. So, we can remove jenkins/data/bindep-fallback.txt in
this change.

Change-Id: I08f23e96defb152946ade1490628f37f0215300d
Depends-On: https://review.openstack.org/543141
Depends-On: https://review.openstack.org/543142
2018-02-20 20:52:47 +01:00
Zuul 83227f5268 Merge "Move translation scripts out of jenkins/scripts" 2018-02-08 14:47:55 +00:00
Zuul 8631464c73 Merge "Remove install-distro-packages.sh from images" 2018-02-08 07:24:06 +00:00
Zuul 8d55c740c2 Merge "Add description to update_upper_constraints patches" 2018-02-08 05:07:17 +00:00
Andreas Jaeger 3d686adc0b rm unused jenkins/scripts/*
These scripts are not used anymore with our current jobs, delete
them.

Change-Id: Ibaa66a7b9b1af867298909b69ace45d21e120a09
2018-02-07 21:17:32 +01:00
Andreas Jaeger e20c9b0d52 Move translation scripts out of jenkins/scripts
No need to install these files on all nodes. Instead copy them over only
when needed and place them together in the translation directory.

Change-Id: Iebffa4f7b5998f2e1a3ee660f1242a11abf1e3f8
2018-02-07 21:05:17 +01:00
Andreas Jaeger cd6f594a0d Remove install-distro-packages.sh from images
This is now installed for legacy jobs only, remove it here.

Change-Id: I663f6b2bd551f4efacf7392477ee90e636c4e636
Depends-On: https://review.openstack.org/541780
2018-02-07 17:27:22 +01:00
Monty Taylor 27a0666385
Change directories to os-service-types during update
The update script framework calls the update function with an argument
of the directory of the project we're trying to update. The update
script needs to pushd into it before curling the new content.

Change-Id: Ic569aee5716ca32e755adda02f149359b532c4fc
2018-02-07 09:54:34 -06:00
Andreas Jaeger 1e48e84b69 Remove zuul-sudo-grep.sh
We copy this file over in legacy jobs:
http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/legacy/pre.yaml#n47

Remove it here, no need to duplicate.

Change-Id: I446537bb2a8d1abbb78104626253dd434d1d0495
2018-02-07 16:19:45 +01:00
Andreas Jaeger 6c189c6117 Remove jenkins/scripts/run-tox.sh
Now all users of this script are gone, time to delete it.

Change-Id: Ic452adf1265b63ef76731700fc8ee4a2058e8f9b
Depends-On: https://review.openstack.org/540603
Depends-On: https://review.openstack.org/540596
Depends-On: https://review.openstack.org/540595
Depends-On: https://review.openstack.org/539207
Depends-On: https://review.openstack.org/537789
Depends-On: https://review.openstack.org/540659
Depends-On: https://review.openstack.org/538344
2018-02-07 07:31:57 +00:00
Ruby Loo eb22332f78 Add description to update_upper_constraints patches
This adds a description to the patches that update the upper
constraints of projects, to indicate that the patch depends
on the branching of the openstack/requirements repository.

Change-Id: I58ab61715ec4876f722caa04869cedb0be822491
2018-02-06 13:58:46 -05:00
Zuul aa71919383 Merge "add a retry loop to clone_repo.sh" 2018-02-06 05:36:59 +00:00
Zuul a53d519c90 Merge "add git timeout setting for clone_repo.sh" 2018-02-06 05:34:58 +00:00
Doug Hellmann 88c097cc97 add a retry loop to clone_repo.sh
Retry expensive remote git operations like cloning and fetching.

Change-Id: Ic68ebf45663ea14c821a565f53a603e1bf8ab5db
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-02-05 18:07:28 -05:00
Doug Hellmann ca1672c0c4 add git timeout setting for clone_repo.sh
Use GIT_HTTP_LOW_SPEED_TIME as a way to time out the git operations
talking to remote servers.

Change-Id: I1d828cf629d5370995b2b88eb8e9763438b03aae
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-02-05 18:07:27 -05:00
Zuul b68ad99ab8 Merge "Use zuul-sudo-grep" 2018-02-05 20:35:39 +00:00
Zuul 4189d21381 Merge "Remove unused xstatic_check_version.py" 2018-02-05 19:50:19 +00:00
Andreas Jaeger 2bd133384e Remove unused xstatic_check_version.py
The file is not used anymore, we have already moved it to
http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/xstatic/xstatic_check_version.py

Remove it.

Change-Id: I40e1317fb814cfdfdaef6ada8370946e4c8129ea
2018-02-03 18:30:47 +01:00
Andreas Jaeger 937906694c Remove unused jenkins/scripts/run-tox-* scripts
We converted some jobs to Zuul v3 native, remove now unused scripts:
* The run-tox-api-ref.sh script is obsolete for some time, remove it.
* the tox-with-X scripts, they are not used anymore (see dependency)

Change-Id: I296102a5fdd0ecf688ed5ae68fa061577814ccf7
Depends-On: https://review.openstack.org/540586
2018-02-03 16:16:04 +01:00
Andreas Jaeger 684bb2682e Use zuul-sudo-grep
Remove jenkins-sudo-grep and use zuul-sudo-grep instead in the only user
which is run-tox.sh.

This allows to cleanup the legacy pre.yaml file in openstack-zuul-jobs.
that file replaces already jenkins-sudo-grep with zuul-sudo-grep.

Remove also now unneeded jenkins-sudo-grep settings.

Change-Id: I73e57eef72b1e5a0ec7be9e57160b24c6076e710
Needed-By: https://review.openstack.org/538344
2018-01-26 22:39:10 +01:00
Andreas Jaeger d8c76a79a7 translations: Fix nodejs install
We're not using local install yet, so make the nodejs function handle
running in a venv.

Currently it fails with:
"Can not perform a '--user' install. User site-packages are not visible
in this virtualenv."

Change-Id: I7a62a6fb236286c967fe2fbfb0266e016d3dcecf
2017-12-21 09:34:56 +01:00
Zuul 6000fd9b6e Merge "Small refactor of translation scripts" 2017-12-20 20:01:03 +00:00
Zuul 8995ddbd42 Merge "Prevent duplicate pull_from_zanata on the same repo" 2017-12-20 17:14:11 +00:00
Andreas Jaeger 235d97b627 Small refactor of translation scripts
The modulename parameter for setup_projects are not needed anymore,
their usage has been removed with retirement of mitaka. Remove these
parameter.

Call setup_project only once per project and not for both python and
django setup. Also, move the zanata pull out to not have to create a
lock.

Change-Id: Iaf58a6882527eee3ce6528b516fe0f24f51cd976
2017-12-19 13:14:01 +01:00
Frank Kloeker f86fd674c4 Prevent duplicate pull_from_zanata on the same repo
If django and python modules are on the same repo, propose translation
script overwrites on the second run with pull_from_zanata files from
other modules. This is not required because the full project files
will already fetched on the first run.

see
http://logs.openstack.org/periodic/git.openstack.org/openstack/networking-bgpvpn/master/propose-translation-update/4826efb/job-output.txt.gz#_2017-12-16_06_38_59_479293

Change-Id: I02f8bf74e9bd8f572877368e3e13d950d6639768
2017-12-17 09:01:05 +01:00
Monty Taylor d261ebdc96 Use ensure-babel to make sure babel is installed
We made an ensure-babel role. Use it instead of the shell script.
From doing that, remove references in the translation update
scripts and instead source the activate script. Ensure-babel currently
installs into a venv, but may want to just do pip install --user in the
future.

Change-Id: I0ea86040b91c1c3de4f5b315183974548e51b5fa
Depends-On: I1334ff1f469061884b222dd99e72a989d72c68be
2017-12-16 15:42:26 +01:00
Andreas Jaeger 139c6eab98 Fix npm publishing
During the conversion, the nodejs4-publish-to-npm template was replaced
with python publishing jobs by accident.

Really use nodejs4-publish-to-npm for the publishing and remove thus
legacy-npm-upload and publish-openstack-python-tarball for npm repos.

Remove also npm-tarball-upload.sh, it is only used by the
legacy-npm-upload job and thus not needed anymore.

For reference, see
http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml?h=before-jenkins-config-removed
and check the templates used by the projects changed.

Needed-By: I696a23792f8e4056b788ad4a8b018c423aa1af68
Change-Id: I5cccd00518afcc50c575a4bb1937345a30fe584f
2017-12-15 12:13:52 +01:00
Andreas Jaeger 6a3e8b03fa Improve translation jobs
Looking at [1], I see "Cannot rebase: You have unstaged changes.", let's
add a "git status" to see what might be the reason.

Looking at [2], an error occurs since releasenotes/source/locale is not
under version control. This can happen if the project has translated
files on a branch from copying. Ignore the failure.

[1]
http://logs.openstack.org/periodic/git.openstack.org/openstack/networking-bgpvpn/master/propose-translation-update/dda4792/job-output.txt.gz
[2]
http://logs.openstack.org/periodic/git.openstack.org/openstack/trove-dashboard/stable/pike/propose-translation-update/8deef07/job-output.txt.gz

Change-Id: Ia4f4863a60a64d5f6333e4550a548b8ed3fe56f5
2017-12-15 07:59:34 +01:00
Zuul 7ab9e5175e Merge "Remove unused jenkins/scripts scripts" 2017-11-28 09:41:20 +00:00
Andreas Jaeger 4b5afc5c82 translations: Use installed os-testr
We do not need to install os-testr for each run, it's already installed
on the nodepool images we use. Thus change the script to use the
installed environment.

Change-Id: Ifa98cb4e7d35c2b45996a7f4fce9eba953f20c99
2017-11-27 08:18:02 +01:00
Andreas Jaeger 5bdc92a620 Remove unused jenkins/scripts scripts
A couple of scripts are not used anymore with Zuul 3, remove them.

Change-Id: I7d7f9dae76545720af8c7c9ed897a630f56a6bba
2017-11-24 19:15:46 +01:00
Ian Wienand e7d3379aaa Remove ruby from bindep-fallback.txt
These have been part of bindep-fallback.txt since the original commit
(I0219e1b8e3c25acebe6a26dd738943c7ed000426) but they really don't seem
like the type of thing we want a) installed in all fallback jobs or b)
projects having hidden dependencies on.  I think there's enough bindep
support out there we can remove this and put it into jobs as required.

Change-Id: Iedb3feabc75715e92c252ed956daf39fd04612bc
2017-11-23 07:33:05 +00:00
Zuul 79ecde3cd4 Merge "Remove some non-standard things from bindep-fallback" 2017-11-22 14:26:27 +00:00
Andreas Jaeger 771cdf437f Follow changed .venv location
.venv is now in $HOME/.venv as said up by ensure-reno.

Change-Id: I21cefac9af96624f5f669b0f31f8b8c3d7d5f4d3
2017-11-22 06:45:29 +01:00
Ian Wienand f3cac1e4a0 Ignore all dot-files for translation updates
We have moved from using tox to installing our own venv in ".venv",
which is now matching.  Modify the excludes to just ignore all
dot-files, as there will be nothing translatable in there.

Change-Id: I0c5f30dc3ea19ee8ee74e02e265f42ad2b54095c
2017-11-20 10:43:00 +11:00
Andreas Jaeger 67c123ea99 Fix translation scripts
The translation scripts need sudo, we cannot use the release notes
pre.yaml which revokes sudo. Instead, create our own pre script and use
it.

Use .venv/bin for sphinx-build since that's where the current role in
openstack-zuul-jobs puts the content.

Depends-On: I23ae1ed7abe7f36d46aae3d72293210ca2a8c0e6
Change-Id: Ia7427bef0408bc5138b6f2a0eb30713ce8fd379d
2017-11-18 14:37:47 +01:00
Monty Taylor 36304438ee
Replace git:// with https://
git:// is not a secure protocol. It's also harder to do apache rewrite
tricks and whatnot when we eventually flatten namespaces. Change
occurances of git://git.openstack.org to https://git.openstack.org.

Change-Id: Ic54f1b84346e37a76baa87deadbb540662889bca
2017-11-17 09:06:21 -06:00