The v3 migration script can handle these just fine - but if we go ahead
and remove them the output can be predominantely shell rather than
script.
Change-Id: I440851a0149e1cc7f8c5c6e8a4e3e0b94088ee82
As part of the docs-migration, we already changed the publish job to
only run on xenial - all official repos have eol'ed mitaka. Thus, remove
the job definition for trusty as well.
Change-Id: I8840ac0f4dafeaedb2743f5f89f876532336bf70
The run-docs.sh script already does some renaming and file moving for
us, so the unified publishing job needs to be more careful about the
renaming it does as it prepares files for publishing. We want everything
for a project to appear under docs.o.o/$project/, with master going to
"latest", other stable branches going to $series, and tagged versions
going to a directory named for the tag.
Change-Id: I4c0350b29ff910e5acdd1c0982d0096a20188495
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Since we're publishing now with every commit, use the unified publish
jobs everywhere. Stop publishing pre-release docs, we don't use
pre-releases and always have current docs up, so can use the template
everywhere.
Also, remove one obsolete comment.
Change-Id: I6cc2cf94a90494959219448d62993bef014714f6
After the doc migration, these templates should no longer be needed.
Change-Id: Id997f88a34501824ea1fcd3ee5cd6fff085225ad
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
The old version of the job converted the tag name to a branch and used
that as the publishing location. We do, however, want to be able to
publish docs for a given version of a deliverable when we tag it. That
will allow us to refer users directly to the guide for the version of
their tool or service.
This change assumes that the job is only configured in the release queue
and will not be run for pre-releases. If it sees that the trigger for
the job is a tag, it uses that as the output location. Otherwise it uses
the branch name. That gives us docs.openstack.org/$project/$version or
docs.openstack.org/$project/latest or
docs.openstack.org/$project/$series as the final destination of the
docs.
Change-Id: I5a3fe2775fc21c23fb1df6f6c987ec63000f1c7c
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This new job will be used after we migrate the content from the
openstack-manuals repo into project trees.
Change-Id: Icfe74c1f9eabe50bbf6fe686c2e948fceca135cb
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Remove branch-git-prep, we can use zuul-cloner everywhere.
Rework zuul-git-branch-prep to follow the logic of
zuul-git-branch-prep-upper-constraints. Note that zuul-git-branch-prep was not
used prior to this patch.
Change occurences of branch-git-prep to zuul-git-branch-prep.
Change-Id: Id938a858a0ff1967b97293405ef41b9b281f407f
The AFS based publishing and serving via files.openstack.org works fine,
we can stop publishing to docs.openstack.org
Change-Id: I4d6b51c7f8065e5cd21942acc64ae56639bb185b
Across the entirety of our job config corpus, we only ever set the
doc-publisher-site parameter to docs.openstack.org. Instead just set
it directly in the publishers where it's needed and reduce
unwarranted confusion for people writing job configs.
Change-Id: I87a17ee7de618361f12a025d23629b7d5d711810
I don't think we can switch on branch name for these release queue jobs
because the ref info will always be fore refs/tags/*. Just move forward
to running this on Xenial since the vast majority of our future releases
that need publishing will have been tested on Xenial.
Change-Id: I855a2242af4644dd86117812a5efab4643a3f771
Create jenkins/jobs/include directory and move the two scripts that we
currently have into that directory.
Adjust builders using the scripts.
Change-Id: If0cda5a6749dd19fdc57b6f64a6f7ed5ac5bfc00
Add a new script to build release notes including translation,
we will run this script as part of each release notes build.
The new script will build releasenotes and all translations for them.
It checks for available translations and only builds for these.
The translated releasenotes are published in-tree, so for
trove-dashboard this would publish:
/releasenotes/trove-dashboard # Original untranslated content
/releasenotes/trove-dashboard/ja/ # Japanese translation
/releasenotes/trove-dashboard/ko_KR/ # Korean translation
The index file gets automatically enhanced to include
links to the translated releasenotes.
A preview of the generated release notes are at:
http://users.suse.com/~aj/horizon-releasenotes/http://users.suse.com/~aj/trove-dashboard-releasenotes/
Include the file directly in JJB so that it can be changed anytime.
For details, see:
http://docs.openstack.org/infra/jenkins-job-builder/definition.html#module-jenkins_jobs.local_yaml
This needs a change for jenkins-projects-checks.py: The yaml is now an
extented yaml, use the JJB yaml loader to parse it.
For now, use a separate job gate-{name}-i18n-releasenotes-nv and add it
only to horizon and openstack-manuals so that we can evaluate that this
works as expected. A followup will add this for all jobs and the
publishing as well.
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Change-Id: Ic64d571a91bc1bd292d45c0f6ba1a67bfee75997
This is so that we can create a list of directories to ignore when we
rsync documentation builds to a publishing site. Some builds are
published as subdirectories underneath the results of superior builds.
This allows those superior builds to be rsynced without removing the
other builds that were published under them.
Also, create a common builder add-docs-root-marker and use it where
possible and update publishing of release notes as well.
Change-Id: I34cd631eb199d77801ad9fa068aa209f02ad1cfd
This is so that we can create a list of directories to ignore when we
rsync documentation builds to a publishing site. Some builds are
published as subdirectories underneath the results of superior builds.
This allows those superior builds to be rsynced without removing the
other builds that were published under them.
Change-Id: Ie2eb32c4853188fa059018dffb0e161fcf95c085
Both builders do basically the same thing: Check out a repository and
the requirements repo and make the upper-constraints file available.
Problem is that zuul-git-prep-upper-constraints cannot use in
tag/release queues. The builder
zuul-release-git-prep-upper-constraints was written for this. It can be
used in all queues, so let's use that one instead of having two
builders.
The change removes zuul-git-prep-upper-constraints, renames
zuul-release-git-prep-upper-constraints to
zuul-git-prep-upper-constraints and changes usage of
zuul-release-git-prep-upper-constraints to
zuul-git-prep-upper-constraints.
So, we now have one builder under the
name zuul-git-prep-upper-constraints that can be used everywhere.
Change-Id: Ifbebd0f8a7b265dfe517b4c180502580b4b0b55f
Use zuul-cloner with constraints for {name}-docs-{node} and
{name}-docs-tags-only jobs. These jobs run in release and post
pipelines, so use zuul-release-git-prep-upper-constraints builder.
Change-Id: I87406f3f42e8d3f2d093b1bccc11f7e479f0aac7
Testing of releasenotes publishing looks fine with the test job, remove
the test job and enable zuul-cloner for the releasenotes jobs.
Since releasenotes are always published from master, run the publish
jobs as well as the check/gate jobs on xenial.
Change-Id: If5d6b11bd795091eef949001cca6e8d4ec723ef2
This is part of our effort to migrate to ubuntu-xenial.
Change-Id: Ida82b5c601bd3c1cc2eb1e52dbdde566777f1a44
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Use zuul-cloner with constraints for python jobs that use venv.
Those jobs all run on ubuntu nodes, so zuul-cloner is available.
Note that we can switch using constraints in projects yet since the
translation jobs use venv environment and run on the proposal node that
does not have constraints setup.
Change-Id: I1cd03efc184a7fba6073afd0dd16e6e9f17facd8
Add more information to see whether all branches have the proper state
and tags are available.
Run the test job also on openstack-manuals for further coverage.
Change-Id: I58a862271e4f4d2bd0ac92ee00e59aac490595b8
Add new job for testing that constraints work with release notes.
This job runs on master, so let's run it directly on xenial.
Change-Id: Iec2af9380534672786ef36fed74b374f99c30859
We defined with the openstack-publish-jobs group both doc and
doc-tags-only jobs, but each repos uses either of these, never both of
them.
Create a new openstack-client-publish-jobs group and use that as
appropriate. Remove the docs-tags-only job from
openstack-publish-jobs.
This removes 387 unused jobs.
Change-Id: I441c75debe9e9f744790a60cc51d93eef6460a9c
Documents that we only publish in the release pipeline, need to get
published to the top level directory as well as to a versioned subdir.
Create docs-tags job for publishing of these repositories, like it was
done for normal python projects.
Remove the storyboard publish jobs, we can use the normal infra publish
jobs for these. ci.openstack.org is not used anymore and the location
http://ci.openstack.org/storyboard
redirects to http://docs.openstack.org/infra/storyboard/
Also, update a misleading description.
Change-Id: Ibef97f8d0c06b9f7f000872de14283e2911d3099
As discussed on the mailing list [1], merging tags between branches
confuses the history as reno sees it and results in invalid release
notes builds.
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-June/096895.html
Change-Id: Ie09158ca0d522905b229251564ff5141e59e2b6c
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
We have some headroom on the logs filesystem and this allows us to go
back to more reliable methods of grabbing logs. Specifically we are not
required to have http access back to the jenkins master from every job
that runs into order to get the console logs then upload them to swift.
Change-Id: Ie8479c224b2fd3c3efca4e1bf4157656540eccff
Ensure that the release notes job that runs after a patch merges or when
a tag is applied always checks out the master branch of the project so
the full set of notes is built, and not just the version from the branch
into which the patch merged.
Change-Id: I7cbb0f60d8120300c438cb44598b9f7b16e8abe5
Closes-Bug: #1552911
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
No projects uses the constraints jobs anymore, we can remove the jobs
from Jenkins setup now.
Remove also conditions for constraints in zuul/layout.yaml that are not
needed anymore since there's no job called -constraints anymore.
Change-Id: I93f2fc2a875ceab9483e9170274761e82d497aed
Depends-On: I48ecd961d68b36e5d418171cc449681092816523
Projects using openstack-server-publish-jobs publish documents both from
post and release pipelines. Now a tag on stable branch will publish the
documents both to developer/$project/$tag and developer/$project. This
overrides the documents published in the post pipeline - replacing
master documents with stable documents.
The publishing to both locations is correct for projects that only
publish as part of releases but not for those useing
openstack-server-publish-jobs.
To fix this:
Create a new job '{name}-docs-tags-only' that adds a new parameter to
run-docs script. If that parameter is passed, tag publishing will only
happen to developer/$project/$tag . Add the new job to the
openstack-publish-jobs so that all projects can use it.
Adapt openstack-server-publish-jobs in zuul to use the new job.
This change also adds constraint version of the publishing job. A new
builder docs-tags-only-env has beend added for the new jobs.
Change-Id: If0d000358f17a7d9ea0b8c3e3aada1c16291b3ee
Use branch-git-prep instead of zuul-git-branch-prep to prepare the local
source repository, since it appears the latter does not work properly in
post jobs, yet.
Change-Id: I8cbcd49ff4bf7234e149985535f394bb686c58c3
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Between the Icehouse and Juno releases, OpenStack changed its
supported Ubuntu LTS platform from Precise to Trusty. In support of
continuing testing stable/icehouse and prior branches on Precise
while starting to test newer branches on Trusty, a branch-based
decision tree was introduced into our Zuul configuration by way of
complex parameter functions with an ever growing list of regular
expression matches on job names.
Now that Icehouse is EOL (in fact, Juno is too) this simply adds
unnecessary complexity and an attractive nuisance for cargo-cult
copying into new job and job-template definitions which don't even
need it. In preparation to remove the associated parameter
functions, get rid of multiple labels in all jobs and set them to
whatever label Zuul would ultimately select.
This change touches a vast number, possibly a majority, of our jobs
and so manual jenkins-jobs updates on all Jenkins masters will be
needed after this merges, before it's safe to approve the
corresponding Zuul configuration cleanup.
Change-Id: Ic952ee02da2c77fe2ace81c4e2fa87531be6119c
Monty Taylor