For some time we experience lots of timeout on our build jobs. Since
some of them are voting we can't merge changes required for release
Change-Id: Ie3d2eae6cff80328fc2c04fc4eb8ecdd3e71de78
Prepare groundwork for upgrade gates and dockerhub publisher, also move
multinode jobs from experimental to regular pipeline
Change-Id: Id6510c454975b5847845f6438c8342d0a9ee0b2e
A few projects used excess '' for node declarations, remove them.
We had 17 'ubuntu-trusty' and 285 ubuntu-trusty,
and 25 'ubuntu-xenial' and 744 ubuntu-xenial.
This changes all node names for consistency, thus making search easier.
Change-Id: I0cf565d1e31ea246f3b9d93bbd8de9980df4c684
kolla-dsvm-build.* related jobs only run since ocata cycle and trusty is
not supported since ocata by Kolla. So remove it in post pipeline.
Change-Id: Ifeff872b23eccd40c044233aacd38b73351b7b44
Kolla jobs had both gerrit-git-prep and zuul-git-prep-upper-constraints.
Only one of these is needed, remove the unneeded gerrit-git-prep.
Change-Id: I1235a52e04be148f70f8ea06da13c903a177ee38
We have now some rather long job-template names, shorten them with
replacing job-suffix with suffix.
Before this change we had 1500 usages of job-suffix and 700 of just
suffix. This change therefore unifies to one setting.
Note that this does not change any job names at all, they are unchanged,
it changes and shortens only the names of job-templates.
Change-Id: I514df039a848d9f6619bca3adc05dfb84a761fc9
Use single one script will be helpful to do more work when jobs is done.
Depends-On: Ic0bbeef4b431132b4f992a696dc5e83c43821b91
Change-Id: I9c92aaab89667239716d2661f798a6b9824a1cac
This is the first step to publish kolla images to tarballs.o.o. For
now, we are using the experimental pipeline, until we can validate
uploads work as expected. Then we'll move the the jobs to the post
pipeline.
Change-Id: I7d0c20a7ea221117ec6a6b0ade3e9e317c13bbd2
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We are struggling to get mirrors added to openstack-infra because
I believe it is a manual proces and reviews are stalling out waiting
on internal openstack-infra training. I realize its the end of the
cycle and everyone in openstack-infra is working 18x7, so rather then
fix this properly now (with mirrors which solves the problem) I'd like
to extend our gate timer to 90 minutes for our centos jobs which are
much slower becaue of a lack of mirrors for our dependent software
packages.
Removing bindep install helped 2 minutes - but that isn't enough. The
reason this has reared its ugly head is likely because we have twice
as many containers in Newton as we had in Mitaka - which puts the build
time at 45 minutes in the gate.
I will follow this up with a stacked review of mirroring for our
repos in Kolla for CentOS (where the gates are failing) in early
Ocata.
Change-Id: If4fbbd84e49e851f3ddac48595a8d8b72ff69db5
* clean up the Kolla jobs
* add the node type to the job name
* add -nv for the none-voting jobs
* enable the ubuntu-xenial on the master branch
Change-Id: Id8e43d7e36f0ecf691fb1abe802823ca6ab84091
This is related to zuulv25, we make sure console-log is the last
publisher to ensure zuul-launcher logs are also uploaded.
Change-Id: I1616b97fad2b97fb9e977ec14543aab4bc673b58
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
I'm not realy sure what the problem is with our gate timing out,
but 90 minutes doesn't fix the problem. It just prolongs the delay
for gating when the gate fails in some mysterious way.
Thanks for working with us while we sort out our gating situation.
This reverts commit 01155c491b.
Change-Id: I45ff5b59697d949e2c9ea8412442b4649486b2ea
This is part of our effort to migrate from devstack-centos7 to
centos-7 dibs.
Change-Id: I2ff5fbb315832b8cf07c93d0dd0cdaacd958631e
Depends-On: Ie30c8437b310d1ea5bc75307375c255131dfe934
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We cannot update the current template without affecting kolla jobs, so
create an experimental one we can use for testing purposes.
Change-Id: Id24226bd88e39ccb0f67b4750504e81b0527f450
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Our centos gates are often timing out because we do not
have a local mirror of centos in the infrastructure. While
I'd like to fix that problem (to match the Ubuntu mirrors we
do have) our gating fails at about 90% complete most of the
time on centos deployment. The problem isn't memory related
but related to the booting of the 1 VM we do (virt on virt).
if 90 minutes isn't acceptable, we could try 75 minutes. We
are pretty close to having a full centos deploy in the gate
complete. See:
http://logs.openstack.org/71/286771/1/check/gate-kolla-dsvm-deploy-centos-source/f552bf8/console.html
Change-Id: Icc416f08d6b27cda58a44dd5b9d93a3d6262a00a
Due to the length of the job name and the tox target we run into an
uncommon limitation; the virtualenv that tox launches is nested in a
path that is too long. This leads to an error on our longest named
job which prevents tox from running properly.
This limitation is the limit for the line length of the first line
in a shell script. In this case <virtualenv>/bin/pip has a path that
exceeds that limit by a few characters. This results in a very
confusing "OSError: [Errno 2] No such file or directory" error.
See `man execve` for more info. A quote from that manpage:
'A maximum line length of 127 characters is allowed for the first line
in a #! executable shell script.'
To reproduce this locally you can just create a very long path and
then run tox in it. Example: http://paste.openstack.org/show/478559/
Change-Id: I51f8f67d9ab139f1d539fce05ce3cc4766aeedff
Depends-On: I43fba2a5ff1890d699045496c9eaee5e849f3e75
clarkb suggested dsvm be in the job names for Kolla to follow the
job naming convention. In the case of the experimental job we have
recently merged, that it as a requirement for the name of the job.
To keep consistency within the project and within the OpenStack
community we are renaming all of the gates that use the dsvm's to the
convention that all other jobnames follow.
Since the tox paramaters do not change in the jobs/kolla.yaml, this
requires no change on the Kolla side.
Change-Id: I9a22434d99b4deb7936ecaf7b714d0dc41666367
This adds three additional gates. These new gates will deploy and test
OpenStack. In contrast to the existing gates, these new gates will
only build a subset of the images Kolla provides. It will only build
those needed to deploy OpenStack.
Change-Id: I2c94eac72c61385fe6472a95ea3c9cafa33bc22d
Depends-On: I7a24a704023cc8f19d42844636e9cb512060f693
Remove the kolla-functional-f21 job because it only runs the
removed-from-kolla bash build scripts. The build gating is now
handled by other jobs which run a Python based build tool. As a
result, evreything related to the kolla-functoinal-f21 job can be
removed.
As stands now this check will always fail because the bash buid
scripts are completely removed from the kolla repo and this check
job depends on these removed bash build scripts.
Change-Id: I4de8fb52c6a32b6999f6be4112fa0b76e9f0ac42
This builds centos in binary mode. The tox job uses Kolla's
next generation build script which is not compatible with the
initial version of our build script. As a result of this we
left the existing job intact so we maintain gating of our
image building process with our initial build script tool.
As part of the new gating with the new build, we are also gating an
entirely new Dockerfile infrastructure which is templated. These
new dockerfiles are not compatible with the legacy build script. As
a result we need the existing kolla-functional-f21 to remain intact
until we finish the build and dockerfile templated implementations.
After that we intend to remove the kolla-function-f21 gate job,
the legacy build script and the legacy dockerfiles.
Co-Authored-By: Steven Dake <stdake@cisco.com>
Change-Id: I2a3d42d34a32a2f514b8c64f17ff54e7b08d6d7b
Apparently the gate needs more work in Kolla to be stable. I
have filed a Critial bug here after which we should be able to
revert this revert ;)
https://bugs.launchpad.net/kolla/+bug/1466677
This reverts commit 6d3e0cac21.
Change-Id: Ie7f7b4b5db019ea6298fe103aff8c26eedcfa214
Because of a recent change, docker image building cannot be done
as the root user. This is a good thing ;) But breaks the current
gate.
Also we have a setup_docker.sh script already in our tools directory,
so rename the script we are running to setup_gate.sh because that is
actually what the script does.
Change-Id: I8f86616fb19b4b03f34fe40acf1689f6b174ff98
We believe the Kolla build will take between 1 and 3 hours. We
are not certain exactly how long it will take, so we plan to
start with 90 minutes and later tune the timer to match our
average run-times + 20%.
Note our gate functional job builds about 40 docker images which
takes quite awhile.
Change-Id: Ifd052464cacbe4ce4d0584277aa729ab151d48fe
Otherwise, the script will fail.
Also add for tox invocation until potential group issue is sorted out
(group membership applied to current running shell).
Change-Id: Ic84d5435761deae0d28909c813cb1d87540fbb21
The setup_docker script downloads and starts the Docker binary while
the tox run performs the actual test of verifying all the Dockerfiles
in the project successfully builds or fails as expected.
Change-Id: I4d05c32a2faa30f0b15487dfd605089f61baf985
Monty Taylor