The opendev project has been moving away from puppet and this is one of
the puppet modules that is no longer used. To simplify things for us we
are taking the extra step of retiring this repo.
Change-Id: I46ccab87f063c51dd3236757dbb0567738803fae
With the ARA 1.0.0 release, it has been rearchitected in ways which
require more extensive changes to the way it's deployed. For now,
pin it to use the previous version.
Change-Id: Iadfc699e6a4ce1de1a90bf4accb916454b788df0
keyring 19.0.0 only supports python3. While we should obviously
get things updated, we're going to need to unwind some things
to do that.
Change-Id: I2ae4fe3f43e5e0e606a58f58a4f6e2bafd3a3930
Our CI jobs' logserver wasn't setting cache headers so browsers were
always downloading the log files on each request. Reduce load on the
logserver and improve performance by users via the setting of a 2 week
ttl on log data.
This should be safe because each job writes to a uuid identified dir and
that data doesn't get overwritten. Once written the data is static.
Change-Id: Ic99df4e4e64a6bf64e68e6f1bc297c91134ab716
The openstack infra team is moving the managmeent of clouds.yaml files
from puppet to ansible. To keep puppet and ansible from fighting with
each other over the contents of this file make the contents optional
here. If provided puppet will manage them otherwise puppet doesn't touch
the clouds.yaml file.
Change-Id: I037abf2fb64a63af160ea6e25ce300197f7f513f
install_nodepool_builder is a variable and needs a $ to get an expected
result. Right now the logical condition always evaluates to true which
is wrong.
Change-Id: I6a94865cf356a4a8802650fe7b76e56204b351ee
ARA running as a WSGI application highlighted the fact that we don't
currently fork succifient processes to handle the different calls.
In our specific context, logs.o.o's mod_wsgi is always busy because
logstash is hitting it. The default settings with the increased load
from ARA aren't able to keep up.
These values are being inspired by the puppet-openstacklib work for
their WSGI implementation [1].
[1]: http://git.openstack.org/cgit/openstack/puppet-openstacklib/tree/manifests/wsgi/apache.pp
Change-Id: If3e338f412771c178f064602081256b6abeffc28
ARA is going to bundle a WSGI middleware that allows an operator
to route an URL (by default ^.*/ara-report) to the ARA web application.
This URL is expected to contain an 'ansible.sqlite' database that
would have been uploaded as part of log collection.
There are no static files generated as part of this process. The
web application reads directly from the database. It is meant
to avoid having to generate a static report for each job which does
not scale very well for the amount of jobs we are running.
For more details, see this code review in ARA [1].
[1]: https://review.openstack.org/#/c/511992/
Change-Id: I3b10c93b4902a9b45e23c227863e472697f662ef
$::project_config::nodepool_config_file_zuulv3 is set as the fqdn of
the host. So far it is only used for nodepool-launchers.
We wish to have the ability to create custom configuration files for
nodepool-builders too. Add the fqdn variable as the first match for
the builder config. If no more specific config, the default should
remain.
Change-Id: Ibdd8bca3f1e0a1f6709be29c37fe9a696a7bb2d4
The dependent change adds a flag to the builder for exporting build
logs, and another flag to the launcher to enable the webapp. Plumb
through so we can enable both.
Also remove deprecated build/upload log arguments that are no longer
relevant.
Change-Id: I1644e41629507b1a7082e70afaba1baf63d620a4
Depends-On: https://review.openstack.org/543671
This arugment doesn't do anything in ::nodepool::builder, and is being
removed.
Depends-On: https://review.openstack.org/543669
Change-Id: I3eaa8ae0cbb3ed039726d3ca47dc81332539f889
Allow a user to toggle zuulv3 settings for nodepool.
Change-Id: Ifdcfadca1a56a48b958cd612fba67bdc9d8d3653
Depends-On: I7fff2565f9e77d737c247f204925f66a0e4aabe8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Both v2 and v3 deployments need to be supported for a bit after zuulv3
feature is merged. The architectures are different, with some parts
missing (jenkins), while new parts being added (zuul-exector). It is
logical to split the configuration into two using zuulv2 flag.
Setting zuulv2 to true would use the old configuration. The choice to
use zuulv2 = true instead of zuulv3 = false is made so that it would be
easier to remove the flag once v2 support is removed.
The support for v3 is not added with this patch. It merely allows for
feature/zuulv3 to be merged to master without forcing users of
puppet-openstackci to pickup the new version.
For zuulv2 using zuul@2.6.0 and nodepool@0.5.0 which represent the
latest master state as of this patch. The default used to be 'master'
for both in single_node_ci class, so this should minimize unintended
consequences for those who used the class with defaults.
Change-Id: I2e7cd8d1c8e662b7e21fa80d4fa62846e2d44a70
Story: #2001367
Task: #5951
- In vhosts templates, iterates each key/value from the hash
by using the proper Ruby syntax: https://ruby-doc.org/core-2.2.0/Hash.html
- In logserver.pp, default readmes to an empty hash instead of an empty
array.
Change-Id: Ib1c70cfd9254bb4da9b6f7477baa48918de16cc2
Set the legacy parameter to default false, now that the
openstack_project::static class provides the same resources. This
way the parameter can be removed gracefully in a subsequent change.
Change-Id: I4656a511aa21b1096e60155e5f7de1ae089a95ae
Depends-On: I354639545c9cbc12f1b844946962946aa2bb4fb0
Add legacy parameter for logserver class, to install logserver resources
specific to OpenStack Infra only when needed (true by default for
backward compatibility).
Change-Id: I65e8ecf39130377c088c053e5623212f7db60f7a
Allow our nodepool installations to be installed under python 3.
Goal is to eventually remove py2 support from nodepool for zuulv3.
Change-Id: Icac77f88e411a3ac08fc54afb4e9746039f908e8
Depends-On: Ie2c223d74b32798e5f0cb0ca8b76d9fa6a4f3095
The old cronjob to delete Elasticsearch indices has been unused and
set absent for a while. Recent changes in Puppet seem to have
started disliking the literal \% sequences in the string, so just go
ahead and remove it.
Change-Id: Ic81ac00935bc3f79f17371ef13ad4d1686ffb028
Puppet is currently failing to deploy on static.openstack.org due to
keyring being unable to build. It depends on the crypography package,
which requires libssl-dev to build.
Change-Id: I9d6451eeecd317d415d08a6bb7ea593a500a3d8a
puppet-lint finally released a new major version that enforces the
not-that-new-anymore puppet style guide. This patch updates the
parameter lists to avoid the "optional parameter listed before required
parameter" lint warnings that are causing the lint jobs to fail.
Change-Id: I3cd3275958e5bdccb353e2a351da5aa9cfca0481
Passing Java arguments to Jenkins puppet allows to tune Jenkins
master Java. This is useful in order to set various Java attributes
like Java heap size virtual memory consumption.
This is most useful for downstream projects.
This patch also enables to workaround Jenkins SECURITY-170 by
setting java arguments through hieradata.
https://issues.jenkins-ci.org/browse/JENKINS-34885
Change-Id: I745a23b0f177f868d5406639bc8c42e3acacea72
Depends-On: Ice0e4ffa2fffd041d8bcc4a0b323ffae7ba2b0c8
The puppet-openstackci beaker tests have been failing for all the puppet
modules since they began running on Ubuntu Xenial nodes. This was
because there was no Xenial-specific nodeset file for beaker to run and
it was simply failing upon being unable to find the file.
This patch adds the ubuntu-xenial nodeset. Further, it forgoes the
install_puppet function because it attempts to install puppet 3 from the
puppetlabs apt repo, which does not support Xenial. Because we're
no longer using the install_puppet function, we also need to set default
values for the host object so that we can access things like
host['distmoduledir']. Finally, we add the libffi-dev package since the
keyring pip package requires it to build and it does not seem to be in
our Xenial images.
Change-Id: I8cc44e78116d6a7c8406e6c03756acb63c70834f
As part of our zuulv3 effort, we want to start testing statsd from
nodepool-launcher. Expose the ability to prefix our requests, as not
to affect production data.
Change-Id: I943df7789cfa6c69f5c5d360c9b80af6328ca878
Depends-On: I8bef6aae5a566ff3a34e816ece14f9cfc1ef289d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
It turns out we don't need a public key, since we don't build images
on nodepool-launcher.
Depends-On: I1318805b466be9416f162280766da14b08998df5
Change-Id: I4e55bce6332e4b4ee6cf5b0b67637b478ee5ff21
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We need nl01.o.o to use nodepoolv3.yaml, while nodepool.o.o still
exists.
Change-Id: Id2b61072d07eae59b36abe69eabb28d892c043ff
Depends-On: Ic04d3f625e4f0b05317ea6348b6d02e730bd61b3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We need to ability to SSH into the nodes we launch.
Change-Id: I148af66c52d515c030c3618b7d276febe97e5b57
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This is part of our feature/zuulv3 effort to scale up our
nodepool-launcher process.
Change-Id: I29e661e38e9a2a56a01a1c30f1dc1eae97a4de7b
Depends-On: I393fa1d8ff080260af772a2f020cca9b9e49b173
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
update the default gerrit_ssh_host_key to the current value that
review.o.o is using. otherwise, zuul-merger will fail to clone
git repos from gerrit because correct review.o.o host key is not
in zuul user's known_hosts.
Change-Id: I0e2ec0e340d63666de2ce0ed72c1043be0ed0662
Since we use DIB elements to create our zuul user, we also need to
have our SSH public key on disk. Other wise, diskimage builds will
fail.
Change-Id: Ic80f2337d42ea228496b4dafcf32303fbc2ef3e5
Depends-On: I6879d095941fe76d151d3bd9e590b1f691c146e2
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
These are required by nodepool::init. They are not needed for the
zuulv3 builder, but go ahead and add them optionally so this works
with the v2 builder for now.
Change-Id: I398b5ba1fdae1844ef8ab3777068d1b96e1270ac
This creates a server that is solely a nodepool builder. For a
combined builder/launcher, use the nodepool manifest.
Change-Id: I6d2a5f2fe3d96857d021b57611a526a7c79e0529
Depends-On: Ia58d7d038e99df2efa18bdaa6a8440f71e036906
We need to start decoupling nodepool more as we plan to start scaling
up nodepool-builder servers.
Change-Id: Ia58d7d038e99df2efa18bdaa6a8440f71e036906
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Allow pinning of Jenkins version to the last known working
with other Third-Party CI services.
Depends-On: I054b27b045e97c39f0706d110c05459e6b9d74a1
Change-Id: Id03c846c5ddb92a221e95533eeb27bb5e598fda4
Allow nodepool to launch 3 process, instead of 1. This is to overcome
capacity issues with a single process.
Change-Id: Ibe41ce3c488da9cd68d8833b61fd42682f0b4e73
Depends-On: Iaee3148d6710f796e9f1ea7cba2962cfbb530fa2
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Clark Boylan