In functional tests we don't ensure that wget exists. Also for Ubuntu
we ensure that curl is present only. So let's replace wget command with
curl and be consistent across repos.
Change-Id: If9106bd0a481079b9e676c5547e244a11a35fb73
Currently TESTING_BRANCH serving 2 purposes - it controls over OSA
branch to use and services branch.
At the same time we need to have these separated from time to time, as
OSA is trailing.
Change-Id: Ib5ca8c7d3300f4e8efd81f87612c821594bebfac
This must be python3.8 as we use this version in the integrated repo
and require it for ansible 2.12
Change-Id: Ia326d92c24b9cdfcf46011e0661a9b5081e11573
Since OSA has trailing releases, we might want to override branch we're
testing. At the moment it's set based on the .gitreview file.
At same time we temporary override branch to yoga, as deploying from
master breaks installation.
Change-Id: Iacef9bec9ab8f2f15a56e3d95f71734277d7a8c7
Ansible-lint prior to v5.3.1 has issue with reach version >=11.0.0.
In order to cherry-pick fix to stable branches we fix rich version.
We can't use requirements as rich is neither in blacklist nor in
global-requirements, so must be handled independently
This should be reverted with ansible-lint version bump.
Change-Id: I7ad62182178dd1eb285d6002b2852b457f678c7b
The intent was to be able to test easily with Depends-On patches with a
repository prepared by Zuul but ara is being retired from
opendev.org/recordsansible/ara and will no longer be available at that location.
Change-Id: I961f58d5b24aa5e11e2a4db39f2daff3c8330063
With the new pip resolver, if a package is constrainted to two
different versions simultaneously then the package cannot be installed.
setuptools has a constraint in both OSA global-requirement-pins and
openstack upper-constraints, so the installation will fail with the
new resolver.
This patch removes any references to setuptools from openstack u-c
and allows the constraint from the OSA repo to be used.
Change-Id: Ib4c3592184973ba9908a1ba17d935a96448969a1
This requires at least ansible version 2.9. A new requirements file
is added to specify collections, and the env-prep script is modified
to install the openstack collection into the default collection
location.
Change-Id: I90b7ca4ea9491c3de197cbc4df83f7f959dd9571
As CentOS 8 is not compatible with old ara, we installing new one isnted
Also providing py3 as default python for CentOS 8
Change-Id: I264e6f8bfa507668dada7b629af52a8a13001a45
Currently, we are overriding 'install_command' to use 'pip'. This is
considered poor behavior and 'python -m pip' should be used instead:
https://snarky.ca/why-you-should-use-python-m-pip/
It turns out that this is the the default value provided by tox:
https://tox.readthedocs.io/en/latest/config.html#conf-install_command
So we can remove the line and simply use the default value - after
adding constraints to deps as is best practice as well.
Mosts tests itself use "pip install" in the shell scripts which sets
also the constraints. Change that to "python -m pip install" like done
for tox.
Change-Id: I27f80d8aa4ac3688ee56e8a9234e34f4e09afe34
The docuementation requirements are only for documentation and not
for other code.
Since https://review.opendev.org/#/c/709222/ the minimum python version
for openstackdocstheme is 3.6 which breaks our legacy functional tests.
This patch stops docs/requirements.txt from being installed for
functional tests.
Change-Id: I7ef5369143ce8cfe3cde1e3df4da528449539e34
In order for Ansible to consume the env var, it needs to
be exported. Without the export, TESTING_BRANCH is not
set and test-vars.yml results in defaulting to 'master'
which is not appropriate for stable branches.
This patch is s stop-gap to unblock testing for stable/stein.
Another patch will follow which will force any tests using
this mechanism to fail if the env var is not set.
Change-Id: I1ff58205c2c0346dfe1744fdf3921b9db03c2774
The SELinux Python package has been renamed to python2-selinux
which means that Fedora jobs are currently failing.
This patch adds a conditional to be able to handle Fedora while
keeping support for CentOS.
Change-Id: I47d252c3db4d0071e90f9b1c566ab26b00635592
Currently the role tests always clone the plugins repo without
specifying a branch. The tests also always use a git clone, even
though in zuul the repo is already cloned.
This patch ensures that the branch is specified when doing the
clone, and also will just symlink to the existing zuul clone if
it is present.
Change-Id: Iaf5ee20e534a8279b5b563c820518bb176f0f4bb
When running role tests, the tox venv does not have the selinux
bindings, and they are not installable via pip. To ensure that
Ansible is able to use the file module which requires these
bindings, we symlink them into the venv.
Change-Id: Ie8426653b4843fdae0946a49c02fa11db139fd4e
It has been discussed and decided at the last two PTG's
that upgrade testing in roles is broken in the current
state and that we need to make use of the integrated
repo for upgrade tests. As such these old tools are no
longer required.
Change-Id: I46756bc0db6baa9487dffccce3e64cd5d035fd3c
Our Vagrantfile is rsync'ing the role to /vagrant. This breaks the
role test since the $WORKING_DIR no longer matches the name of the
OSA project. As such, when we are using vagrant to test the plugins
tests and ops repositories we end up using the HEAD of the repositories
instead of our local copy. We fix this by introducing a new
OSA_PROJECT_NAME and then we test against this to determine the actual
OSA project we are testing
Change-Id: Ic8b99b4f2d241144e1200ddf7c33742e9bc91c5b
Some roles need to make use of extra requirements, but
only for tests. For example, the octavia role needs to
use the octavia client for some test tasks.
Instead of duplicating the whole test requirements file
in that repo, we change the mechanism to make it additive
so that the repo test-requirements are added into the
test runtime venv.
Change-Id: If670a32e79bfb4623fa427925cdc992219742d58
Using tox for requirements management requires in-repo
requirements files for all our repositories. Rather than
do that, we make use of the tests repo to capture our
common requirements and use this to install them.
This reduces our review requirement rate and simplifies
maintenance for us for the tox config. It also makes it
usable with 'Depends-On', which is marvellous!
The tox changes in this review will need to be replicated
to all other roles. All other roles will also need their
'test-requirements.txt' file removed unless they have
something special in them which is not common.
The tox requirements definitions for docs/releasenotes
builds are left in-place as those are standard entries
across the community. If that changes at some point, we
can re-assess those entries too.
Change-Id: I7c209dc69eb893b2448a7d70cf195e55b628604b
There is absolutely no chance that zuul will be reverted
back to v2, so let's remove the extra code.
Change-Id: I4010515bd867761b0c7fca4e4d93f8320eaa608f
The ceph-ansible project has created a new library which
is held in the root folder. We need to ensure that the
library path is included.
Additionally a callback plugins directory inside plugins/callback has
been added, along with an action plugins directory inside
plugins/actions.
Finally, the sed command we use to replace the default roles directory
was causing issues in that it applied to any lines that were not
necessarily the roles_path. This PR adjusts the sed to ensure it is only
the roles_path line that is replaced.
Change-Id: I0d30d749ef2e00c1d199e53a837462619a9b2650
Currently lint tests are failing due to ARA bringing in
extra packages unconstrained. Here we ensure that the
upper constraints are applied when installing ARA.
Change-Id: I35e3688ae632d5f7747f32dafc41d23883c033c5
This change adds a common test role requirements file to the
preparation process whch will ensure all required common roles are
available prior to running tests.
As we move to use common roles throughout our deliverables this change
will help ensure we're not having to map complex common dependencies
throughout our supported roles, which is prone to human error and can
cause cascading failures throughout out stack should a requirement be
missed.
Change-Id: If3c57d5c864aee3a6bc461dead367eac7f3eede2
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The plugin path was using the ansible root however to allow the plugins
to be loaded as a dependency it needs to be in the roles path, which is
handled in the integratged gate. This change pulls our test repo in
allignment with the integrated repo.
Change-Id: I0c914e2430ee609a6093482834eb145c3894dd42
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Right now, ARA is being setup when we are running functional tests.
However, we could use it for other type of tests such as the upgrade
ones. As such, we can setup ARA by default as part of the environment
preparation steps.
Change-Id: I49052809c4970592c40a394a31a4d8eca12116e5
In order to decouple changes in the roles repositories from changes
needed in other repositories due to the ansible-lint test, the roles
repositories will skip all dependent roles when executing the lint test.
The roles can be updated in time as patches are submitted to those
repositories. In order to do that, we need to stop symlinking the
role into the same directory where its dependent roles are and place
it to its own directory so we can ignore the dependent roles in
ansible-lint.
Change-Id: I49c01fb63054e45bae5ae45a89cce986579959de
Closes-Bug: #1737310
Currently the ansible-syntax and ansible-lint tests
assume the playbook to be tested against, the role
requirements file location and a single path for all
roles. This patch allows these to be modified so that
the tests can be used for the integrated repo which
does not have the same structure as the roles.
Various lint failures are no being picked up with
these modifications, so they're resolved using quick
hacks.
Change-Id: I8bea90082dbde7de49c5e2e86d298c017b16591d
This patch implements an initial set of jobs intended to match
the current job execution method. It does not intend to improve
how the jobs are executed - only to replicate what is currently
in openstack-infra/openstack-zuul-jobs and provide the platform
to iterate on.
Change-Id: Iafe9f3622037d7ba71f9ba8b73217f9e2070f246
In order to be able to simultaneously work with the
differing implementations of zuul-cloner in zuul v2
and zuul v3 we implement a detection routine to
behave appropriately for each version.
This is very much a band-aid just to help us transition
because once the transition to zuul v3 is complete we
should be able to ditch this whole playbook for
something built into zuul v3, and use something very
simple for executing tests when not using zuul.
Change-Id: Ic824e7ca7c564d5de7a27c77a79014ad274afade
To keep the script simpler and support cross-repo
testing, the entire openstack-ansible-ops repo is
cloned.
To match the tests repo clone process, the
clonemap creation is moved into a function.
Change-Id: Ia0159aa512f50083b4c47c7fb2a92615adc480a9
Currently the plugins are git clones for the plugins
repo. This patch instruments the ability to use
zuul-cloner instead when appropriate.
Change-Id: I9fb0366b4e233d931e121ba11d9cf0b95d7d4d96
Commit cf6bea2acc ("Enable cross-repo testing for OSA role tests")
added the ability to test patchsets along with their dependencies.
However, this breaks regular testing since the 'grep ^ZUUL' will return
false in case we are not in an zuul environment. As such, fix this
command to always return true.
Fixes: cf6bea2acc ("Enable cross-repo testing for OSA role tests")
Change-Id: Id8c020264e61bd77a04b41957ee1313f693328eb
When 'Depends-On: <change-id>' is used in a patch,
Zuul cloner must be used in order to ensure that
both patches are actually used for the gate test.
This patch implements the use of zuul-cloner [1]
for gate tests and falls back to using git if
zuul-cloner is not present.
Additionally the playbook execution requires the
file based inventory to be used instead of the
CLI string so that the python interpreter is set.
This is required in order to ensure that the copy
module can act against CentOS with access to the
libselinux-python library on the host. See [2] for
more details.
[1] http://docs.openstack.org/infra/zuul/cloner.html
[2] https://goo.gl/VCKKfp
Change-Id: I8cf4d7d30284e5ef1e70897f2ecfe1da0b11c0c7
This commit mutually benefits both OSA and ARA: it allows ARA to
leverage the func jobs of OSA as a mean to test itself in it's
own gate and make sure one of it's patches does not break OSA.
It also moves the ARA callback setup from env-prep to functional
which is the only place where it makes sense to set up ARA in the
first place.
Change-Id: Ia7b46424740da94b82c57b828b53c7993fdaa76b
The callback path at /ara/callback was removed in
https://review.openstack.org/#/c/432795/ after being deprecated
since October 2016. This took effect in the ARA 0.11 release.
Update to the new location at /ara/plugins/callbacks.
Change-Id: Iab871aecfdd34c08fdab2d5aad7e41170ad72e79
The ara link happens even though the conditionals should exclude it.
This is a quick fix to allow the tasks to be run on consecutive tries
without having to take manual action.
Change-Id: I221551d6e62afe515d88fe576e4dcb99889b07bf
Following on from https://review.openstack.org/#/c/396324/
Let's set this up for the central test repo (which will filter down for
other role tests).
Change-Id: Ie592aa161890e7b3ef7e586726ce301e24964dd9
The current idempotency check is broken because Ansible
does not produce a log output without the environment
variable being exported.
This patch ensures that each playbook run produces an
individual log, and ensures that the idempotency check
uses the correct log output.
The following are also included:
- The script outputs the full CLI used to execute the test
playbook. This is useful for debugging purposes.
- The parameters are re-ordered in alphabetical order to
make them easier to find.
- The log collection now also collects the Ansible logs.
- Instead of repeating the rsync command in the log collection
a variable is used, reducing the line length and making it
easier to change/re-use.
Change-Id: Ie379de765c6ebba958ce8e7f9dc27b7a3af74ff8
This patch implements test scripts intended for use by all
OpenStack-Ansible role tests.
The intent is to simplify the role tox.ini configuration
and ensure that as many changes to role testing configuration
can be managed from the centralised tests repo instead of
individually in each repository.
This patch implements the scripts to centralise the
ansible-lint, ansible-syntax, and functional Ansible tests.
Functionality included:
- For a simple functional test, the defaults will be allow
the execution of the test without any parameters set.
- For a scenario test the scripts allow the inventory,
extra vars and any other CLI parameters for Ansible to be
set via environment variables.
- Both check mode and an idempotence test.
In addition to this functionality, the bash scripts are all
set to fail on error to ensure that tox shows a failure.
Change-Id: I23c24146485da340d4f046f80e4814652e6e3876
Added get-ansible-role-requirements.yml to a testing repo that
uses git clone to download the roles instead of downloading the
roles using ansible-galaxy.
Closes-Bug: 1626113
Change-Id: I241de348310a7a3beba6181a271f5351c9460318
This patch implements test scripts intended for use by all
OpenStack-Ansible role tests.
The intent is to simplify the role tox.ini configuration
and ensure that as many changes to role testing configuration
can be managed from the centralised tests repo instead of
individually in each repository.
The common tests repo target location when cloned is changed
from 'tests/playbooks' to 'tests/common' to better reflect
the content and purpose.
Change-Id: I1a93329ada24670c87588ce5a07f92c9bce4e50b
Dmitriy Rabotyagov