With tox release of 4.0, some parameters were deprecated and are ignored now
which causes tox failures. One of the most spread issues we have is using
`whitelist_externals` isntead of `allowlist_externals`
Change-Id: I4967f301398621ae6e7b47b22d9a4d52037f6a3b
We are now much clearer about the setting of ansible_python_interpreter
and even though ansible is executing inside the venv it should
be deploying against the host with ansible, not the venv.
Depends-On: https://review.opendev.org/735289
Change-Id: I583f2236acbc02b33e6fc192faa8cee62cd10b89
This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove setup.* files, these are not needed for this repo
- Cleanup */source/conf.py to remove now obsolete content.
- Remove install_command from tox.ini, the default is fine,
move constraints into deps, cleanup a bit
- Enable warnings for docs building
- Correct ansible search/match tests for ansible 2.9
Depends-On: https://review.opendev.org/726645
Change-Id: Ia5aaa8f41172fe200d1d7ce0c7b6f26f7834e38a
1. Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.
2. Remove the unnecessary blank line for tox.ini
[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045
Change-Id: Id99c8c66f156dadc9912eaf3ddcb28413094a359
This patch adds a `pdf-docs` tox target that will build
PDF versions of our docs. As per the Train community goal:
https://governance.openstack.org/tc/goals/selected/train/pdf-doc-generation.html
Add sphinxcontrib-svg2pdfconverter to doc/requirements.txt
to convert our SVGs.
Change-Id: Ie582a8d9d1a9fcd5a9766c351c9edb58ade47fd0
Story: 2006105
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: Ia3f91544b5f60592c44a2970e983df8bbe829e3a
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I0af955e386a4050a49a35f411094e24a11f66ca8
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
The TESTING_BRANCH environment variable is provided by the
run_tests.sh script and is derived from the .gitreview file.
This ensures that once the master branch becomes a stable
branch, the constraints from the stable branch in the
integrated repository will automatically get used once the
.gitreview file is updated.
To ensure that the required environment variables are present
we export them appropriately in run_tests.sh and modify the
tox configuration to pass them into the tox test.
Change-Id: I096778341f2022de5604e9624eff23a6e4c5f9a8
Needed-By: https://review.openstack.org/579371
Using tox for requirements management requires in-repo
requirements files for all our repositories. Rather than
do that, we make use of the tests repo to capture our
common requirements and use this to install them.
This reduces our review requirement rate and simplifies
maintenance for us for the tox config. It also makes it
usable with 'Depends-On', which is marvellous!
The tox requirements definitions for docs/releasenotes
builds are left in-place as those are standard entries
across the community. If that changes at some point, we
can re-assess those entries too.
Depends-On: https://review.openstack.org/579208
Change-Id: I483410dc4415b880a781215b171616aecb8f6a18
Now that run_tests.sh handles the tests repo clone, we can
remove the use of the older tests-repo-clone.sh script.
Change-Id: I2da0609609e8f70671b0499f0a5911746985c917
In order to allow the use of the environment variable which informs
Ansible which user executed the playbook, we pass the USER env var
into the environment that tox builds.
Change-Id: Ic67117a2a0b20dfcad01686276d91fa66a196fe0
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Currently the role tests use whatever versions of pip,
setuptools and wheel are already installed on the host.
When a version of these tools changes it often causes
problems for our testing.
This will ensure that we use a known good set of pins
which is maintained in the general SHA bumping process.
Change-Id: Ic87ed4f9f3d60f05d3c1d9828a6b63f5c5626138
When executing the tests repo clone in OpenStack-CI,
use zuul-cloner instead of git to enable cross-repo
testing. This ensures that if a dependent patch from
the tests repo is noted using 'Depends-On: <change-id>'
in the commit message, that patch will be included.
Change-Id: I87314f8c46554a9b6856078bda61c4775e20243b
Depends-On: Idce7abebf32f24c356a27e099fbca954d917402b
With the implementation of https://review.openstack.org/388087 all
tox targets may now use upper constraints.
Change-Id: I61c36be4184c15e484007d85511ed072125f5ede
This patch consumes the centralised Ansible test scripts
implemented in https://review.openstack.org/381853
Change-Id: Ie7ad735d16af74922cbc712f59e343bba3050e61
With https://review.openstack.org/381479 merged, setting this
variable in tox.ini via an extra var is unnecessary.
Change-Id: I313a4d69ee6e165238d5b459cacda3b29f800c8a
This patch consumes the test scripts implemented by
https://review.openstack.org/375061 to ensure that
the tests and test preparation is consistent and
more maintainable.
Change-Id: I8ad59bdb2dab0e79bc73d7e4cdb63ef49cbdddba
The volume of logs we collect in CI jobs are extensive. This
patch ensures that the logs collected are compressed to reduce
the space taken in CI systems.
PYTHONUNBUFFERED is also set to ensure that the console log
from the CI jobs is recorded in the exact order of execution.
Change-Id: Ied2a2f817a88a1326d446fe77a0f666a63a6e3d2
Related-Bug: #1620849
This change enables log collection within the gate so that further analysis
on gate tasks can be performed post build. This is very useful when
debugging problems.
Change-Id: Ibf510274bc065d5db37781601941e2a2f5d833a5
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
As per [1] all linting tests can now use upper-constraints. This patch
removes all instances of the install_command override relating to lint
testing which were needed to negate the use of upper-constraints.
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101474.html
Change-Id: Id2996bfb7c7b2b34714ec56bf264afc759f1060b
The plugin repo needs to be cloned for ansible-lint to
understand ansible plugins. The commands currently
reside in tox.ini under the ansible section and are not
currently included. This commit fixes that error.
Change-Id: If46565f2e58de9e3d228377c5b10e6d4bf4b3aa1
The 'docs' tox target executes the doc8 lint test which may result in
failures when testing documentation builds, but OpenStack-CI does not
execute that tox target.
In order to ensure that we catch all standard documentation syntax
errors and prevent them from merging, this patch includes the docs
target in the 'linters' chain of tests.
Fixes for any failures which result from executing this test are also
included in the patch.
Change-Id: Ifb66e109667ae14f56f83a91301b0595319a24ca
With the implementation of https://review.openstack.org/321331 the
human_log callback plugin is now part of the plugins repo.
This patch removes the retrofitted version in tox in favor of using
the version from the plugins repo instead.
Change-Id: I8e20ac8ffc64265057e2c52e8e162943c9ba8038
In order to prepare for the move of detailed configuration
content from the install guide to the roles, doc8 checks
are being implemented in the roles to ensure that any new
content submitted is properly checked.
Change-Id: Ibeed486250a1a6d0323d8469d6432b7e3575cb6b
Partially-Implements: blueprint osa-install-guide-overhaul
In order to prepare for implementing requirements management by the
OpenStack requirements management process, and to improve the
reliability and effectiveness of test execution, this patch implements
some changes to the tox configuration:
- The minimum tox version is increased in order to be able to use
constraints for the python packages.
- The OpenStack upper-constraints are used when preparing the test
venv for the linters checks.
- Any proxy environment variables set on the test host are passed
into the venv to enable testing from behind a proxy.
- The environment variables used by Ansible tests are moved into
a new venv called 'ansible' and this environment is inherited
by all Ansible-related tests.
- The docs test will clean-up an existing build directory before
executing the docs build.
- The releasenotes build cannot use upper-constraints at this point,
so it doesn't.
- The Ansible role download will no longer ignore errors so that any
problems discovered will result in a failed test.
- The human readable logging callback plugin is implemented for
functional testing.
- The ansible test requirements are moved into tox.ini to ensure
compliance for requirements.txt/test-requirements.txt for the
global-requirements management contract.
- The ~/.ansible directory as a whole is not deleted. Instead only
the plugins and roles folders are deleted to ensure that zuul's
Ansible artifacts are left in-place.
- The ansible-lint version is updated to support execution against a
folder, and the test now executes against the entire role to ensure
that it captures all applicable files for lint testing.
This is a combined port of the following:
- https://review.openstack.org/323507
- https://review.openstack.org/338193
- https://review.openstack.org/332443
- https://review.openstack.org/338193
- https://review.openstack.org/339493
Change-Id: I6e90dab524c3e6ae34af26dcbd49f3d669da3a65
Currently the container cache preparation process uses a pre-prepared
LXC base image which includes its own package repository configuration.
This presents a few problems:
- The first packages installed will make use of the base image's
package repo configuration, resulting in a bypass of local mirrors
to install the first set of packages.
- A set of vars need to be set in order to have the containers use a
local mirror, otherwise it'll use the mirrors set in the role's vars
files. This is counterintuitive.
Another problem introduced by I95c210c83ca968d11ba6f6a36b634bb798fa291f
as a result of the package repository vars moving from the role defaults
to the vars files is that the precedence has changed. The change in
precedence means that a task which sets a fact can't be used to override
the defaults set in the vars file. This method is used in all the role
tests to ensure that the OpenStack-CI repositories can be discovered from
the host and then used.
This patch changes the image cache preparation process to ensure that
the container package repository configuration matches the host
configuration. This is simpler and more intuitive.
Additionally the copy task from the deployment host into the container
cache is set to assume the same destination in the container as the
source (to reduce configuration verbosity), appropriately sets the
leading '0' for the mode (to prevent unexpected surprises), and
appropriately quotes the variable (to ensure forward compatibility
with Ansible 2.0).
Finally, the use of lxc_container_caches in the test configuration
has been removed as it is no longer used.
Change-Id: I420382fd3bbbb5fcae90ae0c6160233202a1a51a
This is a common pattern in many other independent role
repositories and should be consistent across all of them.
Change-Id: I8ca656ceebf04aade2b3e9a677cf2ccd2c52f388
The change moves several tasks around and adds packages to the install
process which were previously assumed to be present on the base OS.
This also updates the lxc-net-bridge template to be more configurable
to address issues where the base OS is more minimal than previously
expected.
Change-Id: I3c8225124a5f18db81259e1d52d0168ef52c3c17
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch updates the tox.ini the same bashate exceptions as are
currently in the OpenStack-Ansible playbook repo.
It also ensures that the linters and all lint targets work
appropriately and normalises the tox.ini configuration to use
uniform formatting.
The use of ansible.cfg is removed as there is no way of being
certain which paths can be used without reverting to an ugly
sed hack in the commands. This is why it is preferred to make
use of environment variables which make use of tox's default
substitutions instead. It's a more reliable way of achieving
the goal for the purpose of gating and testing.
The switch to using a git clone instead of ansible-galaxy to
download the plugins is due to the path spec not being able to
work in Ansible 2.x. [1]
[1] https://github.com/ansible/ansible/issues/13563
Change-Id: I1df29fe1f9c2ee2e5370c4dd9f733ae2c697608e
OpenStack-Infra has requested that lint tests be pipelined into the 'linters'
tox target in order to make more efficient use of nodepool resources.
This patch:
- Ensures that a complete set of lints tests is available in individual
tox targets.
- Adds a 'linters' target which executes all the other lints tests.
Change-Id: I5c844b8f9e8bdfd10dc1a9a799b0e55c45a5a42f
This brings our tox.ini files inline with other OpenStack projects.
Change-Id: Ied8a032326141df5e1a7cb0c790f6db738da1efe
Depends-On: I376e51824c46f9b5eb4a4cff01aaa176632087dd
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Instead of launching two separate build jobs in our CI infrastructure,
one for bashate and one for pep8, launch a single one. While the jobs
are short, the setup of the images takes a long time, so this is
friendlier to our CI infrastructure.
Use the pep8 environment as common lint target and merge the bashate job
into it.
This patch also renamed dev-requirements to test-requirements to bring it
into line with all other OpenStack projects.
Change-Id: I7268b76b48c95e110f141e678d8ad87e0be6b057
This patch updates the run_tests convenience script to make it
execute all test environments using tox, but also ensures that
all the correct pre-requisites are in place before doing so.
Change-Id: I7c269971f453a79787c3b9fc8d56f513bf2d91da
The change moves the role out from the main repo lxc_host
repository and into its own standalone repository.
Items within this change:
* The role has been updated to ensure it runs standalone.
* Tests added to the role within tox.
* Functional tests added to the role that can either be run
via the run_tests.sh script or using tox.
* dev requirements have been updated for testing usecases.
* Docs added to both the README.rst file as well as the docs
folder.
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Dmitriy Rabotyagov