Right now we are not using any constraints for docs and releasenotes builds.
This has resulted in docs job failures once Sphinx 7.2.0 has been released.
The patch will ensure that constraints are used an we should not face
simmilar issue again.
TOX_CONSTRAINTS_FILE is updated by Release bot once new branch is created,
so it should always track relevant constraints.
Some extra syntax-related changes can apply, since patch is being passed
through ConfigParser, that does not preserve comments and align indenting.
Change-Id: Ic65b9abc930e151f06d69fb9474a813cb504d7ed
Glance-registry service has been removed in V cycle with [1]
We do all necessary cleanup to fully remove service deployment.
[1] https://review.opendev.org/738671/
Change-Id: I0b2e2e39040fd0daef04724f94a39f2d11e4d105
This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove python 2.7 stanza from setup.py
- Remove obsolete sections from setup.cfg
- Cleanup doc/source/conf.py to remove now obsolete content.
Change-Id: Ic2b03b0ac599dce3a84e589d5808bd989c2a453e
1. Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.
2. Remove the unnecessary blank line for tox.ini
[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045
Depends-On: Ie837d376bb1a1d93367369d25140dd8a9d5ac14e
Change-Id: I1f7337188c99433e8c8560dc174824f5016d93cb
This patch adds a `pdf-docs` tox target that will build
PDF versions of our docs. As per the Train community goal:
https://governance.openstack.org/tc/goals/selected/train/pdf-doc-generation.html
Add sphinxcontrib-svg2pdfconverter to doc/requirements.txt
to convert our SVGs.
Change-Id: I2efe015a695bc45efe1ae63f62dd55329aa94c31
Story: 2006105
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I8cceae74275c7d6a5dda17d42614dc6478f10299
Closes-Bug: #1801657
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I4b24bf9ddd1c009d23eb601de09be1fd0c2bda6f
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Now that we're using the general templates, we can slim down
the role test definitions. We can also remove tests which are
not being watched, or which are fundamentally broken. With
this we can also remove unused scripts/plays.
We do the following:
1. We remove the 'ssl' job, given that the person working on
that is no longer doing so, and no-one else has picked up
the work.
2. We remove the 'upgrade' job, given that it's been broken
all cycle due to the way the job executes and we need to
regroup to figure out another way to do it.
3. We promote the distro_install jobs, given that they are
all now working.
Change-Id: I9d33daeb6d591c6919f131aafd62c4a33e4ff68f
The TESTING_BRANCH environment variable is provided by the
run_tests.sh script and is derived from the .gitreview file.
This ensures that once the master branch becomes a stable
branch, the constraints from the stable branch in the
integrated repository will automatically get used once the
.gitreview file is updated.
To ensure that the required environment variables are present
we export them appropriately in run_tests.sh and modify the
tox configuration to pass them into the tox test.
Change-Id: I24046adaa8a4d1da9c29496422d61ce7e2de1ccf
Needed-By: https://review.openstack.org/579371
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I24bb866f53bc48d65c65a2251d50502c386a8ed0
Using tox for requirements management requires in-repo
requirements files for all our repositories. Rather than
do that, we make use of the tests repo to capture our
common requirements and use this to install them.
This reduces our review requirement rate and simplifies
maintenance for us for the tox config. It also makes it
usable with 'Depends-On', which is marvellous!
The tox requirements definitions for docs/releasenotes
builds are left in-place as those are standard entries
across the community. If that changes at some point, we
can re-assess those entries too.
Depends-On: https://review.openstack.org/579208
Change-Id: If570f3f6b423e83dc51e0b089e0e5b45a83272d2
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones.
Change-Id: I026a440b6a0fda43b613e30f359b2a23c3c1151f
Depends-On: I5a78e2120e596d36629b4ba978b2b5df76b149b0
Implements: blueprint openstack-distribution-packages
Now that run_tests.sh handles the tests repo clone, we can
remove the use of the older tests-repo-clone.sh script.
Change-Id: I045bd91dd77c911b1b77b152b6feaa318adbe0ce
In order to allow the use of the environment variable which informs
Ansible which user executed the playbook, we pass the USER env var
into the environment that tox builds.
Change-Id: I23fb5da80312458c22de461f1d5b5d2a07332a6f
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
When 'glance_galera_use_ssl' is True, use an encrypted connection to
the database using either a self-signed or user-provided CA certificate.
A new non-voting test has been added to verify that the role remains
functional when enabling SSL features.
Partial-Bug: 1667789
Change-Id: I18e9d47e88e61ff287e5120dead49b02cdf1f8ac
Depends-On: I95cc994df5118fce7ce588fc0bff979bc283a6f3
Systemd has the ability to manage mounts and ensure functionality
/ resource management. Using a systemd mount has the benifit of not
requiring writes to the legacy fstab file which can impact OS
functionality especially when deploying on baremetal. This change
moves the glance NFS mount to a systemd unit file allowing systemd
to manage it independently with no potentially breaking impact to
the underlying operating system.
Changes:
- This PR corrects a long standing issue when using Glance+NFS where
initial deployment would work but if the playbooks were run again
it would fail due to the glance images location being an NFS mount
point with a potentially different UID/GID. To correct this we stat
the directory and if it does NOT exist it is created.
- Following the nova pattern options have been provided to set the UID
and GID of the glance user.
- To ensure out NFS backend solution works with the installation of
glance a test has been added to deploy glance using an NFS backend.
- An upgrade task has been added to this commit to clean up legacy
mounts, This task should be removed in R.
Change-Id: I716c9fe35391629532e67e212d45ea27a5422d1b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
If a deployer sets the boolean vars to 'yes', 'no' or
other equivalent booleans valid for ansible, the glance
services will currently use those in the .conf files
and it will break.
This patch ensures that we cast the output into the
.conf files properly so that regardless of the input,
the output is correct.
The tests are adjusted to deliberately use one of the
Ansible boolean values instead of a python boolean to
validate that this works.
Change-Id: I267b97da618bbda05140d2a0332798fc77db06a5
Rather than the previous basic functional tests, this
implements the use of tempest to exercise the API's
properly to validate that they're working properly.
Included is the normalisation of the way the upgrade
test is implemented to make it exactly in line with
the way the original API upgrade test was implemented
in the keystone repository. This keeps it consistent.
Change-Id: Id491d018beb30bce748240b50e9d66dcc1a249fa
When using the glance v2 API the glance-registry service is
optional, and the intention is to remove the glance-registry
service in the S cycle.
The glance v1 API is scheduled to be removed in Queens.
This patch therefore disables the v1 API by default to give
us as much time as possible to identify the impact of that
and to get the issues resolved before it is removed from
the code-base.
The patch also cleans up the glance-registry init files
to handle the transition in an existing environment.
Tests are added to validate that enabling the v1 API still
works, and enabling the v2 registry still works.
Change-Id: I4c27aa0ca5b649e4fa76cfd0f326d80f50074db1
The glance v1 API is deprecated and intended to be removed
from the glance code within the Queens or Rocky cycles.
When using the glance v2 API the glance-registry service is
optional, and the intention is to remove the glance-registry
service in the S cycle. The glance-registry service is required
when using the v1 API though.
Furthermore, when using the glance-registry service it is not
possible to execute a rolling upgrade without losing API
transactions.
Given the above information, this patch enables the deployment
of glance with only the v2 API enabled, and without the
glance-registry service. It adds a per-commit test to validate
that this configuration works.
This patch also corrects a previous misconfiguration which
enabled the v2 registry service, but did not set the data_api
correctly for the API service to inform it that the registry
was operating.
The glance_enable_v1_registry variable is also removed as it
is meaningless. The v1 API *requires* the registry to be
enabled, so we just enable it if glance_enable_v1_api is
enabled.
Change-Id: Ie95daed286798d139f0a35ffdd2a4dd1cdda6ff9
Currently the role tests use whatever versions of pip,
setuptools and wheel are already installed on the host.
When a version of these tools changes it often causes
problems for our testing.
This will ensure that we use a known good set of pins
which is maintained in the general SHA bumping process.
Change-Id: I8b9366e75e97232409d6e1204f517243794dd975
When executing the tests repo clone in OpenStack-CI,
use zuul-cloner instead of git to enable cross-repo
testing. This ensures that if a dependent patch from
the tests repo is noted using 'Depends-On: <change-id>'
in the commit message, that patch will be included.
Change-Id: I1eef3d34dd96250df7817e330dd3e654037bd692
Depends-On: Idce7abebf32f24c356a27e099fbca954d917402b
This will separate out the runs so that variables and modules are not
re-used across runs - ensuring the latest modules and versions are used,
and ensuring multiple versions of the tests playbook aren't required on
non-upgrade runs.
Change-Id: Ia79bee91950a841c4ee1fd17fe20d1f7ee994166
To avoid requiring all the os_previous_glance tasks to be evaluated, we
should split the 2 tests up. This will help make the gate more efficient
and avoids unnecessary issues.
Change-Id: Iae43ff48a6656e9fff0952d6d388e6699461b336
Ansible 2.2.1.0 requires included vars files to exist even if the
task/play that includes the file is never run (based on a conditional).
This patch fixes that by ensuring the "upgrade" previous version of
test-vars is fetched for every test.
Change-Id: Id2b0c1409c7ef676238e2879167407041c6f1d38
Implement upgrade testing for os_glance role.
This patch adds an upgrade testenv, which configures a previous version
of glance, and the test-vars from the stable/newton branch to be used by
the previous version of glance.
We don't run functional tests after the initial stable/newton deploy
because a working stable/newton branch is already tested, and it's
assumed it worked if the upgrade works too - which is more efficient.
This follows the pattern setup for Keystone in:
https://review.openstack.org/#/c/384269/
Change-Id: I8b2418ffc9197814b0eb85da647916323f9cefd5
Implements: blueprint upgrade-testing
With the implementation of https://review.openstack.org/388087 all
tox targets may now use upper constraints.
Change-Id: I3876ef912b72b8288db715bb6db6d5ef5f511414
This patch consumes the centralised Ansible test scripts
implemented in https://review.openstack.org/381853
Change-Id: I02fa50b9ab5460b37d5d28d50836b0392d86efe4
With https://review.openstack.org/381479 merged, setting this
variable in tox.ini via an extra var is unnecessary.
Change-Id: I71578985c65bc35fdf84d98c22cd0ed373fdf0a6
This patch consumes the test scripts implemented by
https://review.openstack.org/375061 to ensure that
the tests and test preparation is consistent and
more maintainable.
Change-Id: Ie2938a023310885a91907707880ed53f139536cf
The volume of logs we collect in CI jobs are extensive. This
patch ensures that the logs collected are compressed to reduce
the space taken in CI systems.
PYTHONUNBUFFERED is also set to ensure that the console log
from the CI jobs is recorded in the exact order of execution.
Change-Id: I2ef51aa6ab35a1a8d079fe395817d9967dfada1c
Related-Bug: #1620849
This change enables log collection within the gate so that further analysis
on gate tasks can be performed post build. This is very useful when
debugging problems.
Change-Id: Ie35dbcac968fefaeab6ca847ab66240807218e1f
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
As per [1] all linting tests can now use upper-constraints. This patch
removes all instances of the install_command override relating to lint
testing which were needed to negate the use of upper-constraints.
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101474.html
Change-Id: Id8ca0b1e12887e1898f97857247042c1bc65ee99
The plugin repo needs to be cloned for ansible-lint to
understand ansible plugins. The commands currently
reside in tox.ini under the ansible section and are not
currently included. This commit fixes that error.
Change-Id: Ia97084a3002f6210430de1056bbe2bae27e68990
The 'docs' tox target executes the doc8 lint test which may result in
failures when testing documentation builds, but OpenStack-CI does not
execute that tox target.
In order to ensure that we catch all standard documentation syntax
errors and prevent them from merging, this patch includes the docs
target in the 'linters' chain of tests.
Fixes for any failures which result from executing this test are also
included in the patch.
Change-Id: Ica629c89c0e9638e96acdf9679edf24726a8e07c
Co-Authored-By: Major Hayden <major@mhtx.net>
With the implementation of https://review.openstack.org/321331 the
human_log callback plugin is now part of the plugins repo.
This patch removes the retrofitted version in tox in favor of using
the version from the plugins repo instead.
Change-Id: I64270d3f9598ce4a44254fff940a922c54dbf90b
In order to prepare for the move of detailed configuration
content from the install guide to the roles, doc8 checks
are being implemented in the roles to ensure that any new
content submitted is properly checked.
Change-Id: I2104f69f7cf8833b0f1fc9ec5ce7109951b3edc2
Partially-Implements: blueprint osa-install-guide-overhaul
In order to prepare for implementing requirements management by the
OpenStack requirements management process, and to improve the
reliability and effectiveness of test execution, this patch implements
some changes to the tox configuration:
- The minimum tox version is increased in order to be able to use
constraints for the python packages.
- The OpenStack upper-constraints are used when preparing the test
venv for the linters checks.
- Any proxy environment variables set on the test host are passed
into the venv to enable testing from behind a proxy.
- The environment variables used by Ansible tests are moved into
a new venv called 'ansible' and this environment is inherited
by all Ansible-related tests.
- The docs test will clean-up an existing build directory before
executing the docs build.
- The releasenotes build cannot use upper-constraints at this point,
so it doesn't.
- The Ansible role download will no longer ignore errors so that any
problems discovered will result in a failed test.
- The human readable logging callback plugin is implemented for
functional testing.
- The ansible test requirements are moved into tox.ini to ensure
compliance for requirements.txt/test-requirements.txt for the
global-requirements management contract.
- The ~/.ansible directory as a whole is not deleted. Instead only
the plugins and roles folders are deleted to ensure that zuul's
Ansible artifacts are left in-place.
- The ansible-lint version is updated to support execution against a
folder, and the test now executes against the entire role to ensure
that it captures all applicable files for lint testing.
This is a combined port of the following:
- https://review.openstack.org/323507
- https://review.openstack.org/338193
- https://review.openstack.org/332443
- https://review.openstack.org/338193
- https://review.openstack.org/339493
Change-Id: I51af7598075995efc3d755a57daf0f82f972a4a4
Dmitriy Rabotyagov