Commit Graph

56 Commits

Author SHA1 Message Date
Dmitriy Rabotyagov 737c9a51d6 Update tox.ini to work with 4.0
With tox release of 4.0, some parameters were deprecated and are ignored now
which causes tox failures. One of the most spread issues we have is using
`whitelist_externals` isntead of `allowlist_externals`


Change-Id: Ic7864bf2150acbeab504d4dce3dd1a920974e5eb
2022-12-27 17:53:26 +01:00
Jonathan Rosser d1ec190e3b Cleanup setup.py config
Change-Id: I91029458f0939f41d9d1075a0fc7829e1f66746b
2022-04-04 09:59:36 +00:00
zhoulinhui 2b4d0634d5 Replace deprecated UPPER_CONSTRAINTS_FILE variable
Change-Id: Ic9503c500aa32be4ee76d5862351b2dbf31b0d4a
2020-11-10 13:12:04 +08:00
Andreas Jaeger 295cd5e3b4 Cleanup py27 support
This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove python 2.7 stanza from setup.py
- Remove obsolete sections from setup.cfg
- Enable warnings for sphinx-build
- Cleanup doc/source/conf.py to remove now obsolete content.

Change-Id: I58849f8ace219a1d23540932c9481f50fc98225c
2020-04-13 20:16:36 +02:00
Ghanshyam Mann 81b13dcf20 [ussuri][goal] Updates for python 2.7 drop
OpenStack is dropping the py2.7 support in ussuri cycle.

Only update required is in tox to define common baspython as py3 os that
all tox env including [testenv:functional] will use py3 instead of py2.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: Ic6300763471d4a1a588b750d05d4f3dd72342704
2020-03-01 19:23:54 -06:00
Zuul 1e6cc45d97 Merge "tox: Keeping going with docs" 2019-12-12 20:56:59 +00:00
Dmitriy Rabotyagov 0832b3a665 Replace git.openstack.org with opendev.org
This patch replaces git.openstack.org with opendev.org as redirection
from old path was enabled.
Also we change upper constraints url due to [1]

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html

Change-Id: I3c897a4aac58aeb8ea641839e554c49d5eacb708
2019-12-04 18:22:58 +00:00
jacky06 9423b20d46 tox: Keeping going with docs
1. Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.
2. Remove the unnecessary blank line for tox.ini

[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045

Depends-On: Ie837d376bb1a1d93367369d25140dd8a9d5ac14e
Change-Id: Id299ea1d104bc95a49efa3c1cb5e43b0a43fcdd3
2019-10-23 17:39:15 +00:00
Dmitriy Rabotyagov 97790114dc PDF Documentation Build tox target
This patch adds a `pdf-docs` tox target that will build
PDF versions of our docs. As per the Train community goal:

  https://governance.openstack.org/tc/goals/selected/train/pdf-doc-generation.html

Add sphinxcontrib-svg2pdfconverter to doc/requirements.txt
to convert our SVGs.

Change-Id: Ic33ce7bd38461dc9b3363f70b1e95baf4e969172
Story: 2006105
2019-10-18 18:05:17 +03:00
Mohammed Naser 4de1219730 swift3: remove support
The swift3 middleware has been deprecated in replacement with the
s3api middleware instead.  This removes all the swift3 references
to hopefully enable someone to cleanly add s3api eventually.

Change-Id: I3a8a1ab861ec81b1f4f8dbc02a6a332d4ce495b8
2019-03-29 09:41:50 -04:00
huang.zhiping e965a6ddcb fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I27b758fed3cdb4e65203d0b0051244fcde72b700
Closes-Bug:  #1801657
2018-11-08 13:46:45 +00:00
Doug Hellmann 8fcf7e4c1a fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I6a2c6bbaa09b220c2b44e2a8d9be6a9a0bfa3453
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-09-26 18:44:24 -04:00
Jesse Pretorius 386cdb51e2 Clean up role tests
Now that we're using the general templates, we can slim down
the role test definitions. We can also remove tests which are
not being watched, or which are fundamentally broken. With
this we can also remove unused scripts/plays.

We do the following:

1. We remove the 'upgrade' job, given that it's been broken
   all cycle due to the way the job executes and we need to
   regroup to figure out another way to do it. The associated
   scripts are also removed.
2. We remove the 'pypy' tox target. The job was removed some
   time ago, but the tox target left behind. The associated
   inventory is also removed.
3. We remove the 'multireg' jobs. The jobs have been failing
   for some time and are very complex. If multi-region tests
   are re-implemented, they should be done in a simpler way -
   probably best using multiple nodes, rather than trying to
   fudge it on a single host with containers. The associated
   scripts and inventory items are also removed.
4. We rename the 'func_swift3' tox target to 'swift3', and
   reconfigure the job appropriately. The verbose logging for
   ansible is also removed, given that we have ARA to provide
   decent diagnostic output.

Change-Id: I411fcaac9a1ddf17996d771f13b2de7e775e4a60
2018-09-09 11:36:17 +01:00
Zuul 5597d9066d Merge "fix tox python3 overrides" 2018-08-28 12:43:06 +00:00
Jesse Pretorius fa8b4d9af9 Use the TESTING_BRANCH env var for constraints
The TESTING_BRANCH environment variable is provided by the
run_tests.sh script and is derived from the .gitreview file.

This ensures that once the master branch becomes a stable
branch, the constraints from the stable branch in the
integrated repository will automatically get used once the
.gitreview file is updated.

To ensure that the required environment variables are present
we export them appropriately in run_tests.sh and modify the
tox configuration to pass them into the tox test.

Change-Id: Id3e0021ffe0ee8008bb0c16b7703ae8a5c34f9f4
Needed-By: https://review.openstack.org/579371
2018-08-16 18:25:27 +01:00
melissaml 6f8fb9fad0 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I7abdeda5e90627776b588e196a72becfd599ebbf
2018-07-23 19:59:20 +08:00
Heba Naser e3f4ab05d1 Use tests repo for common role test requirements
Using tox for requirements management requires in-repo
requirements files for all our repositories. Rather than
do that, we make use of the tests repo to capture our
common requirements and use this to install them.

This reduces our review requirement rate and simplifies
maintenance for us for the tox config. It also makes it
usable with 'Depends-On', which is marvellous!

The tox requirements definitions for docs/releasenotes
builds are left in-place as those are standard entries
across the community. If that changes at some point, we
can re-assess those entries too.

Depends-On: https://review.openstack.org/579208
Change-Id: I963ccbb3353387adb724ee5f5630978ac9fd4f9e
2018-07-02 11:35:28 -04:00
Markos Chandras 2ae3bdb459 tox.ini: Use common vars file for distibution installations
Change-Id: I944369645135bb446b4f447aaf55c6115ba1332d
Implements: blueprint openstack-distribution-packages
2018-06-14 16:02:52 +01:00
Markos Chandras 51ea328b7c Add support for using distribution packages for OpenStack services
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones. However, functional
testing is not complete yet since it requires tempest to be in the swift
virtual environment which doesn't exist for distro installs. As such,
for functional testing to cover this method of installation it needs
to be changed quite a bit.

Change-Id: Id75e56440685df407a9991018dc07b5a75429082
Implements: blueprint openstack-distribution-packages
2018-06-04 08:41:19 +01:00
Jesse Pretorius 97a448da3e Remove tests-repo-clone.sh
Now that run_tests.sh handles the tests repo clone, we can
remove the use of the older tests-repo-clone.sh script.

Change-Id: I7d2961028c2182f3ec763c31dd0bd32badf88944
2018-03-28 10:16:20 +01:00
Jesse Pretorius 073e3c4506 tox.ini: Expose USER environment variable to execution environment
In order to allow the use of the environment variable which informs
Ansible which user executed the playbook, we pass the USER env var
into the environment that tox builds.

Change-Id: I06e1528995924ca3c59405b04db28281a572cf6d
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
2018-03-15 18:02:06 +00:00
ZhongShengping effff15606 Follow the new PTI for document build
For compliance with the Project Testing Interface as described in:
https://governance.openstack.org/tc/reference/project-testing-interface.html

For more detials information, please refer to:
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Change-Id: I934aa82f8a46989caa9eeae25cbe808f2deb722a
2018-03-09 12:01:27 +08:00
Charles Farquhar ea0d7bdb3e Add basic functional testing for swift3
This commit adds the following swift3 functional testing:

 1. Install and configure `s3cmd` utility
 2. Verify bucket creation
 3. Verify object upload
 4. Verify object listing
 5. Verify object and bucket deletion

Change-Id: I6290e8b02004f6c518983700a6439d28fbd127a4
Co-Authored-By: Jean-Philippe Evrard <jean-philippe@evrard.me>
2018-02-16 09:48:06 +00:00
Jesse Pretorius 062260a55f Ensure that role tests pin pip/setuptools/wheel
Currently the role tests use whatever versions of pip,
setuptools and wheel are already installed on the host.

When a version of these tools changes it often causes
problems for our testing.

This will ensure that we use a known good set of pins
which is maintained in the general SHA bumping process.

Change-Id: Ib62f4c84bcfe62442e94671422860b5eec1222a3
2017-06-04 11:22:59 +01:00
Jesse Pretorius 49d9c4a71a Use zuul-cloner for tests repo in OpenStack-CI
When executing the tests repo clone in OpenStack-CI,
use zuul-cloner instead of git to enable cross-repo
testing. This ensures that if a dependent patch from
the tests repo is noted using 'Depends-On: <change-id>'
in the commit message, that patch will be included.

Change-Id: Ie1a48685f48cd5074031d58478a541967a01be42
Depends-On: Idce7abebf32f24c356a27e099fbca954d917402b
2017-05-04 17:40:00 +00:00
Andy McCrae 44cdfed99d Move swift upgrades into its own shell script
The upgrade path becomes complicated as previous variables are stored
and we have to start a clean run.

The best way to achieve this is to separate the run steps out in a shell
script for upgrades only.

Change-Id: Ib2d2c8bad5241779d52cdc9428d7057c770d2d7c
2017-02-16 17:04:15 +00:00
Jesse Pretorius 34d2e4de85 Update upgrade test to use stable/ocata
Change-Id: I93ff2d05fe25a0712cbf700060856fa4748bbb2d
2017-02-08 14:26:53 +00:00
Cao Xuan Hoang 42cc0a4212 Use https instead of http for git.openstack.org
Trivialfix

Change-Id: If7efc916e31cea7ff3afb2da55339232e89babbf
2017-02-06 17:09:07 +07:00
Andy McCrae 0e45afbd91 Split upgrade test into its own play
To avoid requiring all the os_previous_swift tasks to be evaluated, we
should split the 2 tests up. This will help make the gate more efficient
and avoids unnecessary issues.

Change-Id: I105d0c49f1b5dc78e1e5a5c7334cd81ecc455056
2017-01-20 16:42:00 +00:00
Andy McCrae c02bf40395 Fix tox.ini to work with ansible 2.2.1.0
Ansible 2.2.1.0 requires included vars files to exist even if the
task/play that includes the file is never run (based on a conditional).

This patch fixes that by ensuring the "upgrade" previous version of
test-vars is fetched for every test.

Change-Id: If9c64f7f1525939aa1bcbccef94752852e975ec8
2017-01-18 17:38:47 +00:00
gecong1973 3a64525c11 Delete deprecated Hacking in tox.ini
Some hacking have been removed,so we can delete them.
More details:
     https://github.com/openstack-dev/hacking/blob/master/setup.cfg

Change-Id: I360fc27dbee4a4651eeebf03815979f021cf96aa
2016-12-04 10:24:41 +08:00
Jenkins f9213a7ae0 Merge "Attempt to fix swift_pypy memory issues" 2016-12-03 03:14:08 +00:00
Andy McCrae 3fb6784459 Attempt to fix swift_pypy memory issues
Move to use tempauth to resolve memory issues resulting from an AIO
swift install running in pypy.

This PR adds some options for using pypy:
* Set a pypy Garbage collection value

This PR includes some tempauth fixes to make it useable:
* Set the tempauth users based on a variable

Testing is changed as follows for pypy:
* Use only memcache within swift-proxy
* Remove galera/keystone
* Add swap for swift-storage hosts
* Use tempauth for pypy
* Reduce to 2 swift hosts

Change-Id: Ic1ed5acc9b20853d9a159035226f97fda088f035
2016-12-01 11:07:10 +00:00
Andy McCrae a5e3fa7180 Swift upgrade testing.
Implement upgrade testing for os_swift role.

This patch adds an upgrade testenv, which configures a previous version
of swift, and the test-vars from the previous stable branch to be used
by the previous version of swift.

We don't run functional tests after the initial stable/newton deploy
because a working stable/newton branch is already tested, and it's
assumed it worked if the upgrade works too - which is more efficient.

This follows the pattern setup for Keystone in:
https://review.openstack.org/#/c/384269/

Change-Id: Icefc1c75b814b9d7dc5e79422e975346b9482e9d
Implements: blueprint upgrade-testing
2016-11-22 09:53:52 +00:00
Jesse Pretorius 4f6c605db6 Use upper constraints for all tox targets
With the implementation of https://review.openstack.org/388087 all
tox targets may now use upper constraints.

Change-Id: Ia19db752aa59ca1e5e992135e4d57054c3dfd5b2
2016-10-19 07:44:26 +01:00
Jesse Pretorius 902f28e16e Use centralised Ansible test scripts
This patch consumes the centralised Ansible test scripts
implemented in https://review.openstack.org/381853

Change-Id: Icf49b6e72f6355903225b1163802967865ee587d
2016-10-12 18:40:22 +00:00
Jesse Pretorius 465bd050ab Remove install_test_packages variable
With https://review.openstack.org/381479 merged, setting this
variable in tox.ini via an extra var is unnecessary.

Change-Id: I6041c6235ed95d9a0d3a247ed2d19465ea7e846d
2016-10-04 11:42:54 +01:00
Jesse Pretorius ff495ded2b Use centralised test scripts
This patch consumes the test scripts implemented by
https://review.openstack.org/375061 to ensure that
the tests and test preparation is consistent and
more maintainable.

Change-Id: I5ec2a2c47169e75ad2dbb801eac1732d51cdd2b8
2016-09-28 13:27:00 +01:00
Andy McCrae 2257044bc0 Use openstack-ansible-tests repository for Swift
Move to use the openstack-ansible-tests repository for Swift.
Make changes use Ansible 2.1.1 (These are required for gating to work
properly, since the majority of repositories have already changed).

Additionally, lets set this up to work with 1 infra host in the MR swift
gate to improve efficiency.

Change-Id: Ic0d35bcf8bafb90a986c7cdda3987f70580af165
2016-09-22 15:59:32 +00:00
Jesse Pretorius 1690a5d512 Compress test execution logs
The volume of logs we collect in CI jobs are extensive. This
patch ensures that the logs collected are compressed to reduce
the space taken in CI systems.

PYTHONUNBUFFERED is also set to ensure that the console log
from the CI jobs is recorded in the exact order of execution.

Change-Id: Ie13ecef2958a926933520cceeee6bfb669b94fce
Related-Bug: #1620849
2016-09-09 10:48:44 +01:00
Andy McCrae 80d2d96a5f Add Multi Region Swift gate scenario
* Fix networking so that we can extend it for MR swift gate.
* Add a vars file for region A and region B.
* Add host_vars and inventory files for region A and region B
* Split out the function-test to allow all scenarios to use the same
test.yml file.
* Use -e @vars-file.yml format in tox.ini since we can't include vars
files based on a variable.
* Remove any references to include_vars: test-vars.yml as a result of
moving to use -e @vars-file.yml
* Add static routes into host_vars for swift-proxy and swift-storage
hosts so that they can route to the other region.
* Make addresses in test-vars more generic based on inventory.

Change-Id: I52b4068acbc80dcb7710d388ccbed2edcb8bfd61
2016-09-06 14:31:16 +01:00
Kevin Carter 43a6244ecc Enable log collection after functional testing
This change enables log collection within the gate so that further analysis
on gate tasks can be performed post build. This is very useful when
debugging problems.

Change-Id: Ib3452141ea81a86484dc47307ddaa984f361ed3d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-08-29 21:37:26 -05:00
Jesse Pretorius aad88fdb7c Make all linting tests use upper-constraints
As per [1] all linting tests can now use upper-constraints. This patch
removes all instances of the install_command override relating to lint
testing which were needed to negate the use of upper-constraints.

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101474.html

Change-Id: Icf7e4a8716bff5bad413b0df0bc0e7cf494b1af4
2016-08-20 16:06:54 +01:00
Jenkins 6abd1d9158 Merge "Relocate Swift configuration docs to os_swift role" 2016-08-16 01:48:12 +00:00
Travis Truman d3305e4be0 Relocate Swift configuration docs to os_swift role
Change-Id: Ic3e6540556eeb0518d8e6c59b32f38a427071799
2016-08-15 12:54:04 -04:00
Jenkins 2516054e6e Merge "Add pypy test Scenario for Swift" 2016-08-15 15:01:52 +00:00
Jean-Philippe Evrard 0b5919e07c Include ansible commands for ansible linting
The plugin repo needs to be cloned for ansible-lint to
understand ansible plugins. The commands currently
reside in tox.ini under the ansible section and are not
currently included. This commit fixes that error.

Change-Id: Ic0d5c1776d8f3ac5f3b7d0fec257556ab0452bc9
2016-08-11 18:10:24 +01:00
Andy McCrae c926f33c47 Add pypy test Scenario for Swift
New func_pypy test scenario whcih will run Swift under pypy. This will
start off as experimental but would be good to have added to the gate
permanently.

Change-Id: Iaddfcb4c165e8c7b7265c2e3058268505d2d41bf
2016-08-09 15:12:40 +00:00
Jesse Pretorius cce137c728 Ensure that doc linting is included in the linters test
The 'docs' tox target executes the doc8 lint test which may result in
failures when testing documentation builds, but OpenStack-CI does not
execute that tox target.

In order to ensure that we catch all standard documentation syntax
errors and prevent them from merging, this patch includes the docs
target in the 'linters' chain of tests.

Fixes for any failures which result from executing this test are also
included in the patch.

Change-Id: I336dae4702abb0a671b940bdc50afeb21a118d6f
2016-07-25 14:11:52 +01:00
Jesse Pretorius 675315289b Use plugins repo version of the human_log callback plugin
With the implementation of https://review.openstack.org/321331 the
human_log callback plugin is now part of the plugins repo.

This patch removes the retrofitted version in tox in favor of using
the version from the plugins repo instead.

Change-Id: I02b8646a867a39c555cf821e4680d93d57b27b23
2016-07-15 14:03:50 +01:00