Commit Graph

381 Commits

Author SHA1 Message Date
Balazs Gibizer 1c8afcd3f1 Make tox.ini tox 4.0.0 compatible
* removed skipsdist=True to make sure placement available in the virtual
  env

* passenv fixed as space separated list is not allowed any more

* improved the doc generation command line to print all the errors
  instead of stopping at the first one

* added functional-py310 for convenience

* removed basepython = python3 as we assume all developer switched to
  python3 in their env already

* removed ignore_basepython_conflict = True as without the basepython
  definition generative targets now work without conflict

* updated the install_command to mimic better what tox uses by default

Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/866943
Change-Id: I1de24b7a98f149fce33d4e5eca82460eea72140b
2023-01-09 16:21:22 +01:00
Zuul d723f225c1 Merge "Change minversion of tox to 3.18.0" 2022-06-24 11:11:22 +00:00
Takashi Kajinami d013b37066 Update python testing as per zed cycle testing runtime
In Zed cycle, we have dropped the python 3.6/3.7[1] testing
and its support. Add release notes and update the python
classifier for the same.

[1] https://governance.openstack.org/tc/reference/runtimes/zed.html

Co-Authored-By: Ghanshyam Mann <gmann@ghanshyammann.com>
Change-Id: I998b34f884a2869d46ff05a179708a7f775e1615
2022-06-08 08:14:53 +09:00
Ghanshyam Mann 03d567928e Drop lower-constraints.txt and its testing
As discussed in TC PTG[1] and TC resolution[2], we are
dropping the lower-constraints.txt file and its testing.
We will keep lower bounds in the requirements.txt file but
with a note that these are not tested lower bounds and we
try our best to keep them updated.

[1] https://etherpad.opendev.org/p/tc-zed-ptg#L326
[2] https://governance.openstack.org/tc/resolutions/20220414-drop-lower-constraints.html#proposal

Change-Id: Ieba7daf39fa3323e8c9a7396747449f24189fcd5
2022-05-01 00:39:43 +00:00
Stephen Finucane 20fdc44bfd tox: Enable SQLAlchemy 2.0 warnings
As described in [1]. Unfortunately it doesn't seem to be possible to
silence these for other modules (cough...oslo.db...cough), presumably
due to how SQLAlchemy has configured warnings. We'll have to ignore
these for now.

[1] https://docs.sqlalchemy.org/en/14/changelog/migration_20.html

Change-Id: I28bc91d515440553828976dd1d3a3aef36a6d6a9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2022-04-07 15:39:27 +01:00
Takashi Natsume ba7cd28931 Change minversion of tox to 3.18.0
The patch bumps min version of tox to 3.18.0 in order to
replace whitelist_externals with allowlist_externals option:
https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23

Signed-off-by: Takashi Natsume <takanattie@gmail.com>
Change-Id: I35c8c1591c8da83eb2c681ef176eb3e3f3c2798f
2022-03-04 17:04:58 +00:00
Stephen Finucane b9e7524cbf tox: Remove psycopg2 warning filter
Our minimum constraint for this dependency is now at the correct
version, meaning we no longer need this filter. Remove it.

Change-Id: I94485a3f16b0cd47ae1e2331efc6b19eb4f6278b
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2022-02-06 02:00:06 +09:00
Ghanshyam Mann 1141597b5a Updating python testing as per Yoga testing runtime
Yoga testing runtime[1] has been updated to add py39
testing as voting. Unit tests update are
handled by the job template change in openstack-zuul-job

- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286

this commit makes other required changes in zuul.yaml and
update the classifier in setup.cfg file.

[1] https://governance.openstack.org/tc/reference/runtimes/yoga.html

Change-Id: I50c125df4ee03377caf16da5824e2ed492d4b7da
2021-12-14 02:24:04 +00:00
Zuul 120069638c Merge "Add functional-py3[89] tox targets" 2021-02-02 21:02:48 +00:00
Ghanshyam Mann a0acd21937 Fix l-c job and move to latest hacking 4.0.0
There are few failure in l-c jobs due to

- The conflict is caused by:
    The user requested pyflakes>=2.1.1
    hacking 0.12.0 depends on pyflakes==0.8.1

- The conflict is caused by:
    The user requested requests>=2.14.2
    keystonemiddleware 4.18.0 depends on requests>=2.14.2
    oslo-config 6.7.0 depends on requests>=2.18.0

Remove the flake8 and pyflake from lower constraints as they are
maintained in hacking side[1]. Maintaining in hacking aas well as
in project side can lead to version conflict. Also few more deps
from l-c as they are not used in placement directly.

Update requests version to 2.25 to fix the http connection failure with
2.18
- https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_dff/766994/6/check/openstack-tox-lower-constraints/dff1943/testr_results.html

This commits also moves the code to latest hacking 4.0.0 with W504 in
ignore list and fix W503 and other checks.

[1] 4069b0c4f5/lower-constraints.txt (L9)

Change-Id: Id2761300060597c1744e76ea77740ed380d98943
2020-12-15 10:21:18 -06:00
Balazs Gibizer 46947d52b8 Add functional-py3[89] tox targets
This also adds a non voting functional-py39 job

Depends-On: https://review.opendev.org/760932

Change-Id: I5557535967a721037731c6108acee4c6f5582bbc
2020-11-04 13:41:45 +01:00
Tetsuro Nakamura 8685ca0e2e Adds py38 functional tests to gate
Python 3.8 is now our highest level supported python runtime,
but there are no functional tests run in the gate.
This patch adds the tests to the gate.

Change-Id: Ia81b3983ea3636beadf0e58c1b584d607bd567b4
2020-09-11 07:00:25 +00:00
qiufossen fcc5ebc705 Bump default tox env from py37 to py38
Python 3.8 is now our highest level supported python runtime.
This updates the default tox target environments to swap out
py37 for py38 to make sure local development testing is
covering this version.

This does not impact zuul jobs in any way, nor prevent local
tests against py37. It just changes the default if none is
explicitly provided.

Change-Id: If9931b519f7bba4faa0529f0a5403c935e36dacb
2020-09-11 06:41:08 +00:00
Chris Dent 8436f45d01 Drop support for python 2
This is modelled on the similar change in nova [1] but is
simpler because Placement was already using python3 for
"integration" style tests.

The 'train' python3 template is replaced with corresponding 'ussuri'.

The python template is removed (it is 2.7) and the pep8 job that was
included in that template is now listed in check and gate jobs.

.zuul.yaml is updated to not do a generic 'funtional' test,
instead versioned tests only.

setup.cfg removes indications of support for python 2.

A reno is added.

tox.ini is changed to make the functional environment a
base environment for specific functional jobs.

[1] Ie1a0cbd82a617dbcc15729647218ac3e9cd0e5a9

Change-Id: Ifce3b0c0ff11553f32b2d02df07fde81efc34100
2019-10-18 12:58:36 +00:00
Zuul 7c93417d14 Merge "Update the constraints url" 2019-09-23 19:58:46 +00:00
pengyuesheng 10fc2d350b Update the constraints url
For more detail, see http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html

Change-Id: I72523f8eb275076e0970f98346add7483d2d46b1
2019-09-23 16:32:03 +08:00
Chris Dent b93200d7d6 Build pdf docs
This follows the instructions [1] in an attempt to build pdf docs.

Several doc/source/conf.py changes are required to get this to work.
The most important one is

    'maxlistdepth': '10',

which prevents the build process from stalling out and dropping the
caller into an interactive session.

[1] https://etherpad.openstack.org/p/train-pdf-support-goal

Change-Id: Icf7c22bf9d1de6fb2a74a756c370930d4c00b0b9
Story: 2006110
Task: 35398
2019-09-20 13:50:56 +00:00
Andreas Jaeger 7b3c179fe4 Update api-ref location
The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.

Note that redirects will be set up as well but let's point now to the
new location.

For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html

Change-Id: Iccddd4356b5d7c28977ad954654d0875972dae97
2019-07-22 19:17:01 +02:00
Zuul 3ba36fb717 Merge "Add Python 3 Train unit tests" 2019-07-16 09:42:51 +00:00
Corey Bryant 7858375439 Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html

Change-Id: I2ef60794a2606a7d7abfd413c6ba50689c52e67b
Story: #2005924
Task: #34236
2019-07-10 21:09:52 +00:00
Stephen Finucane 8c4ad849fd tox: Stop building api-ref docs with the main docs
We have an 'api-ref' docs target - use that instead.

Change-Id: Ie94add90b507d631a74f4617de7feba56fb0c646
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2019-07-08 09:18:31 +01:00
Stephen Finucane 4b4dd5ecf8 Add whereto for testing redirect rules
Use whereto to test that the redirect rules do what we expect.

Change-Id: Id3a8f3b9f372ebe9176f1df917f7a6aac30a8e92
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2019-07-08 09:18:18 +01:00
Tetsuro Nakamura 7f4b79b7e1 Cache provider ids in requested aggregates
Getting allocation candidates for one request group, placement
internally calls provider_ids_matching_aggregates() several times with
the same arguments. This is redundant and can degrade performance.

This patch changes to call it on initialization of
`RequestGroupSearchContext` object and cache the results so that we can
refer the results later on demand.

Change-Id: I93e1fa81a7345b651ea2d58dfa0d5508e726f43f
Story: 2005712
Task: 31038
2019-05-29 15:22:38 -05:00
Zuul 7f92fdb729 Merge "Replace git.openstack.org URLs with opendev.org URLs" 2019-05-05 14:34:16 +00:00
caoyuan a2513d8ea6 Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I62921ef9ee51c391bbd2b4e39941df5b68a9a908
2019-04-29 13:06:10 +08:00
Ghanshyam Mann da3e8795f3 Dropping the py35 testing
All the integration testing has been moved to
Bionic now[1] and py3.5 is not tested runtime for
Train or stable/stein[2].

As per below ML thread, we are good to drop the py35
testing now:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005097.html

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004647.html
[2]
https://governance.openstack.org/tc/reference/runtimes/stein.html
https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: I5e4a9148c0e0649ff0e4657ea4c87e50c1b3fd2d
2019-04-22 23:12:02 +00:00
Chris Dent da1a588b8d Remove use of oslo.i18n and translation
Nothing is being translated in placement so, for the sake of being clean
and tidy, this patch removes the framework for translation and the
import of oslo.i18n.

Originally the hope was we could remove the dependency on oslo.i18n and
Babel entirely to save some disk space but many other oslo-related libs
depend on oslo.i18n so they are present anyway. [1]

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004220.html

Change-Id: Ia965d028b6f7c9f04d1f29beb12f4862585631d5
2019-03-25 16:59:21 +00:00
Eric Fried 849c89d0e5 Remove pep8 whitespace ignores
We had carried over the full list of pep8 whitespace ignores from nova.
Trying to address them all in the entire nova repository would have been
too big a task; but it's tenable here in placement.

Do it now rather than letting these whitespace issues compound.

This change removes the E* whitespace ignores and fixes the pep8 issues
they were masking.

Change-Id: Icbabdb9b56fd5a3e9fd14ab537abf3d0d6456fee
2019-03-05 18:18:30 -06:00
Eric Fried 0b8475db68 Use tox 3.1.1 fixes
By 3.1.1, tox had fixed:
- [1] which mishandled the basepython directive.
- ignoring PYTHONDONTWRITEBYTECODE [2]

Require this version and remove the workarounds.

See ML thread [3] for some details

[1] https://github.com/tox-dev/tox/issues/477
[2] 336f4f6bd8
[3] http://lists.openstack.org/pipermail/openstack-dev/2018-July/thread.html#132075

Change-Id: I87cbf4b26bdd40a8b7a699966e1ace838a2491de
2019-02-14 09:59:07 -06:00
Eric Fried 9e5436efab tox: Don't write byte code (maybe)
In tox versions after 3.0.0rc1 [1], setting the environment variable
PYTHONDONTWRITEBYTECODE will cause tox not to write .pyc files, which
means you don't have to delete them, which makes things faster.

In older tox versions, the env var is ignored.

If we bump the minimum tox version to something later than 3.0.0rc1, we
can remove the commands that find and remove .pyc files.

[1] 336f4f6bd8

Change-Id: Id9a97af032a2dbefdc50057270368ad087e6bb5f
2019-02-14 09:54:44 -06:00
Chris Dent 4469a813cb Correct lower-constraints.txt and the related tox job
While exploring removing unused packages from lower-constraints.txt it
became clear that the lower-constraints job was not working as expected:
Because our tox config has usedevelop=True 'setup.py develop' is called
to install the placement package after the install command is called.
This means that the lower-constraints are clobbered.

I had mistakenly assumed that turning off 'usedevelop', which causes
'setup.py install' would not make any difference, because it usually
installs dependencies too. It turns out however, that when using pbr
and within a git working dir, it does not. That took some time to
figure out. Oh well.

This change makes it so that we create the tox environment using
usedevelop=False and with our own install_command, to avoid upper
constraints conflicting with lower constraints.

This flagged up a few changes, the main one being that we did not have
a new enough version of keystonemiddleware in order to require use of
www_authenticate_uri. requirements.txt is updated for this as well.

And PasteDeploy needed to be updated to work with Python 3's notion
of namespace packages.

psycopg2 need a newer version to work with Postgresql 10.

oslotest needs to be raised to 3.4.0 because the tests in
cmd.test_manage use features to control what is capture by the Output
fixture from oslotest. Note that the lower-constraints job found
this problem and also demonstrates why we must run the lower-constraints
job without upper-constraints being involved. upper-constraints will
"win" and we don't want that. The point of the job is find packages
where lower-constraints are wrong, so it must "win".

The end result here is a lower-constraints.txt file that starts from
the lower-constraints.txt defined by nova, and then is adapted to
update the versions of packages that were not up to date, remove
those packages which are no longer present, and add some that are
now required.

Change-Id: Id66a28f7ace6fc2adf0e1201d9de5f901234d870
2018-11-27 17:41:02 +00:00
Chris Dent 3b569321cc Clean up and clarify tox.ini
While looking into making py36 available when running tox without '-e'
and making py37 more available for both unit testing and functional
testing, it became clear that our tox.ini was a bit messy. This change
cleans it up to:

* Removes all py* jobs and lets tox define those automatically. Their
  commands come from the base [testenv].

  Non-unit tests need to define their commands specifically.

* move pep-related tox below both unit and functional

* remove some out of date comments

* set the default when using tox without -e to unit and functional
  in 2.7 and 3.6, and pep8

* removes the 'stestr slowest' from the unit and functional test env's.
  It's not used consistently and the output is far from consistent so
  it doesn't seem to have any particular value (for this project)

This is one of several ways to do this cleanup. This way assumes some
knowledge of the fact that tox defines a lot default jobs with default
python versions (via factors[1]).

Note that the releasenotes job does not currently work and did
not work before these changes, a conf.py is missing.

[1] https://tox.readthedocs.io/en/latest/config.html#factors-and-factor-conditional-settings

Change-Id: I9a877439e13eb753c7f98dc02e41db891ce33690
2018-11-06 16:18:14 +00:00
Zuul ab62e9f4da Merge "Reduce max-complexity to 15" 2018-10-11 15:19:30 +00:00
Zuul 8c85050960 Merge "max-complexity=>16: refactor GET /a_c qs parsing" 2018-10-02 11:22:02 +00:00
Zuul baaa0d023d Merge "Put stestr group_regex in .stestr.conf" 2018-09-25 21:52:49 +00:00
Zuul 28a5beaec3 Merge "Use both unit and functional for coverage testing" 2018-09-25 21:47:57 +00:00
Takashi NATSUME e04ff2395b Fix missing specifying doctrees directory
In tox.ini file, the doctrees directory is specified
as an argument of sphinx-build command in api-ref,
releasenotes targets.
But it is missing in docs. So add it.

TrivialFix
Change-Id: I767b675b503cd627e80c7c1ff434d4a8ad53203a
2018-09-21 09:40:20 +09:00
Eric Fried 7ddf015238 Reduce max-complexity to 15
The dependencies knock down the four most complex methods in the
project, bringing our most complex method to 14 (and hence our
max-complexity limit to 15).

Ife303d018a26ddcf8ea41d5950dc774c5e898a56
placement/util.py: 'parse_qs_request_groups' (17)

I9f3aa66b4806cf2c34f5271a5f4426839cb872ce
placement/objects/resource_provider.py:
  'ResourceProviderList._get_all_by_filters_from_db' (16)
placement/objects/resource_provider.py:
  '_get_provider_ids_matching' (16)

I80f8b1a74b8afdabbb85afc6caef1abef8ac3751
placement/objects/resource_provider.py:
  'AllocationCandidates._get_by_requests' (16)

Change-Id: I11d3e3c0e1baf729f2acceafbe2f115212a08385
2018-09-18 14:47:23 +00:00
Eric Fried 95f87ca109 max-complexity=>16: refactor GET /a_c qs parsing
Before this patch, the project's max-complexity of 17 was perpetrated by
RequestGroup.dict_from_request, the request querystring parsing method
used for GET /allocation_candidates. This refactor-only patch splits
that method into smaller chunks, the most complex of which now comes in
at 8.

There is no functional change, hence no deltas to tests.

Change-Id: Ife303d018a26ddcf8ea41d5950dc774c5e898a56
2018-09-14 16:15:53 -06:00
Chris Dent 12a9e321bd Put stestr group_regex in .stestr.conf
This dries up tox.ini a bit while preserving the explanatory
comment. Having the grouping expression on test jobs that don't
require it has only minor impact: the regex is applied to the
list of tests and no groups are found.

Change-Id: I89330cf5b0ace25db6a7575cfa4f93d9575b08ac
2018-09-14 13:30:14 -06:00
Chris Dent 09905baab2 Use both unit and functional for coverage testing
Placement has always relied heavily on functional testing for both API
and database testing, so not including those tests in 'coverage' leads
to misleading results.

This change adjusts the cover job in tox to use a test path of
placement/tests, resulting in all the tests running.

In the process .coveragerc is updated to be placement oriented
and make sure we do not ignore branches.

This is in preparation for turning on a check job for coverage.
See this email for more on that:

http://lists.openstack.org/pipermail/openstack-dev/2018-September/134385.html

A followup patch will turn on a check job for it.

Change-Id: I6c122facc9c9f6e0b2a1d69c6aaf5faeee7cf253
2018-09-14 13:30:14 -06:00
Zuul f2a242d457 Merge "s/placement-config-generator/config-generator/" 2018-09-14 16:34:23 +00:00
Eric Fried f5a25d5fd6 s/placement-config-generator/config-generator/
Change [1] added support for tox -e genconfig via a configuration file
called placement-config-generator.conf. Since we're in the placement
project and only have one config generator, the placement- part is
redundant. In keeping with similar pieces (e.g. policy-generator.conf)
this patch renames the file and fixes references.

[1] https://review.openstack.org/#/c/601838/

Change-Id: I8ea9c00d26539feab141ada493bc825987046b3d
2018-09-14 16:10:58 +00:00
Zuul 3c30ed3cb4 Merge "Set upper bound on max-complexity in pep8" 2018-09-13 23:16:14 +00:00
Chris Dent 6cd3d800b9 Set upper bound on max-complexity in pep8
The current max-complexity in placement code is 17, so set the
upper bound in the pep8 tests to max of 18 so we get an error
when we're going to make it worse. The current high complexity
(but still under 18) functions are:

./placement/util.py:415:1: C901 'parse_qs_request_groups' is too complex (17)
./placement/objects/resource_provider.py:1341:1: C901 'ResourceProviderList._get_all_by_filters_from_db' is too complex (16)
./placement/objects/resource_provider.py:2848:1: C901 '_get_provider_ids_matching' is too complex (16)
./placement/objects/resource_provider.py:4076:1: C901 'AllocationCandidates._get_by_requests' is too complex (16)

Change-Id: I58dc5acb9bb13dcdb5623d13515820591ec5f974
2018-09-13 16:45:59 -06:00
Zuul 7e7dc3b930 Merge "Rename files to remove 'placement'" 2018-09-11 15:04:21 +00:00
Zuul c53e4a92f5 Merge "Make docs build" 2018-09-07 20:34:48 +00:00
EdLeafe 5794b857b4 Rename files to remove 'placement'
When part of Nova, many files were named in such a way as to indicate
that they were separate from the nova files that did the same thing. Now
that that distinction is no longer needed, we can rename them to sound
more natural.

Additionally, references to these file names in other parts of the code
have been updated.

Change-Id: Id4439b4af827b64bbac213085ac74d1283c96db0
2018-09-07 20:10:06 +00:00
Eric Fried 9b584fcd1d Make docs build
This patch does pretty much the minimum possible to get `tox -e docs`
building and gating. Much, much more needs to be done to make the
content good.

Also corrects a link in the api-ref which, once these are publishing,
would have pointed to the wrong place.

Change-Id: I5cbc3a3cceeaeaa7be5593658b6a03fa25fb69d0
2018-09-06 11:45:11 -05:00
Chris Dent 15ca8bc758 Removing non-existent job from tox envlist
The pip-missing-reqs job does not exist but because of the way
we have tox configured, running 'tox' by itself will run what
amount to a null job, consuming time but testing nothing.

See also: I8355372da93b93cc1c7f7a1501af0fbd01eee615

Change-Id: I0969908ca7f5acbcde4a3d561366295bf2403c83
2018-09-04 20:49:54 +00:00