Commit Graph

249 Commits

Author SHA1 Message Date
Zuul b6223f3210 Merge "tests: Enable SQLAlchemy 2.0 deprecation warnings" 2023-08-28 18:12:50 +00:00
Zuul 93e14113a5 Merge "Clean generated source RST files" 2023-08-15 03:31:18 +00:00
Stephen Finucane d409b27a79 tests: Enable SQLAlchemy 2.0 deprecation warnings
Well, sort of. We enable them but immediately filter out the ones we're
actually seeing, the rationale being that we can address these in a
piecemeal fashion without the risk of introducing new issues.

There's a lot more to be done here. However, the work done in oslo.db
[1] and other projects should provide a guide for how to resolve the
outstanding issues.

[1] https://review.opendev.org/q/topic:%2522sqlalchemy-20%2522+project:openstack/oslo.db

Change-Id: I7a59714b104659b64f46f4c1437cfc4018356815
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 17:29:01 +01:00
Stephen Finucane f136d103fb tox: Unset basepython
This is a common source of bugs in tox 4 and is no longer necessary now
that we are all using Python 3 everywhere.

Change-Id: I3733f779075c189c9a630416181b7d4c14ba599b
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 17:14:22 +01:00
Brian Rosmaita 0bc55d4f83 Clean generated source RST files
The tox docs env generates RST files in doc/source/contributor/api/
and these aren't updated when modules are removed from the glance
source code, and can cause errors trying to build the docs.  So
clean this dir out (like we do with doc/build/) before building
the docs.

Change-Id: I47e8ceccecadb469c27cbf203fe77528cc38d95d
2023-07-19 15:41:53 +00:00
Brian Rosmaita 084c8a32f5 Update functional jobs for 2023.2
Based on the python runtimes for 2023.2 [0], we should be running
functional jobs on python 3.9 and python 3.10.  This patch adds
functional testenvs for python 3.10 and 3.11 (the latter for local
testing) to tox.ini so the appropriate zuul jobs can be defined.

The functional-py38 testenv is not removed from tox.ini, as it may
be useful locally.

Functional py39 and py310 jobs are added to the check and the gate.

The current openstack-functional-py38-fips job is not removed; a
py39 job has not yet been defined to replace it [1].

[0] https://governance.openstack.org/tc/reference/runtimes/2023.2.html
[1] b3cad4f7a3/zuul.d/jobs.yaml

Change-Id: Ibc21b107878f5ba50137da4082a7cbc6342d2aa9
2023-04-17 18:00:24 -04:00
Cyril Roelandt 41da684fe4 Fix functional test failures with PasteDeploy 3.x
PasteDeploy replaced pkg_resources with importlib_metadata[1] in 3.0.
This causes functional test failures because the line breaks in the
example configs from glance/tests/functional/__init__.py are read as a
"prefix" by PasteDeploy. For instance,

paste.filter_factory =
  glance.api.middleware.version_negotiation:VersionNegotiationFilter.factory

is read as "\nglance.api.middleware.version_negotiation:VersionNegotiationFilter.factory".

This might be a bug in PasteDeploy or (quite unlikely) in
configparser.ConfigParser rather than in our code, so this issue should
be revisited at some point. For now, remove the cosmetic line breaks in
order to fix our gates.

[1] ef58c20961

Change-Id: I1e557e731cceb3cfbc4ebc3f0887adf372a91d1d
2023-04-12 20:37:47 +02:00
Abhishek Kekane d07187f710 Remove migration constant job and test
Since openstack release naming conventions has changed, current test
which check data migration version will not work as expected and same
is also blocking our gate. Removing this job and test to unblock the
gate.

NOTE: Going forward glance PTL/team needs to change the database
migration version without fail at the start of release cycle.

Change-Id: Idcb12a6c450d4ce4ee859e6e1f02fb71adf8c1d5
2023-01-16 14:40:29 +00:00
Ghanshyam Mann e2cd7251f4 Fix tox4 error
tox.ini started failing with Tox4 which had some
incompatible changes. One is passenv where we need
to pass each value in newline otherwise, it fails with
error:

 failed with pass_env values cannot contain whitespace,
 use comma to have multiple values in a single line,
 invalid values found.....

Fixing tox.ini for tox4 changes.

Change-Id: I0a377c9329cef8b251b800018fc0d7f784008329
2022-12-26 00:23:59 +00:00
Zuul 6c504b3210 Merge "Update python testing as per zed cycle teting runtime" 2022-05-19 21:47:33 +00:00
Ghanshyam Mann d55d250d33 Update python testing as per zed cycle teting runtime
In Zed cycle, we have dropped the python 3.6/3.7[1] testing
and its support. Moving the py36 job to py38 based as well as
updating the python classifier also to reflect the same.

openstack-tox-functional-py36-fips job is left which can be moved
to py38|py39 based once that job is defined in openatck-zuul-jobs
repo.

openstack-tox-functional-py36-fips job will be migrated to py38 or py39
in followup patch as that need openstack-zuul-config changes too.

[1] https://governance.openstack.org/tc/reference/runtimes/zed.html

Change-Id: Id0813d9dc553dd424732079039349b42f6f3201b
2022-05-19 12:56:56 -05:00
Cyril Roelandt 1962f47ed2 Delay string interpolations at logging calls
This was already fixed in 4889dc1814 but
we did not enforce this rule and reintroduced "bad" string
interpolations. This patch adds a hacking rule to prevent us from doing
this again in the future.

Change-Id: I96d9a157d3887286542859d67138ffdae5a589f1
2022-05-14 03:17:34 +02:00
Elod Illes 36aedba20e [CI] Add upper constraints to install command
In cases when a package is installed in tox's develop-inst phase upper
constraints are not used if it were not defined in 'deps' of the tox
target. This caused for example 'tox -e docs' to fail in case a package
introduced some branch incompatible change.

There are two option to fix this:
* add the upper constraints to 'deps' of the target (and pre-install
  requirements for some cases) OR
* use it in the install_command

In the past the latter needed some extra attention (due to e.g. lower
constraints tests) and projects were encouraged to remove the
install_command, but nowadays this seems better fitting for some
project and easier to use. So this patch fixes the upper constraints
usage via the custom install_command and by removing it from 'deps'.

Change-Id: Iae07e1c5abde09874928951ac55c2d0355bcd244
2022-05-05 17:16:57 +02:00
Brian Rosmaita c3fb128d78 Update doc8 ignore-path
Ignore the .eggs directory that is created by (among other things)
pbr.  It contains eggs downloaded by setuptools to build, test, and
run plugins, but more to the point, it contains a README.TXT that
violates D001 (Line too long).

Change-Id: I60a5692f0061ae7568c5cdd4b8482f6b2a963261
2022-03-11 13:27:26 -05:00
Stephen Finucane 931809c037 Remove remaining six.moves usage
Change-Id: Ibca3884e1ea3d0fb170bcc9e70a176d144ee24cc
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2022-01-27 16:37:11 +00:00
Stephen Finucane 6bd7c188ee Remove six.text_type, six.binary_type usage
Change-Id: I2ed464202f8b645aed11490e111c61d3c7423c11
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2022-01-27 14:54:00 +00:00
Zuul e0413eb56c Merge "Updating python testing as per Yoga testing runtime" 2022-01-21 10:50:44 +00:00
Takashi Kajinami 0a5b511729 Use LOG.warning instead of deprecated LOG.warn
The LOG.warn method is deprecated[1] and the LOG.warning method should
be used instead.

[1] https://docs.python.org/3/library/logging.html#logging.warning

Change-Id: Ie0cac63dedf69728392f293f4551e495aebf4d40
2022-01-19 11:36:12 +09:00
Stephen Finucane 502fa0ffc8 Remove six.iteritems usage
We also remove hacking tests for this, along with those for iterkeys and
itervalues (no usage of these).

Change-Id: If5b46580078eb756651ac6118f502eccdc693646
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-12-21 13:01:56 +00:00
Ghanshyam Mann ed1183919a Updating python testing as per Yoga testing runtime
Yoga testing runtime[1] has been updated to add py39
testing as voting. Unit tests update are
handled by the job template change in openstack-zuul-job

- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286

this commit updates the functional and tips jobs to test
on py39, also update the classifier in setup.cfg file.

[1] https://governance.openstack.org/tc/reference/runtimes/yoga.html

Change-Id: I6e1c64961f0feb920e45ef6a6ed29f2e32265768
2021-12-13 20:57:28 -06:00
Zuul 91ed096ec0 Merge "Add a nonvoting functional job with RBAC defaults" 2021-07-01 15:19:36 +00:00
Dan Smith 8201f5f101 Add a nonvoting functional job with RBAC defaults
This adds a new tox target called functional-py38-rbac which enables
the new secure RBAC policy defaults for all functional tests. To do
this, the functional tests needed a little bit of extra work to
actually set those, and a new non-voting job is added to run these
in CI.

Related to blueprint policy-refactor

Change-Id: Id376193521671bdb0ebc08ea8e563578bbaa541f
2021-06-30 11:49:30 -07:00
Zuul 7b0eb314cc Merge "Drop lower-constraints jobs" 2021-06-28 18:56:51 +00:00
Dan Smith fa1bc2f904 Drop lower-constraints jobs
Change-Id: Id030b56b721cb62255936d56f0409c4be9ce21d4
2021-06-21 07:19:01 -07:00
wu.shiming bba5014c1a Changed minversion in tox to 3.18.0
The patch bumps min version of tox to 3.18.0 in order to
replace tox's whitelist_externals by allowlist_externals option:
https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23

Change-Id: I7263ff1498dfdf51a919aa678adde413c1935bc6
2021-06-03 06:58:44 +00:00
Cyril Roelandt 8a166d8e30 Tox.ini: add py39
This allows us to run unit tests and functional tests using Python3.9.
We also remove all testenv:py3* environments, since tox is able to infer
what version of Python to use from the environment name.

Change-Id: If9ce1475ddfc66c9b37cd3500af982bf3814bf60
2021-01-14 14:00:54 +01:00
Erno Kuvaja 673666cbf4 Deprecation cleanout Registry and related
This patch removes majority of the registry and it's related
endpoints and config options that has been deprecated for
removal in various releases.

Change-Id: I75014bd50bf382efebe56bd89c20ffefbdde25f5
2020-06-30 20:41:30 +01:00
Brian Rosmaita 890b740adc Add tests to lower-constraints job
The current job simply installs requirements constrained by
lower-constraints.txt.  While it is nice to know that the
requirements files are compatible, it would be even better to
run some tests to actually check that the code runs using the
current lower-constraints.txt file.

Change-Id: I7a191ac44b004f840c986ded61341e43b35f1d49
2020-05-13 09:29:06 -04:00
Sean McGinnis 38f3348298
Bump default tox env from py37 to py38
Python 3.8 is now our highest level supported python runtime.
This updates the default tox target environments to swap out
py37 for py38 to make sure local development testing is
covering this version.

This does not impact zuul jobs in any way, nor prevent local
tests against py37. It just changes the default if none is
explicitly provided.

Change-Id: I08f4d4e3845bb98e208168f7c632e1e10263e022
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-01 08:44:44 -05:00
Sean McGinnis b4eabc062a
Install all deps in venv creation
The venv target was inheriting the default deps, which appropriately
just includes the main project requirements and our test requirements.
We also use venv for reno creation and other miscellaneous things, so it
is good to get all potential requirements installed. This adds the
installation of our docs requirements in the creation of the venv
environment.

Change-Id: I8b232116e54e39243a009a195090f12ef1a56bd4
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-13 15:34:11 -05:00
Andreas Jaeger 89df136bc4 Update hacking and reenable local checks
Update to hacking 3.0.

Enable local hacking checks again, newer flake has a different
interface.

Remove hacking and friends from lower-constraints, they are not needed
for installation.

Unbreak docs:
Don't build apidocs for hacking - blacklist in doc/source/conf.py.

Change-Id: Ib230d72be9f0288d77cecd2c5ee0ff7aa91fc086
2020-04-02 18:00:44 +02:00
Zuul 2143060833 Merge "Raise hacking to latest 2.0.0 release" 2020-03-30 14:50:31 +00:00
Andreas Jaeger 176d24de94 Cleanup old cruft
Make a few cleanups:
- Remove python 2.7 stanza from setup.py
- Update requirements: Remove sphinx, it's not needed for testing, add
  pygments which is really needed.
- Update doc/requirements: Remove python 2.7 support
- tox.ini: Remove testing of po files, the infra scripts do this since
  a long time
- Update conf.py, no need to import openstackdocstheme anymore.

Change-Id: I9d030eb450f2c7ae74c25b7564a01b8785503e5e
2020-03-28 08:18:01 +01:00
Dirk Mueller 595c1b17ff
Raise hacking to latest 2.0.0 release
We were capped at a very old version of hacking. Hacking itself caps the
various linters it uses to remain consistent, so our pep8 job was not
checking quite a bit that current versions have added.

This raises that limit to the latest to get up to the level of other
projects and addresses the errors the updated linters uncovered.

Change-Id: I89a9d73fbd59606a649e26077acebc5c42873d67
Co-authored-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-03-27 14:11:08 -05:00
Zuul c632586ef3 Merge "Move policy defaults into code" 2020-01-09 14:00:55 +00:00
Zane Bitter 5c17e4c7ef Move policy defaults into code
Instead of a default policy.json file, policy defaults are now defined
in code. An operator need not supply policy.json data except to the
extent they want to override the defaults. Currently an empty
policy.json is still shipped because it is expected by devstack, but
this can be removed later. A sample policy.yaml file can be generated
using the genpolicy tox environment.

This partly fulfils the requirements of the policy in code goal[1].
However, because policies don't map 1:1 with APIs, it will not be
possible to fully document the policies until changes are made in how
policies are applied as proposed in https://review.opendev.org/528021

Due to the fact that existing policy files may rely on a rule named
"default" to specifiy policies not explicitly listed in the policy.json
file, all policies that are not admin-only by default now default to
"rule:default", so that the "default" rule will continue to apply to
those policies that are not listed in policy.json.

To ensure that this yields the expected policy in a standard
policy-in-code config file, the default value of the "default" rule is
now the empty string "". This is a change; between the Queens release
and now the default was set to "role:admin" to match the value specified
in the default policy.json file. An installation relying on both the
"default" rule for some policies and the default value of the default
rule may end up with a more permissive policy after upgrading. It's
likely that no such policies exist in the wild, because prior to the
Queens release the default value for the "default" rule was "@" (allow
all requests), so anybody relying on this rule will surely have
specified it explicitly in their policy.json.

Policies whose default is "role:admin" no longer use the "default" rule.
Therefore existing policy.json files that rely on the "default" rule for
those policies, and who have specified a value for the "default" rule
that is more permissive, will result in a more restrictive policy after
upgrading. It is unlikely that any of these policies exist in the wild
either.

[1] https://governance.openstack.org/tc/goals/selected/queens/policy-in-code.html

Change-Id: I8d1ccf5844078cc0b1652fb1130794daf07cedbc
2020-01-06 12:56:30 -05:00
Brian Rosmaita 40bd312410 Remove py2 testing, jobs
Change tox.ini so that all jobs are python3, and add explicit
unit/functional testenvs for the two ussuri python runtimes
(3.6 and 3.7).

Remove py2 jobs from .zuul.yaml and redefine the periodic "tips"
jobs so that they are testing with the ussuri python runtimes.
Added nodeset: ubuntu-bionic to the abstract base jobs so that
py3.7 is available.

Co-authored-by: Ghanshyam Mann <gmann@ghanshyammann.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>

Closes-bug: #1853693
Change-Id: I15467c34fc8466fa1724c5401dc879ae77c1922d
2019-12-17 14:24:24 -05:00
Erno Kuvaja 06b2465f59 Remove native ssl support
As eventlet ssl termination is broken with python 3 and
we won't be supporting python 2.7 anymore we will just
remove ssl termination to glance-api and expect the
termination being handled by something else, like HAProxy.

This patch also removes the broken ssl test job as the
non-existing feature is not broken anymore.

Change-Id: Iaf16dfcfdb3a2c93312dcad1ea1229e6b3c8caaa
2019-12-09 15:17:03 +00:00
Brian Rosmaita d2d67ae3a1 Install requirements during the correct tox phase
Glance isn't explicitly installing requirements, so add requirements
as tox deps in such a way that they'll be properly constrained by the
current openstack upper-constraints.

This change also bumps the version of psycopg to that of the current
upper-constraints in test-requirements and lower-constraints.  (The
version in our current lower-constraints is subject to the "could not
determine PostgreSQL version from '10.x'" error, which causes the
lower-constraints job to fail.)

Change-Id: I1bb5759e6b1e3f8545ab417617072a61323b59a7
Closes-bug: #1852356
2019-11-14 08:20:29 -05:00
Zuul 0be23bdc8c Merge "Dropping the py35 testing" 2019-09-05 17:49:48 +00:00
Zuul 9788aeab58 Merge "Modify the url of upper_constraints_file" 2019-08-07 12:20:29 +00:00
Andreas Jaeger 15609a11bd Update api-ref location
The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.

Note that redirects will be set up as well but let's point now to the
new location.

For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html

Change-Id: I17c97cb7655bdedc3927f4883b45ba4dcfc55516
2019-07-22 19:15:05 +02:00
pengyuesheng b7e9db7046 Dropping the py35 testing
All the integration testing has been moved to
Bionic now[1] and py3.5 is not tested runtime for
Train or stable/stein[2].

As per below ML thread, we are good to drop the py35
testing now:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005097.html

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004647.html
[2]
https://governance.openstack.org/tc/reference/runtimes/stein.html
https://governance.openstack.org/tc/reference/runtimes/train.html

Story: #2005924
Task: #34208

Change-Id: I2dee048b71e3758208fae438423cceae661c8f00
2019-07-15 09:37:43 +08:00
pengyuesheng 5cab93f8c3 Modify the url of upper_constraints_file
Depends-On: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html

Change-Id: I641f6bb61e545617b9ad971b1215294cf335adbc
2019-06-19 12:19:34 +08:00
Erno Kuvaja 3c57b78653 Bring py27 tests back
Since defaulting to py3 our 2.7 jobs have actually ran against python 3
due to basepython not declared on our 27 evironments. This patch will
change that.

Change-Id: I4d97c0577460949e8636e9f26b1d4b60b109f5d1
2019-06-13 15:49:37 +01:00
Erno Kuvaja 8aea36403d Don't target broken ssl tests to specific py3.minor
Change-Id: I37d881315e801926bd5a7ae46402038fd69e060b
2019-05-14 11:34:04 +00:00
ZhongShengping 8e16e3a857 Replace git.openstack.org URLs with opendev.org URLs
Thorough replacement of git.openstack.org URLs with their opendev.org
counterparts.

Change-Id: Ibe48a4ea537915bc6ce142812354b1d3b3ab1cf7
2019-04-28 09:26:48 +08:00
Chuck Short 0c9a419d8f
Switch to using stestr
According to Openstack summit session [1] stestr is maintained
project to which all Openstack projects should migrate.
Let's switch it then.

[1] https://etherpad.openstack.org/p/YVR-python-pti

Change-Id: I3c5d4e068ee5f2919a4061e8f7b78f724cd61724
Signed-off-by: Chuck Short <chucks@redhat.com>
2019-04-15 10:30:30 -05:00
Zuul 6cd76cb591 Merge "Separate out configuration tests" 2019-04-12 13:02:27 +00:00
Abhishek Kekane 79e4adaf43 Periodic jobs are failing for python 3.5
Periodic tips jobs are failing because python 3.5 interpreter
is not available on testing nodes. It is decided that Stein
branch should be tested with python 3.6 version.

Renamed periodic tips jobs to 3.6 and added python 3.6 enviornment
in tox.ini to run periodic tips jobs against python 3.6
environment.

Change-Id: I8a763eff7c6524969b1fabf98ab0a5624954d8b3
Closes-Bug: #1822238
2019-03-29 05:56:43 +00:00