Commit Graph

17 Commits

Author SHA1 Message Date
Stephen Finucane ede0f9c1b4 db: Don't inherit cache for DeleteFromSelect
Resolve the following SAWarning warning:

  Class DeleteFromSelect will not make use of SQL compilation caching as
  it does not set the 'inherit_cache' attribute to ``True``.  This can
  have significant performance implications including some performance
  degradations in comparison to prior SQLAlchemy versions.  Set this
  attribute to True if this object can make use of the cache key
  generated by the superclass.  Alternatively, this attribute may be set
  to False which will disable this warning.

This uses different arguments so the cache presumably does not make
sense.

Change-Id: I958ba79ca7ac6709e8c16eceb212f329c807aea8
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-25 09:05:44 +00:00
Stephen Finucane 8c5091cd48 db: Drop use of implicit auto-commit
Resolve the following RemovedIn20Warning warning:

  The current statement is being autocommitted using implicit
  autocommit, which will be removed in SQLAlchemy 2.0. Use the .begin()
  method of Engine or Connection in order to use an explicit transaction
  for DML and DDL statements.

This is the trickiest one hence it's left to last. We really need to get
off of the legacy EngineFacade pattern since it's harder to track
transactions when they're not local (the request context-based binding
of sessions allows us to decorate individual methods) but that's a
larger work item that we're not going to tackle here.

Change-Id: Ie16965a8e3e2f316b807068a868cee6fbb2f33ba
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-25 10:03:11 +01:00
Stephen Finucane ce5056ad3a db: Replace dict-style access of Row attributes
Resolve the following RemovedIn20Warning:

  Using non-integer/slice indices on Row is deprecated and will be
  removed in version 2.0; please use row._mapping[<key>], or the
  mappings() accessor on the Result object.

Change-Id: Ife8bdf766eb6d6fe9748565c9c61aa971d1af6f8
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 21:34:11 +00:00
Stephen Finucane 25dcacea7c db: Don't pass strings to Connection.execute()
Resolve the following RemovedIn20Warning warning:

  Passing a string to Connection.execute() is deprecated and will be
  removed in version 2.0.  Use the text() construct, or the
  Connection.exec_driver_sql() method to invoke a driver-level SQL
  string.

We also resolve two additional RemovedIn20Warning warnings along the
way, since we're already touching this code:

  The Engine.execute() method is considered legacy as of the 1.x series
  of SQLAlchemy and will be removed in 2.0. All statement execution in
  SQLAlchemy 2.0 is performed by the Connection.execute() method of
  Connection, or in the ORM by the Session.execute() method of Session.

  The Executable.execute() method is considered legacy as of the 1.x
  series of SQLAlchemy and will be removed in 2.0. All statement
  execution in SQLAlchemy 2.0 is performed by the Connection.execute()
  method of Connection, or in the ORM by the Session.execute() method of
  Session.

Change-Id: I547bd2f441085fcdc5dc4a0e58d27d11e8a8b223
2023-07-24 21:34:03 +00:00
Stephen Finucane 44050a2f75 db: Replace use of aliased keyword arguments
Resolve the following RemovedIn20Warning warning:

  The ``aliased`` and ``from_joinpoint`` keyword arguments to
  Query.join() are deprecated and will be removed in SQLAlchemy 2.0.

This one is a little trickier to grok, but the SQLAlchemy docs explain
it rather well [1] (Look for "Legacy Features of Query.join()" and
"Joins to a Target with an ON Clause"). Essentially, because we are
joining e.g. the ImageTag table multiple times, we need to create an
alias and use it for both the join and resulting queries.

[1] https://docs.sqlalchemy.org/en/14/orm/query.html#sqlalchemy.orm.Query.join

Change-Id: Ifb818df8ca8500adcf1354eff0671f5762473110
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 21:33:58 +00:00
Stephen Finucane e5a8918e87 db: Pass case.whens as positionals, not a list
Resolve the following RemovedIn20Warning warnings:

  The "whens" argument to case(), when referring to a sequence of items,
  is now passed as a series of positional elements, rather than as a
  list.

Change-Id: I4df3a52333d71809d6ff305a786ba6ae03a230ca
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 17:08:56 +00:00
Stephen Finucane c70349d823 db: Replace use of 'Query.values()'
Resolve the following RemovedIn20Warning warnings:

  Query.values() is deprecated and will be removed in a future release.
  Please use Query.with_entities() (deprecated since: 1.4)

Change-Id: Id231036ae122c5878205fbb3d591cee53b817613
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 17:08:42 +00:00
Stephen Finucane 36329c0a75 db: Don't invoke and_() without arguments
Resolve the following RemovedIn20Warning warnings:

  Invoking and_() without arguments is deprecated, and will be
  disallowed in a future release. For an empty and_() construct, use
  and_(True, *args).

Change-Id: I7f34d8079975e57df904e59e325bee21ca1417ed
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 17:08:34 +00:00
Stephen Finucane 1f5bbf0bd1 db: Remove use of 'bind' arguments
Resolve the following RemovedIn20Warning warnings:

  The MetaData.bind argument is deprecated and will be removed in
  SQLAlchemy 2.0.

  The ``bind`` argument for schema methods that invoke SQL against an
  engine or connection will be required in SQLAlchemy 2.0.

We are also able to resolve the following RemovedIn20Warning warning,
since the migration away from the bind parameter requires updating these
calls.

  The autoload parameter is deprecated and will be removed in version
  2.0.  Please use the autoload_with parameter, passing an engine or
  connection.

Similarly, we can also resolve the following RemovedIn20Warning warning,
since the calls we're updating here are the only ones affected by this
deprecation:

  The insert.values parameter will be removed in SQLAlchemy 2.0.  Please
  refer to the Insert.values() method.

Change-Id: Ic956980a03e1a0e1b6f90c492d8a03a64ea275cc
2023-07-24 17:47:39 +01:00
Stephen Finucane 6c150194c6 db: Replace use of legacy select() calling style
Resolve the following RemovedIn20Warning warning:

  The legacy calling style of select() is deprecated and will be removed
  in SQLAlchemy 2.0.  Please use the new calling style described at
  select().

Change-Id: Iaebe4bf97b99a0f8f3bb8983ee0a3748f7cd482a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 17:45:39 +01:00
Stephen Finucane d409b27a79 tests: Enable SQLAlchemy 2.0 deprecation warnings
Well, sort of. We enable them but immediately filter out the ones we're
actually seeing, the rationale being that we can address these in a
piecemeal fashion without the risk of introducing new issues.

There's a lot more to be done here. However, the work done in oslo.db
[1] and other projects should provide a guide for how to resolve the
outstanding issues.

[1] https://review.opendev.org/q/topic:%2522sqlalchemy-20%2522+project:openstack/oslo.db

Change-Id: I7a59714b104659b64f46f4c1437cfc4018356815
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 17:29:01 +01:00
Stephen Finucane cbabd5757d tests: Restore - don't reset - warning filters
There are more various warning filters pre-configured in a typical
Python environment, including a few from third-party libraries such as
requests [1][2] and urllib3 [3] as well as stdlib [4]. Our fixture to
configure warnings, 'WarningsFixture', called 'warnings.resetwarnings'
which *reset* all the warning filters [5]. This is clearly not something
we want to do, and resulted in tests puking warnings after the initial
test run.

Resolve this by backing up the existing warning filters before applying
the filter, and then *restoring* this original list of warning filters
after the test run.

This was previously done in Nova [6].

[1] https://github.com/psf/requests/blob/v2.26.0/requests/__init__.py#L127
[2] https://github.com/psf/requests/blob/v2.26.0/requests/__init__.py#L152
[3] https://github.com/urllib3/urllib3/blob/1.26.7/src/urllib3/__init__.py#L68-L78
[4] https://docs.python.org/3.8/library/warnings.html#default-warning-filter
[5] https://docs.python.org/3.8/library/warnings.html#warnings.resetwarnings
[6] https://review.opendev.org/c/openstack/nova/+/817641

Change-Id: I3808640c4b99f6aba4bdae6e36bbada3b3a8ff84
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 16:35:36 +01:00
Dan Smith 06e6542f15 Add unified quotas infrastructure
This adds some infrastructure to be able to query and honor limits
declared in keystone. It adds a single initial quota value for the
total size of all active images for bootstrapping the tests.

Checking these values is controlled by a new configuration option
that globally enables and disables the checking, defaulting to
False.

Related to blueprint glance-unified-quotas
Change-Id: I8d8f4aaed465486e80be85bc9a5d2c2be7f1ecad
2021-06-21 10:58:55 -07:00
Hervé Beraud 64e3c41167 Stop to use the __future__ module.
The __future__ module [1] was used in this context to ensure compatibility
between python 2 and python 3.

We previously dropped the support of python 2.7 [2] and now we only support
python 3 so we don't need to continue to use this module and the imports
listed below.

Imports commonly used and their related PEPs:
- `division` is related to PEP 238 [3]
- `print_function` is related to PEP 3105 [4]
- `unicode_literals` is related to PEP 3112 [5]
- `with_statement` is related to PEP 343 [6]
- `absolute_import` is related to PEP 328 [7]

[1] https://docs.python.org/3/library/__future__.html
[2] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html
[3] https://www.python.org/dev/peps/pep-0238
[4] https://www.python.org/dev/peps/pep-3105
[5] https://www.python.org/dev/peps/pep-3112
[6] https://www.python.org/dev/peps/pep-0343
[7] https://www.python.org/dev/peps/pep-0328

Change-Id: I82b61a2d7687ff109051815e01619eb612ea9073
2020-06-02 20:14:31 +02:00
Matt Riedemann 88a8ad7823 Fix DeprecationWarnings for RequestContext.tenant/user usage
RequestContext.tenant and user fields are deprecated in favor
of project_id and user_id respectively.

This change modifies the glance.context.RequestContext constructor
to transition usage of tenant/user to project_id/user_id until
all tests are moved over to the new attributes. Runtime usage of
the old fiels is updated.

To prevent new code from using the deprecated fields, a warnings
filter is added which will make tests fail if they hit code using
the old fields.

Co-Authored-By: Abhishek Kekane <akekane@redhat.com>
Change-Id: I351380840308a24769ece93abc6d1a9a6d6aa06f
2019-02-06 09:59:36 -05:00
Sean McGinnis 3b5ecbe032 Add fixture to limit unit test log output
This follows changes in Nova and Cinder to add a logging fixture that
allows the log level to be raised to limit the amount of junk output
in the unit test logs, while still processing debug level logs to
catch formatting error issues.

Change-Id: If740108db5291ff3fe0963127f9a3efadf3346c3
2018-04-09 21:17:54 +00:00
Sean McGinnis 96334ad951 Add fixture to only emit DeprecationWarning once
We have a ton of DeprecationWarning messages in our unit test runs.
Most of these are out of our control from third party libs. This
adds a WarningsFixture to limit warning output to once per test
run. In local py35 unit testing, this went from 14549 warnings to
8913.

Also including ignorning a policy 'is_admin' deprecation warning
that was added before a clear plan or replacement had been put
in place. Other projects have added this rather than fixing it
at the source since it is currently being reworked.

Based on work previously done in Nova and Cinder.

Change-Id: I4d97f74ed37b7b0e9a613ecfe33c4b26216ca768
2017-12-12 15:49:04 +00:00