Added required database API methods for cache related
operations.
Related blueprint centralized-cache-db
Change-Id: Iae1f1927566f4bda74c5aeccf464df820cfae9d4
Added expand script and model to define `cached_images` and
`node_reference` database tables.
Related blueprint centralized-cache-db
Change-Id: I9991a4e945bd4d231e942096445fd5374ff7ff2a
This aligns the metadef DB APIs with the rest of the APIs and simplifies
our life when we decide to move off the legacy engine facade in the
future.
Change-Id: Ifc1605e8e89c48b071ea505e40a9488df208fb37
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Align '_image_update' with 'image_update'. This caught me out once or
twice.
Change-Id: I78e005b51485a224e59c075a00798de5fa5a3896
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We already got rid of implicit autocommit. Now get rid of explicit
autocommit. This is far more involved than the prior patch but the basic
strategy here is that no public method should take a 'session' argument
and instead should call 'get_session'. In addition, all private methods
should take both a 'context' and 'session' argument and they should be
the first and second arguments, respectively. Finally, public methods
should only call private methods. Combined, this ensures we only ever
have one session in play and the transactions are correctly started and
ended as expected.
Change-Id: I5245c402699e179c219039514b626a52fa0028d8
Resolve the following RemovedIn20Warning warning:
The update.values parameter will be removed in SQLAlchemy 2.0. Please
refer to the Update.values() method.
Curiously this didn't raise an explicit warning and was only spotted
when updating to SQLAlchemy 2.0 itself.
Change-Id: I1ecb144553992966824420af2121f2f3d64836c6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Recent versions of Alembic are now able to detect the difference in
these types. Given only SQLite is affected and SQLite is only
supported for testing purposes, we simply modify the existing
migration rather than create a new migration.
Change-Id: Idea46240fa17f44442a1da0992d3e073a22205e6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following SQLAlchemy deprecation warning:
The convert_unicode on Engine and String as well as the unicode_error
flag on String are deprecated. All modern DBAPIs now support Python
Unicode natively under Python 2, and under Python 3 all strings are
inherently Unicode. These flags will be removed in a future release.
Change-Id: I0de54294cc311b32f0956942a62deb2e47729a4f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following SAWarning warning:
Class DeleteFromSelect will not make use of SQL compilation caching as
it does not set the 'inherit_cache' attribute to ``True``. This can
have significant performance implications including some performance
degradations in comparison to prior SQLAlchemy versions. Set this
attribute to True if this object can make use of the cache key
generated by the superclass. Alternatively, this attribute may be set
to False which will disable this warning.
This uses different arguments so the cache presumably does not make
sense.
Change-Id: I958ba79ca7ac6709e8c16eceb212f329c807aea8
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Rather than creating multiple connections, use the same connection but
multiple transactions. This should be slightly faster.
Change-Id: I02dc85729c3fbb6d775a32e3c2924e79abe3761c
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following RemovedIn20Warning warning:
The current statement is being autocommitted using implicit
autocommit, which will be removed in SQLAlchemy 2.0. Use the .begin()
method of Engine or Connection in order to use an explicit transaction
for DML and DDL statements.
This is the trickiest one hence it's left to last. We really need to get
off of the legacy EngineFacade pattern since it's harder to track
transactions when they're not local (the request context-based binding
of sessions allows us to decorate individual methods) but that's a
larger work item that we're not going to tackle here.
Change-Id: Ie16965a8e3e2f316b807068a868cee6fbb2f33ba
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We were doing this adhoc in tests, which doesn't seem to be possible in
SQLAlchemy 2.0. Do it everywhere instead.
This highlights a small issue that we have to workaround in the tests.
Change-Id: I9dd70c9976f83a00a8cb270422e545ff2c77f12f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following RemovedIn20Warning warning:
Passing a string to Connection.execute() is deprecated and will be
removed in version 2.0. Use the text() construct, or the
Connection.exec_driver_sql() method to invoke a driver-level SQL
string.
We also resolve two additional RemovedIn20Warning warnings along the
way, since we're already touching this code:
The Engine.execute() method is considered legacy as of the 1.x series
of SQLAlchemy and will be removed in 2.0. All statement execution in
SQLAlchemy 2.0 is performed by the Connection.execute() method of
Connection, or in the ORM by the Session.execute() method of Session.
The Executable.execute() method is considered legacy as of the 1.x
series of SQLAlchemy and will be removed in 2.0. All statement
execution in SQLAlchemy 2.0 is performed by the Connection.execute()
method of Connection, or in the ORM by the Session.execute() method of
Session.
Change-Id: I547bd2f441085fcdc5dc4a0e58d27d11e8a8b223
Resolve the following RemovedIn20Warning warning:
The ``aliased`` and ``from_joinpoint`` keyword arguments to
Query.join() are deprecated and will be removed in SQLAlchemy 2.0.
This one is a little trickier to grok, but the SQLAlchemy docs explain
it rather well [1] (Look for "Legacy Features of Query.join()" and
"Joins to a Target with an ON Clause"). Essentially, because we are
joining e.g. the ImageTag table multiple times, we need to create an
alias and use it for both the join and resulting queries.
[1] https://docs.sqlalchemy.org/en/14/orm/query.html#sqlalchemy.orm.Query.join
Change-Id: Ifb818df8ca8500adcf1354eff0671f5762473110
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following RemovedIn20Warning warnings:
The "whens" argument to case(), when referring to a sequence of items,
is now passed as a series of positional elements, rather than as a
list.
Change-Id: I4df3a52333d71809d6ff305a786ba6ae03a230ca
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following RemovedIn20Warning warnings:
Query.values() is deprecated and will be removed in a future release.
Please use Query.with_entities() (deprecated since: 1.4)
Change-Id: Id231036ae122c5878205fbb3d591cee53b817613
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following RemovedIn20Warning warnings:
Invoking and_() without arguments is deprecated, and will be
disallowed in a future release. For an empty and_() construct, use
and_(True, *args).
Change-Id: I7f34d8079975e57df904e59e325bee21ca1417ed
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We don't want alembic overriding our logging: we can configure it
ourselves. This removes a whole lot of noise from our test output.
Change-Id: Ic1bc4ab967913a8adaac559482d330d72e47a4b5
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following RemovedIn20Warning warnings:
The MetaData.bind argument is deprecated and will be removed in
SQLAlchemy 2.0.
The ``bind`` argument for schema methods that invoke SQL against an
engine or connection will be required in SQLAlchemy 2.0.
We are also able to resolve the following RemovedIn20Warning warning,
since the migration away from the bind parameter requires updating these
calls.
The autoload parameter is deprecated and will be removed in version
2.0. Please use the autoload_with parameter, passing an engine or
connection.
Similarly, we can also resolve the following RemovedIn20Warning warning,
since the calls we're updating here are the only ones affected by this
deprecation:
The insert.values parameter will be removed in SQLAlchemy 2.0. Please
refer to the Insert.values() method.
Change-Id: Ic956980a03e1a0e1b6f90c492d8a03a64ea275cc
Resolve the following RemovedIn20Warning warning:
The legacy calling style of select() is deprecated and will be removed
in SQLAlchemy 2.0. Please use the new calling style described at
select().
Change-Id: Iaebe4bf97b99a0f8f3bb8983ee0a3748f7cd482a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
These methods were not being tested. Add some basic tests to validate
loading and unloading of our stock definitions, preventing regressions
during e.g. the SQLAlchemy 2.0 preparation series.
Implementation of this test highlights an minor issue in the DB API,
whereby the default values for some methods with 'sort_key' and
'sort_dir' parameters are not valid. This means you can't call these
without overriding the parameters. This is corrected.
Change-Id: I6e76dcacc32f05a70757df28ab2f119b25da440b
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
'session' is a widely used variable name in the glance.db.sqlalchemy.api
module and this is only going to increase over time. This shadowing
prevents easy identification of e.g. missing function parameters. Alias
the import and update the one call to avoid the need to rename all other
instances of the variable name.
Change-Id: I6b2cbf0b97bb3af7354f2c10a8d83909db963d43
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Zuul