This has been deprecated for some time now.
Change-Id: Ia8b4ed8cd755d283bb773e55293457190b34c482
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
If an option is deprecated, there's a good chance we want to ignore it
unless explicitly requested by the user. Start ignoring options that are
deprecated and unset.
Change-Id: I3008562db52a5625504b262d7148528a832246dd
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-Bug: #2008720
Remaining issues encountered when running with SQLAlchemy 2.0 for real:
* Never call str() on a URL and expect it to be meaningful anymore.
The password is aggressively obfuscated now (users absolultely
wouldn't let us leave it as is)
* More utilities and fixtures that were calling begin() within a
block that would have already begun
* isnot is now called is_not; mocking "isnot" leads into too many
weird compat layers
* ORM InstrumentedAttribute and internals use __slots__ now, mock
seems to not be able to patch methods. Ideally these tests would use
a comparator subclass or something
* Connection.connection.connection is now called driver_connection,
SQLAlchemy keeps the old name available however oslo.db test suite
does not appear to tolerate the deprecation warning emitted,
so add a compat layer
* mapper() is fully removed from 2.0, not sure if there is another
not-yet-committed gerrit that removes mapper()
[1] https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.create_engine.params.pool_pre_ping
[2] https://docs.sqlalchemy.org/en/20/changelog/changelog_20.html#change-2fe37eaf2295cebd3bb4ee8e5b8c575c
[3] https://github.com/sqlalchemy/sqlalchemy/issues/5648
Change-Id: Ifaca67c07f008d8bc0febeecd3e200cc7ee7a4b0
The default value for wsrep_sync_wait option should be `None`.
However, since 0 is set incorrectly, an unintended process is executed.
This patch fixes default value for wsrep_sync_wait option to `None`
instead of `0`.
Change-Id: Ifb1dc7ddcb127a69ea01234922caa7ca5ab111ce
Resolve the following RemovedIn20Warning warning:
The Session.autocommit parameter is deprecated and will be removed in
SQLAlchemy version 2.0. The Session now features "autobegin" behavior
such that the Session.begin() method may be called if a transaction
has not yet been started yet. See the section session_explicit_begin
for background.
Change-Id: I7867cdcea115b13f2e45e0674bb9ef2ad138aae9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Resolve the following RemovedIn20Warning:
Calling the mapper() function directly outside of a declarative
registry is deprecated. Please use the
sqlalchemy.orm.registry.map_imperatively() function for a classical
mapping.
Change-Id: I92a7ccdd48eedd4c788384033743daf50a9dc113
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Resolve the following SADeprecationWarning warning:
The legacy calling style of select() is deprecated and will be removed
in SQLAlchemy 2.0. Please use the new calling style described at
select().
Change-Id: Ic5f7240e790425d2689c6870483748650a49bc3d
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The final step in cleaning up our many base classes. This will allow us
to do things consistently across the test suite in future changes.
Change-Id: I0bf663fdfd3c8be93e5658493e221d0a7db78832
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I3e1e5ed4e72837d45b78e5ccb9ce8cca416e5c11
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Today, when a user uses enginefacade reader/writer decorators
he/she is forced to pass the context argument as the
first positional argument. performing a decorated function
call with this argument passed as a keyword would lead to
an out of range exception.
This patch proposes to add support for the context argument
to be passed as a keyword argument in decorated functions.
e.g
@enginefacade.reader
def foo(context):
...
foo(context=ctxt) <- will now be possible
Change-Id: Ief0b71bf9a7eb75935612431bdcc26d33bce852d
Some module reloading scenarios such as that which occurs
within mod_wsgi mean that an existing module-level enginefacade
is already in the "started" state, however initialization
routines from the calling application may still attempt to
call the ``.configure`` method. Add a new flag is_started
to both _TransactionContextManager and _TransactionFactory so
that calling code can check for this state ahead of time;
additionally, promote the TypeError raised to a specific subclass
enginefacade.AlreadyStartedError to allow for better optimistic
schemes.
Change-Id: I2f5a9e35c2fae0c28b78beef3dcd2c4794362766
References: I704196711d30c1124e713ac31111a8ea6fa2f1ba
Raising warnings from an exception module can result in a rather
confusing message:
Running command: '/usr/bin/nova-scheduler '
/.../sqlalchemy/enginefacade.py:332: NotSupportedWarning: Configuration option(s) ['use_tpool'] not supported
exception.NotSupportedWarning
For someone not used to Python's 'warning' infrastructure, this can look
worse than it is, as seen at [1].
While this particular warning was resolved in change I4a5f8958c3e, it's
possible that a similar warning could be raised for other options. Best
to move this out of the 'exception' module and into something more
sensible.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1639423
Change-Id: I8e0480fb11123067a111ed6aeda6b47614615645
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Python 3.7 now makes "async" a keyword, so deprecate the
"async" name and replace with "async_". Additionally,
use attribute name access only in order to implement and
test "async" as it won't work under Python 3.7.
Closes-Bug: #1775866
Change-Id: Ifb2c59c2cfbd615b04570b31fcfe868f4ffea3c6
Downstream projects have started using the fixtures inside
of oslo_db.tests in response to the deprecations added in
I0163e637ffef6d45d2573ebe29b5438911d01fce. This was not
the intent of these deprecations, so add messsaging to
this effect and ensure fixtures used by the test suite
itself are noted as private.
Change-Id: I3afe0a440a3ab66904aaecf556948df9d4e16b8e
Repaired the "synchronous_reader" modifier of enginefacade so that it
refers to the "writer" engine when set to True, thereby allowing
"synchronous" behavior with the writer. When set to False, this is
"asynchronous", so this should be associated with the async engines.
The flag had the reverse behavior previously.
Change-Id: Id7fea7562ba90eb710176d497af103303f230531
Closes-bug: #1746116
In Python3, assertRegexpMatches & assertNotRegexpMatches
are deprecated in favor of assertRegex and assertNotRegex
Change-Id: I35fd27e9618dcc6e229267a9af215ca15a47db69
The "idle_timeout" name and documentation are misleading to users.
Rename the option to connection_recycle_time to more closely
match that of the underlying SQLAlchemy argument name
"pool_recycle", which refers to a time period for which
a connection present in the pool for longer than that number
of seconds will be closed and replaced with a new one *upon checkout*,
to prevent usage of stale connections that may have been closed by
server-side reapers.
Change-Id: I5e39d5ac81eaa8d85b1dfb18862fe27695ab8b96
Maybe we need to know the original exception when exception occurs on
method 'session.rollback()'.
Change-Id: Iccda37a3279d6f24de69cfccef3e29787f1ac694
Closes-Bug: #1684329
Attaching a reference to the context to the session/connection
is useful for SQLAlchemy event handlers that only get access to
the session and need access to the context for additional details.
The use case driving this is the related bug where we want to pass
conditional update constraints set in the context via the HTTP API
down into an SQLAlchemy event listener.
Related-Bug: #1493714
Change-Id: I5c08e672782344e8778187e257e03c1a1d1b019a
When converting existing session context manager code to
use enginefacade, normally a flush() is implicit on
a "subtransaction" when it closes. Enginefacade
has not maintained this behavior. Ideally, this should
have been on by default when enginefacade was first
released, but for now add this as a behavioral
option for projects that require it.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Closes-Bug: #1664643
Change-Id: I6f4f439928588cff954e749dfa938425892e0931
Fixed two issues in patch_engine() as used in test suites
where facade-level arguments would not be propagated to the
internal _TestTransactionFactory, and also if the parent
factory weren't "started" the patch operation would fail
as it assumes a started factory.
This removes the "synchronous_reader" argument from
_TestTransactionFactory as this is propagated from the parent
_TransactionFactory, where synchronous_reader defaults to True.
However, arbitrary keyword arguments are currently allowed to
allow for outside projects calling _TestTransactionFactory
(currently Neutron).
Change-Id: I15f85a0ebfd732c09d7cb92d885c1773c393aabd
The original idea of enginefacade was that applications
would call upon the global _TransactionContextManager
given in oslo_db.sqlalchemy.enginefacade. However, as it
turns out, virtually no Openstack projects seem to be
using that technique, and instead, everyone is creating
their own ad-hoc _TransactionContextManager objects
and usually establishing it as a module-level global.
Nova has two of them.
Additionally, projects add configuration to these
enginefacades (which IS part of the original idea), and
this configuration in some cases is necessary to be present
for tests that run as well, a key example being the
sqlite_fks flag. The original DbFixture integration
provided no way of reusing this configuration.
Finally, projects very much tend to use custom fixtures
in order to define their database communication.
Test classes themselves don't really make use of
oslo_db's DbTestCase anymore.
This patch introduces a modernized fixture system
which, in conjunction with the recent provisioning
patch, addresses these use cases. Applications will typically
create their own subclasses of these fixtures up front
to suit the various testing cases they have, including
SQLite fixed, SQLite ad-hoc, and opportunistic.
In order to accommodate the fixture-based flow
along with the use of testresources for opportunistic
database provisioning, a mixin class OpportunisticDbTestMixin
is still needed when a test needs to use "opportunistic"
testing in order to provide the .resources attribute.
The calculation of .resources is moved into the fixture
system, but because this attribute is consulted before
setUp(), the "opportunistic" fixture must be created
early and stored.
Closes-Bug: #1548960
Change-Id: I0163e637ffef6d45d2573ebe29b5438911d01fce
Adds new methods make_new_manager(), patch_factory(),
patch_engine(), to provide support to test fixtures that
wish to generate new managers from an existing one, and to
cross-patch managers, factories and engines into other managers.
Change-Id: I83c6296d992f08117fff1438bc079bbf71db787b
Add on_engine_create argument to transaction_context to enable users
to take some actions with newly created engines. Main intended usage
is providing oslo.db and osprofiler users with a way to hook them
together without making them depend on each other.
Closes-bug: #1600739
Change-Id: I78bef4979c2000d05658ce17d0348cd0a10c24d9
Many assertEqual sentences don't follow assertEqual(expected, actual),
These misplaces have 2 impacts:
1, giving confusing messages when some tests failed.
2, mislead other developers, new test modules may follow these wrong pattern.
This patch fix all of them.
Change-Id: Ia5155114c60f5e27cb00b8aba6de5cf32e046dd7
Closes-Bug: #1604213
Openstack applications such as Neutron will refer to
the engine.pool.dispose() method in order to remove
any remaining connections, typically as part of
a test suite. Add a public-facing method to the context
and its underlying factory so this can be called without
underscore access.
Change-Id: I18b16e002a8507eb2eaa3a7f168e853b0ca17aa5
The decorator form can now be used with bound methods after a fix
for the related bug was merged. Mention this in docs and release
notes, so that people are aware of it.
A new test is added to make sure this also works with class methods
(as well as to check that the proposed decorator applying order
is actually correct).
Related-Bug: #1520195
Change-Id: Ifea08114d6d89de9d67fcae397eb94c0afc4d339
The previous behavior was to assume that a RequestContext was the first
argument of the wrapped method. This has meant creating staticmethods
within classes in order to be wrapped properly. It's not always
desirable to do so and should not be necessary.
Change-Id: I76b0d9c80dd7841c1333f8b8a2ccdd15e55dfcff
Closes-bug: 1520195
In Nova we need a decorator that is READER by default
but can be used as ASYNC if asynchronous session is started before.
Co-Authored-By: Sergey Nikitin <snikitin@mirantis.com>
Change-Id: I0c0f332d87bf6b1213c800763918340b1e226934
When LegacyEngineFacade is passed a _TransactionFactory
that is not started, or one that fails to start, the
_start() method should be called again each time the
factory is called upon; this ensures that the failed
start either can be made to succeed, or at least re-raises
the same error as earlier, instead of falling into attribute
access for non-existent attributes.
Change-Id: I7484f837d455c1f69c448e5ccadc277d19af47e7
Closes-Bug: #1515326
This public method was added to be able to create several
legacy facades from several transaction context managers.
It could be used in Nova where we have two DBs.
There we need some legal way to create two legacy facades
because right now in Nova we create facades by using
private field _factory. It is not quite right.
Change-Id: I96b3dbcbc5a05ac025c247ae6d8abfc988d0295e
In order to pickle a context object which refers
to an active _TransactionContextTLocal, we have to
use __reduce__ so that no database, engine, or session
state is pickled, and only a constructor remains. Upon
unpickling, the constructor will install a new, empty
_TransactionContextTLocal on the new context, as is
appropriate since this is a totally different context object
than the original one.
Change-Id: Ia50cf10ff91a013ee24773c095a3df69ae06847b
Check explicitly for the cases where no "sql_connection"
attribute was set when running _start(), so that the
lack of this parameter is documented by the
exception rather than failing into create_engine()
with an unclear failure mode.
If _start() fails as it will here, make sure _started
stays False so that repeated calls to _start() continue
to raise the same exception, rather than raising
attribute errors. When accessing the "session" or
"connection" attributes of the context when these
attributes were not requested by the decorator or
context manager, raise explicit exceptions
for each, rather than returning None which leads to
hard-to-debug NoneType errors.
Change-Id: Iadfbf4707daed4140285a3a472009f6863b18275
Closes-bug: 1477080
Some projects are calling EngineFacade._session_maker for
some use cases. To suit these cases more cleanly on the legacy
API, add a public get_sessionmaker() method.
Change-Id: Ica9574d70a6b8be80bd1a5e7cf250828455c5ade
This change removes RequestContext implicit decoration as some
applications have their own session management for the moment or don't
need this feature. Moreover this implicit decoration doesn't help
debugging.
This change doesn't disallow RequestContext decoration but applications
should require it explicitly:
from oslo_db.sqlalchemy import enginefacade
enginefacade.transaction_context_provider(oslo_context.RequestContext)
Closes-Bug: #1468707
Change-Id: I143f30c41e788c7aa9887c0e994f49ee55c94651
This module presents a replacement for the EngineFacade
system. At the center is the oslo.db.sqlalchemy.enginefacade
module, which when imported, provides decorators and context
managers which perform all database and ORM connectivity
functions transparently. The docstrings as well
as the blueprint provide an introduction.
The patch includes a refactoring of sqlalchemy/session.py
into three dependent modules engines.py, orm.py and
enginefacade.py. This is to maintain a non-cyclical import
structure as well as to maintain the import behavior of
oslo.db overall, as some projects such as glance currently
have dependencies on this structure.
There is also a slimming down and attempt at modernizing
some very old documentation in session.py. The enginefacade
system should be preferred moving forward.
Implements: blueprint make-enginefacade-a-facade
Change-Id: I9a3d0c26bb727eb2c0bd823b9a12fde57cc7c9c3
Stephen Finucane