Currently InternalError is being handled for detecting MariaDB/Galera
deadlocks but recently we have seen a variant that raises
OperationalError instead. Because it's not being detected, usage of the
@retry_on_deadlock decorator is not performing retries in those cases.
This adds handling of OperationalError for detecting this deadlock.
Closes-Bug: #2057987
Change-Id: I6ff3667b35ea38a2d3c258f810a55eda9abe465e
The previous attempt did not update the version in pre commit config
so the old version is still used by pep8 target.
Change-Id: Ib4b2ee82bab8410b0b1f3a1e8a4e53cfb165e1d5
So we know if we can remove this in the future, or at least have context
around why it's there.
Change-Id: I4e4bdcd6a01e8c1071045bbe1310cbdf4195ec27
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This has been deprecated for some time now.
Change-Id: Ia8b4ed8cd755d283bb773e55293457190b34c482
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Resolve the following SAWarning warning:
SAWarning: Attribute '_expr_to_appease_mox' on class <class
'oslo_db.tests.sqlalchemy.test_utils.FakeTable'> appears to be a
non-schema SQLAlchemy expression object; this won't be part of the
declarative mapping. To map arbitrary expressions, use
``column_property()`` or a similar function such as ``deferred()``,
``query_expression()`` etc.
Change-Id: I7baac158e5987d95f6272d9849c5ec6a2071a25f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We're seeing a weird condition whereby the exceptions raised by our
exception handler and being "rehandled", resulting in 'DBError' being
raised instead of e.g. 'DBReferenceError'. This is hard to reproduce and
thus far has only be spotted in Barbican unit tests. Prevent the issue
by checking if the exception being handled is one of our own and skip
handling it if so.
Change-Id: Ibd3f665a3ed3aedf9d1f33edcab35a46c27ea3dc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Alembic was previously modifying the metadata server default when
comparing it in the autogenerate process. This was removed in Alembic
1.11. As noted in the commit message [1], we implement our own
'compare_server_default' which means we now need to handle this.
[1] https://github.com/sqlalchemy/alembic/commit/230a2932f6
Change-Id: Ib5f8f78b917d79c4c234cc689c13acff011c2403
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
If an option is deprecated, there's a good chance we want to ignore it
unless explicitly requested by the user. Start ignoring options that are
deprecated and unset.
Change-Id: I3008562db52a5625504b262d7148528a832246dd
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-Bug: #2008720
In change I1e71150ba6daeba464b6ed8d46163f1f34959db3 we removed the
legacy base test classes, first deprecated in 2015. We forgot to include
a release note, however. Address this now.
Change-Id: I4d66f0308b89a187143ef6c8495383fe60043c14
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change I8629225eeb51d95264d8a3e4b719268bb1597f4f bumped the minimum
version for SQLAlchemy to 1.4, meaning this logic is now dead. Remove
it.
Change-Id: I4d4a58e15e840ecfa63e15c709617a65642c8323
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We were creating regexes without the DOTALL flag, which meant '.' wasn't
matching newlines. This meant exceptions that contained multiple lines
would not be caught. For example, in my environment where Kerberos is
used, I see the following otherwise harmless message:
(psycopg2.OperationalError) connection to server at "localhost" (::1),
port 5432 failed: could not initiate GSSAPI security context: No
credentials were supplied, or the credentials were unavailable or
inaccessible: Configuration file does not specify default realm
connection to server at "localhost" (::1), port 5432 failed: FATAL:
database "non_existent_database" does not exist
The presence of that newline causes our matchers to fail and the
exception is not wrapped. Correct this.
In the meanwhile, we reformat the function that does the wrapping to
make it a little flatter. This was difficult to modify (for debugging
purposes) due to the level of indentation.
Change-Id: I5396a5a3272e6984954d819cfc71507283c775db
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This should have been removed in change
Ic3d6bd318038d723b0d50d39e45f8e26289e9a57 but was missed.
Change-Id: I4e1faa2c617ac19e7c9766e99ee9012ad9298d31
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Mock it out, reducing test run of ~3 and ~15 seconds to milliseconds.
Change-Id: Ice3a0c0d0a5b8c2920c7f775ff8ce974b572c66e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This functionality has been available upstream since SQLALchemy 1.2 [1].
However, for oslo.db to use this feature while maintaining full
behavior, we need at least SQLAlchemy 2.0.5 to provide complete event
support. In particular, oslo.db adds several new "is disconnect"
conditions including those specific to Galera.
Behavior of the handle_error event is modified to expect the "pre-ping"
calling form, which may not have an "engine" present (only a dialect),
and additionally takes advantage of the new is_pre_ping attribute which
informs on the correct way to affect the disconnection status within the
ping handler.
Change-Id: I50d862d3cbb126987a63209795352c6e801ed919
Remaining issues encountered when running with SQLAlchemy 2.0 for real:
* Never call str() on a URL and expect it to be meaningful anymore.
The password is aggressively obfuscated now (users absolultely
wouldn't let us leave it as is)
* More utilities and fixtures that were calling begin() within a
block that would have already begun
* isnot is now called is_not; mocking "isnot" leads into too many
weird compat layers
* ORM InstrumentedAttribute and internals use __slots__ now, mock
seems to not be able to patch methods. Ideally these tests would use
a comparator subclass or something
* Connection.connection.connection is now called driver_connection,
SQLAlchemy keeps the old name available however oslo.db test suite
does not appear to tolerate the deprecation warning emitted,
so add a compat layer
* mapper() is fully removed from 2.0, not sure if there is another
not-yet-committed gerrit that removes mapper()
[1] https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.create_engine.params.pool_pre_ping
[2] https://docs.sqlalchemy.org/en/20/changelog/changelog_20.html#change-2fe37eaf2295cebd3bb4ee8e5b8c575c
[3] https://github.com/sqlalchemy/sqlalchemy/issues/5648
Change-Id: Ifaca67c07f008d8bc0febeecd3e200cc7ee7a4b0
The SQLAlchemy method ``url.make_url`` accepts strings and ``url.URL``
object types. By default, oslo.db was converting any object to string
before calling ``url.make_url``, that parses this string. Since
SQLAlchemy 2.0, the ``url.URL.__str__`` method is removed and the
``url.URL.__repr__`` method returns a string with the password hidden.
The new utility method checks what type of object is passed and only
if the object is not a string nor a ``url.URL`` object, is converted
to a string.
Closes-Bug: #2012928
Change-Id: I84f13f378f83e2a55078370ae2b4787f00982c23
The fixtures have been available for a very long time now. We can drop
these.
Change-Id: I1e71150ba6daeba464b6ed8d46163f1f34959db3
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
sqlalchemy-migrate does not support SQLAlchemy 2.x and we're not going
to invest the time in changing this. Remove integration of
sqlalchemy-migrate in oslo.db, allowing us to support SQLAlchemy 2.x
fully.
Note that we do not remove the 'migration_cli' module entirely yet.
While this is deprecated, it is possible to use this with alembic.
New users shouldn't be switching to it, but any existing users can
continue to use this module for some time yet.
Change-Id: Ic3d6bd318038d723b0d50d39e45f8e26289e9a57
Sem-Ver: api-break
In the method ``engines._connect_ping_listener``, the connection should
be rolled back after the ping execution. The rollback will revert the
transaction and delete it.
Closes-Bug: #2008209
Change-Id: Iba29ded227634e02795052acfd89b572bf21f54c
The default value for wsrep_sync_wait option should be `None`.
However, since 0 is set incorrectly, an unintended process is executed.
This patch fixes default value for wsrep_sync_wait option to `None`
instead of `0`.
Change-Id: Ifb1dc7ddcb127a69ea01234922caa7ca5ab111ce
When using Galera, the wsrep_sync_wait option [1]
can change the behavior of a variety of Galera DQL/DML statements
such that a particular operation will pause until outstanding
write-sets are fully persisted to the local node. The setting
supersedes the previous boolean parameter wsrep_causal_reads
which only affected SELECT, with an updated approach that allows
for fine-grained control of so-called "causality checks"
on individual statement types. The legacy-compatible setting
of '1' indicates that READ/SELECT/BEGIN operations should
proceed only after any pending writesets are fully available.
The use case for this setting is for an application that
is running operations on multiple Galera nodes simultaenously.
An application that commits data on one node, and then immediately
uses a different connection (on a potentially different node)
to SELECT that data, may fail to see those changes if
"causality checks" for SELECT are not enabled. While
a COMMIT operation in Galera will block locally until all other
nodes approve of the writeset, the operation does not block
for the subsequent period of time when other nodes are actually
persisting that writeset. Setting up "causal reads"
in this case indicates that a SELECT operation will wait until
any writesets in progress are available, thus maintaining
serialization between the COMMIT and subsequent SELECT.
As the name implies, wsrep_sync_wait adds...waiting! to the
operation, and thus directly impacts performance by adding
latency to SELECT operations or to the operations that have
been selected for causality checks, to the degree that
concurrent writesets are expected to be present.
Since it's not expected that most if any Openstack applications
actually need this setting in order to be effective with
Galera multi-master operation, and as the setting is available
within client session scope and also impacts performance,
making it available on a per-application basis means that
specific applications which may see issues under load can
choose to enable this setting, much in the way any other
"transaction isolation" settings might be made, without having
to add a cluster-wide performance penalty by setting it at the
Galera server level.
[1] https://mariadb.com/docs/ent/ref/mdb/system-variables/wsrep_sync_wait/
Change-Id: Iee7afcac8ba952a2d67a9ad9dd0e4eae3f42518e
Avoids the following warning:
SAWarning: Could not assemble any primary keys for locally mapped
table 'fake_table_inh' - no rows will be persisted in this Table.
class FakeTableJoinedInh(FakeTable):
Change-Id: Ia7795be05e6364d8b2d70be70f5154094a55161a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
It seems postgres-client has changed the format of error messages.
Previously we saw messages like:
fatal: database "non_existent_database" does not exist
These are now prefixed. For example:
connection to server at "localhost" (::1), port 5432 failed: fatal:
database "non_existent_database" does not exist
You can see this in the docs. Compare the "Client Connection Problems"
section for Postgres 13 [1] to Postgres 14 [2].
[1] https://www.postgresql.org/docs/13/server-start.html
[2] https://www.postgresql.org/docs/14/server-start.html
Change-Id: Id2c8eec202d128d142b8a8a8f904fcc14b6f52d7
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-bug: #1989208
The 'assert_has_calls' is a method, not a variable.
Change-Id: I6aea978b8fc9adf4a062c149c84d6562f5adec04
Closes-Bug: 1989280
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
In change Idf6fd858fad9521c7c5ba82c31b6d3077756abd9, we defined the
'cache_ok' key on a number of custom types. However, we relied on this
attribute being inheritable which is not the case due to how that
attribute is checked for [1]. As a result, we must set this attribute on
every one of our custom types.
[1] https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_41/lib/sqlalchemy/sql/type_api.py#L984
Change-Id: I005af8c7afe3b5104068ca153aecbfe54e163ca3
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Traditionally, the MySQL support in oslo.db has assumed use of the
InnoDB storage engine. However, this isn't the only storage engine
available and a few years ago an effort was made to add support for
another storage engine, MySQL Cluster (NDB). The oslo.db aspects of this
effort were tracked via bug 1564110 [1] and from reading this bug and
looking at other patches related to this effort [2], it becomes obvious
that this was never seen through to the completion and the
OpenStack-wide effort never took off [3]. As a result, much of what is
here is in-effect dead code now.
Given no one is using this engine, there's no reason to keep it around.
Deprecate it with an eye on removing it sooner rather than later.
[1] https://bugs.launchpad.net/oslo.db/+bug/1564110
[2] https://review.opendev.org/q/owner:octave.orgeron%2540oracle.com
[3] https://review.opendev.org/c/openstack/openstack-specs/+/429940
Change-Id: Id5ddf1d6f47b8a572001f58ad8b9b8a7dbe4e8ac
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We've deprecated a number of modules recently. We don't need to emit
these warnings when running unit tests. Silence things.
Change-Id: I7aed7789584bf0070f11c22b5eaa0e80c42dfc9c
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We shouldn't be raising warnings from SQLAlchemy. Where we are
intentionally doing so, we should capture these warnings at the test
level. This requires some minor fixes.
Change-Id: I9d4512dc337153edc48a2cc3bf95ab2b31c39ccf
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Resolve the following RemovedIn20Warning warning:
The Session.autocommit parameter is deprecated and will be removed in
SQLAlchemy version 2.0. The Session now features "autobegin" behavior
such that the Session.begin() method may be called if a transaction
has not yet been started yet. See the section session_explicit_begin
for background.
Change-Id: I7867cdcea115b13f2e45e0674bb9ef2ad138aae9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This was missed in change Ib789cd4d11a3d5dd01fcdb99822025b11bbc234e
("Don't rely on implicit autocommit")
Change-Id: I9ec27650ae5e36099a6d2b2d59bb66cd820e8ffc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Resolve the following RemovedIn20Warning warning:
Calling .begin() when a transaction is already begun, creating a 'sub'
transaction, is deprecated and will be removed in 2.0. See the
documentation section 'Migrating from the nesting pattern' for
background on how to migrate from this pattern.
Change-Id: I59e45dfea8fcbf72fc6e34345e510554cbdd138e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
These are not compatible with SQLAlchemy 2.0 due to their reliance on
nested transactions. We should deprecate them first but doing so would
push the boat out further wrt how long we have to wait before achieving
compatibility with this new version.
Change-Id: If3db4e8c1b681c0c62d3f04a57f92802639b3b9b
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
melanie witt