Commit Graph

1387 Commits

Author SHA1 Message Date
melanie witt 8e1f869910 exc_filters: Handle OperationalError for MariaDB/Galera
Currently InternalError is being handled for detecting MariaDB/Galera
deadlocks but recently we have seen a variant that raises
OperationalError instead. Because it's not being detected, usage of the
@retry_on_deadlock decorator is not performing retries in those cases.

This adds handling of OperationalError for detecting this deadlock.

Closes-Bug: #2057987

Change-Id: I6ff3667b35ea38a2d3c258f810a55eda9abe465e
2024-03-15 02:16:17 +00:00
Zuul 3a94baa0e2 Merge "reno: Update master for unmaintained/xena" 2024-03-12 14:32:08 +00:00
Zuul a555bae981 Merge "reno: Update master for unmaintained/wallaby" 2024-03-12 14:24:26 +00:00
Zuul 07fd65099d Merge "reno: Update master for unmaintained/victoria" 2024-03-12 13:11:35 +00:00
OpenStack Release Bot 3e2dab780c Update master for stable/2024.1
Add file to the reno documentation build to show release notes for
stable/2024.1.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2024.1.

Sem-Ver: feature
Change-Id: I66efd50c02dc7aa582a6398f23e8230221baa912
2024-03-08 14:22:17 +00:00
OpenStack Release Bot add9b72dc8 reno: Update master for unmaintained/xena
Update the xena release notes configuration to build from
unmaintained/xena.

Change-Id: I1009da8cc0784d9a6834760ca7e5e454915ef7b5
2024-03-07 14:25:26 +00:00
OpenStack Release Bot f1e544d72e reno: Update master for unmaintained/wallaby
Update the wallaby release notes configuration to build from
unmaintained/wallaby.

Change-Id: I622a1a0523fc5143505807ca7fe14c41fefdc8a5
2024-03-07 14:21:12 +00:00
OpenStack Release Bot 249350b55f reno: Update master for unmaintained/victoria
Update the victoria release notes configuration to build from
unmaintained/victoria.

Change-Id: I96da4f7b629ce62726b53cff686f7e5d319db6ba
2024-03-07 14:15:37 +00:00
Zuul 3e1618fb99 Merge "Display coverage report" 2024-02-07 06:21:16 +00:00
Takashi Kajinami c66bc22b3e Display coverage report
... for easy reference. Also make sure old data is purged.

Change-Id: I4de110655db80f111fa88bdcdbc0665ebbc40cfc
2024-02-07 12:13:08 +09:00
OpenStack Release Bot 41fcc96812 reno: Update master for unmaintained/yoga
Update the yoga release notes configuration to build from
unmaintained/yoga.

Change-Id: I1ec6eec49561f235091236aa46c5fc01a8d54f9b
2024-02-06 14:27:51 +00:00
Takashi Kajinami 05d8f82c80 Bump hacking (again)
The previous attempt did not update the version in pre commit config
so the old version is still used by pep8 target.

Change-Id: Ib4b2ee82bab8410b0b1f3a1e8a4e53cfb165e1d5
2024-01-26 10:00:50 +09:00
Zuul c02d93fd21 Merge "Bump hacking" 2024-01-25 15:52:02 +00:00
Zuul 2376924760 Merge "options: Remove deprecated option aliases" 2024-01-25 15:52:01 +00:00
Takashi Kajinami 300a41c151 Bump hacking
hacking 3.0.x is too old.

Change-Id: Id95477ce65868e49ea4c1ab1e165f9c69596f5da
2024-01-16 22:24:57 +09:00
Ghanshyam Mann 7811687769 Update python classifier in setup.cfg
As per the current release tested runtime, we test
python version from 3.8 to 3.11 so updating the
same in python classifier in setup.cfg

Change-Id: I303a0fbe36a1ca7ae7b9b6e82972da3107f58d0c
2024-01-11 16:18:27 -08:00
Takashi Kajinami 45dd78c812 Fix zuul config warning
This change resolves the following warning detected by zuul.

  All regular expressions must conform to RE2 syntax, but an
  expression using the deprecated Perl-style syntax has been detected.
  Adjust the configuration to conform to RE2 syntax.

  The RE2 syntax error is: invalid perl operator: (?!

Change-Id: Iddb5224b6e2f32a0c26f348ae3acc884b8d9b53b
2023-11-25 12:00:05 +09:00
Zuul 48ff2a0e43 Merge "Add link to bug" 2023-10-27 15:34:58 +00:00
Stephen Finucane 5854e6b0bb bindep: Use new mysql-* package names
Debian Bookworm switched from MySQL to MariaDB.

Change-Id: Id5f64a5cedd45769188be12fb5df8e43b2abcb1d
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-09-08 12:18:46 +01:00
OpenStack Release Bot 1ae346824c Update master for stable/2023.2
Add file to the reno documentation build to show release notes for
stable/2023.2.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2023.2.

Sem-Ver: feature
Change-Id: Icbb93217de7bb305596077eaee3c4f862b0894e6
2023-09-07 09:36:33 +00:00
Stephen Finucane 6c780a475e Add link to bug
So we know if we can remove this in the future, or at least have context
around why it's there.

Change-Id: I4e4bdcd6a01e8c1071045bbe1310cbdf4195ec27
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-07-20 12:37:23 +01:00
Stephen Finucane 5363ca11c9 Remove MySQL NDB Cluster Support
This has been deprecated for some time now.

Change-Id: Ia8b4ed8cd755d283bb773e55293457190b34c482
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-07-20 11:16:22 +01:00
Stephen Finucane c42eb7ec07 trivial: Reflow docstrings
Change-Id: I3148122069fb10c647356ffda3d742cb32128d84
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-07-19 16:44:39 +01:00
Zuul f2ebe65c2f Merge "exc_filters: Do not reraise" 2023-07-19 13:46:00 +00:00
Stephen Finucane 64389d3537 tests: Use column_property
Resolve the following SAWarning warning:

  SAWarning: Attribute '_expr_to_appease_mox' on class <class
  'oslo_db.tests.sqlalchemy.test_utils.FakeTable'> appears to be a
  non-schema SQLAlchemy expression object; this won't be part of the
  declarative mapping. To map arbitrary expressions, use
  ``column_property()`` or a similar function such as ``deferred()``,
  ``query_expression()`` etc.

Change-Id: I7baac158e5987d95f6272d9849c5ec6a2071a25f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-07-17 10:43:05 +01:00
Stephen Finucane 8dcd47383d exc_filters: Do not reraise
We're seeing a weird condition whereby the exceptions raised by our
exception handler and being "rehandled", resulting in 'DBError' being
raised instead of e.g. 'DBReferenceError'. This is hard to reproduce and
thus far has only be spotted in Barbican unit tests. Prevent the issue
by checking if the exception being handled is one of our own and skip
handling it if so.

Change-Id: Ibd3f665a3ed3aedf9d1f33edcab35a46c27ea3dc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-07-14 15:41:37 +01:00
Zuul 6285f2b4e2 Merge "Ignore unset opts when deprecated" 2023-06-23 13:35:45 +00:00
Zuul c9e31f718f Merge "Bump bandit" 2023-06-21 14:13:10 +00:00
Zuul 9ef739e505 Merge "Imported Translations from Zanata" 2023-06-20 12:02:55 +00:00
Stephen Finucane c69e5e5f89 Handle server default quoting
Alembic was previously modifying the metadata server default when
comparing it in the autogenerate process. This was removed in Alembic
1.11. As noted in the commit message [1], we implement our own
'compare_server_default' which means we now need to handle this.

[1] https://github.com/sqlalchemy/alembic/commit/230a2932f6

Change-Id: Ib5f8f78b917d79c4c234cc689c13acff011c2403
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-06-14 16:08:47 -07:00
Hervé Beraud be31cd0a5d Bump bandit
Change-Id: Ia9b9dcbda0c6be800319e80099d6d59461afc51b
2023-05-17 16:45:50 +02:00
OpenStack Proposal Bot 54565201e9 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: Icf53e13a93d4fc2a610cc57ef4fefd62ab5de711
2023-05-09 04:28:20 +00:00
Zuul bbb5fbe47b Merge "Revert "Moves supported python runtimes from version 3.8 to 3.10"" 2023-05-08 10:43:50 +00:00
Ghanshyam c6e57cf2a1 Revert "Moves supported python runtimes from version 3.8 to 3.10"
This reverts commit 481936a822.

Keeping Python 3.10 in setup.cfg classifier.

Reason for revert:

Needed-By: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/882175

TC has been discussing about re-adding the python 3.8
testing in current master 2023.2 release testing.

- https://meetings.opendev.org/meetings/tc/2023/tc.2023-04-25-18.00.log.html#l-191
- https://lists.openstack.org/pipermail/openstack-discuss/2023-April/033469.html

While governance changes are under review, TC agreed to add py3.8 testing
so that we do not see more project/lib dropping python 3.8 and make them
uninstalable on python 3.8

- https://meetings.opendev.org/meetings/tc/2023/tc.2023-05-02-18.00.log.html#l-17
- https://review.opendev.org/c/openstack/governance/+/882165

Also adding py3.8 testing back in job https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/882175

Change-Id: I8b17440c9d36b36fe50183e61d6f1c22ece6d3d4
2023-05-05 18:55:51 +00:00
Zuul a7c03ba9de Merge "Add release note for base test class removal" 2023-04-17 15:38:16 +00:00
Stephen Finucane f6ba070657 Ignore unset opts when deprecated
If an option is deprecated, there's a good chance we want to ignore it
unless explicitly requested by the user. Start ignoring options that are
deprecated and unset.

Change-Id: I3008562db52a5625504b262d7148528a832246dd
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-Bug: #2008720
2023-04-17 14:47:53 +00:00
Zuul b557c6a30c Merge "Remove logic for SQLAlchemy < 1.4" 2023-04-17 14:13:42 +00:00
Zuul f67df4a166 Merge "Match exceptions with multiple lines" 2023-04-17 14:05:43 +00:00
Zuul 50a1d90e54 Merge "Remove dead code" 2023-04-17 14:04:54 +00:00
Zuul 60dc68ef41 Merge "Don't sleep in tests" 2023-04-17 14:04:52 +00:00
Stephen Finucane af5392bf3f Add release note for base test class removal
In change I1e71150ba6daeba464b6ed8d46163f1f34959db3 we removed the
legacy base test classes, first deprecated in 2015. We forgot to include
a release note, however. Address this now.

Change-Id: I4d66f0308b89a187143ef6c8495383fe60043c14
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-04-17 13:34:40 +01:00
Zuul c3305cb282 Merge "Remove legacy base test classes" 2023-04-17 11:11:39 +00:00
Stephen Finucane 56d79bf378 Remove logic for SQLAlchemy < 1.4
Change I8629225eeb51d95264d8a3e4b719268bb1597f4f bumped the minimum
version for SQLAlchemy to 1.4, meaning this logic is now dead. Remove
it.

Change-Id: I4d4a58e15e840ecfa63e15c709617a65642c8323
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-04-17 10:17:44 +01:00
Stephen Finucane 7d62b3664e Match exceptions with multiple lines
We were creating regexes without the DOTALL flag, which meant '.' wasn't
matching newlines. This meant exceptions that contained multiple lines
would not be caught. For example, in my environment where Kerberos is
used, I see the following otherwise harmless message:

  (psycopg2.OperationalError) connection to server at "localhost" (::1),
  port 5432 failed: could not initiate GSSAPI security context: No
  credentials were supplied, or the credentials were unavailable or
  inaccessible: Configuration file does not specify default realm

  connection to server at "localhost" (::1), port 5432 failed: FATAL:
  database "non_existent_database" does not exist

The presence of that newline causes our matchers to fail and the
exception is not wrapped. Correct this.

In the meanwhile, we reformat the function that does the wrapping to
make it a little flatter. This was difficult to modify (for debugging
purposes) due to the level of indentation.

Change-Id: I5396a5a3272e6984954d819cfc71507283c775db
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-04-17 10:17:44 +01:00
Stephen Finucane 04acea8cbb Remove dead code
This should have been removed in change
Ic3d6bd318038d723b0d50d39e45f8e26289e9a57 but was missed.

Change-Id: I4e1faa2c617ac19e7c9766e99ee9012ad9298d31
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-04-17 10:17:38 +01:00
Stephen Finucane 672dd056aa Don't sleep in tests
Mock it out, reducing test run of ~3 and ~15 seconds to milliseconds.

Change-Id: Ice3a0c0d0a5b8c2920c7f775ff8ce974b572c66e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-04-17 10:17:35 +01:00
Hervé Beraud 481936a822 Moves supported python runtimes from version 3.8 to 3.10
Within 2023.2 python version 3.9 and 3.10 are the
supported python runtimes [1].
[1] https: //review.opendev.org/c/openstack/governance/+/872232

Change-Id: I0cb0d4e03858a4f394aed484508b305683ce7863
2023-04-10 13:22:48 +09:00
Stephen Finucane 99cba2dd7d Run unit tests against main branch of sqlalchemy, alembic
Add two new jobs to verify the main branches of these two projects. This
should keep us on top of things.

Change-Id: Iaa955e6d5563c97ab5cc64fe9133db63d6489a0e
Depends-on: https://review.opendev.org/c/openstack/project-config/+/879549
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-04-06 14:55:03 +01:00
Stephen Finucane 64e50494f2 Use SQLAlchemy native pre-ping
This functionality has been available upstream since SQLALchemy 1.2 [1].
However, for oslo.db to use this feature while maintaining full
behavior, we need at least SQLAlchemy 2.0.5 to provide complete event
support.  In particular, oslo.db adds several new "is disconnect"
conditions including those specific to Galera.

Behavior of the handle_error event is modified to expect the "pre-ping"
calling form, which may not have an "engine" present (only a dialect),
and additionally takes advantage of the new is_pre_ping attribute which
informs on the correct way to affect the disconnection status within the
ping handler.

Change-Id: I50d862d3cbb126987a63209795352c6e801ed919
2023-04-06 14:55:03 +01:00
Stephen Finucane 1f003bcb0b Get test suite to full pass with SQLAlchemy 2.0
Remaining issues encountered when running with SQLAlchemy 2.0 for real:

* Never call str() on a URL and expect it to be meaningful anymore.
  The password is aggressively obfuscated now (users absolultely
  wouldn't let us leave it as is)
* More utilities and fixtures that were calling begin() within a
  block that would have already begun
* isnot is now called is_not; mocking "isnot" leads into too many
  weird compat layers
* ORM InstrumentedAttribute and internals use __slots__ now, mock
  seems to not be able to patch methods.  Ideally these tests would use
  a comparator subclass or something
* Connection.connection.connection is now called driver_connection,
  SQLAlchemy keeps the old name available however oslo.db test suite
  does not appear to tolerate the deprecation warning emitted,
  so add a compat layer
* mapper() is fully removed from 2.0, not sure if there is another
  not-yet-committed gerrit that removes mapper()

[1] https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.create_engine.params.pool_pre_ping
[2] https://docs.sqlalchemy.org/en/20/changelog/changelog_20.html#change-2fe37eaf2295cebd3bb4ee8e5b8c575c
[3] https://github.com/sqlalchemy/sqlalchemy/issues/5648

Change-Id: Ifaca67c07f008d8bc0febeecd3e200cc7ee7a4b0
2023-04-06 14:54:40 +01:00