Per the new process, we will not be keeping graduated files around
on the master branch of incubator.
Closes-Bug: #1371734
Change-Id: I6471c9f6708403ce5e07332a0ce07b94a18c4214
Fixed violations:
* E128 continuation line under-indented for visual indent
* E251 unexpected spaces around keyword / parameter equals
* E265 block comment should start with '# '
* E713 test for membership should be 'not in'
* F402 import shadowed by loop variable
* H305 imports not grouped correctly
* H307 like imports should be grouped together'
* H402 one line docstring needs punctuation'
* H703 Multiple positional placeholders
Also enabled H803 check that didn't have any violations.
Change-Id: I957ab273ddc61b02763d6b60b21d11ed4e73d472
During the build of source code reference of Ceilometer Sphinx reported
the following error: Unknown directive type "code". The correct directive
is '.. code-block::' instead of '.. code::'. This was fixed in the
session.py. The other occurences of this directive were correct.
Change-Id: I222a11d8a2101600e7cbb96f587515a32f697c2e
When a disconnect happens (e.g. when VIP moves to another node), all
existing connections in the pool become invalid. There is no sense to
check every single connection in the pool, as this may take a long
time (e.g. when python-mysqldb driver is used with eventlet green
threads, there is no context switch on db IO, so the whole process is
blocked until socket read times out). Dispose all connections in the
pool when disconnect is detected, they will be recreated on demand
then.
Closes-Bug: #1288438
Change-Id: Ia357da2b3092d306a86b6d1787bab374a335f28c
There's no need to set the sql_mode on every checkout of a
connection - once when the connection is created is sufficient.
Change-Id: If34db31d9646c9a75d591e556a7abcc0bf8a7ab1
Closes-Bug: #1288444
A message about the MySQL server mode was logged at the info level
every time a connection was checked out. This happens often, and
logging every time is unnecessary.
Here's an example of the message when using the default
configuration:
MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,...
The fix is to send the message once per engine creation. Also,
this message is now logged the debug level since there's a warning
logged if the mode isn't a "safe" one already.
Closes-Bug: #1288443
Co-Authored-By: Ben Nemec <bnemec@redhat.com>
Change-Id: I13b24d04c28e69a8534aaa8d3a4463ca1d65570f
Patch Ia527e850671ab9542db5864617384cee21e91bf6 added mysql_sql_mode
option to config, so we should get it's value from CONF object, not
from function argument.
Closes-Bug: #1288445
Change-Id: I4be81718399393d817185abe36f4cc4d3da15000
_wrap_db_error() has an implicit contract that it can be applied only
to methods of SQLAlchemy Session subclasses. This seems to be a good
task for an assert statement (as it's something we want to check
during development and debugging, but not something that an end user
can trigger, which would require a general runtime check).
Change-Id: I0a15bf4a90b0aea5fa715027d13d8af816f9e72d
Add a helper that allows to create an EngineFacade instance given an
oslo.config instance. This helper is completely optional, but it was
requested by a few people syncing the latest oslo.db code to target
projects.
Note, that oslo.config is mocked in tests, so oslo.db still has no
runtime dependency on oslo.config.
Blueprint: oslo-db-lib
Change-Id: Icab476386b859be0b0e984a94cbfcf3d581ab6d2
Since there are no users of the special-case TRADITIONAL mode set,
it's OK to drop those references and use only the more flexible
one tuned via a config option.
Update the TRADITIONAL mode unit test, and also add one for
STRICT_ALL_TABLES mode.
Change-Id: Ie9df3aaa5f48d992b61a9ea480d1e17519f16422
Introduce a config option, mysql_sql_mode, that allows the user
to set the desired SQL mode for MySQL sessions.
For MySQL sessions, set the SQL mode session variable
accordingly.
For compatibility purposes, retain the mysql_traditional_mode
argument in method signatures, but only use it to override the
config option as if it were set to 'TRADITIONAL'.
Change-Id: Ia527e850671ab9542db5864617384cee21e91bf6
Closes-bug: 1271706
Rather than implement a method just for the special case of setting
MySQL TRADITIONAL mode, implement one that can set any supported
server mode, and reimplement the old method as a wrapper around that.
In addition, check the actual effective SQL mode
(as per SHOW VARIABLES LIKE 'sql_mode'), and log a warning if it's
deemed unsafe.
Co-author: Ben Nemec <openstack@nemebean.com>
Change-Id: I90139b915b2ff169878fd93625e5320d39094ed3
This patch adds duplicate entry integrity error handling for the
ibm_db_sa driver to the common sqlalchemy db session.
Closes-Bug: #1227321
Part of nova blueprint: db2-database
Change-Id: I2f0d3d660373e4f647615f121fed0da9442f8cd2
There was some invalid rst in the db.sqlalchemy docstrings which
causes warnings to be generated during the doc build and also
causes the docs to be rendered incorrectly.
Change-Id: I5c14b714b3ab0186150e8f3e481bac4c0bb58c25
Remove setting and passing echo parameter to create_engine()
function and replace it with configuring log level directly on
logger sqlalchemy.engine (this prevents adding of multiple
handlers writing to stderr to the root logger).
Change-Id: I799341a347cfcacdd27c70b91ed9b9e3cc70727f
Closes-Bug: #1278860
Update the incubator code to use different domains for log
messages at different levels.
Update the import exceptions setting for hacking to allow
multiple functions to be imported from gettextutils on one
line.
bp log-messages-translation-domain
Change-Id: I6ce0f4a59438612ce74c46b3ee9398bef24c0c19
Using of a global config object is something we want to avoid when
making libraries. Parameters must be passed as function/class methods
arguments instead.
In order not to duplicate options declaration in each project, they
have been put into separate options module (which should not be a
part of oslo.db when it's released; we'll probably want to find a
place for it within oslo-incubator).
Partial-Bug: #1263908
Co-authored-by: Victor Sergeyev <vsergeyev@mirantis.com>
Change-Id: Ib5a4e31b4e78e2b4cb807a8c88b8072f4207eb22
oslo.db is meant to be a collection of helper utilities for
SQLAlchemy. In order to behave like a 'good' library, we must
stop using global state.
This patch ensures we don't store any engine instances globally in
oslo.db. It's up to end applications to decide how to cope with
engines, not oslo.db.
Partial-Bug: #1263908
Co-authored-by: Victor Sergeyev <vsergeyev@mirantis.com>
Change-Id: I330467ec1317e1a13ff9f0237e2d8900d718e379
Closes-bug: #1276510
Although this code error is part of the `ping_listener` tests, it was
missing from the list of known - and ignored - connection errors.
The code error 2013 refers to a connection lost during a query, see:
https://dev.mysql.com/doc/refman/5.0/en/error-messages-client.html#error_cr_server_lost
Without this error code, the session won't try to reconnect to mysql,
which makes `max_retries` useless.
Change-Id: If2667cf83ac656666ffe00aa90e3454516b080db
Openstack common log module registers a few oslo.config options,
which can possibly lead to unpleasant issues (DuplicateOptError)
when we split oslo.db from oslo-incubator into a separate library
and reuse both log and oslo.db in target projects.
We don't use for example, any of the special formatting supported by
ContextAdaptor so we can use logging module from python standart library
instead of common log module.
Related to blueprint oslo-db-lib
Change-Id: I6ec426274eb3b603dbbc894762edb97515a618b2
Current patch allows us to retry db.api method call if
database connection was lost temporally.
It can be done in a such way:
- We should use annotate all methods which are safe to retry with
@safe_for_db_retry, if it's totally safe to retry these functions
- And we must allow db-retry feature in config (`use_db_reconnect` option
in a `database` group). It's will allow db-retry for all db.api
methods.
If function suppose to be retried and it's got `db-gone-away` error,
it's will calls in a loop until it succeeds, or until maximum retries
count will be reached.
Maximum retries count and timeout between retries attempts are configurable.
We use SQLA engine.dialect.is_disconnect() function to determine
broken db connection so it's should work for all supported SQLA
backends (MySQL, PostgreSQL, DB2 and SQLite supported as well).
Added wrapper for session and query methods. This wrapper catch
exceptions caused by broken db connection and re-raise it as
DBConnectionError exception.
Note, that it's safely to use db-retry feature only if all db.api
queries use single transaction.
At the moment this feature is experimental, so disabled by default.
Previous attepmt - https://review.openstack.org/#/c/10797/
'Proof-of-concept' patch in Nova https://review.openstack.org/#/c/35610/
blueprint db-reconnect
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>
Change-Id: Ifd848ea70a62fb9f7ccbc8bfd22a10ab9d3135c9
When generating the docs, a couple of warnings were generated for
invalid RST in the module-level docstring for
db.sqlalchemy.session. This change cleans up the docstring.
Change-Id: Id08c876c6698b4ef329247c1d034033f0bfa86cb
Change I2960435c81d21f5a65ea7fa4dc5c100880cf6aef, which enabled
MySQL TRADITIONAL mode, erroneously would attempt to enable that
MySQL-specific mode on both MySQL and DB2 connections. Only enable
TRADITIONAL mode when actually running against MySQL.
Change-Id: I28c40c47e2515112555f2abdd2ecc4dc5c3399f6
The new version of libsqlite changed the format of error messages.
Code of oslo.db helpers must be updated appropriately. This was meant
to be fixed by I86e9673ac409b3b3e881bea7cb8b3516f320388d, but that
change doesn't cope well with the new format of column names.
Closes-Bug: #1259915
Change-Id: Iec39c8804ea47307a98d1b79899ff74af9404bf6
These were initially noted during import into the manuals at
https://review.openstack.org/67502
Fixes include rework of text, missing or extra spaces, wrong
capitalization and no sentence style capitalization.
Change-Id: I56a47de73e0e494a6f770e29dae8065b398ef5af
Co-Authored-By: Diane Fleming <diane.fleming@rackspace.com>
The database connection string was not marked as secret, so it
would be printed out in cleartext in the logs when config settings
were logged.
The database connection string typically contains the password
that's used to connect to the database, so it should be marked as
secret so that it doesn't get logged.
Change-Id: If6e1026814262c961a4f22d90f546cec34831bc8
Closes-Bug: #1266590
In code review, some projects are using try/except within the
context managed by session. That's wrong, add the note in
docstring, hope others put try/except in right place.
Database Errors like IntegrityError will be raised in session's
__exit__ handler, and any try/except within the context managed
by session will not be triggered. And catching other non-database
errors in the session will not trigger the ROLLBACK, so exception
handlers should always be outside the session, except developer
want to commit parts of stuff on purpose.
Closes-Bug: #1262065
Change-Id: I23223fbe9b6d9fa8e92e59894fb3b1fa195be159
The MySQL server can operate in different SQL modes, and can apply these
modes differently for different clients, depending on the value of the
sql_mode system variable. Traditional mode makes MySQL behave like a
"traditional" SQL database system. A simple description of this mode is
"give an error instead of a warning" when inserting an incorrect value
into a column including the case when the input string is surpassing the
maximum data length of the field.
Closes-Bug: #1224898
Change-Id: I2960435c81d21f5a65ea7fa4dc5c100880cf6aef
SQLite has new error messages since 3.7.16.
DBDuplicateEntry is now recognized correctly.
Change-Id: I86e9673ac409b3b3e881bea7cb8b3516f320388d
Fixes: bug #1259915
Davanum Srinivas