We don't want alembic overriding our logging: we can configure it
ourselves. This removes a whole lot of noise from our test output.
Change-Id: Ic1bc4ab967913a8adaac559482d330d72e47a4b5
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following RemovedIn20Warning warnings:
The MetaData.bind argument is deprecated and will be removed in
SQLAlchemy 2.0.
The ``bind`` argument for schema methods that invoke SQL against an
engine or connection will be required in SQLAlchemy 2.0.
We are also able to resolve the following RemovedIn20Warning warning,
since the migration away from the bind parameter requires updating these
calls.
The autoload parameter is deprecated and will be removed in version
2.0. Please use the autoload_with parameter, passing an engine or
connection.
Similarly, we can also resolve the following RemovedIn20Warning warning,
since the calls we're updating here are the only ones affected by this
deprecation:
The insert.values parameter will be removed in SQLAlchemy 2.0. Please
refer to the Insert.values() method.
Change-Id: Ic956980a03e1a0e1b6f90c492d8a03a64ea275cc
This has been in place since Ocata. If any version since then has been
deployed, this will have already been deployed. Time to drop this.
UpgradeImpact
Change-Id: I5412d78c63cf3381782f5c6fc059641489f89053
Implements: blueprint remove-sqlalchemy-migrate
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
In the bug, a user tried setting a devstack password with a "@" in it.
As it turns out, sqlalchmey turns the connection-string into a
sqlalchemy.engine.url.URL object [1] which returns a RFC1738 quoted
string.
However, alembic's set_main_option [2] uses python
string-interpolation which interprets '%' characters. This means you
end up with an interpolation traceback when using any quoted character
(':@/') in a user/password (more likely password).
Avoid this by ensuring the URL is safe for python interpolation in
set_main_option by replacing '%' -> '%%'.
I convinced myself this is safe because sqlalchemy correctly parses
the quoted and unquoted versions just the same
---
>>> str(sqlalchemy.engine.url.make_url('mysql+pymysql://foo:crazy:@/pw@/moo'))
'mysql+pymysql://foo:crazy%3A%40%2Fpw@/moo'
>>> str(sqlalchemy.engine.url.make_url('mysql+pymysql://foo:crazy%3A%40%2Fpw@/moo'))
'mysql+pymysql://foo:crazy%3A%40%2Fpw@/moo'
---
A test is added
[1] https://github.com/zzzeek/sqlalchemy/blob/master/lib/sqlalchemy/engine/url.py
[2] http://alembic.zzzcomputing.com/en/latest/api/config.html#alembic.config.Config.set_main_option
Change-Id: I3ef7e3e539e35ce040573f2044ab6eb3c990200a
Closes-Bug: #1695299
Modified 'glance-manage db_sync' operation to use
expand, migrate and contract operations.
Added test queens scripts for testing purpose only.
This patch removes use of "monolithic" upgrade scripts
and resolve the issue while upgrading from ocata to pike.
Co-Authored-By: Shilpa Devharakar <Shilpa.Devharakar@nttdata.com>
Closes-Bug: #1723001
Change-Id: I2653560d637a6696f936b49e87f16326fd601dfe
DbMigrationError is deprecated and will NOT be thrown in oslo.db
since oslo.db >=4.27.0, DBMigrationError will be thrown instead.
Consumers should catch DBMigrationError instead of DbMigrationError
Depends-On: Iab0566cf9f4552e91fa417e64472fa106e8bc86d
Depends-On: I0ebd69c3d778acb5bec9e136627e345e7fcf2bd3
Change-Id: Ic2fe5bd6be16d61df3257560c2072b075603672f
The parent of this patch introduced the change to Alembic-based
migrations. This commit builds on top of that by adding expand,
migrate and contract commands to the glance-manage tool.
Appropriate documentation is updated and failing tests are adjusted
to accomodate the new database versioning schema.
Data migrations are expected to be run in the background with older
Glance services being active during the upgrade process.
Partially-Implements: blueprint database-strategy-for-rolling-upgrades
Co-Authored-By: Hemanth Makkapati <hemanth.makkapati@rackspace.com>
Change-Id: Ie839e0f240436dce7b151de5b464373516ff5a64
Depends-On: I77921366a05ba6f9841143af89c1f4059d8454c6
* Functional tests now use alembic instead of sqlalchmey-migrate
to build and destroy test database.
* All tests now use a file-based sqlite db as opposed to an in-memory
database.
Partially-Implements: blueprint alembic-migrations
Change-Id: I77921366a05ba6f9841143af89c1f4059d8454c6
Depends-On: Ie8594ff339a13bf190aefa308f54e97ee20ecfa2
This change proposes the use of Alembic to manage Glance migrations.
* Introduce new directory ``alembic_migrations`` under
``glance/db/sqlalchemy``. This directory is the home for all glance
migrations henceforth. All the migration scripts reside under
``versions`` directory.
* All the migrations up to Liberty are consolidated into one migration
called ``liberty_initial`` as those migrations are not supported
any more. Mitaka migrations are retained but under a different naming
convention.
* All the glance manage db commands are changed appropriately. They now
use alembic to perform operations such as ``version``, ``upgrade``,
``sync`` and ``version_control``.
* The database versions are not numerical any more. They are the revision
ID of the last migration applied on the database. Since we don't
support migrations before Mitaka, the Liberty version ``42`` will now
appear as ``liberty``. Migration ``43`` and ``44`` in Mitaka appear as
``mitaka01`` and ``mitaka02`` respectively.
* When one performs a ``sync`` or ``upgrade`` command, the database is
first stamped with an equivalent alembic version before upgrading.
* The older migration scripts are retained so that users can correlate
with the new migrations. Also, it is probably safe to retain them until
the alembic migrations become stable. Similarly, the ``migrate_version``
table is not removed yet.
Partially-Implements: blueprint alembic-migrations
Change-Id: Ie8594ff339a13bf190aefa308f54e97ee20ecfa2
Co-Authored-By: Alexander Bashmakov <alexander.bashmakov@intel.com>
Depends-On: I1596499529af249bc48dfe859bbd31e90c48a5e0
Stephen Finucane