Added expand script and model to define `cached_images` and
`node_reference` database tables.
Related blueprint centralized-cache-db
Change-Id: I9991a4e945bd4d231e942096445fd5374ff7ff2a
Update the data migration current release to '2023_1'. Added
empty migration scripts to rule out regression scenarios as well.
Change-Id: Ic97825dd9c38ec1c759e5ca610a19c93fe4ca6a6
Update the data migration current release to 'zed'. Include a
semver pseudo-header in this commit message so that pbr will
increment the major version number, otherwise
glance.tests.unit.gate.test_data_migration_version will break.
Change-Id: Iad32226cea6f81cb0a18a46f4f89ebd3dc42d9a9
Sem-Ver: api-break
Update the data migration current release to 'yoga'. Include a
semver pseudo-header in this commit message so that pbr will
increment the major version number, otherwise
glance.tests.unit.gate.test_data_migration_version will break.
Also add a reminder about this to the release cycle tasks list.
Change-Id: Ibdbeb752d29afeb48628587442577ab139be9ac9
Sem-Ver: api-break
This doesn't actually do anything, and hasn't since we merged change
I77921366a05ba6f9841143af89c1f4059d8454c6 way back in Ocata. Drop it.
Change-Id: I623b2ffb8bdd72525241835b5d1164c275f9fe73
Implements: blueprint remove-sqlalchemy-migrate
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This has been in place since Ocata. If any version since then has been
deployed, this will have already been deployed. Time to drop this.
UpgradeImpact
Change-Id: I5412d78c63cf3381782f5c6fc059641489f89053
Implements: blueprint remove-sqlalchemy-migrate
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Added 3 new columns 'image_id', 'request_id' and 'user_id' to task
table. This is a base patch for adding new API to show task
information to the user.
Partially-Implements: blueprint messages-api
Change-Id: I4946c25375f1dc54d8f550f21c97bdf5b3005877
As glance is changing metadata value of location from 'backend' to
'store', adding migration script which will update old image locations
to use 'store' as a metadata value.
NOTE:
Bump master (train development) to 19
Change-Id: I1386c535bc8ff4519e6b0bb879026b05c930b791
Sem-Ver: api-break
Modified 'glance-manage db_sync' operation to use
expand, migrate and contract operations.
Added test queens scripts for testing purpose only.
This patch removes use of "monolithic" upgrade scripts
and resolve the issue while upgrading from ocata to pike.
Co-Authored-By: Shilpa Devharakar <Shilpa.Devharakar@nttdata.com>
Closes-Bug: #1723001
Change-Id: I2653560d637a6696f936b49e87f16326fd601dfe
Now that Pike has been released, we can change CURRENT_RELEASE from
pike to queens to open Queens for data migrations.
Change-Id: I3bb21fd9aa2134cf3e37c90c278e82beedf88810
Glare became a separate project during Newton. The code was copied
out of the Glance tree, but remained in the Glance repository. It
is no longer being maintained, and that has begun to cause some
problems, for example, blocking a recent stevedore upper constraints
change; see I141b17f9dd2acebe2b23f8fc93206e23bc70b568
This patch removes the Glare code from the Glance repository. It
includes Alembic database migration scripts, in both the all-in-one
and E-M-C format. It also includes release notes.
Implements: blueprint glare-ectomy
Change-Id: I3026ca6287a65ab5287bf3843f2a9d756ce15139
Change CURRENT_RELEASE from ocata to pike to open Pike for data
migrations.
Change-Id: Id4ffcaaa79b6abe7c23b8d2f7ab628a99014a3a3
Depends-On: I2a7dc86b88afaf0e5b878d79607f54125a35eb16
The parent of this patch introduced the change to Alembic-based
migrations. This commit builds on top of that by adding expand,
migrate and contract commands to the glance-manage tool.
Appropriate documentation is updated and failing tests are adjusted
to accomodate the new database versioning schema.
Data migrations are expected to be run in the background with older
Glance services being active during the upgrade process.
Partially-Implements: blueprint database-strategy-for-rolling-upgrades
Co-Authored-By: Hemanth Makkapati <hemanth.makkapati@rackspace.com>
Change-Id: Ie839e0f240436dce7b151de5b464373516ff5a64
Depends-On: I77921366a05ba6f9841143af89c1f4059d8454c6
* Functional tests now use alembic instead of sqlalchmey-migrate
to build and destroy test database.
* All tests now use a file-based sqlite db as opposed to an in-memory
database.
Partially-Implements: blueprint alembic-migrations
Change-Id: I77921366a05ba6f9841143af89c1f4059d8454c6
Depends-On: Ie8594ff339a13bf190aefa308f54e97ee20ecfa2
This change proposes the use of Alembic to manage Glance migrations.
* Introduce new directory ``alembic_migrations`` under
``glance/db/sqlalchemy``. This directory is the home for all glance
migrations henceforth. All the migration scripts reside under
``versions`` directory.
* All the migrations up to Liberty are consolidated into one migration
called ``liberty_initial`` as those migrations are not supported
any more. Mitaka migrations are retained but under a different naming
convention.
* All the glance manage db commands are changed appropriately. They now
use alembic to perform operations such as ``version``, ``upgrade``,
``sync`` and ``version_control``.
* The database versions are not numerical any more. They are the revision
ID of the last migration applied on the database. Since we don't
support migrations before Mitaka, the Liberty version ``42`` will now
appear as ``liberty``. Migration ``43`` and ``44`` in Mitaka appear as
``mitaka01`` and ``mitaka02`` respectively.
* When one performs a ``sync`` or ``upgrade`` command, the database is
first stamped with an equivalent alembic version before upgrading.
* The older migration scripts are retained so that users can correlate
with the new migrations. Also, it is probably safe to retain them until
the alembic migrations become stable. Similarly, the ``migrate_version``
table is not removed yet.
Partially-Implements: blueprint alembic-migrations
Change-Id: Ie8594ff339a13bf190aefa308f54e97ee20ecfa2
Co-Authored-By: Alexander Bashmakov <alexander.bashmakov@intel.com>
Depends-On: I1596499529af249bc48dfe859bbd31e90c48a5e0
oslo.db mandates all tables should be created with utf-8 charset
when using MySQL DB. We added an option (deprecated) to skip this
sanity check and it was subsequently removed in Liberty.
This patch fixes existing migration scripts to create tables with
utf-8 charset. Since oslo.db does a sanity check before running
the migrations, it's safe to assume that only new tables created
since the previous migration will be affected.
Closes-bug: #1279000
Change-Id: Ia31dc9e5d4494d1b9633f916fa6e2704d33dffb0
In newer versions of oslo.db, a sanity check is run to verify table's
collations and ensure they're utf8. In order to un-block the gate, we
need to land this patch, which is broken by this sanity check.
I've disabled the sanity_check in this patch to help moving it forward
and I've filed an issue on LP (#1453047) to track the explicit switch to
utf-8. I preferred not to do the switch in this patch because it
deserves its own commit so we can track it properly (and revert if
needed).
Partially-Fixes: #1453047
Change-Id: Ifffbdd50c38a1d25215faa6a966c2d2a63935711
oslo_db was moved out of the oslo namespace in
oslo.db>=1.4.0.
Change-Id: I13620d26ff12c55b2294b3b051669e0e98513a10
Related-to: blueprint drop-namespace-packages
oslo_config was moved out of the oslo namespace in oslo.config>=1.6.0.
Related-to: blueprint drop-namespace-packages
Change-Id: I30ecbf8f9de77496fcb643c7ad9738d79ad359f0
This change fixes a couple of import statements that rely on some import
order and a bit of magic to make the oslo.db.options module visible
under the oslo.db package without importing it explicitly.
The old version of the imports relies on Python's import machinery to
attach sub-modules to packages after import (see
http://paste.openstack.org/show/148711/). It is safer to explicitly
import the module we need, to ensure that we are not relying on
side-effects of importing other modules to configure global state for
the application.
Change-Id: Ia2772030915061ff373e7b7dc77f161146b56e59
Now we have own implementation for loading
extensions. stevedore avoids creating yet
another extension mechanism by building on
top of setuptools entry points.
Change-Id: I7fcb981b58b20fb49d10eea3d477f962e4bf3565
Refactored migration tests to use OpportunisticTestCase, removed
unused code and ``test_migrations.conf`` file.
The main feature of this approach is to create a new database with
random name for each migration test. This will avoid migration tests of
race conditions and reduce tests intersection. After this change, database
``openstack_citest`` will be used only for initial connection to the database.
``test_migrations.conf`` file not required anymore, because we create test
database for migration test, so we no longer need to keep database credentials.
Partial-Bug: #1368274
Change-Id: Ib1d6dfae62cc60f814c01d07adc53f68e7c234f6
Replace common oslo code glance.openstack.common.db
by usage of oslo.db library and remove common code.
Change-Id: I27216e03763f72e8665121d4440ba2828349eac5
This sync contains commit ``ce69e7f Don't store engine instances in oslo.db ``
which removed global engine from oslo.db code. So, added code
for work with sqla engines and sessions to Glance - get_engine()
and get_session() functions in glance.db.sqlalchemy.api module.
Also this remove database slave connection and tpool features, because
they was removed from oslo code and seeps to be unused in Glance.
Full list of changes:
7959826 db: move all options into database group
dda24eb Introduce mysql_sql_mode option, remove old warning
0b5af67 Introduce a method to set any MySQL session SQL mode
8dccc7b Handle ibm_db_sa DBDuplicateEntry integrity errors
0f24d82 Fix migration.db_version when no tables
ac84a40 Update log translation domains
c0d357b Add model_query() to db.sqlalchemy.utils module
84254fc Fix a small typo in api.py
b8a676c Remove CONF.database.connection default value
86707cd Remove None for dict.get()
0545121 Fix duplicating of SQL queries in logs
fcf517d Update oslo log messages with translation domains
fa05b7c Restore the ability to load the DB backend lazily
630d395 Don't use cfg.CONF in oslo.db
ce69e7f Don't store engine instances in oslo.db
35dc1d7 py3kcompat: remove
b4f72b2 Don't raise MySQL 2013 'Lost connection' errors
271adfb Format sql in db.sqlalchemy.session docstring
0334cb3 Handle exception messages with six.text_type
eff69ce Drop dependency on log from oslo db code
7a11a04 Automatic retry db.api query if db connection lost
11f2add Clean up docstring in db.sqlalchemy.session
1b5147f Only enable MySQL TRADITIONAL mode if we're running against MySQL
39e1c5c Move db tests base.py to common code
986dafd Fix parsing of UC errors in sqlite 3.7.16+/3.8.2+
9a203e6 Use dialect rather than a particular DB API driver
1779029 Move helper DB functions to db.sqlalchemy.utils
bcf6d5e Small edits on help strings
ae01e9a Transition from migrate to alembic
70ebb19 Fix mocking of utcnow() for model datetime cols
7aa94df Add a db check for CHARSET=utf8
aff0171 Remove "vim: tabstop=4 shiftwidth=4 softtabstop=4" from headers
fa0f36f Fix database connection string is secret
8575d87 Removed copyright from empty files
d08d27f Fix the obsolete exception message
8b2b0b7 Use hacking import_exceptions for gettextutils._
9bc593e Add docstring for exception handlers of session
855644a Removal of _REPOSITORY global variable.
ea6caf9 Remove string.lowercase usage
a33989e Remove eventlet tpool from common db.api
e40903b Database hook enabling traditional mode at MySQL
f2115a0 Replace xrange in for loop with range
c802fa6 SQLAlchemy error patterns improved
1c1f199 Remove unused import
97d8cf4 Remove lazy loading of database backend
Co-authored-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Related to blueprint db-use-oslo-common-code
Change-Id: Ia73abba8309ccc2ad10a0f636b410984c6b6e5d8
Changes due to use common migration module instead of Glance
implementation.
Removed glance.db.sqlalchemy.migration module and related code.
blueprint db-use-oslo-common-code
Change-Id: Ib7e2e478a84a90cc9c773f2733dc60b88834fd23
This adds a migration.py file for setup and migration commands
similar to the file in nova. This was needed for porting the
new test migration framework.
Change-Id: Ic4d3a5e471b5fb8ee38b22b97df2e99b9fb649a1
Move the core of glance.tests.unit.utils:FakeDB to glance.db.simple.
This 'simple' driver is an alternative to the traditional sqlalchemy
driver. Additionally, the sqlalchemy driver has been moved from
glance.db to glance.db.sqlalchemy. The simple db driver is only
available to be used by tests for now.
Related to bp refactor-db-layer
Change-Id: I9d33a433c0c03e53fb5a3491076086427ae694b3