Commit Graph

38 Commits

Author SHA1 Message Date
Abhishek Kekane b6bd03d4bc Add new tables for cache operations
Added expand script and model to define `cached_images` and
`node_reference` database tables.

Related blueprint centralized-cache-db

Change-Id: I9991a4e945bd4d231e942096445fd5374ff7ff2a
2024-02-09 08:40:46 +00:00
Pranali Deore 969de4ef07 Change DB migration constant to 2023_2
Change-Id: I12af2e6e5eede2f1027682ab46fc69e1aeaa317c
2023-04-10 09:39:35 +00:00
Abhishek Kekane 9786936915 Update migration constant
Update the data migration current release to '2023_1'. Added
empty migration scripts to rule out regression scenarios as well.

Change-Id: Ic97825dd9c38ec1c759e5ca610a19c93fe4ca6a6
2023-01-18 04:57:34 +00:00
Abhishek Kekane 14bd3eb1bb Update migration constant
Update the data migration current release to 'zed'.  Include a
semver pseudo-header in this commit message so that pbr will
increment the major version number, otherwise
glance.tests.unit.gate.test_data_migration_version will break.

Change-Id: Iad32226cea6f81cb0a18a46f4f89ebd3dc42d9a9
Sem-Ver: api-break
2022-04-21 05:50:44 +00:00
Brian Rosmaita ba5f556d27 Update migration constant
Update the data migration current release to 'yoga'.  Include a
semver pseudo-header in this commit message so that pbr will
increment the major version number, otherwise
glance.tests.unit.gate.test_data_migration_version will break.

Also add a reminder about this to the release cycle tasks list.

Change-Id: Ibdbeb752d29afeb48628587442577ab139be9ac9
Sem-Ver: api-break
2022-02-10 09:31:16 -05:00
Zuul fa55888550 Merge "db: Remove "support" for configurable migration backends" 2021-07-22 16:16:21 +00:00
Zuul 11fd47a380 Merge "db: Stop checking for DB under sqlalchemy-migrate control" 2021-07-22 08:08:32 +00:00
Abhishek Kekane e33e0debb1 Change database migration version to xena
Change-Id: I3f42e37e783fdeefe60a25211a97d54600c97021
2021-07-15 17:55:13 +00:00
Stephen Finucane ae43625781 db: Remove "support" for configurable migration backends
This doesn't actually do anything, and hasn't since we merged change
I77921366a05ba6f9841143af89c1f4059d8454c6 way back in Ocata. Drop it.

Change-Id: I623b2ffb8bdd72525241835b5d1164c275f9fe73
Implements: blueprint remove-sqlalchemy-migrate
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-07-13 17:19:33 +01:00
Stephen Finucane 7fb274f6db db: Stop checking for DB under sqlalchemy-migrate control
This has been in place since Ocata. If any version since then has been
deployed, this will have already been deployed. Time to drop this.

UpgradeImpact

Change-Id: I5412d78c63cf3381782f5c6fc059641489f89053
Implements: blueprint remove-sqlalchemy-migrate
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-07-13 17:18:52 +01:00
Abhishek Kekane 6f20983ff8 Expand tasks database table to add more columns
Added 3 new columns 'image_id', 'request_id' and 'user_id' to task
table. This is a base patch for adding new API to show task
information to the user.

Partially-Implements: blueprint messages-api
Change-Id: I4946c25375f1dc54d8f550f21c97bdf5b3005877
2021-02-24 05:13:23 +00:00
Abhishek Kekane 03bf00ee3b Change database migration version to wallaby
Change-Id: I5c57696827719d752879cb11044d9e969c853411
2021-01-25 12:30:50 -08:00
Abhishek Kekane f2276f9a73 Change database migration version to victoria
Change-Id: I709a5d5b164031ffaabbf4164f4da4b080200543
2020-07-30 15:58:42 +00:00
Abhishek Kekane d413625172 Change database migration version to ussuri
Change-Id: Ic80b169d34ae9adc6d1329785f80a0d27d2cb241
2020-01-03 13:02:14 +00:00
Abhishek Kekane b4e3cb65ad Add migration script to change backend to store
As glance is changing metadata value of location from 'backend' to
'store', adding migration script which will update old image locations
to use 'store' as a metadata value.

NOTE:
Bump master (train development) to 19

Change-Id: I1386c535bc8ff4519e6b0bb879026b05c930b791
Sem-Ver: api-break
2019-08-06 05:48:50 +00:00
Brian Rosmaita c98bcd4f21 Open Stein for data migrations
Change CURRENT_RELEASE from 'rocky' to 'stein' to open Stein for
data migrations.

Change-Id: Ie01c2e29d8fa238d682ad7c925e38304f5a9ea8e
2019-01-21 18:00:57 -05:00
Brian Rosmaita 54d179a80d Open Rocky for data migrations
Change CURRENT_RELEASE from 'queens' to 'rocky' to open Rocky for
data migrations.

Change-Id: I86b6b873d4a7403868ea9e8dcfafe47b24524e06
2018-03-08 10:36:01 -05:00
shilpa.devharakar f268df1cbc Modify glance manage db sync to use EMC
Modified 'glance-manage db_sync' operation to use
expand, migrate and contract operations.
Added test queens scripts for testing purpose only.

This patch removes use of "monolithic" upgrade scripts
and resolve the issue while upgrading from ocata to pike.

Co-Authored-By: Shilpa Devharakar <Shilpa.Devharakar@nttdata.com>
Closes-Bug: #1723001

Change-Id: I2653560d637a6696f936b49e87f16326fd601dfe
2018-02-05 17:00:57 +05:30
Brian Rosmaita 97158c3463 Open Queens for data migrations
Now that Pike has been released, we can change CURRENT_RELEASE from
pike to queens to open Queens for data migrations.

Change-Id: I3bb21fd9aa2134cf3e37c90c278e82beedf88810
2017-08-30 09:42:00 -04:00
Brian Rosmaita 4b8026714f Glare-ectomy
Glare became a separate project during Newton.  The code was copied
out of the Glance tree, but remained in the Glance repository.  It
is no longer being maintained, and that has begun to cause some
problems, for example, blocking a recent stevedore upper constraints
change; see I141b17f9dd2acebe2b23f8fc93206e23bc70b568

This patch removes the Glare code from the Glance repository.  It
includes Alembic database migration scripts, in both the all-in-one
and E-M-C format.  It also includes release notes.

Implements: blueprint glare-ectomy
Change-Id: I3026ca6287a65ab5287bf3843f2a9d756ce15139
2017-03-01 21:46:52 +00:00
Hemanth Makkapati 0a400d07a4 Open Pike for data migrations
Change CURRENT_RELEASE from ocata to pike to open Pike for data
migrations.

Change-Id: Id4ffcaaa79b6abe7c23b8d2f7ab628a99014a3a3
Depends-On: I2a7dc86b88afaf0e5b878d79607f54125a35eb16
2017-02-28 11:21:13 -06:00
Alexander Bashmakov 0f0354a8b8 Add expand/migrate/contract commands to glance-manage CLI
The parent of this patch introduced the change to Alembic-based
migrations. This commit builds on top of that by adding expand,
migrate and contract commands to the glance-manage tool.
Appropriate documentation is updated and failing tests are adjusted
to accomodate the new database versioning schema.

Data migrations are expected to be run in the background with older
Glance services being active during the upgrade process.

Partially-Implements: blueprint database-strategy-for-rolling-upgrades
Co-Authored-By: Hemanth Makkapati <hemanth.makkapati@rackspace.com>
Change-Id: Ie839e0f240436dce7b151de5b464373516ff5a64
Depends-On: I77921366a05ba6f9841143af89c1f4059d8454c6
2017-02-01 23:23:11 +00:00
Hemanth Makkapati 95c7c1b753 Refactor tests to use Alembic to run migrations
* Functional tests now use alembic instead of sqlalchmey-migrate
  to build and destroy test database.
* All tests now use a file-based sqlite db as opposed to an in-memory
  database.

Partially-Implements: blueprint alembic-migrations
Change-Id: I77921366a05ba6f9841143af89c1f4059d8454c6
Depends-On: Ie8594ff339a13bf190aefa308f54e97ee20ecfa2
2017-02-01 16:08:17 -06:00
Hemanth Makkapati 21d431013f Port Glance Migrations to Alembic
This change proposes the use of Alembic to manage Glance migrations.
* Introduce new directory ``alembic_migrations`` under
  ``glance/db/sqlalchemy``. This directory is the home for all glance
  migrations henceforth. All the migration scripts reside under
  ``versions`` directory.
* All the migrations up to Liberty are consolidated into one migration
  called ``liberty_initial`` as those migrations are not supported
  any more. Mitaka migrations are retained but under a different naming
  convention.
* All the glance manage db commands are changed appropriately. They now
  use alembic to perform operations such as ``version``, ``upgrade``,
  ``sync`` and ``version_control``.
* The database versions are not numerical any more. They are the revision
  ID of the last migration applied on the database. Since we don't
  support migrations before Mitaka, the Liberty version ``42`` will now
  appear as ``liberty``. Migration ``43`` and ``44`` in Mitaka appear as
  ``mitaka01`` and ``mitaka02`` respectively.
* When one performs a ``sync`` or ``upgrade`` command, the database is
  first stamped with an equivalent alembic version before upgrading.
* The older migration scripts are retained so that users can correlate
  with the new migrations. Also, it is probably safe to retain them until
  the alembic migrations become stable. Similarly, the ``migrate_version``
  table is not removed yet.

Partially-Implements: blueprint alembic-migrations

Change-Id: Ie8594ff339a13bf190aefa308f54e97ee20ecfa2
Co-Authored-By: Alexander Bashmakov <alexander.bashmakov@intel.com>
Depends-On: I1596499529af249bc48dfe859bbd31e90c48a5e0
2017-01-31 22:42:43 -06:00
Sabari Kumar Murugesan e5d10c3c1b Fix existing migrations to create utf-8 tables for MySQL DB
oslo.db mandates all tables should be created with utf-8 charset
when using MySQL DB. We added an option (deprecated) to skip this
sanity check and it was subsequently removed in Liberty.

This patch fixes existing migration scripts to create tables with
utf-8 charset. Since oslo.db does a sanity check before running
the migrations, it's safe to assume that only new tables created
since the previous migration will be affected.

Closes-bug: #1279000

Change-Id: Ia31dc9e5d4494d1b9633f916fa6e2704d33dffb0
2015-07-29 22:20:28 -07:00
OpenStack Proposal Bot 9b8e2867d9 Updated from global requirements
In newer versions of oslo.db, a sanity check is run to verify table's
collations and ensure they're utf8. In order to un-block the gate, we
need to land this patch, which is broken by this sanity check.

I've disabled the sanity_check in this patch to help moving it forward
and I've filed an issue on LP (#1453047) to track the explicit switch to
utf-8. I preferred not to do the switch in this patch because it
deserves its own commit so we can track it properly (and revert if
needed).

Partially-Fixes: #1453047
Change-Id: Ifffbdd50c38a1d25215faa6a966c2d2a63935711
2015-05-08 11:08:35 +02:00
Louis Taylor e42fad7385 Move from oslo.db to oslo_db
oslo_db was moved out of the oslo namespace in
oslo.db>=1.4.0.

Change-Id: I13620d26ff12c55b2294b3b051669e0e98513a10
Related-to: blueprint drop-namespace-packages
2015-01-12 19:03:20 +00:00
Louis Taylor d759a6fd1a Move from oslo.config to oslo_config
oslo_config was moved out of the oslo namespace in oslo.config>=1.6.0.

Related-to: blueprint drop-namespace-packages

Change-Id: I30ecbf8f9de77496fcb643c7ad9738d79ad359f0
2015-01-12 14:34:22 +00:00
Doug Hellmann d4371c5dff Remove reliance on import order of oslo.db mods
This change fixes a couple of import statements that rely on some import
order and a bit of magic to make the oslo.db.options module visible
under the oslo.db package without importing it explicitly.

The old version of the imports relies on Python's import machinery to
attach sub-modules to packages after import (see
http://paste.openstack.org/show/148711/). It is safer to explicitly
import the module we need, to ensure that we are not relying on
side-effects of importing other modules to configure global state for
the application.

Change-Id: Ia2772030915061ff373e7b7dc77f161146b56e59
2014-12-10 09:39:00 -05:00
Oleksii Chuprykov d40185f248 Replace custom lazy loading by stevedore
Now we have own implementation for loading
extensions. stevedore avoids creating yet
another extension mechanism by building on
top of setuptools entry points.

Change-Id: I7fcb981b58b20fb49d10eea3d477f962e4bf3565
2014-11-10 13:58:00 +00:00
Oleksii Chuprykov e518ab629b Refactor test_migrations module
Refactored migration tests to use OpportunisticTestCase, removed
unused code and ``test_migrations.conf`` file.

The main feature of this approach is to create a new database with
random name for each migration test.  This will avoid migration tests of
race conditions and reduce tests intersection. After this change, database
``openstack_citest`` will be used only for initial connection to the database.

``test_migrations.conf`` file not required anymore, because we create test
database for migration test, so we no longer need to keep database credentials.

Partial-Bug: #1368274

Change-Id: Ib1d6dfae62cc60f814c01d07adc53f68e7c234f6
2014-09-30 14:32:40 +03:00
Eugeniya Kudryashova 6185e9d97e Move to oslo.db
Replace common oslo code glance.openstack.common.db
by usage of oslo.db library and remove common code.

Change-Id: I27216e03763f72e8665121d4440ba2828349eac5
2014-07-21 12:31:10 +03:00
Victor Sergeyev 99f4ad8172 Sync common db code from Oslo
This sync contains commit ``ce69e7f Don't store engine instances in oslo.db ``
which removed global engine from oslo.db code. So, added code
for work with sqla engines and sessions to Glance - get_engine()
and get_session() functions in glance.db.sqlalchemy.api module.

Also this remove database slave connection and tpool features, because
they was removed from oslo code and seeps to be unused in Glance.

Full list of changes:

7959826 db: move all options into database group
dda24eb Introduce mysql_sql_mode option, remove old warning
0b5af67 Introduce a method to set any MySQL session SQL mode
8dccc7b Handle ibm_db_sa DBDuplicateEntry integrity errors
0f24d82 Fix migration.db_version when no tables
ac84a40 Update log translation domains
c0d357b Add model_query() to db.sqlalchemy.utils module
84254fc Fix a small typo in api.py
b8a676c Remove CONF.database.connection default value
86707cd Remove None for dict.get()
0545121 Fix duplicating of SQL queries in logs
fcf517d Update oslo log messages with translation domains
fa05b7c Restore the ability to load the DB backend lazily
630d395 Don't use cfg.CONF in oslo.db
ce69e7f Don't store engine instances in oslo.db
35dc1d7 py3kcompat: remove
b4f72b2 Don't raise MySQL 2013 'Lost connection' errors
271adfb Format sql in db.sqlalchemy.session docstring
0334cb3 Handle exception messages with six.text_type
eff69ce Drop dependency on log from oslo db code
7a11a04 Automatic retry db.api query if db connection lost
11f2add Clean up docstring in db.sqlalchemy.session
1b5147f Only enable MySQL TRADITIONAL mode if we're running against MySQL
39e1c5c Move db tests base.py to common code
986dafd Fix parsing of UC errors in sqlite 3.7.16+/3.8.2+
9a203e6 Use dialect rather than a particular DB API driver
1779029 Move helper DB functions to db.sqlalchemy.utils
bcf6d5e Small edits on help strings
ae01e9a Transition from migrate to alembic
70ebb19 Fix mocking of utcnow() for model datetime cols
7aa94df Add a db check for CHARSET=utf8
aff0171 Remove "vim: tabstop=4 shiftwidth=4 softtabstop=4" from headers
fa0f36f Fix database connection string is secret
8575d87 Removed copyright from empty files
d08d27f Fix the obsolete exception message
8b2b0b7 Use hacking import_exceptions for gettextutils._
9bc593e Add docstring for exception handlers of session
855644a Removal of _REPOSITORY global variable.
ea6caf9 Remove string.lowercase usage
a33989e Remove eventlet tpool from common db.api
e40903b Database hook enabling traditional mode at MySQL
f2115a0 Replace xrange in for loop with range
c802fa6 SQLAlchemy error patterns improved
1c1f199 Remove unused import
97d8cf4 Remove lazy loading of database backend

Co-authored-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>

Related to blueprint db-use-oslo-common-code

Change-Id: Ia73abba8309ccc2ad10a0f636b410984c6b6e5d8
2014-02-27 14:14:21 +02:00
Victor Sergeyev 99fb66ca80 Use common db migrations module from Oslo
Changes due to use common migration module instead of Glance
implementation.
Removed glance.db.sqlalchemy.migration module and related code.

blueprint db-use-oslo-common-code

Change-Id: Ib7e2e478a84a90cc9c773f2733dc60b88834fd23
2014-01-25 11:04:55 +02:00
Noboru arai 8de10b0f34 Remove vim header
No need to set tabstop tons of times, this can be set in your vimrc
file instead.

More disucssion:
http://openstack.10931.n7.nabble.com/Remove-vim-modelines-td21780.html

Change-Id: I3b830a5bb831a63c188109e6fad66ba48884fff3
Partial-Bug: #1229324
2014-01-14 16:03:28 +00:00
Matthew Treinish b584b83830 Add migration.py based on the one in nova.
This adds a migration.py file for setup and migration commands
similar to the file in nova. This was needed for porting the
new test migration framework.

Change-Id: Ic4d3a5e471b5fb8ee38b22b97df2e99b9fb649a1
2013-02-14 10:04:15 -05:00
Brian Waldon a68eb24431 Create 'simple' db driver
Move the core of glance.tests.unit.utils:FakeDB to glance.db.simple.
This 'simple' driver is an alternative to the traditional sqlalchemy
driver. Additionally, the sqlalchemy driver has been moved from
glance.db to glance.db.sqlalchemy. The simple db driver is only
available to be used by tests for now.

Related to bp refactor-db-layer

Change-Id: I9d33a433c0c03e53fb5a3491076086427ae694b3
2012-06-07 09:20:50 -07:00
Brian Waldon dfd9e207f2 Move glance.registry.db to glance.db
Related to bp refactor-db-layer

Change-Id: Ia7c5aea29eef8068a9b67ac5fd051b45a38a20de
2012-06-04 09:29:42 -07:00