Commit Graph

18 Commits

Author SHA1 Message Date
Abhishek Kekane b6bd03d4bc Add new tables for cache operations
Added expand script and model to define `cached_images` and
`node_reference` database tables.

Related blueprint centralized-cache-db

Change-Id: I9991a4e945bd4d231e942096445fd5374ff7ff2a
2024-02-09 08:40:46 +00:00
Stephen Finucane 707ac69914 db: Replace use of update.values parameter
Resolve the following RemovedIn20Warning warning:

  The update.values parameter will be removed in SQLAlchemy 2.0. Please
  refer to the Update.values() method.

Curiously this didn't raise an explicit warning and was only spotted
when updating to SQLAlchemy 2.0 itself.

Change-Id: I1ecb144553992966824420af2121f2f3d64836c6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-25 09:06:52 +00:00
Stephen Finucane 1f5bbf0bd1 db: Remove use of 'bind' arguments
Resolve the following RemovedIn20Warning warnings:

  The MetaData.bind argument is deprecated and will be removed in
  SQLAlchemy 2.0.

  The ``bind`` argument for schema methods that invoke SQL against an
  engine or connection will be required in SQLAlchemy 2.0.

We are also able to resolve the following RemovedIn20Warning warning,
since the migration away from the bind parameter requires updating these
calls.

  The autoload parameter is deprecated and will be removed in version
  2.0.  Please use the autoload_with parameter, passing an engine or
  connection.

Similarly, we can also resolve the following RemovedIn20Warning warning,
since the calls we're updating here are the only ones affected by this
deprecation:

  The insert.values parameter will be removed in SQLAlchemy 2.0.  Please
  refer to the Insert.values() method.

Change-Id: Ic956980a03e1a0e1b6f90c492d8a03a64ea275cc
2023-07-24 17:47:39 +01:00
Abhishek Kekane 9786936915 Update migration constant
Update the data migration current release to '2023_1'. Added
empty migration scripts to rule out regression scenarios as well.

Change-Id: Ic97825dd9c38ec1c759e5ca610a19c93fe4ca6a6
2023-01-18 04:57:34 +00:00
Stephen Finucane 77676e833f db: Replace sqlalchemy-migrate call with alembic equivalent
The 'ocata_expand01_add_visibility' alembic migration was calling
'alter' on a table column, something that is only possible once you've
monkey patched sqlalchemy by importing the 'migrate' module. Resolve
this.

Change-Id: Ic71710aebafa38dceeead104b3e97fafa4bb4362
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-07-13 17:15:41 +01:00
Abhishek Kekane 6f20983ff8 Expand tasks database table to add more columns
Added 3 new columns 'image_id', 'request_id' and 'user_id' to task
table. This is a base patch for adding new API to show task
information to the user.

Partially-Implements: blueprint messages-api
Change-Id: I4946c25375f1dc54d8f550f21c97bdf5b3005877
2021-02-24 05:13:23 +00:00
Abhishek Kekane d413625172 Change database migration version to ussuri
Change-Id: Ic80b169d34ae9adc6d1329785f80a0d27d2cb241
2020-01-03 13:02:14 +00:00
Abhishek Kekane b4e3cb65ad Add migration script to change backend to store
As glance is changing metadata value of location from 'backend' to
'store', adding migration script which will update old image locations
to use 'store' as a metadata value.

NOTE:
Bump master (train development) to 19

Change-Id: I1386c535bc8ff4519e6b0bb879026b05c930b791
Sem-Ver: api-break
2019-08-06 05:48:50 +00:00
Brian Rosmaita 0b24dbd620 Multihash implementation for Glance
Partially implements blueprint multihash.

Requires glance_store 0.26.1

Co-authored-by: Scott McClymont <scott.mcclymont@verizonwireless.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>

Change-Id: Ib28ea1f6c431db6434dbab2a234018e82d5a6d1a
2018-07-31 21:28:38 -04:00
Abhishek Kekane a308c44406 Hide old images
Added new boolean column "os_hidden" in images table. Images where
"os_hidden" = True will be omitted from the image list presented
to the user. This will apply to all image visibilities. However,
the images will continue to be discoverable. User can use
filter "os_hidden=true" in GET v2/images call to see all hidden
images.

Implements: blueprint hidden-images
Change-Id: If8f02ca94fdb8e1ac7a81853cd392988900172d1
2018-07-25 16:37:56 +00:00
Abhishek Kekane 14e8a7b53b Triggers shouldn't be execute in offline migration
Recently this change [1] in glance-manage db_sync is internally
using Expand, Migrate and Contract. EMC is explicitly used for
online migration for which glance uses triggers to sync data
between old columns and new columns. DB Sync is used for
offline migartion for which adding triggers is not required.

Made provision to execute triggers explicitly in case of
online migration (EMC pattern) and skip the same in
case of offline migration (db sync).

[1] https://review.openstack.org/#/c/433934/

Closes-Bug: #1749640
Change-Id: I816c73405dd61d933182ad5efc24445a0add4eea
2018-02-21 03:40:13 +00:00
shilpa.devharakar f268df1cbc Modify glance manage db sync to use EMC
Modified 'glance-manage db_sync' operation to use
expand, migrate and contract operations.
Added test queens scripts for testing purpose only.

This patch removes use of "monolithic" upgrade scripts
and resolve the issue while upgrading from ocata to pike.

Co-Authored-By: Shilpa Devharakar <Shilpa.Devharakar@nttdata.com>
Closes-Bug: #1723001

Change-Id: I2653560d637a6696f936b49e87f16326fd601dfe
2018-02-05 17:00:57 +05:30
Jenkins 30148b2be9 Merge "Fix experimental E-M-C migrations" 2017-04-05 21:16:22 +00:00
Alexander Bashmakov 15817166ab Fix experimental E-M-C migrations
Two closely related issues were discovered in the code which prevent
the experimental E-M-C (rolling upgrade) migration feature introduced
in Ocata from working: a malformed select statement in the ocata
community images data migration (missing function parentheses)
and an incorrect revision dependency in the DB migration (should be
'ocata_expand01' instead of 'expand'. Also added a test for the case
where the Glance database is new/empty which illuminated this bug.

Closes-Bug: 1670409
Change-Id: I40ffaaa3fa2bae7a555a86d9f31422d79fb9bb19
2017-03-06 21:45:56 +00:00
Brian Rosmaita 4b8026714f Glare-ectomy
Glare became a separate project during Newton.  The code was copied
out of the Glance tree, but remained in the Glance repository.  It
is no longer being maintained, and that has begun to cause some
problems, for example, blocking a recent stevedore upper constraints
change; see I141b17f9dd2acebe2b23f8fc93206e23bc70b568

This patch removes the Glare code from the Glance repository.  It
includes Alembic database migration scripts, in both the all-in-one
and E-M-C format.  It also includes release notes.

Implements: blueprint glare-ectomy
Change-Id: I3026ca6287a65ab5287bf3843f2a9d756ce15139
2017-03-01 21:46:52 +00:00
Hemanth Makkapati 9859df2d1f Add expand/migrate/contract migrations for CI
This patch adds equivalent expand/migrate/contract migration scripts for
Community Images. The expand migration
'ocata_expand01_add_visibility.py' creates a new migration branch 'expand'
from the last known migration i.e., 'mitaka02'. Similarly, the contract
migration 'ocata_contract01_drop_is_public.py' creates another new
migration branch called 'contract' from the last known migration.

The data migration 'ocata_migrate01_community_images.py' migrates
all rows in the database at once. There is possibility of performance
degradation while the data migrations are running.

Change-Id: I34f5623d6804e9fe594e6b5b196ea4a162578196
Partially-Implements: blueprint database-strategy-for-rolling-upgrades
Co-Authored-By: Hemanth Makkapati <hemanth.makkapati@rackspace.com>
Depends-On: Ie839e0f240436dce7b151de5b464373516ff5a64
2017-02-02 00:16:04 +00:00
Hemanth Makkapati 95c7c1b753 Refactor tests to use Alembic to run migrations
* Functional tests now use alembic instead of sqlalchmey-migrate
  to build and destroy test database.
* All tests now use a file-based sqlite db as opposed to an in-memory
  database.

Partially-Implements: blueprint alembic-migrations
Change-Id: I77921366a05ba6f9841143af89c1f4059d8454c6
Depends-On: Ie8594ff339a13bf190aefa308f54e97ee20ecfa2
2017-02-01 16:08:17 -06:00
Hemanth Makkapati 21d431013f Port Glance Migrations to Alembic
This change proposes the use of Alembic to manage Glance migrations.
* Introduce new directory ``alembic_migrations`` under
  ``glance/db/sqlalchemy``. This directory is the home for all glance
  migrations henceforth. All the migration scripts reside under
  ``versions`` directory.
* All the migrations up to Liberty are consolidated into one migration
  called ``liberty_initial`` as those migrations are not supported
  any more. Mitaka migrations are retained but under a different naming
  convention.
* All the glance manage db commands are changed appropriately. They now
  use alembic to perform operations such as ``version``, ``upgrade``,
  ``sync`` and ``version_control``.
* The database versions are not numerical any more. They are the revision
  ID of the last migration applied on the database. Since we don't
  support migrations before Mitaka, the Liberty version ``42`` will now
  appear as ``liberty``. Migration ``43`` and ``44`` in Mitaka appear as
  ``mitaka01`` and ``mitaka02`` respectively.
* When one performs a ``sync`` or ``upgrade`` command, the database is
  first stamped with an equivalent alembic version before upgrading.
* The older migration scripts are retained so that users can correlate
  with the new migrations. Also, it is probably safe to retain them until
  the alembic migrations become stable. Similarly, the ``migrate_version``
  table is not removed yet.

Partially-Implements: blueprint alembic-migrations

Change-Id: Ie8594ff339a13bf190aefa308f54e97ee20ecfa2
Co-Authored-By: Alexander Bashmakov <alexander.bashmakov@intel.com>
Depends-On: I1596499529af249bc48dfe859bbd31e90c48a5e0
2017-01-31 22:42:43 -06:00