Add expand/migrate/contract commands to glance-manage CLI
The parent of this patch introduced the change to Alembic-based migrations. This commit builds on top of that by adding expand, migrate and contract commands to the glance-manage tool. Appropriate documentation is updated and failing tests are adjusted to accomodate the new database versioning schema. Data migrations are expected to be run in the background with older Glance services being active during the upgrade process. Partially-Implements: blueprint database-strategy-for-rolling-upgrades Co-Authored-By: Hemanth Makkapati <hemanth.makkapati@rackspace.com> Change-Id: Ie839e0f240436dce7b151de5b464373516ff5a64 Depends-On: I77921366a05ba6f9841143af89c1f4059d8454c6
This commit is contained in:
Alexander Bashmakov
Hemanth Makkapati
parent
95c7c1b753
commit
0f0354a8b8
|
|
@ -29,11 +29,31 @@ The commands should be executed as a subcommand of 'db':
|
|||
Sync the Database
|
||||
-----------------
|
||||
|
||||
glance-manage db sync <VERSION>
|
||||
glance-manage db sync [VERSION]
|
||||
|
||||
Place an existing database under migration control and upgrade it to the
|
||||
specified VERSION.
|
||||
specified VERSION or to the latest migration level if VERSION is not specified.
|
||||
|
||||
.. note:: Prior to Ocata release the database version was a numeric value.
|
||||
For example: for the Newton release, the latest migration level was ``44``.
|
||||
Starting with Ocata, database version will be a revision name
|
||||
corresponding to the latest migration included in the release. For the
|
||||
Ocata release, there is only one database migration and it is identified
|
||||
by revision ``ocata01``. So, the database version for Ocata release would
|
||||
be ``ocata01``.
|
||||
|
||||
However, with the introduction of zero-downtime upgrades, database version
|
||||
will be a composite version including both expand and contract revisions.
|
||||
To achieve zero-downtime upgrades, we split the ``ocata01`` migration into
|
||||
``ocata_expand01`` and ``ocata_contract01``. During the upgrade process,
|
||||
the database would initially be marked with ``ocata_expand01`` and
|
||||
eventually after completing the full upgrade process, the database will be
|
||||
marked with ``ocata_contract01``. So, instead of one database version, an
|
||||
operator will see a composite database version that will have both expand
|
||||
and contract versions. A database will be considered at Ocata version only
|
||||
when both expand and contract revisions are at the latest revisions
|
||||
possible. For a successful Ocata rolling upgrade, the database should be
|
||||
marked with both ``ocata_expand01``, ``ocata_contract01``.
|
||||
|
||||
Determining the Database Version
|
||||
--------------------------------
|
||||
|
|
@ -46,11 +66,40 @@ This will print the current migration level of a Glance database.
|
|||
Upgrading an Existing Database
|
||||
------------------------------
|
||||
|
||||
glance-manage db upgrade <VERSION>
|
||||
glance-manage db upgrade [VERSION]
|
||||
|
||||
This will take an existing database and upgrade it to the specified VERSION.
|
||||
|
||||
|
||||
Expanding the Database
|
||||
----------------------
|
||||
|
||||
glance-manage db expand
|
||||
|
||||
This will run the expansion phase of a rolling upgrade process.
|
||||
Database expansion should be run as the first step in the rolling upgrade
|
||||
process before any new services are started.
|
||||
|
||||
|
||||
Migrating the Data
|
||||
------------------
|
||||
|
||||
glance-manage db migrate
|
||||
|
||||
This will run the data migrate phase of a rolling upgrade process.
|
||||
Database migration should be run after database expansion and before
|
||||
database contraction has been performed.
|
||||
|
||||
|
||||
Contracting the Database
|
||||
------------------------
|
||||
|
||||
glance-manage db contract
|
||||
|
||||
This will run the contraction phase of a rolling upgrade process.
|
||||
Database contraction should be run as the last step of the rolling upgrade
|
||||
process after all old services are upgraded to new ones.
|
||||
|
||||
Downgrading an Existing Database
|
||||
--------------------------------
|
||||
|
||||
|
|
|
|||
|
|
@ -57,6 +57,18 @@ COMMANDS
|
|||
Place an existing database under migration control and upgrade it to
|
||||
the specified VERSION.
|
||||
|
||||
**db_expand**
|
||||
Run this command to expand the database as the first step of a rolling
|
||||
upgrade process.
|
||||
|
||||
**db_migrate**
|
||||
Run this command to migrate the database as the second step of a
|
||||
rolling upgrade process.
|
||||
|
||||
**db_contract**
|
||||
Run this command to contract the database as the last step of a rolling
|
||||
upgrade process.
|
||||
|
||||
**db_export_metadefs [PATH | PREFIX]**
|
||||
Export the metadata definitions into json format. By default the
|
||||
definitions are exported to /etc/glance/metadefs directory.
|
||||
|
|
|
|||
|
|
@ -51,6 +51,7 @@ from glance.common import exception
|
|||
from glance import context
|
||||
from glance.db import migration as db_migration
|
||||
from glance.db.sqlalchemy import alembic_migrations
|
||||
from glance.db.sqlalchemy.alembic_migrations import data_migrations
|
||||
from glance.db.sqlalchemy import api as db_api
|
||||
from glance.db.sqlalchemy import metadata
|
||||
from glance.i18n import _
|
||||
|
|
@ -88,7 +89,7 @@ class DbCommands(object):
|
|||
'alembic migration control.'))
|
||||
|
||||
@args('--version', metavar='<version>', help='Database version')
|
||||
def upgrade(self, version='heads'):
|
||||
def upgrade(self, version=db_migration.LATEST_REVISION):
|
||||
"""Upgrade the database's migration level"""
|
||||
self.sync(version)
|
||||
|
||||
|
|
@ -105,12 +106,12 @@ class DbCommands(object):
|
|||
"revision:"), version)
|
||||
|
||||
@args('--version', metavar='<version>', help='Database version')
|
||||
def sync(self, version='heads'):
|
||||
def sync(self, version=db_migration.LATEST_REVISION):
|
||||
"""
|
||||
Place an existing database under migration control and upgrade it.
|
||||
"""
|
||||
if version is None:
|
||||
version = 'heads'
|
||||
version = db_migration.LATEST_REVISION
|
||||
|
||||
alembic_migrations.place_database_under_alembic_control()
|
||||
|
||||
|
|
@ -118,14 +119,76 @@ class DbCommands(object):
|
|||
alembic_command.upgrade(a_config, version)
|
||||
heads = alembic_migrations.get_current_alembic_heads()
|
||||
if heads is None:
|
||||
raise Exception("Database sync failed")
|
||||
raise exception.GlanceException("Database sync failed")
|
||||
revs = ", ".join(heads)
|
||||
if version is 'heads':
|
||||
if version == 'heads':
|
||||
print(_("Upgraded database, current revision(s):"), revs)
|
||||
else:
|
||||
print(_('Upgraded database to: %(v)s, current revision(s): %(r)s')
|
||||
% {'v': version, 'r': revs})
|
||||
|
||||
def expand(self):
|
||||
"""Run the expansion phase of a rolling upgrade procedure."""
|
||||
expand_head = alembic_migrations.get_alembic_branch_head(
|
||||
db_migration.EXPAND_BRANCH)
|
||||
if not expand_head:
|
||||
sys.exit(_('Database expansion failed. Couldn\'t find head '
|
||||
'revision of expand branch.'))
|
||||
|
||||
self.sync(version=expand_head)
|
||||
|
||||
curr_heads = alembic_migrations.get_current_alembic_heads()
|
||||
if expand_head not in curr_heads:
|
||||
sys.exit(_('Database expansion failed. Database expansion should '
|
||||
'have brought the database version up to "%(e_rev)s" '
|
||||
'revision. But, current revisions are: %(curr_revs)s ')
|
||||
% {'e_rev': expand_head, 'curr_revs': curr_heads})
|
||||
|
||||
def contract(self):
|
||||
"""Run the contraction phase of a rolling upgrade procedure."""
|
||||
contract_head = alembic_migrations.get_alembic_branch_head(
|
||||
db_migration.CONTRACT_BRANCH)
|
||||
if not contract_head:
|
||||
sys.exit(_('Database contraction failed. Couldn\'t find head '
|
||||
'revision of contract branch.'))
|
||||
|
||||
curr_heads = alembic_migrations.get_current_alembic_heads()
|
||||
expand_head = alembic_migrations.get_alembic_branch_head(
|
||||
db_migration.EXPAND_BRANCH)
|
||||
if expand_head not in curr_heads:
|
||||
sys.exit(_('Database contraction did not run. Database '
|
||||
'contraction cannot be run before database expansion. '
|
||||
'Run database expansion first using '
|
||||
'"glance-manage db expand"'))
|
||||
|
||||
if data_migrations.has_pending_migrations(db_api.get_engine()):
|
||||
sys.exit(_('Database contraction did not run. Database '
|
||||
'contraction cannot be run before data migration is '
|
||||
'complete. Run data migration using "glance-manage db '
|
||||
'migrate".'))
|
||||
|
||||
self.sync(version=contract_head)
|
||||
|
||||
curr_heads = alembic_migrations.get_current_alembic_heads()
|
||||
if contract_head not in curr_heads:
|
||||
sys.exit(_('Database contraction failed. Database contraction '
|
||||
'should have brought the database version up to '
|
||||
'"%(e_rev)s" revision. But, current revisions are: '
|
||||
'%(curr_revs)s ') % {'e_rev': expand_head,
|
||||
'curr_revs': curr_heads})
|
||||
|
||||
def migrate(self):
|
||||
curr_heads = alembic_migrations.get_current_alembic_heads()
|
||||
expand_head = alembic_migrations.get_alembic_branch_head(
|
||||
db_migration.EXPAND_BRANCH)
|
||||
if expand_head not in curr_heads:
|
||||
sys.exit(_('Data migration did not run. Data migration cannot be '
|
||||
'run before database expansion. Run database '
|
||||
'expansion first using "glance-manage db expand"'))
|
||||
|
||||
rows_migrated = data_migrations.migrate(db_api.get_engine())
|
||||
print(_('Migrated %s rows') % rows_migrated)
|
||||
|
||||
@args('--path', metavar='<path>', help='Path to the directory or file '
|
||||
'where json metadata is stored')
|
||||
@args('--merge', action='store_true',
|
||||
|
|
@ -198,15 +261,24 @@ class DbLegacyCommands(object):
|
|||
def version(self):
|
||||
self.command_object.version()
|
||||
|
||||
def upgrade(self, version='heads'):
|
||||
def upgrade(self, version=db_migration.LATEST_REVISION):
|
||||
self.command_object.upgrade(CONF.command.version)
|
||||
|
||||
def version_control(self, version=db_migration.ALEMBIC_INIT_VERSION):
|
||||
self.command_object.version_control(CONF.command.version)
|
||||
|
||||
def sync(self, version='heads'):
|
||||
def sync(self, version=db_migration.LATEST_REVISION):
|
||||
self.command_object.sync(CONF.command.version)
|
||||
|
||||
def expand(self):
|
||||
self.command_object.expand()
|
||||
|
||||
def contract(self):
|
||||
self.command_object.contract()
|
||||
|
||||
def migrate(self):
|
||||
self.command_object.migrate()
|
||||
|
||||
def load_metadefs(self, path=None, merge=False,
|
||||
prefer_new=False, overwrite=False):
|
||||
self.command_object.load_metadefs(CONF.command.path,
|
||||
|
|
@ -244,6 +316,18 @@ def add_legacy_command_parsers(command_object, subparsers):
|
|||
parser.add_argument('version', nargs='?')
|
||||
parser.set_defaults(action='db_sync')
|
||||
|
||||
parser = subparsers.add_parser('db_expand')
|
||||
parser.set_defaults(action_fn=legacy_command_object.expand)
|
||||
parser.set_defaults(action='db_expand')
|
||||
|
||||
parser = subparsers.add_parser('db_contract')
|
||||
parser.set_defaults(action_fn=legacy_command_object.contract)
|
||||
parser.set_defaults(action='db_contract')
|
||||
|
||||
parser = subparsers.add_parser('db_migrate')
|
||||
parser.set_defaults(action_fn=legacy_command_object.migrate)
|
||||
parser.set_defaults(action='db_migrate')
|
||||
|
||||
parser = subparsers.add_parser('db_load_metadefs')
|
||||
parser.set_defaults(action_fn=legacy_command_object.load_metadefs)
|
||||
parser.add_argument('path', nargs='?')
|
||||
|
|
|
|||
|
|
@ -557,3 +557,9 @@ class InvalidJsonPatchPath(JsonPatchException):
|
|||
def __init__(self, message=None, *args, **kwargs):
|
||||
self.explanation = kwargs.get("explanation")
|
||||
super(InvalidJsonPatchPath, self).__init__(message, *args, **kwargs)
|
||||
|
||||
|
||||
class InvalidDataMigrationScript(GlanceException):
|
||||
message = _("Invalid data migration script '%(script)s'. A valid data "
|
||||
"migration script must implement functions 'has_migrations' "
|
||||
"and 'migrate'.")
|
||||
|
|
|
|||
|
|
@ -43,7 +43,13 @@ def get_backend():
|
|||
cfg.CONF.database.backend).driver
|
||||
return _IMPL
|
||||
|
||||
|
||||
# Migration-related constants
|
||||
EXPAND_BRANCH = 'expand'
|
||||
CONTRACT_BRANCH = 'contract'
|
||||
CURRENT_RELEASE = 'ocata'
|
||||
ALEMBIC_INIT_VERSION = 'liberty'
|
||||
LATEST_REVISION = 'ocata01'
|
||||
INIT_VERSION = 0
|
||||
|
||||
MIGRATE_REPO_PATH = os.path.join(
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ import sys
|
|||
from alembic import command as alembic_command
|
||||
from alembic import config as alembic_config
|
||||
from alembic import migration as alembic_migration
|
||||
from alembic import script as alembic_script
|
||||
from oslo_db import exception as db_exception
|
||||
from oslo_db.sqlalchemy import migration as sqla_migration
|
||||
|
||||
|
|
@ -98,3 +99,10 @@ def place_database_under_alembic_control():
|
|||
print(_("Placing database under Alembic's migration control at "
|
||||
"revision:"), alembic_version)
|
||||
alembic_command.stamp(a_config, alembic_version)
|
||||
|
||||
|
||||
def get_alembic_branch_head(branch):
|
||||
"""Return head revision name for particular branch"""
|
||||
a_config = get_alembic_config()
|
||||
script = alembic_script.ScriptDirectory.from_config(a_config)
|
||||
return script.revision_map.get_current_head(branch)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,70 @@
|
|||
# Copyright 2016 Rackspace
|
||||
# Copyright 2016 Intel Corporation
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import importlib
|
||||
import os.path
|
||||
import pkgutil
|
||||
|
||||
from glance.common import exception
|
||||
from glance.db import migration as db_migrations
|
||||
from glance.db.sqlalchemy import api as db_api
|
||||
|
||||
|
||||
def _find_migration_modules(release):
|
||||
migrations = list()
|
||||
for _, module_name, _ in pkgutil.iter_modules([os.path.dirname(__file__)]):
|
||||
if module_name.startswith(release):
|
||||
migrations.append(module_name)
|
||||
|
||||
migration_modules = list()
|
||||
for migration in sorted(migrations):
|
||||
module = importlib.import_module('.'.join([__package__, migration]))
|
||||
has_migrations_function = getattr(module, 'has_migrations', None)
|
||||
migrate_function = getattr(module, 'migrate', None)
|
||||
|
||||
if has_migrations_function is None or migrate_function is None:
|
||||
raise exception.InvalidDataMigrationScript(script=module.__name__)
|
||||
|
||||
migration_modules.append(module)
|
||||
|
||||
return migration_modules
|
||||
|
||||
|
||||
def _run_migrations(engine, migrations):
|
||||
rows_migrated = 0
|
||||
for migration in migrations:
|
||||
if migration.has_migrations(engine):
|
||||
rows_migrated += migration.migrate(engine)
|
||||
|
||||
return rows_migrated
|
||||
|
||||
|
||||
def has_pending_migrations(engine=None):
|
||||
if not engine:
|
||||
engine = db_api.get_engine()
|
||||
|
||||
migrations = _find_migration_modules(db_migrations.CURRENT_RELEASE)
|
||||
if not migrations:
|
||||
return False
|
||||
return any([x.has_migrations(engine) for x in migrations])
|
||||
|
||||
|
||||
def migrate(engine=None):
|
||||
if not engine:
|
||||
engine = db_api.get_engine()
|
||||
|
||||
migrations = _find_migration_modules(db_migrations.CURRENT_RELEASE)
|
||||
rows_migrated = _run_migrations(engine, migrations)
|
||||
return rows_migrated
|
||||
|
|
@ -23,6 +23,7 @@ from oslo_db.sqlalchemy import test_base
|
|||
from oslo_db.sqlalchemy import test_migrations
|
||||
import sqlalchemy.types as types
|
||||
|
||||
from glance.db import migration as dm
|
||||
from glance.db.sqlalchemy import alembic_migrations
|
||||
from glance.db.sqlalchemy.alembic_migrations import versions
|
||||
from glance.db.sqlalchemy import models
|
||||
|
|
@ -35,7 +36,8 @@ class AlembicMigrationsMixin(object):
|
|||
|
||||
def _get_revisions(self, config):
|
||||
scripts_dir = alembic_script.ScriptDirectory.from_config(config)
|
||||
revisions = list(scripts_dir.walk_revisions(base='base', head='heads'))
|
||||
revisions = list(scripts_dir.walk_revisions(base='base',
|
||||
head=dm.LATEST_REVISION))
|
||||
revisions = list(reversed(revisions))
|
||||
revisions = [rev.revision for rev in revisions]
|
||||
return revisions
|
||||
|
|
|
|||
|
|
@ -47,28 +47,20 @@ class TestGlanceManage(functional.FunctionalTest):
|
|||
(sys.executable, self.conf_filepath))
|
||||
execute(cmd, raise_error=True)
|
||||
|
||||
def _assert_tables(self):
|
||||
cmd = "sqlite3 %s '.schema'" % self.db_filepath
|
||||
def _assert_table_exists(self, db_table):
|
||||
cmd = ("sqlite3 {0} \"SELECT name FROM sqlite_master WHERE "
|
||||
"type='table' AND name='{1}'\"").format(self.db_filepath,
|
||||
db_table)
|
||||
exitcode, out, err = execute(cmd, raise_error=True)
|
||||
|
||||
self.assertIn('CREATE TABLE images', out)
|
||||
self.assertIn('CREATE TABLE image_tags', out)
|
||||
self.assertIn('CREATE TABLE image_locations', out)
|
||||
|
||||
# NOTE(bcwaldon): For some reason we need double-quotes around
|
||||
# these two table names
|
||||
# NOTE(vsergeyev): There are some cases when we have no double-quotes
|
||||
self.assertTrue(
|
||||
'CREATE TABLE "image_members"' in out or
|
||||
'CREATE TABLE image_members' in out)
|
||||
self.assertTrue(
|
||||
'CREATE TABLE "image_properties"' in out or
|
||||
'CREATE TABLE image_properties' in out)
|
||||
msg = "Expected table {0} was not found in the schema".format(db_table)
|
||||
self.assertEqual(out.rstrip(), db_table, msg)
|
||||
|
||||
@depends_on_exe('sqlite3')
|
||||
@skip_if_disabled
|
||||
def test_db_creation(self):
|
||||
"""Test DB creation by db_sync on a fresh DB"""
|
||||
"""Test schema creation by db_sync on a fresh DB"""
|
||||
self._sync_db()
|
||||
|
||||
self._assert_tables()
|
||||
for table in ['images', 'image_tags', 'image_locations',
|
||||
'image_members', 'image_properties']:
|
||||
self._assert_table_exists(table)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,204 @@
|
|||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import mock
|
||||
|
||||
from glance.db.sqlalchemy.alembic_migrations import data_migrations
|
||||
from glance.tests import utils as test_utils
|
||||
|
||||
|
||||
class TestDataMigrationFramework(test_utils.BaseTestCase):
|
||||
|
||||
@mock.patch('glance.db.sqlalchemy.alembic_migrations.data_migrations'
|
||||
'._find_migration_modules')
|
||||
def test_has_pending_migrations_no_migrations(self, mock_find):
|
||||
mock_find.return_value = None
|
||||
self.assertFalse(data_migrations.has_pending_migrations(mock.Mock()))
|
||||
|
||||
@mock.patch('glance.db.sqlalchemy.alembic_migrations.data_migrations'
|
||||
'._find_migration_modules')
|
||||
def test_has_pending_migrations_one_migration_no_pending(self, mock_find):
|
||||
mock_migration1 = mock.Mock()
|
||||
mock_migration1.has_migrations.return_value = False
|
||||
mock_find.return_value = [mock_migration1]
|
||||
|
||||
self.assertFalse(data_migrations.has_pending_migrations(mock.Mock()))
|
||||
|
||||
@mock.patch('glance.db.sqlalchemy.alembic_migrations.data_migrations'
|
||||
'._find_migration_modules')
|
||||
def test_has_pending_migrations_one_migration_with_pending(self,
|
||||
mock_find):
|
||||
mock_migration1 = mock.Mock()
|
||||
mock_migration1.has_migrations.return_value = True
|
||||
mock_find.return_value = [mock_migration1]
|
||||
|
||||
self.assertTrue(data_migrations.has_pending_migrations(mock.Mock()))
|
||||
|
||||
@mock.patch('glance.db.sqlalchemy.alembic_migrations.data_migrations'
|
||||
'._find_migration_modules')
|
||||
def test_has_pending_migrations_mult_migration_no_pending(self, mock_find):
|
||||
mock_migration1 = mock.Mock()
|
||||
mock_migration1.has_migrations.return_value = False
|
||||
mock_migration2 = mock.Mock()
|
||||
mock_migration2.has_migrations.return_value = False
|
||||
mock_migration3 = mock.Mock()
|
||||
mock_migration3.has_migrations.return_value = False
|
||||
|
||||
mock_find.return_value = [mock_migration1, mock_migration2,
|
||||
mock_migration3]
|
||||
|
||||
self.assertFalse(data_migrations.has_pending_migrations(mock.Mock()))
|
||||
|
||||
@mock.patch('glance.db.sqlalchemy.alembic_migrations.data_migrations'
|
||||
'._find_migration_modules')
|
||||
def test_has_pending_migrations_mult_migration_one_pending(self,
|
||||
mock_find):
|
||||
mock_migration1 = mock.Mock()
|
||||
mock_migration1.has_migrations.return_value = False
|
||||
mock_migration2 = mock.Mock()
|
||||
mock_migration2.has_migrations.return_value = True
|
||||
mock_migration3 = mock.Mock()
|
||||
mock_migration3.has_migrations.return_value = False
|
||||
|
||||
mock_find.return_value = [mock_migration1, mock_migration2,
|
||||
mock_migration3]
|
||||
|
||||
self.assertTrue(data_migrations.has_pending_migrations(mock.Mock()))
|
||||
|
||||
@mock.patch('glance.db.sqlalchemy.alembic_migrations.data_migrations'
|
||||
'._find_migration_modules')
|
||||
def test_has_pending_migrations_mult_migration_some_pending(self,
|
||||
mock_find):
|
||||
mock_migration1 = mock.Mock()
|
||||
mock_migration1.has_migrations.return_value = False
|
||||
mock_migration2 = mock.Mock()
|
||||
mock_migration2.has_migrations.return_value = True
|
||||
mock_migration3 = mock.Mock()
|
||||
mock_migration3.has_migrations.return_value = False
|
||||
mock_migration4 = mock.Mock()
|
||||
mock_migration4.has_migrations.return_value = True
|
||||
|
||||
mock_find.return_value = [mock_migration1, mock_migration2,
|
||||
mock_migration3, mock_migration4]
|
||||
|
||||
self.assertTrue(data_migrations.has_pending_migrations(mock.Mock()))
|
||||
|
||||
@mock.patch('importlib.import_module')
|
||||
@mock.patch('pkgutil.iter_modules')
|
||||
def test_find_migrations(self, mock_iter, mock_import):
|
||||
def fake_iter_modules(blah):
|
||||
yield 'blah', 'ocata01', 'blah'
|
||||
yield 'blah', 'ocata02', 'blah'
|
||||
yield 'blah', 'pike01', 'blah'
|
||||
yield 'blah', 'newton', 'blah'
|
||||
yield 'blah', 'mitaka456', 'blah'
|
||||
|
||||
mock_iter.side_effect = fake_iter_modules
|
||||
|
||||
ocata1 = mock.Mock()
|
||||
ocata1.has_migrations.return_value = mock.Mock()
|
||||
ocata1.migrate.return_value = mock.Mock()
|
||||
ocata2 = mock.Mock()
|
||||
ocata2.has_migrations.return_value = mock.Mock()
|
||||
ocata2.migrate.return_value = mock.Mock()
|
||||
|
||||
fake_imported_modules = [ocata1, ocata2]
|
||||
mock_import.side_effect = fake_imported_modules
|
||||
|
||||
actual = data_migrations._find_migration_modules('ocata')
|
||||
self.assertEqual(2, len(actual))
|
||||
self.assertEqual(fake_imported_modules, actual)
|
||||
|
||||
@mock.patch('pkgutil.iter_modules')
|
||||
def test_find_migrations_no_migrations(self, mock_iter):
|
||||
def fake_iter_modules(blah):
|
||||
yield 'blah', 'liberty01', 'blah'
|
||||
yield 'blah', 'kilo01', 'blah'
|
||||
yield 'blah', 'mitaka01', 'blah'
|
||||
yield 'blah', 'newton01', 'blah'
|
||||
yield 'blah', 'pike01', 'blah'
|
||||
|
||||
mock_iter.side_effect = fake_iter_modules
|
||||
|
||||
actual = data_migrations._find_migration_modules('ocata')
|
||||
self.assertEqual(0, len(actual))
|
||||
self.assertEqual([], actual)
|
||||
|
||||
def test_run_migrations(self):
|
||||
ocata1 = mock.Mock()
|
||||
ocata1.has_migrations.return_value = True
|
||||
ocata1.migrate.return_value = 100
|
||||
ocata2 = mock.Mock()
|
||||
ocata2.has_migrations.return_value = True
|
||||
ocata2.migrate.return_value = 50
|
||||
migrations = [ocata1, ocata2]
|
||||
|
||||
engine = mock.Mock()
|
||||
actual = data_migrations._run_migrations(engine, migrations)
|
||||
self.assertEqual(150, actual)
|
||||
ocata1.has_migrations.assert_called_once_with(engine)
|
||||
ocata1.migrate.assert_called_once_with(engine)
|
||||
ocata2.has_migrations.assert_called_once_with(engine)
|
||||
ocata2.migrate.assert_called_once_with(engine)
|
||||
|
||||
def test_run_migrations_with_one_pending_migration(self):
|
||||
ocata1 = mock.Mock()
|
||||
ocata1.has_migrations.return_value = False
|
||||
ocata1.migrate.return_value = 0
|
||||
ocata2 = mock.Mock()
|
||||
ocata2.has_migrations.return_value = True
|
||||
ocata2.migrate.return_value = 50
|
||||
migrations = [ocata1, ocata2]
|
||||
|
||||
engine = mock.Mock()
|
||||
actual = data_migrations._run_migrations(engine, migrations)
|
||||
self.assertEqual(50, actual)
|
||||
ocata1.has_migrations.assert_called_once_with(engine)
|
||||
ocata1.migrate.assert_not_called()
|
||||
ocata2.has_migrations.assert_called_once_with(engine)
|
||||
ocata2.migrate.assert_called_once_with(engine)
|
||||
|
||||
def test_run_migrations_with_no_migrations(self):
|
||||
migrations = []
|
||||
|
||||
actual = data_migrations._run_migrations(mock.Mock(), migrations)
|
||||
self.assertEqual(0, actual)
|
||||
|
||||
@mock.patch('importlib.import_module')
|
||||
@mock.patch('pkgutil.iter_modules')
|
||||
def test_migrate(self, mock_iter, mock_import):
|
||||
def fake_iter_modules(blah):
|
||||
yield 'blah', 'ocata01', 'blah'
|
||||
yield 'blah', 'ocata02', 'blah'
|
||||
yield 'blah', 'pike01', 'blah'
|
||||
yield 'blah', 'newton', 'blah'
|
||||
yield 'blah', 'mitaka456', 'blah'
|
||||
|
||||
mock_iter.side_effect = fake_iter_modules
|
||||
|
||||
ocata1 = mock.Mock()
|
||||
ocata1.has_migrations.return_value = True
|
||||
ocata1.migrate.return_value = 100
|
||||
ocata2 = mock.Mock()
|
||||
ocata2.has_migrations.return_value = True
|
||||
ocata2.migrate.return_value = 50
|
||||
|
||||
fake_imported_modules = [ocata1, ocata2]
|
||||
mock_import.side_effect = fake_imported_modules
|
||||
|
||||
engine = mock.Mock()
|
||||
actual = data_migrations.migrate(engine)
|
||||
self.assertEqual(150, actual)
|
||||
ocata1.has_migrations.assert_called_once_with(engine)
|
||||
ocata1.migrate.assert_called_once_with(engine)
|
||||
ocata2.has_migrations.assert_called_once_with(engine)
|
||||
ocata2.migrate.assert_called_once_with(engine)
|
||||
|
|
@ -78,6 +78,21 @@ class TestLegacyManage(TestManageBase):
|
|||
self._main_test_helper(['glance.cmd.manage', 'db_upgrade', 'liberty'],
|
||||
manage.DbCommands.upgrade, 'liberty')
|
||||
|
||||
@mock.patch.object(manage.DbCommands, 'expand')
|
||||
def test_legacy_db_expand(self, db_expand):
|
||||
self._main_test_helper(['glance.cmd.manage', 'db_expand'],
|
||||
manage.DbCommands.expand)
|
||||
|
||||
@mock.patch.object(manage.DbCommands, 'migrate')
|
||||
def test_legacy_db_migrate(self, db_migrate):
|
||||
self._main_test_helper(['glance.cmd.manage', 'db_migrate'],
|
||||
manage.DbCommands.migrate)
|
||||
|
||||
@mock.patch.object(manage.DbCommands, 'contract')
|
||||
def test_legacy_db_contract(self, db_contract):
|
||||
self._main_test_helper(['glance.cmd.manage', 'db_contract'],
|
||||
manage.DbCommands.contract)
|
||||
|
||||
def test_db_metadefs_unload(self):
|
||||
db_metadata.db_unload_metadefs = mock.Mock()
|
||||
self._main_test_helper(['glance.cmd.manage', 'db_unload_metadefs'],
|
||||
|
|
@ -172,6 +187,21 @@ class TestManage(TestManageBase):
|
|||
'upgrade', 'liberty'],
|
||||
manage.DbCommands.upgrade, 'liberty')
|
||||
|
||||
@mock.patch.object(manage.DbCommands, 'expand')
|
||||
def test_db_expand(self, expand):
|
||||
self._main_test_helper(['glance.cmd.manage', 'db', 'expand'],
|
||||
manage.DbCommands.expand)
|
||||
|
||||
@mock.patch.object(manage.DbCommands, 'migrate')
|
||||
def test_db_migrate(self, migrate):
|
||||
self._main_test_helper(['glance.cmd.manage', 'db', 'migrate'],
|
||||
manage.DbCommands.migrate)
|
||||
|
||||
@mock.patch.object(manage.DbCommands, 'contract')
|
||||
def test_db_contract(self, contract):
|
||||
self._main_test_helper(['glance.cmd.manage', 'db', 'contract'],
|
||||
manage.DbCommands.contract)
|
||||
|
||||
def test_db_metadefs_unload(self):
|
||||
db_metadata.db_unload_metadefs = mock.Mock()
|
||||
self._main_test_helper(['glance.cmd.manage', 'db', 'unload_metadefs'],
|
||||
|
|
|
|||
|
|
@ -43,6 +43,7 @@ from glance.common import timeutils
|
|||
from glance.common import utils
|
||||
from glance.common import wsgi
|
||||
from glance import context
|
||||
from glance.db import migration as db_migration
|
||||
from glance.db.sqlalchemy import alembic_migrations
|
||||
from glance.db.sqlalchemy import api as db_api
|
||||
from glance.db.sqlalchemy import models as db_models
|
||||
|
|
@ -677,7 +678,7 @@ class HttplibWsgiAdapter(object):
|
|||
def db_sync(version=None, engine=None):
|
||||
"""Migrate the database to `version` or the most recent version."""
|
||||
if version is None:
|
||||
version = 'heads'
|
||||
version = db_migration.LATEST_REVISION
|
||||
if engine is None:
|
||||
engine = db_api.get_engine()
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue