Compacts Icehouse database migrations into a single migration,
'234_icehouse.py'.
Users will now need to update to Icehouse before updating to Juno or
later.
Specific changes include:
- Drop all references to dump tables entirely, since they were added in
216 and dropped in 232.
- Drop the 'compute_node_stats' table in favour of a 'stats' column on
the 'compute_nodes' table
- Add 'metrics' and 'extra_resources' to 'compute_nodes'
- Add 'ephemeral_key_uuid' to 'instances'
- Add 'host' and 'details' to 'instance_actions_events'
- Add an index to the 'reservations' table covering the 'deleted' and
'expire' columns
Change-Id: I17db7cdaad2c6368092b4fb00d5959711ad249f9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We were simply referencing this function - not calling it. This has been
this way since commit 817baeb937 was first merged way back in 2013 so
it clearly has not been missed.
Change-Id: I2623bd4fe91906e928e891fb3250c02873d7fb09
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We're going to be flattening these migrations is short order. Start by
cleaning up the existing base migration to remove some of the oddities
and generally make merging of migrations as painless as possible.
A comparison of schemas before and after this [1] shows no significant
differences.
[1] https://that.guru/blog/comparing-nova-db-migrations/
Change-Id: I883562754f9330f9c797813714d8fe5b74227027
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
If the name of the database used includes a dash, it will fail the
migrations with an error like this:
You have an error in your SQL syntax; check the manual that corresponds to
your MySQL server version for the right syntax to use near '-cell0 DEFAULT
CHARACTER SET utf8' at line 1
Even more difficult, any future migration fails with the instances table
already existing making it very hard to troubleshoot. This patch fixes
it by wrapping the schema name in the query.
Closes-Bug: #1892370
Change-Id: Ic96dc415396985737b5bdeffed61077af5f245e6
A few places in the code had conditionals including:
if something in (element):
which was clearly intended to be
if something in (element,):
(I.e. `in $tuple`, not `in element` with redundant parens) or just
if something == element:
Fix those [1] and introduce hacking rule N363 to disallow this kind of
thing in the future.
[1] NOTE: These weren't actually latent bugs because
'foo' in ('foo')
which is the same as
'foo' in 'foo'
returns True. In order to be a bug, the left operand would have to be
able to be a substring of the right:
'foo' in ('foobar') # True
'foo' in ('foobar',) # False
...which I don't think is possible in any of the scenarios found.
Change-Id: I950d07eb533e0d43466c58e36b314aaaf8560251
The i18n team has decided not to translate the logs because it
seems like it not very useful; operators prefer to have them in
English so that they can search for those strings on the internet.
Partially fix on nova/conductor, nova/console, nova/consoleauth,
nova/db and nova/image other paths will be fixed on next commits
Change-Id: I132f60cb4743f85c96ec5231d6f860cfb0815758
This removes db2 support from tree completely. This is an oddball non
open database that made doing live data migrations difficult. It is
used by 0% of users in the OpenStack User Survey.
Supporting commercial software that doesn't have users at the cost of
delivering features and fixes to our community is the wrong
tradeoff. This corrects that.
It rewrites migrations, which we typically don't ever do, but it is
better if newton fresh environments fail early in db creation instead
of very deep in the data migration process.
Change-Id: Ifeb9929e4515e3483eb65d371126afd7672b92a4
In a time long ago and a land far far away, someone thought it was a
good idea to populate the database with default flavors. That was
probably reasonable at the time, but it no longer makes sense and
in fact causes us some pain now.
This patch removes those default flavors from the database. That means
that new deploys will not have them, but doesn't actually rewrite
history in any way.
This will require changes to our docs, which largely assume the presence
of these default flavors from time zero.
DocImpact
Depends-On: Ic275887e97221d9ce5ce6f12cdcfb5ac94e300b0
Change-Id: I80b63ce1ebca01be61ac0f43d26a2992ecf16678
Notes on migration changes/tests:
1. The initial havana migration does not create any foreign keys which
require the instances.uuid column since it's nullable and DB2 won't
allow creating a unique constraint over a nullable column. To create
the foreign keys on instances.uuid, this change depends on commit
e07a2b2d4b to make instances.uuid
non-nullable and create a unique constraint on that column. Then
the new migration here creates the missing instances.uuid
related foreign keys for DB2.
2. Commit b930fb3a6b changed
test_migrations.py to use oslo.db's opportunistic test fixture
which currently only supports sqlite/mysql/postgresql. Unit test
support for DB2 therefore needs to be added to oslo.db and then
hooked into nova, but that's targeted for the 2016.1 'M' release
so is not part of this change.
3. The 247_nullable_mismatch migration is updated for DB2 because
making the pci_devices.deleted column nullable=True fails since
the column is in a UniqueConstraint already (DB2 would require
it to be in a unique index instead for the column to be nullable).
There is a foreign key involved (created in 246) so for DB2 to
have the same FKey it requires a UC and that requires the deleted
column to be non-null, which doesn't work with the SoftDeleteMixin.
So in this case the schema will diverge for DB2 until we make
the deleted column non-nullable for all tables.
4. The 252_add_instance_extra_table migration is updated for DB2
because a foreign key is created between the instance_extra
table and the instances.uuid column (which is nullable at the
time) so we have to handle that foreign key constraint creation
in the 296 migration with the other foreign keys to
instances.uuid.
5. The 271 migration is updated to add ibm_db_sa the same as
sqlite and postgresql since it works the same in this case for
DB2.
6. The API migrations need checks for DB2 to avoid a duplicate index
SQL0605W error since DB2 implicitly creates an Index when a
UniqueConstraint is created and will fail if trying to add a
duplicate index on the same column(s) that are in the UC.
Implements blueprint db2-database
Change-Id: Ic4224e2545bcdfeb236b071642f9f16d9ee3b99f
* Replace dict(obj.iteritems()) with dict(obj) where obj is a dictionary
or an oslo.db object
* Replace obj.iteritems() with obj.items() where obj is a dictionary:
obj.items() works on Python 2 and Python 3.
* Replace filter() and map() with list-comprehension using if when a list is
expected. On Python 3, filter() and map() return an iterator.
* Replace obj.keys() with list(obj.keys()) when a list is expected.
Replace obj.keys()[0] with list(obj.keys())[0]. On Python 3,
dict.keys() now returns an iterator.
* Replace (int, long) with six.integer_types
* Get the name of a function using the __name__ attribute, instead of
the func_name attribute. The func_name attribute was removed
in Python 3.
* InstanceTypeTestCase: use dict.update() to combine dictionaries
instead of using dict.items()+reduce()+dict()
* matchers.py: replace StringIO.StringIO with six.StringIO.
* tox.ini: add nova.tests.unit.db.test_db_api to Python 3.4
Blueprint nova-python3
Change-Id: Iae4f6d971818d5b38e838a83753a0707e954bb04
The TC approved a cross project spec to stop putting db downgrade into
project migration repos:
https://github.com/openstack/openstack-specs/blob/master/specs/no-downward-sql-migration.rst
This was done after quite a bit of feedback from the operator
community, and everyone was good with it.
This implements that spec for Nova, adjust the tests to pass
accordingly, and changes the test which requires downgrade functions
to one that prohibits them.
Implements bp nova-no-downward-sql-migration
Change-Id: I9b164b748998530d62bbc6baf356a9c3f61584cc
Convert the use of the incubated version of the log module
to the new oslo.log library.
Sync oslo-incubator modules to update their imports as well.
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Change-Id: Ic4932e3f58191869c30bd07a010a6e9fdcb2a12c
oslo.i18n uses different marker functions to separate the
translatable messages into different catalogs, which the translation
teams can prioritize translating. For details, please refer to:
http://docs.openstack.org/developer/oslo.i18n/guidelines.html#guidelines-for-use-in-openstack
There were not marker fuctions some places in directory network.
This commit makes changes:
* Add missing marker functions
* Use ',' instead of '%' while adding variables to log messages
Added a hacking rule for the log exception about checking
translation for it.
Change-Id: If80ea6f177bb65afcdffce71550bb38fedcc54eb
Some sqlalchemy drivers (eg: mysqlconnector) don't support
engine.execute("sql stmt 1; sql stmt 2;") and require a separate
execute() call for each SQL statement. After discussions with
sqlalchemy author, he confirmed it would be better to fix callers rather
than attempt to patch in support for multiple statements.
With this patch, nova-manage db sync succeeds using mysqlconnector.
Change-Id: I57e6ecdafe90110eaffe757f0ef0bf7b41b0f3e3
oslo.i18n provides the i18n functions that were provided by
oslo-incubator's gettextutils module. Some tests that were
using internal details of the library were removed.
Change-Id: I44cfd5552e0dd86af21073419d31622f5fdb28e0
We don't need to have the vi modelines in each source file,
it can be set in a user's vimrc if required.
Also a check is added to hacking to detect if they are re-added.
Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6
Closes-Bug: #1229324
Compacts the pre-Icehouse database migrations into a
single migration (216_havana.py).
Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.
Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.
Specific changes include:
1) Add aggregate metadata unique constraint.
2) Fix migrations index (add deleted column).
3) Add pci_devices table.
4) fix migrations_instance_uuid_and_status_idx index.
Implements: blueprint compact-havana-db-migrations
Change-Id: Id96e13b0ed65837761bd173c9e32ebcc1ff5f376
Stephen Finucane