Under Python 3.12 this results in a syntax error rather than
M329 being detected on line 8 of the test code.
Change-Id: Ia1bb4dfb43b00115cde1158681d34d6ad7c7d744
Resolve the following RemovedIn20Warning warning:
The current statement is being autocommitted using implicit
autocommit, which will be removed in SQLAlchemy 2.0. Use the .begin()
method of Engine or Connection in order to use an explicit transaction
for DML and DDL statements.
This is the beefiest one due to our extensive reliance on auto-commit.
Change-Id: Iebf9d022c312b8f5457ff34eb497cdb851aa4901
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Resolve the following RemovedIn20Warning warning:
The legacy calling style of select() is deprecated and will be removed
in SQLAlchemy 2.0. Please use the new calling style described at
select().
Change-Id: I00206489e97468d98a3a516c7c081c70348e3998
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Resolve the following RemovedIn20Warning warning:
Using strings to indicate column or relationship paths in loader
options is deprecated and will be removed in SQLAlchemy 2.0. Please
use the class-bound attribute directly.
Change-Id: I1b71331796c2730713797043489b6c11710b9851
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Resolve the following RemovedIn20Warning warning:
Passing a string to Connection.execute() is deprecated and will be
removed in version 2.0. Use the text() construct, or the
Connection.exec_driver_sql() method to invoke a driver-level SQL
string.
We do the latter.
Change-Id: I0ee518022772beeb0298e12b5b36279fd384fb4a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
These are highlighted by the new WarningsFixture.
Change-Id: If9a27784e4f4fde61dff73c1f6fde99324e1aa07
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
These are no longer used. Curiously, sqlalchemy-migrate is not in the
dependencies and was instead being pulled in as a transitive dependency
of oslo.db. This will therefore disappear when oslo.db is bumped to 13.x
or later in upper-constraints, since that version removes
sqlalchemy-migrate support.
Change-Id: I4ea8ae36c9f8d3e5e5a9ac9548c5fe8f975cd666
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This is easier than Cinder, Nova etc. but more difficult than Heat.
Masakari hasn't had many database migrations in recent cycles but it did
have on in Antelope. This means we need to handle people doing
skip-level upgrades and validate which version of the legacy migrations
they are currently on. We support users coming from both Zed and
Antelope and anyone else will need to go through an older version of
Masakari to update their database migrations first. Other than this
difference, the logic is pretty similar: as with other projects, we
simply determine if we're upgrading a deployment that was previously
using sqlalchemy-migrate, upgrading a deployment that has already
migrated to alembic, or deploying a new deployment, and adjust
accordingly.
In addition, we also have to consider Taskflow's migrations. These were
previously being run once as part of the legacy
006_add_persistence_tables migrations. Since Taskflow uses Alembic under
the hood, it's safe to run every time. The presence of Taskflow does
force us to use a different table name in Masakari though.
Note that one curious side-effect of this is that the order than table
rows are purged change. This appears to be happening because the
notification table is now being created in the initial Alembic
migration, which alters the return value of 'MetaData.sorted_tables'.
In any case, the fix is simply a case of adjusting this order in the
tests.
Change-Id: I5285d7cc3c6da0059c0908cedc195b2262cb1fce
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Alembic's auto-generate functionality doesn't preserve index names or
column order. This causes comparisons between databases to fail. Update
the initial alembic migration to match the schema's generated by
sqlalchemy-migrate.
Change-Id: I16763a82cc2db9138882ace2dad6f0cae330b9e8
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This was generated using the same approach first used in heat [1], but
with a difference to account for the fact that, unlike heat, we have a
migration added in recent times. We generate our initial schema
automatically based on our models using alembic's auto-generate
functionality:
alembic --config masakari/db/sqlalchemy/alembic.ini \
revision -m 'Initial empty revision'
alembic --config masakari/db/sqlalchemy/alembic.ini \
upgrade head
alembic --config masakari/db/sqlalchemy/alembic.ini \
revision -m 'Auto-generated revision' --autogenerate
The two files were then combined, formatting and imports adjusted. We
then *remove* the parts of the migration related to the 'vm_moves' table
and run the autogeneration step again:
alembic --config masakari/db/sqlalchemy/alembic.ini \
revision -m 'Add vm moves table' --autogenerate
As noted above, this is necessary since we're going to need to support
alembic-based migrations for users jumping from e.g. Zed to Bobcat.
Once done, the schema of these migrations and the sqlalchemy-migrate
migrations are compared. This step was done by comparing the schemas of
a database created by the sqlalchemy-migrate tool to the one created by
alembic. First, we compared the initial alembic migration which
corresponds to the state of the database after sqlalchemy-migrate
migration 007:
alembic --config masakari/db/sqlalchemy/alembic.ini \
upgrade 8f848eb45d03
python masakari/db/sqlalchemy/migrate_repo/manage.py version_control \
sqlite:///masakari-old.db 000
python masakari/db/sqlalchemy/migrate_repo/manage.py upgrade \
sqlite:///masakari-old.db 007
With the two databases created, we can compare them using the
methodologies described in [2]. Once this is done, we do the same things
for the final migration to generate its own alembic variation of the
migration.
alembic --config masakari/db/sqlalchemy/alembic.ini \
upgrade head
python masakari/db/sqlalchemy/migrate_repo/manage.py upgrade \
sqlite:///masakari-old.db
These last steps highlight some small differences between the schemas.
These changes are kept to a separate change to make them more obvious.
[1] https://review.opendev.org/c/openstack/heat/+/878350
[2] https://that.guru/blog/comparing-nova-db-migrations/
Change-Id: I6d0f27ba1d81e75010e8b56c70172ccf32c1abb3
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
No migrations yet: this is simply the output of 'alembic init' with some
minor tweaks.
Change-Id: Ib9423c2f751d7ec0a0dec89bdc39f9b6ab043655
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
oslo.db has killed the idea of the migration wrappers. This knob is
therefore no longer useful.
Change-Id: Ia61b3291668b17b13d5f674c0dceff29a7b9b4cf
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We also bump the hacking version and remove some unnecessary notes from
the top of the requirements files: these don't apply with the dependency
resolver introduced in pip 20.3.
Change-Id: Ifebaec916264bfd10eec13040295719fd47ae107
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
1.Add sqlalchemy-migrate dependence in test-requirements.txt.
It will remove the dependence after this project switchs to alembic.
Remove "check-requirements" temporarily.
2.Fix unit tests broken by olso.utils
Some of the object unit tests grub Mock object unintentionally, and
that results in failure during initializing an versioned object,
because the Mock object does not present its version correctly.
3.make functional jobs not voting
Fix functional jobs later.
Change-Id: Id3c952a54e77952cfd0d76d259f49a14edca1d8b
A recent change to oslo.messaging to redact contexts before sending them
to a notification bus broke masakari due to the invalid default for
read_deleted. This ensures that a "None" value for read_deleted ends up
being interpreted as a 'none'.
Related-bug: #2030976
Change-Id: Ic9b895601f301c1e9ba135766493d4234fa9b50b
This feature is mainly to persist vm moves information of one host
notification into the database. It also provides a new 'VMove' api,
which could help users to insight the process or result of the host
recovery workflow, such as which vms evacuated succeed or failed, and
which ones are still evacuating.
Implements: BP vm-evacuations-for-host-recovery
Change-Id: I334af06ef526bf11dfe5030e8cba210b98f1ceea
For instance or process failure workflow, the failure notification
would stuck into running status if timeout.
Closes-Bug: #1996835
Change-Id: I61e941ab9dd831369fcc46a132ae2b11c1dd23ba
If an user sends host notification without host status:
openstack notification create COMPUTE_HOST <host> <date> '{"event": "STOPPED"}'
logging registers silly error like object None has not method 'upper' and
notification gets status 'running', host is set in maintenance status.
It's impossible to set manually host status to non-maintanence
until there are host-related running notifications.
Running notifications are expired in 24 hours by default.
The patch makes engine to set 'ignore' status for such
notifications.
Closes-Bug: 1960619
Change-Id: Id873b3300b6de49082805654c85414a8868dd925
Nova-compute would periodically sync the instance status. So in the
instance failure recovery workflow,it would stop instance random failed
because of conflict, which will terminate the recovery workflow.
This patch can catch the Confict exception and continue the recovery
workflow if the instance already stopped.
Closes-Bug: 1980736
Change-Id: I59a1f9d7078614c1ddc8f4c362e967a15b8ec5e8
Two concurrent events that host is down
might be accepted by different instances of API
which triggers two evacuation workflows for instances.
Patch introduces distributed lock for creation of
host type notifications. It solves the issue.
Closes-Bug: 1961110
Change-Id: Ie8f10b14f29a8548181560cd8a26b4dc79afc3dc
This change fixes the remaining issue with oslo.context >= 4.0.0.
In oslo.context < 4.0, RequestContext.from_dict looks up 'tenant' key
from dict to set project_id while the method looks up 'project_id' key
since oslo.context 4.0. This change implements a logic to make unit
tests compatible with the both behavior, but the one for older version
can be removed once we get oslo.context >= 4.0.0 in upper-constraints.
Change-Id: I463e34ef36891c5ef29e88cb645c38a6cccac2f2
The tenant argument of RequestContext is longer available since
oslo.context 4.0.0. This change removes usage of the deprecated
argument to prepare for version bump.
Change-Id: If237c6abf791526a7c20e393525e83c0a2d1a9ed
Masakari never sets reason why compute service was disabled.
"disabled reason" was added in config.
Closes-Bug: 1936181
Change-Id: I998f7884195b93927773c7186d61c13670a53662
The various test cases and mixins in 'oslo_db.sqlalchemy.test_base' have
been deprecated since oslo.db 4.34.0 (March 2018). Remove use of these.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I8f4d55c397f61777bec830855f229e5fe5af90dc
The patch fix a spelling mistakes in unit test.In notification unit test,
test use fake host named 'fake_host1',normally it is 'fake_host_1'.
Change-Id: Ic607edef6ce6c8b90eb46a840f483210ba6019f6
The .count() method is gone from the Table class.
Let's use func.count in its stead.
Closes-Bug: #1926399
Change-Id: Ica81271fe85478fdffae4ea0b6f4ed3b168f4552
James Page