Commit Graph

7 Commits

Author SHA1 Message Date
Ghanshyam Mann 0a1d21ff47 [goal] Migrate testing to ubuntu focal
As per victoria cycle testing runtime and community goal[1]
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).

Fixing:
- bug#1885825
Fix user creation with GRANT in MySQL 8.0(Ubuntu Focal)
Ubuntu Focal (20.04) has mysql 8.0 and with mysql 8.0 there
is no implicit user creation with GRANT. We need to
create the user first before using GRANT command.

- bug#1886298
Bump the lower constraints for required deps which added python3.8 support
in their later version.

- bug#1886296
pyflake issue and some pep8 fixes.

Story: #2007865
Task: #40209

Related-Bug: #1885825
Related-Bug: #1886298

[1] https://governance.openstack.org/tc/goals/selected/victoria/migrate-ci-cd-jobs-to-ubuntu-focal

Change-Id: Iac9652b0d36ef208075da506957cc87f22912c45
2020-08-16 21:07:26 +00:00
Chris Dent c62ed6ccba Package db migration scripts in placement pypi dist
Having the db migration scripts within the openstack-placement pypi
distribution is desirable for deployment tools, such as
openstack-ansible. It provides a known good location for the script,
available with a pip install.

There are several ways to distribute files with a python package. The
method used here was chosen because it works both with tarballs and
wheels (the files are already in the tarball, as a result of the way pbr
works, but not in the wheel).

Here's what's done:

* The db migrate scripts are put in their own direcory,
  placement_db_tools, so that only they are packaged, not the other
  tools.
* To preserve how grenade interacts with these files as well as not
  disrupt the docs, symlinks from tools to placement_db_tools have been
  created.
* placement_db_tools is added to the list of packages included in the
  openstack-placement distro. This means that when 'pip install
  openstack-placement' happens, the python environment will then include
  placement and placement_db_tools directories.

The end result is that the true path to the script can be found with:

  pkg_resources.resource_filename('placement_db_tools', 'mysql-migrate-db.sh')

This has been noted in the to-stein.rst document.

A different package was chosen to not muddy the waters of what is
"actually placement".

Similarly, the 'data_files' functionality provided by pbr was not used,
because that requires the file be written to a location on the local
filesystem, relative to the install prefix. Dirtying the filesystem
outside the python lib with this sort of thing is inappropriate.

Change-Id: Ie326ce8a2a0692d20793bc18be606e034fa94a44
Story: 2005535
Task: 30671
2019-05-07 00:08:39 +00:00
Tetsuro Nakamura bfd6528c75 Add stamp DB version to the migration script
The existing nova-api to placement database migration shell scripts
don't include the post task to stamp the placement database to the
(b4ed3a175331) version. Without that stamping, syncing the DB to the
current version fails on having the initial upgrade: (initial) ->
(b4ed3a175331). This is because it assumes the initial DB has no
contents and tries to create the tables again.

This patch changes the shell scripts to include that stamping task
rather than leaving it as operators' manual duties to safely bring
the placement DB under alembic version control.

With this change, the shell scripts will need to be executed under the
following condition:
 - The placement is already installed to execute `placement-manage *`
 - The placement cli can access the placement's database, for example,
   by reffering to the `[placement_database]` section in the
   `placement.conf`.

Depends-On: https://review.openstack.org/620485
Change-Id: I75926b0efb3983d62603f2fd30b5a8cc30203d46
2018-12-12 00:26:57 +00:00
EdLeafe c8ab9ff906 Added alembic environment
Added alembic, and created an initial database revision.

For the sake of testing with real databases, tools/test-setup.sh is added
so that CI can start up a mysql and/or postgresql database on which to
test migrations. The OpportunisticDBMixin from oslo_db was originally
used for this, but because of the way placement configures the database
this proved hard to debug and get correct, so we've used something
that leverages the available oslo_db tools, but with more visibility.

Because these tests mix sqlite, mysql and postgresql settings in the
potentially the same process we need a way to insure that global
settings for databases do not leak into other tests. This is done with
a reset() on the placement db fixture, called by the msyql and
postgresql tests before and after they run. We also need careful
management of that cleanup when these tests are skipped (because db
server or database is not there).

Those tests will confirm that the models match the migrations so we also
need to remove model files that no longer matter.

Since we no longer need to distinguish among multiple database files, we
can simplify the naming of these files.

Co-Authored-By: Chris Dent <cdent@anticdent.org>
Change-Id: I51ed1e4e7dbb76a3eab23af7d0d106f716059112
2018-11-14 15:26:13 +00:00
Yikun Jiang e7fe0ab862 Add nova database migration script for postgresql
This is based on the mysql-migrate-db.sh script but
with the necessary changes for interacting with postgresql.

Long-term it might be possible to refactor both the mysql
and postgresql scripts into a generic script but the
potential complexity in doing that makes it a lower priority
than simply getting pg support for the nova_api/placement
table migration done in Stein.

Change-Id: I1cab3183067ab2e41663ca369509d753442e775c
2018-10-17 11:17:19 -04:00
Dan Smith cec0b71a2d Add nova database migration script
This adds a script that operators, grenade and deployment projects can
use to extract placement data from the nova_api database into its own
location.

Running this with --mkconfig will write out migrate-db.rc with variables
to be filled out and used to push an actual migration. Unless the
caller passes --migrate we will assume dry-run behavior.

This will run with or without existing placement schema (from
placement-manage db sync) and does not migrate/synthesize the migrate_version
table. This will also refuse to run if it looks like migration has
already been performed with a reliable return code. A reliable code is
also provided for the "this looks like a new deployment" case.

Change-Id: I88172d9527681d5456bf080e19e36d343f43c518
2018-09-21 11:30:55 -07:00
Daniel P. Berrange 61206c1c3c tox: make it possible to run pep8 on current patch only
The 'tox -epep8' command takes a long time as it checks
every single file in the Nova git repository (~1,400 at
time of writing).

This makes tox use a simple wrapper around flake8 which
can be told to restrict the check to only files changed
in the current command. This can be invoked in a simple
manner with 'tox -epep8 -- -HEAD'. Since most commits
only touch a handful of files, this will usually be
far faster than checking all 1,400 source files.

To check an entire branch for bisectability it can be
automated via

  git rebase -i master -x 'tox -epep8 -- -HEAD'

Change-Id: I157d1ccb883ca02402eee51fd7d6a50f86079389
2015-07-24 16:15:38 +01:00