Importing some modules lead to registering config options that may
collide with config options from a project that calls the import. This
patch wraps the side effect that registers config options into a
function that needs to be called in case the caller wants to register
the options.
This solution is also not perfect as it guards the common options to be
registered only once even if the function is called multiple times. This
is to solve problems in unittests, ideally we should always call the
function just once even in our testing suites.
Resolves-Bug: #1968606
Change-Id: Ic1532eb8de887ff1b1085206df11f53e22f7f524
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
This reverts commit f0be5f1a83.
We broke with this reverted patch db migration for stadium
projects like neutron-vpnaas, networking-bgpvpn,
networking-bagpipe and all others which have neutron
related db migration scripts from Liberty and older releases.
Change-Id: Ib77cdf6f7cb2e975581aeb2106690d35e798dc7c
This patch squashes the DB alchemy migration from Liberty
branch. The contract and expand migration steps are added
to the initial operation files. The unneeded tables or
parameters are not created.
Now the starting migration milestone is Mitaka.
Change-Id: Ia5bd25411149d1c475c142a60814c6daa8566cae
With python 3.x, six.text_type and six.string_type
are just str.
Also removed a six.integer_type since it was the only
one left in a file.
Another step in removing all of six usage from neutron.
Change-Id: I5208dc41bff1983ecd323286f427296b722da62a
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Instead of using alembic.utils logging functions which
prints output to stdout, lets use in functional tests
oslo_log logger to log all such commands.
This is needed to limit amount of output on stdout during
functional tests running on python3.5.
It was previously switched to use oslo_log in this migration
scripts always by [1] but because of issue in OSA we had to
revert this patch in [2].
So now let's try to change it only in functional tests.
[1] https://review.openstack.org/#/c/625555/
[2] https://review.openstack.org/#/c/626565/
Change-Id: I88fde6d107a1f1b5c48c5c89cd2e1c07fd12c755
This reverts commit dffcf4b54a.
Prior to this patch, output to stdout would look as follows:
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
Need to apply migrations from neutron contract branch. This will require all Neutron server instances to be shutdown before
proceeding with the upgrade.
However, with this patch, now only the following shows up:
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
The message to notify the deployer to run migrations does not appear to be
present in any log files either.
The message and return code are used by deployment projects to determine
whether migrations must be run, for example:
e34b27e5ae/tasks/neutron_db_setup.yml (L59-L69)
Change-Id: I9c9959d4e8407cdc5f7a24a76a222b2bdfc87d4b
Instead of using alembic.utils messages function which
prints output to stdout, lets use oslo_log logger to log
all such commands.
For logging error message alembic.utils module is still used
as those messages are quite important and should be displayed
to user in case of error.
This is needed to limit amount of output on stdout during
functional tests running on python3.5.
Change-Id: I20327bd1d5f784e021e9e1f1cbba1ef654271bfb
Fixed all pep8 E265 errors and changed tox.ini to no longer
ignore them. Also removed an N536 comment missed from a
previous change.
Change-Id: Ie6db8406c3b884c95b2a54a7598ea83476b8dba1
Refactoring neutron db config opts to be in neutron/conf/db/ and
neutron/conf/agent/database/ so that all the configuration options
reside in a centralized location. This simplifies the process of
looking up the config opts and provides an easy way to import.
Change-Id: I4da9bb48d49b99e8c2b34a5c1b83e7eb95b70b82
Partial-Bug: #1563069
A problem was found with the --expand (and --contract) options, see the bug report. I am reverting this patch so that I can redo it and have one single patch that can be back-ported.
Closes-Bug: #1599574
This reverts commit 88359e3fe9.
Change-Id: Iebca469c6fec32bac0480be6627ad5e9dedaad9d
We plan to have all projects that integrate with Neutron use the same
alembic environment (Neutron's) instead of having a separate alembic
env.py in each repo. This means that each project will need to label
their contract and expand branches uniquely.
With this patch we add support for branch labels of the form
'networking-foo-contract'
'networking-foo-expand'
We also support the old labels ('contract' and 'expand') with the
intent of backporting this patch to Mitaka so that projects can be
backwards compatible when they update their alembic branch labels. A
best effort will be made to backport this patch to Liberty also.
Support has been removed for label names starting with a "liberty_"
prefix. Such labels temporarily existed during the Liberty cycle but
have now all been cleaned up.
Partial-Bug: #1497830
Change-Id: Ibf38b3125d5d404d57f3e8ef0c7ecfa56eaf8042
We used to support subprojects with a single alembic branch.
Now the plan is to switch to a single alembic env for Neutron
and all installed subprojects. In a single alembic env, all
subprojects must use split (contract and expand) alembic branches.
The first step of removing single branch support was done in [1]
where we removed support for single HEAD files.
The --split_branches option was used by developers only. By removing
it we do not break any subproject, but force them to use split
branches for any new migration scripts.
[1] https://review.openstack.org/295905
Partial-Bug: #1497830
Change-Id: I62a191e1d22a1af4061c25fdc4a8df965d585419
Remove these CLI options:
--core_plugin : deprecated, not used
--service : deprecated in favor of --subproject
--sql_connection : deprecated in favor of --database-connection
Related-Bug: #1501380
Change-Id: Iff1babbb29d43dd0d4fed653de4714ca0ff75a99
Turns out the patch with Git commit hash prefix 13993764
disabled functional tests logging completely. This patch
fixes that by moving the neutron-db-manage logging setup
from import to the main function. Fixing that, it looks like
patch with Git commit hash prefix 4980f031fe turned off
DEBUG level logging for functional tests. I changed the
tests default logging from INFO to DEBUG to fix that.
Closes-Bug: #1567613
Change-Id: If02c18af20a236514409f37fa993224903877a8b
When running 'neutron-db-manage upgrade heads', I get:
No handlers could be found for logger "oslo_config.cfg" <--
INFO [alembic.runtime.migration] Context impl MySQLImpl.
...
This patch moves the logging configuration from env.py to cli.py,
so the configuration is done when the CLI is invoked and not
when Alembic imports env.py.
Change-Id: I767e72f513699f21dd629241faa04cb2d8fa416c
If branch specification (--expand/--contract) is passed to CLI while
also passing --autogenerate, fail explicitly instead of going into
autogeneration code and generating both scripts, as we currently do.
Change-Id: Ibf829fdd0b66a8d312c314487cfd8122a7721ef6
Related-bug: #1501380
Autogeneration code already enforces the branch for new scripts, but
explicit mode [where you pass --expand or --contract to get empty script
stubs] was not doing it. This resulted in new scripts being created in
the same directory where the current branch head is located. This is a
problem when we want to switch to a new release directory to store
future scripts.
To fix the problem, we now enforce the branch as per --contract/--expand
CLI arguments passed. We also ensure that the new directory is present
before attempting to generate a new stub script.
Related-bug: #1501380
Change-Id: I01cb6fed6dd246bedf945c323372e53991973fdc
When using neutron-db-manage to upgrade to a milestone tag,
the script was not ensuring that the expand branch was
upgraded before the contract branch. This broke projects
where contract migrations depend on expand migrations.
Fixes-Bug: #1550434
Change-Id: I0e6fc31dfa062c689936b2fe982147335ad9dce3
This command should be used by operators and deployment tools to
determine whether full neutron-server shutdown is needed for database
upgrade.
The change also makes neutron-db-manage tool to return the cumulative
result of commands being issued (in most cases it will still be 0 only,
since our command handlers implicitly return None).
DocImpact: Update doc to add new command 'has_offline_migrations' to
'neutron-db-manage' tool. The command determines whether full
neutron-server shutdown is needed for database upgrade.
Closes-Bug: #1519118
Change-Id: I7c5a4882ad4f80459ebe69c9a9c43cc60ce50200
Co-Authored-By: Martin Hickey <martin.hickey@ie.ibm.com>
This adds a new tox environment, genconfig, which generates sample
neutron core configuration file using oslo-config-generator.
Updates to some configuration option help messages to reflect useful
details that were missing in the code but were present in config files.
It also adds details to devref on how to update config files.
Partially-Implements: blueprint autogen-neutron-conf-file
DocImpact
Change-Id: I1c6dc4e7d479f1b7c755597caded24a0f018c712
Closes-bug: #1199963
Co-Authored-By: Louis Taylor <louis@kragniz.eu>
I believe we don't rely on --core_plugin or --quota_driver for the tool
because database contents are independent of chosen plugins and drivers
since ~Juno.
Also marked --service with deprecated_for_removal=True.
Related-Bug: #1501380
Change-Id: I2b4d98242a0d38843d931a71414ef2560a7ddb8f
This command shows current heads in all alembic branches. Even without
the command, we *can* get the heads values by looking into *_HEAD files
in tree. Still, those files may be missing; and the command is a tiny
bit more easy to use. Also, it gives access to more details on head
revisions if used with --verbose.
Change-Id: I6e7b7b5cd6f704d5d4bb4d845bf5098d4045924a
We have pep8 check for validation revisions, but it allows
outdated changes go into merge queue. To prevent this added
CONTRACT_HEAD, EXPAND_HEAD files.
Closes-bug: #1505701
Change-Id: Ie4b727e55a0b1ecb12e915a0037094a928d8f975
Rodolfo Alonso Hernandez