The APIs our consumers are using from neutron.db_utils were rehomed into
neutron-lib with https://review.openstack.org/#/c/540161/
This patch consumes them by removing the rehomed APIs and using lib's
implementation where applicable.
NeutronLibImpact
Change-Id: I7ee53bce917feae8e37bf278eb3121a5af47131c
In case of service_provider, selection according to service type is
necessary. Currently there was no filtering logic, so if there are two
or more service plugins using service_provider, there is a duplicate
service_provider.
New argument 'svc_type' is added to ProviderConfiguration so that only
the service_provider matching the service type is shown.
From caller side of ProviderConfiguration, one should specify
'svc_type' since ProviderConfiguration class have new 'svc_type'
argument to find service provider. Although netron code base using
ProviderConfiguration changed, existed code out of newtron tree should
be also modified following the change becuase if not, there would be
duplicated entries problem currently appeared. But there is no
difference without 'svc_type' argument because matching is effective
only when the argument is specified.
A new test case added in test_get_service_providers() in
neutron/tests/unit/extensions/test_servicetype.py which does not
have any filter options. Without this patch, this test case would be
failed having duplicated results.
Change-Id: I6ad9897dd174b45c7f2315699d25d38d4c060abc
Closes-Bug: #1763627
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
Remove The following _MAX_LEN constants from
neutron/api/v2/attributes.py and use the corresponding DB field size
constants from neutron_lib.db.constants instead.
NAME_MAX_LEN --> NAME_FIELD_SIZE
TENANT_ID_MAX_LEN --> PROJECT_ID_FIELD_SIZE
DESCRIPTION_MAX_LEN --> DESCRIPTION_FIELD_SIZE
LONG_DESCRIPTION_MAX_LEN --> LONG_DESCRIPTION_FIELD_SIZE
DEVICE_ID_MAX_LEN --> DEVICE_ID_FIELD_SIZE
DEVICE_OWNER_MAX_LEN --> DEVICE_NAME_FIELD_SIZE
In alembic migration scripts, the raw numerical value is used.
For more information, see:
http://lists.openstack.org/pipermail/openstack-dev/2016-October/105789.html
NeutronLibImpact
Change-Id: I734890372584fe27e5d6ec38c0cad2de882ff11c
Extract all the common utils from common_db_mixin.py in preparation
for moving them to neutron-lib.
This is a preliminary step in preparation for refactoring the
CommonDbMixin class and moving it to neutron-lib also.
Partial Blueprint: neutron-lib
Change-Id: I3cba375a8162cb68e8f988f22f5c8b1ce7915180
If providers is not found in the CONF.service_providers.service_provider
it looks them up from implicit conf files. If nothing is defined there
it still emits a deprecation warning.
Maybe a warning should be emitted if this value is empty, however the
deprecation warning that is provided is incorrect and misleading.
Change-Id: I3166a445b69d854b294843814a081e74c1aadb3b
Closes-Bug: #1633293
In some places, indentation was wrong.
In this patchset, they are fixed.
Also, I did some readibility improvements which I've found
during indentation fixing.
Change-Id: I298b9b0c557c07848c69ed23b0faa3a3147d055d
Remove deprecation warnings for various constants
and exceptions that have moved to neutron_lib.
Fix miscellaneous other deprecations.
Uses constants instead of l3_constants when importing
neutron-lib constants.
Co-Authored By: Henry Gessau <gessau@gmail.com>
Co-Authored By: Gary Kotton <gkotton@vmware.com>
Change-Id: Ib0e8ff5c3e23677c1009241a1818cbc8a3430c38
Refactoring neutron services config opts to be in neutron/conf/services
so that all the configuration options for services reside in a
centralized location. This simplifies the process of looking up the
extension config opts and provides an easy way to import.
Change-Id: Iad255b020910ee54bc6cce25f4f786376dfe3705
Partial-Bug: #1563069
This feature is redundant, since it duplicates oslo.config functionality. If
neutron-server needs to load the configuration option from multiple files, it
should be achieved by passing multiple --config-dir/--config-file CLI
arguments, not by relying on implicit loading for files with names matching the
neutron_*.conf pattern.
The plan is to stop implicit loading in Ocata.
More details:
http://lists.openstack.org/pipermail/openstack-dev/2016-May/096033.html
Related-Bug: #1492069
Related-Bug: #1599936
Change-Id: I743e181532d285303c8d5c2a73f4fd674626b478
Turned out that if the code extracts config_dirs value from ConfigOpts
objects before config files are parsed, then oslo.config will raise
NoSuchOptError exception.
This is not a usual mode of operation for the code, since main()
function of the process using it is expected to parse CLI and config
files before using it, it may nevertheless happen in some test code.
This patch guards against those exceptions, falling back to
/etc/neutron, as we already do when --config-dir is not specified.
Change-Id: I00cf824baa8580b7aa7ec4518a4741e49c998364
Closes-Bug: #1587359
Since oslo.config 3.8.0 (that included
Ibd0566f11df62da031afb128c9687c5e8c7b27ae), config_dir option is a list,
not a string. While our custom provider configuration parser for
multistring options assumes the latter.
It makes all installations that 1) pass at least one --config-dir option
in CLI and 2) enable any service plugin that relies on provider
definitions, to fail to start neutron-server. For example, this affects
any RDO Mitaka installation with *aas service plugins enabled.
Since Newton requires >=3.9.0, we are fine to switch to the list type
without any code to support backwards compatibility with older option
type. For Mitaka backport, we will need to handle both cases.
Change-Id: I10e399a852d9fba0fd1aea79a10e2e7c906e4b3c
Closes-Bug: #1585102
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
- callers should not explicitly pass exceptions into LOG.exception
because it's already implicitly included in the message by stdlib
logging module.
- callers should not call to LOG.exception when there is no exception to
log about (known to fail in Python 3.x < 3.5).
Change-Id: I58e7e01ed152028ad43bb3ada87d719caa2ab08d
Related-Bug: #1504053
1. Scope mock of 'open' to module
By mocking 'open' at the module level, we can avoid affecting
'open' calls from other modules.
2. Stop using LOG.exception in contexts with no sys.exc_info set
Python 3.4 logger fills in record.exc_info with sys.exc_info() result
[1], and then it uses it to determine the current exception [2] to
append to the log message. Since there is no exception, exc_info[1] is
None, and we get AttributeError inside traceback module.
It's actually a bug in Python interpreter that it attempt to access the
attribute when there is no exception. It turns out that it's fixed in
latest master of cPython [3] (the intent of the patch does not seem
relevant, but it removes the offending code while reshuffling the code).
Note that now cPython correctly checks the exception value before
accessing its attributes [4].
The patch in cPython that resulted in the failure is [5] and is present
since initial Python 3k releases.
The patch in fixtures that broke us is [6].
[1]: https://hg.python.org/cpython/file/tip/Lib/logging/__init__.py#l1412
[2]: https://hg.python.org/cpython/file/tip/Lib/logging/__init__.py#l575
[3]: https://hg.python.org/cpython/rev/73afda5a4e4c
[4]: https://hg.python.org/cpython/rev/73afda5a4e4c#l6.484
[5]: https://hg.python.org/cpython/rev/2ee09afee126
[6]: 67dd295694
Closes-Bug: #1503847
Closes-Bug: #1504053
Co-Authored-By: Ihar Hrachyshka <ihrachys@redhat.com>
Change-Id: I456b7846b8a53e4d3f8c91583685e0e1eaa84719
service_provider is a 'special' configuration. It is a MultiStr
option that used to be in neutron.conf, but moved into its own *-aas
config files after the service split. We allow for the definition to
be available 'anywhere': either in neutron.conf or in the *-aas
service's config file.
The list of 'service_provider' can include drivers from within the *-aas
tree, or from elsewhere, and can apply to different service types. Due to
the polymorphic nature of this variable it is very tricky to identify only
the drivers that pertain a specific service module: the service module
may as well implement more than one service type and may have support
from drivers out of tree.
For this reason it is best to relax this check and rely on query filters
when the ServiceManager.get_service_providers() is invoked. Furthermore,
without this fix there was a situation where the value returned by
'service_providers' may be differ depending on how the configuration is
passed on the CLI, and this inconsistency may only cause headaches.
Closes-bug: #1492069
Change-Id: I4db4ce0b287c984d86181227edf769531ecb7bb8
We may be unable to get rid of this code, but at least we can
make it slimmer and more oslo_config friendly.
This patch also fixes an issue where plain ConfigParser cannot
handle config options with the same name under same section.
Related-bug: #1492069
Change-Id: I596613eb7b2fc3e894a3c1cc7b7cb24d7137cbc5
This patch fixes a regression where, if neutron was loaded using
--config-dir, the service_providers option was no longer available.
We bring the logic back (removed by 61121c5f2a), alongside the ability
to load the option auto-magically. This is especially required for DevStack
deployments as of today, because neutron-server is only loaded by passing
--config-file (...)neutron.conf and --config-file (...)ml2_conf.ini
Change-Id: I9bfaed9e19a5506e27795a0b7ad47f4c31fefa40
Closes-bug: #1490990
After the service split, some of the configuration, parsing and
validation was kept in the neutron core; ultimately this needs to
get closer to the services where it belongs.
This patch starts from ProviderConfiguration and ServiceTypeManager
classes, and aims at removing the hard-coded elements, like the list
of known advanced services, so that in the long run we can make
Neutron easier to plug with external services.
Partial-bug: #1473110
Depends-on: I44edcceba37ac58efcc0a53c9d1f835d9530344a
Depends-on: I8924234aadf786801ffc100d7daa27acc145a195
Change-Id: Ia4cad678e6c722ca05821dbdbf05d61523246a86
Sometime during the split, code was added to fixup driver paths,
which imports service providers even for plugins which are not
in use. That, combined with neutron including default service
providers for VPN and LOADBALANCER, resulted in a really messy
mess in terms of removing VPN from the main neutron test suites.
This change stops the imports, so that if one of the services is
missing, neutron server can still start. It likely breaks the driver
path fixup, which can be fixed outside of this gate blockage.
Closes-Bug: #1483266
Change-Id: I23f9007357d8cbbae599997c244561a4e2f32ce1
We can derive the services from EXT_TO_SERVICE_MAPPING, therefore
there is no need for duplicating the service labels into ALLOWED_SERVICES.
Change-Id: If92e0ea3dea4480588141a2819ea4036c527c9bc
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
Certain tests still put providers into neutron.conf, and upgrade scenarios
will have more correct providers in neutron.conf than the new locations.
If we find providers in neutron.conf, use them in favor of the
split config files during the transition.
Closes-Bug: 1422895
Change-Id: I731f7c80c2df78fa521a5140e450972119f4a105
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
- Extensions will automatically be loaded from service repos in addition
to neutron proper, but neutron proper will take precedence.
- Config entries for service repos will be read out of neutron-{service}.conf
first, and then neutron.conf. After Kilo, they will be read only from
neutron-{service}.conf.
- Service providers for drivers will be collected from all neutron conf files.
This is review 1 of 3. The second set will be in the server repos, moving
the extensions. The third will be in neutron, removing the service exts.
Change-Id: I16b5e5b2bb70717166da14faa975fa2ab9129049
Partially-Implements: blueprint services-split
Patch implements translation from class paths to neutron to class paths
to neutron_<adv_service>. It's achieved by defining entry point in
setup.cfg which is translated by stevedore.
There will be needed patches in advanced services tree calling
get_provider_driver_class() function before importing class.
This patch specifically fixes loading service plugins and
drivers for service plugin. Patches for agents are still needed in
neutron repo and adv services repos.
Alternative and better solution would be implementing new DriverType
to oslo.config, which will have callback to
get_provider_driver_class()-like function.
Co-Authored-By: Ihar Hrachyshka <ihrachys@redhat.com>
Change-Id: I76af175c4387326a4e5ff95c2f15d8b866dedab3
Partial-Bug: 1401895
The patch makes the following changes:
* adds new attribute of the pool: provider, which is provider name
as it is written in configuration
* adds support for multiple plugin drivers for loadbalancer
* cleans up healthmonitor-related plugin driver API
Drivers should work with healthmonitor associations only
* adds ability to update provider attribute for the pool used
to reassociate pools with new providers in case their providers
were removed from configuration
implements blueprint lbaas-integration-with-service-types
DocImpact
Change-Id: I4295c9bcceb38e60f813d5596af48bd8194c1c9b
implements blueprint service-type-framework-cleanup
* Defines logic and API for ServiceProvider - read-only entity
that admins provide in configuration and which is stored in memory
* ServiceType entity which maps to ServiceOfferings in new terms
is removed for now.
* Routed service insertion fixed to not to refer to service providers.
* In case configuration changes and some service providers are removed
then the resources must be cleanup in a special way (undeploy logical
resources). This is a matter of future work
* Add migration.
Change-Id: I400ad8f544ec8bdc7d2efb597c995f284ff05829
Zuul