This patch switches over to the payload style of callbacks for
NETWORK based events. As part of this change a few shims are needed
to handle cases where some callbacks don't yet use payloads and others
do. Once we move over to payloads for all callbacks the shims can be
removed.
NeutronLibImpact
Change-Id: I889364b5d184d47a79fe6ed604ce13a4b334acfa
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
The functionality within neutron.db.common_db_mixin is available via
neutron-lib APIs. This patch removes common_db_mixin and updates any
uses of it to use neutron-lib instead.
Depends-On: https://review.openstack.org/#/c/636159/
NeutronLibImpact
Change-Id: I2388f90b37abb09408809dda8c21da551bcd94bb
Today a number of classes define their supported_extension_aliases
using static strings rather than API definition ALIASes. This patch
switches them to use the ALIAS where applicable.
Change-Id: I716270c68a9fcd850c3c26de31bc13ea16def23d
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.
NeutronLibImpact
Change-Id: I9704f20eb21da85d2cf024d83338b3d94593671e
The _resource_extend module is already rehomed into neutron-lib and is
shimmed in neutron. This patch removes the module as no active
consumers are using it.
NeutronLibImpact
Change-Id: I1550075fa5fa2aa2f1a88ee7189d311a1fe78391
The retry_if_session_inactive decorator was rehomed into neutron-lib
[1]. This patch consumes it by removing the function from neutron and
using neutron-libs version where appropriate.
NeutronLibImpact
[1] https://review.openstack.org/#/c/557040/
Change-Id: I3e3289f33e62d45933d0fbf165bb4b25078f22d5
Enforce validation on filter parameters on list requests.
If an API request contains an unknown or unsupported parameter,
the server will return a 400 response instead of silently ignoring
the invalid input.
In resource attributes map, all filter parameters are annotated by
the ``is_filter`` keyword. Attributes with is_filter set to True
are candidates for validation.
Enabling filter validation requires support from core plugin and
all service plugins so each plugin need to indicate if it supports
the validation by setting ``__filter_validation_support`` to True.
If this field is not set, the default is False and validation is
turned off. Right now, the ML2 plugin and all the in-tree service
plugin support filter validation. Out-of-tree plugins will have
filter validation disabled by default.
An API extension is introduced to allow API users to discover this
new API behavior. This feature can be disabled by cloud operators
if they choose to do that. If it is disabled, the extension won't
be presented.
Depends-On: Ic3ab5b3ffdc378d570678b9c967cb42b0c7a8a9b
Depends-On: I4397df1c35463a8b532afdc9c5d28b37224a37b4
Depends-On: I3f2e6e861adaeef81a1a5819a57b28f5c6281d80
Depends-On: I1189bc9a50308df5c7e18c329f3a1262c90b9e12
Depends-On: I057cd917628c77dd20c0ff7747936c3fec7b4844
Depends-On: I0b24a304cc3466a2c05426cdbb6f9d99f1797edd
Change-Id: I21bf8a752813802822fd9966dda6ab3b6c4abfdc
Partial-Bug: #1749820
The remainder of the neutron.plugins.common.utils were rehomed into
neutron-lib with [1][2]. This patch consumes them by using the functions
from neutron-lib, and removing the neutron.plugins.common.utils module
all together as it's fully rehomed now.
NeutronLibImpact
[1] https://review.openstack.org/#/c/560950/
[2] https://review.openstack.org/#/c/554546/
Change-Id: Ic0f7b37861f078ce8c5ee92d97e977b8d2b468ad
The APIs our consumers are using from neutron.db_utils were rehomed into
neutron-lib with https://review.openstack.org/#/c/540161/
This patch consumes them by removing the rehomed APIs and using lib's
implementation where applicable.
NeutronLibImpact
Change-Id: I7ee53bce917feae8e37bf278eb3121a5af47131c
This patch switches callbacks over to the payload object style events
[1] for PRECOMMIT_UPDATE based notifications. To do so a DBEventPayload
object is used with the publish() method to pass along the related data.
In addition a few UTs are updated to work with the changes. Finally
a few shims are put into place to allow PRECOMMIT_UPDATE based events to
use payloads while still supporting the existing kwarg style events.
NeutronLibImpact
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads
Change-Id: Ie6d27df01cd7b87894efc80946d41eb1ebe25bef
Commit I81748aa0e48b1275df3e1ea41b1d36a117d0097d added the l3 extension
API definition to neutron-lib and commit
I2324a3a02789c798248cab41c278a2d9981d24be rehomed the l3 exceptions,
while Ifd79eb1a92853e49bd4ef028e7a7bd89811c6957 shims the l3
exceptions.
This patch consumes the l3 api def by:
- Removing the code from neutron that's now in lib.
- Using lib's version of the code where applicable.
- Tidying up the related unit tests as now that the l3 api def from lib
is used the necessary fixture is already setup in the parent chain when
setting up the unit test class.
NeutronLibImpact
Change-Id: If2e66e06b83e15ee2851ea2bc3b64ad366e675dd
neutron-lib contains the OVO exception module. This patch switches all
such reference's over to lib's version of the module and removes it
from neutron.
Change-Id: Ic3d435eef51f970598bf50486894838498b2d438
Note: We don't currently have any out-of-tree consumers of this module.
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
neutron-lib now contains the API definitions for neutron's core
resources. This patch removes the constant core resource and collection
variables and uses them from lib. Subsequent patches will consume the
actual core resource attribute definitions.
NeutronLibImpact
Change-Id: Ia9afdf620cd538b2aa420593277d6403a45c996b
The well known service type constants are in
neutron_lib.plugins.constants, but for legacy reasons a few still exist
and are referenced from neutron_lib.constants that we'd like to remove.
This patch switches references over to neutron_lib's plugin constants.
Change-Id: I1861448cec303725b30cef8f42029f467f9e03a3
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
By registering functions directly we cut off the dependency of the
"resource extend" functions on the plugin. This is a step towards
the goal of removing the CommonDbMixin mixin class.
Also, we register all "resource extend" functions at plugin create
(in __new__) instead of in the class definition (which caused the
hooks to be registered on import). This ensures the "resource
extend" functions are only registered for the plugins/mixins that
are actually used.
Note that decorators are used to register "resource extend" methods,
similar to the callback receiver decorators.
Related-Blueprint: neutron-lib
Change-Id: I128cfda773d5f9597df9cd61261fdc05f2a174aa
This adjusts the get-me-a-network code to just
rely on the regular NETWORK precommit events rather
than the EXTERNAL_NETWORK events that the following
patch eliminates.
Partial-Bug: #1678041
Change-Id: Id2db458f4688e4aa4c70ccdb09867b9a66de28ea
Move the model query hook registration and resource extend funcs
registration methods out of the CommonDbMixin class and make them
regular utility functions.
This is a step in refactoring the CommonDbMixin class.
Change-Id: Iec1bb7f7098c83640ae695fd7cf2f4736f414ad2
This patch introduces and implements Olso-Versioned Objects for
network extension External Networks.
There were joined performed to order the fetching of external
networks by standard attribute which seems useless because,
in networks/services/auto_allocated/db.py while fetching
external networks it logs error when multiple networks are
returned. Expected default external network there is one, so
ordering does not make much sense.
Co-Authored-By: Manjeet Singh Bhatia <manjeet.s.bhatia@intel.com>
Co-Authored-By: Victor Morales <victor.morales@intel.com>
Change-Id: Iad609f72945b84df7881b43d1fdf9a188e5816bc
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
This addresses a TODO to convert the notification event to
PRECOMMIT_CREATE instead of BEFORE_CREATE and AFTER_CREATE since
both of these are occuring in a transaction which violates the AFTER
semantics.
Change-Id: Idba21275ade85a1a4afadee00de9ccbaec49b2f9
This patch integrates and implements Oslo-Versioned Objects for Auto
Allocated Topology.
Change-Id: I478af7aa118a947cc9eab36b047cecea6ddd5170
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Co-Authored-By: Anindita Das <anindita.das@intel.com>
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
Extract all the common utils from common_db_mixin.py in preparation
for moving them to neutron-lib.
This is a preliminary step in preparation for refactoring the
CommonDbMixin class and moving it to neutron-lib also.
Partial Blueprint: neutron-lib
Change-Id: I3cba375a8162cb68e8f988f22f5c8b1ce7915180
This patch will relocate external network db models
and separate it from mixins for OVO implementation.
Change-Id: I83e2c7bbab3bd59cc9d2af71c66c0662c7724ecc
Partial-Bug: #1597913
This makes the notifier subscribe to core resource events
and leverage them if they are available. This solves the
issue where internal core plugin calls from service plugins
were not generating DHCP agent notifications.
Closes-Bug: #1621345
Change-Id: I607635601caff0322fd0c80c9023f5c4f663ca25
Based on how Nova calls into this extension, it is clear that this
'error' shows during normal conditions of booting multiple VMs for
the same tenant. Plus, there is nothing the operator can do as the
plugin hopefully takes care of it all.
Change-Id: I6747e64903f8e57ddadcf2d1c74d223fb4fdf97d
No longer necessary on cleanup since individual plugin operations
are protected but it should be on _ensure_network_default_value
callback since it can happen outside of a transaction.
Partial-Bug: #1612798
Change-Id: Iab4f2216e6b138296e8245eb0b1e3e6b5e46561b
Returning a DB object for the external network and passing
that to the create_router operation makes the life of the
DB object transition many operations. If any of those operations
detech it from the session or expire it, it can result in
an exception down in the router operations.
This patch just returns the network ID since that's all we need.
Closes-Bug: #1616126
Change-Id: Ib9c4ba9f00ca7f7a713a89be4f75a230b086ac38
Uncaught exceptions in the core or L3 plugin layers can cause the
auto_allocate plugin to fail unexpectedly. This patch ensures that
any unexpected error is properly handled by cleaning up half
provisioned deployment.
Related-bug: #1612798
Closes-bug: #1619497
Change-Id: I3eb9efd33363045f7b2ccd97fe4a48891f48b161
In the event of a router create failure we need to make sure that
the allocated resources are cleaned up.
When router allocation fails it is difficult for an admin to
understand the root cause. Adding the exception here will provide
a little additional information.
Closes-bug: #1617707
Change-Id: I41940bea0327ee31494bd95867d0e60d9b5a24b8
The model_base file is going to move to Neutron lib in
I2087c6f5f66154cdaa4d8fa3d86f5e493f1d24d9. This will mainly leave
behind only the standard attributes related stuff so the name
'model_base' won't make much sense. This moves the standard attribute
related things into its own module so model_base can conceivably be
eliminated entirely.
Change-Id: Icaf3220fbc5723f2b5421a494371ef274d7073c7
When uncaught, the auto_allocated_network stays behind. This patch
ensures the error is handled properly and the network is indeed
cleared when the subnetpool runs out of space.
Closes-bug: #1616250
Change-Id: Ic6e06d3e6c16a58977e84b371c3faaba66c9603b
Review [1] showed how tricky it can be to let the client
side deal with auto-network-topology cleanups. Rather than
pushing this complexity to the client, we should implement
the DELETE method for this extension, as it's rather
trival to do on this server side.
Since the DELETE method is exposed, but it fails with 500,
it is reasonable to deal with this as a bug fix, rather than
having to go through yet another extension. The neutronclient
side support should be added, but since the first user of this
is Tempest, we can safely assume they can leverage this directly
without depending on a python-neutronclient version bump.
[1] https://review.openstack.org/#/c/327191/
Closes-bug: #1614872
Change-Id: I2fba51bdf8c781fcc0449e1e9947de976c96eec4
This adds a retry decorator to the auto allocate cleanup logic
so it can handle retriable errors during cleanup. It also adds
catches for notfound errors to make the function idempotent so
it works on retries.
Change-Id: I3d62af3018fb834ec85771d8bc8e7379cc80b72a
Closes-Bug: #1615710
DB errors or retry exceptions that bubble up to the
plugin may get masked by integrity violation errors
due to partial provisioning of resources. When that
happens, we should make sure any pending resource
is cleaned up before reattempting the operation.
Closes-bug: #1612615
Change-Id: I76e9f8e4b61fb3566d70af4236c19e4c5a523646
All occurences of ``tenant_id`` across the database are renamed
to ``project_id``. Both options are equally valid, but ``project_id``
is preferred.
To inform external users about the change, HasTenant class was
deprecated.
UpgradeImpact
Partially-Implements: blueprint keystone-v3
Change-Id: I87a8ef342ccea004731ba0192b23a8e79bc382dc
Under particular circumstances, multiple requests to the
auto-allocated-topology extension may lead to the transient
creation of duplicated resources. This is dealt with by the
service plugin code, which cleans them up once the condition
is detected. However the client may accidentally be impacted
and potentially left in error (recoverable on retry).
In order to address this error condition, the logic to
provision the network for any given tenant is tweaked
slightly so that the network is created in disabled state
and re-enabled when it is safe to do so. A Neutron client
should check the network status to see if the network is
ready for use before getting its hands on it.
Closes-bug: #1591766
Change-Id: Ia6ff5ad975673875216eb470080dfc0dcf6b9ab2
Any service plugin that implements the get_plugin_type method should
make it a classmethod. It should not need to be instantiated to
retrieve the simple constant it returns in every case.
Change-Id: Ia3a1237a5e07169ebc9378b1cd4188085e20d71c
Closes-Bug: #1590117
Nurmatov Mamatisa