Some files are using strings access_as_shared or access_as_external
instead of using defined constants ACCESS_SHARED and ACCESS_EXTERNAL.
This commit is doing the cleaning it does not bring any functional
change.
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: Ib75326c762776c5259740cb2f0abc1163842f95d
Taking example of a network that have multiple rbacs. In a situation
of selecting networks that are shared to a project.
If we could could already match the one that are owned by the context
we can expect les rbacs to scan.
https://bugs.launchpad.net/neutron/+bug/1918145/comments/54
In an environement with about 200 00 rbacs and 200 networks this
reduce time of the request from more than 50s to less than a second.
Related-bug: #1918145
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: I54808cbd4cdccfee97eb59053418f55ba57e11a6
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: Ib155fbb3f6b325d10e3fbea201677dc218111c17
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/db.
Trivialfix
Change-Id: I9311cfe5efc51552008072d84aa238e5d0c9de60
The goal of this patch is to make the Neutron code compliant
with SQLAlchemy 2.0.
All SQL transactions must be executed inside an explicit
writer/reader context. SQLAlchemy no longer will create an
implicit transaction if the session has no active transaction.
A warning message, only available in debug mode, is added. When
an ORM session calls "do_orm_execute", if there is no active
transaction, a warning message with a traceback will be logged
to help to debug the regression introduced.
Related-Bug: #1964575
Change-Id: I3da37fee205b8d67d10673075b9130147d9eab5f
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
This patch switches over to the payload style kwargs for RBAC_POLICY
callback events.
NeutronLibImpact
Change-Id: Ibf39013bfec7f03f76be7decf63000df3f0f6ad3
The functionality from model_query is already in neutron-lib and
consumers are using it. This patch removes the _model_query module
from neutron and updates all imports to use neutron-lib's version of
it instead.
NeutronLibImpact
Change-Id: Ib2eae9edb009a93e60b3b0d63ca365056138566b
The _resource_extend module is already rehomed into neutron-lib and is
shimmed in neutron. This patch removes the module as no active
consumers are using it.
NeutronLibImpact
Change-Id: I1550075fa5fa2aa2f1a88ee7189d311a1fe78391
Migrate to use NetworkRBAC OVO in External_net_db_mixin.
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: I083a611d73bb82d0e28b4d2b2cf8a667670a850b
This patch switches callbacks to use payloads for BEFORE_DELETE events
of NETWORK resources. To do so the DBEventPayload payload class is used
to encapsulate event data and the respective callback handles are
updated to use the payload kwarg.
NeutronLibImpact
Change-Id: I4843073799d4a2478a2583664d972c52457c695c
Michael Bayer while analysing neutron process function call trace,
suggested to run queries against specific columns rather than full
ORM entities as it can help reduce load both at the DB level and
in the Python level since they are much faster to fetch as
non-ORM entities. In this patch we are trying that on simpler
queries to improve neutron performance.
Co-Authored-By: Joe Talerico <jtaleric@redhat.com>
Change-Id: I6a41e9487a4427f876442bbeeae61974e892225e
The APIs our consumers are using from neutron.db_utils were rehomed into
neutron-lib with https://review.openstack.org/#/c/540161/
This patch consumes them by removing the rehomed APIs and using lib's
implementation where applicable.
NeutronLibImpact
Change-Id: I7ee53bce917feae8e37bf278eb3121a5af47131c
Router OVO is created in patch [1].
This patch uses Router object in external_net_db
[1] https://review.openstack.org/#/c/516961/
Co-Authored-By: Cao Xuan Hoang <hoangcx@vn.fujitsu.com>
Change-Id: Id3aaf473737ed70ab4e16a122c82457cc5e1f307
neutron-lib contains callback rbac-policy. This patch removes them
from Neutron and uses from lib side.
NeutronLibImpact
Change-Id: Iff4ad20a16270d4cb208afbece585fd6aef8c115
If a network's RBAC external attribute is deleted, we
should update the router:external attribute to False
if there is no other access_as_external rbac policy on the network.
Tempest API test patch: https://review.openstack.org/#/c/520255/
Change-Id: Ibdbe8a88581e54250259825bbf1c77485fd09f89
Closes-Bug: #1692472
The external network extension's API definition was rehomed into
neutron-lib with I9933b91d1e82db3891b3b72f06e94316e56a4f15. This patch
consumes it, switch over to neutron-lib's modules and removing the
rehomed code in neutron.
NeutronLibImpact
Change-Id: I696b52265b9528082cd2524f05febe2338376488
neutron-lib now contains the API definitions for neutron's core
resources. This patch removes the constant core resource and collection
variables and uses them from lib. Subsequent patches will consume the
actual core resource attribute definitions.
NeutronLibImpact
Change-Id: Ia9afdf620cd538b2aa420593277d6403a45c996b
The well known service type constants are in
neutron_lib.plugins.constants, but for legacy reasons a few still exist
and are referenced from neutron_lib.constants that we'd like to remove.
This patch switches references over to neutron_lib's plugin constants.
Change-Id: I1861448cec303725b30cef8f42029f467f9e03a3
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
By registering functions directly we cut off the dependency of the
"resource extend" functions on the plugin. This is a step towards
the goal of removing the CommonDbMixin mixin class.
Also, we register all "resource extend" functions at plugin create
(in __new__) instead of in the class definition (which caused the
hooks to be registered on import). This ensures the "resource
extend" functions are only registered for the plugins/mixins that
are actually used.
Note that decorators are used to register "resource extend" methods,
similar to the callback receiver decorators.
Related-Blueprint: neutron-lib
Change-Id: I128cfda773d5f9597df9cd61261fdc05f2a174aa
external networks are not really independent resources that
have their own lifecycles independent from networks. The only
use cases for this callback can be achieved with just regular
NETWORK callbacks.
The only users of this event have been adjusted to use NETWORK
callbacks (according to codesearch.openstack.org).
Closes-Bug: #1678041
Change-Id: Ie8249ddc3de643bfcb5c97480f30dc3500699256
By registering functions directly we cut off the dependency of the
query hooks on the plugin. This is a step towards the goal of
removing the CommonDbMixin mixin class.
Also, we register all query hooks at plugin create (in __new__)
instead of in the class definition (which caused the hooks to be
registered on import). This ensures the query hooks are only
registered for the plugins/mixins that are actually used.
Since the query hooks are decoupled from the plugin, we remove them
from the extension mixins (make them global in their module). This is
a step towards refactoring all extension mixins for removal.
Extra: In this patch we also remove the CommonDbMixinHooksFixture test
fixture and instead just clear out the hooks after each test.
Related-Blueprint: neutron-lib
Change-Id: Ib6c2134d29e1764de627c3355f6cdee789d6301e
Move the model query hook registration and resource extend funcs
registration methods out of the CommonDbMixin class and make them
regular utility functions.
This is a step in refactoring the CommonDbMixin class.
Change-Id: Iec1bb7f7098c83640ae695fd7cf2f4736f414ad2
This patch introduces and implements Olso-Versioned Objects for
network extension External Networks.
There were joined performed to order the fetching of external
networks by standard attribute which seems useless because,
in networks/services/auto_allocated/db.py while fetching
external networks it logs error when multiple networks are
returned. Expected default external network there is one, so
ordering does not make much sense.
Co-Authored-By: Manjeet Singh Bhatia <manjeet.s.bhatia@intel.com>
Co-Authored-By: Victor Morales <victor.morales@intel.com>
Change-Id: Iad609f72945b84df7881b43d1fdf9a188e5816bc
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
In an effort to simplify the codebase, this eliminates
all of the custom code in the ML2 delete_network
method and leverages registry callbacks for network
events.
The only purpose of the remaining inheritance of the
method is to leave the transaction guard on.
Partial-Bug: #1666424
Change-Id: Ib7d1c2081654dda82b5e015c71d20f378f3308f7
This addresses a TODO to convert the notification event to
PRECOMMIT_CREATE instead of BEFORE_CREATE and AFTER_CREATE since
both of these are occuring in a transaction which violates the AFTER
semantics.
Change-Id: Idba21275ade85a1a4afadee00de9ccbaec49b2f9
Those were moved under neutron.db.models start of Ocata. We should be
able to clean them up now.
NeutronLibImpact
Change-Id: Iadbf44d52ee8e30712807384152a29ce1a8b8f72
The network already has a joined relationship to the external
network table so we can leverage that instead of causing an
additional join for the filtering criteria.
Partial-Bug: #1649317
Change-Id: Idfee69b124f4ab8e2998da8492c5fa627f705bb9
This can prevent revision bumping on a network update if the
only thing that happened was the removal of the external network
model.
This also stops the use of .delete() for the external network
RBAC entries because they are tracked by the quota engine and the
quota engine is based on SQLAlchemy event handlers.
Change-Id: Iafd1a07e693d5719437c10325739693a88e2a52f
Ensure that only one period ends the data. The exception adds the
period so no need to pass the error_message with a period.
TrivialFix
Change-Id: I411534fc7ea56946a72890808154cc43ba3b6c1d
Closes-bug: #1618881
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
Extract all the common utils from common_db_mixin.py in preparation
for moving them to neutron-lib.
This is a preliminary step in preparation for refactoring the
CommonDbMixin class and moving it to neutron-lib also.
Partial Blueprint: neutron-lib
Change-Id: I3cba375a8162cb68e8f988f22f5c8b1ce7915180
This patch will relocate external network db models
and separate it from mixins for OVO implementation.
Change-Id: I83e2c7bbab3bd59cc9d2af71c66c0662c7724ecc
Partial-Bug: #1597913
Remove deprecation warnings for various constants
and exceptions that have moved to neutron_lib.
Fix miscellaneous other deprecations.
Uses constants instead of l3_constants when importing
neutron-lib constants.
Co-Authored By: Henry Gessau <gessau@gmail.com>
Co-Authored By: Gary Kotton <gkotton@vmware.com>
Change-Id: Ib0e8ff5c3e23677c1009241a1818cbc8a3430c38
This fixes the logic to allow updates to wildcard RBAC external
policies. It was broken for two reasons: first, it was using the
wrong kwarg, second, it wasn't considering the target tenant when
determining if the policy was required.
This patch fixes both issues and adds an API test exercising the
update path.
Closes-Bug: #1577100
Change-Id: Id7441ab5c3f3667aa1cc48100286a2a9d480e201
* Detect neutron.i18n import (neutron._i18n is recommended)
* Check builtins _ usage
* 'builtins = _' in tox.ini is no longer required.
* Introduce hacking rule doctest framework.
Newly added check_builtins_gettext() hacking check takes
token as argument. It is not a good idea to pass a tokenized
line manually. Instead it is reasonable to use docstring based
tests used in hacking repo.
Change-Id: Ib7464658fc4c8a6f1b03af6ab46f0bd3ee0bfb18
Sahid Orentino Ferdjaoui