The goal of this series of patches is to make the Neutron code
compliant with SQLAlchemy 2.0.
This patch adds the missing database contexts in the execution of
some queries in the L3 code, reported in the following bugs. This
patch also refactors the ``neutron.db.l3_db`` methods, using the
method decorators instead of the inline context builders.
Closes-Bug: #1974144
Closes-Bug: #1974142
Related-Bug: #1964575
Change-Id: I7e18db7f1fa6b8878c13df09895c993704401815
Save order by in port query when not require fixed_ips,
and save some useless query for dvr subnet mac.
Closes-Bug: #1834308
Change-Id: I6836840edcaa5a21fd2ba9f65ffd24f7e5038fa3
For large scale deployment, the dvr router will be installed to
the scheduled DHCP host. This will definitely increase the l3
agent service pressure, especially in large number of concurrent
updates, creation, or agent restart.
This patch adds a config ``host_dvr_for_dhcp`` for the DHCP port
device_owner filter during DVR host query. Then if we set
``host_dvr_for_dhcp = False``, L3-agent will not host the DVR router
namespace in its connected networks' DHCP agent hosts.
Closes-Bug: #1609217
Change-Id: I53e20be9b306bf9d3b34ec6a31e3afabd5a0fd6f
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
The retry_if_session_inactive decorator was rehomed into neutron-lib
[1]. This patch consumes it by removing the function from neutron and
using neutron-libs version where appropriate.
NeutronLibImpact
[1] https://review.openstack.org/#/c/557040/
Change-Id: I3e3289f33e62d45933d0fbf165bb4b25078f22d5
This patch switches BEFORE_DELETE callback events for AGENT resources
over to the payload style args use a DBEventPayload object. In addition,
the _delete_mac_associated_with_agent is now refactored into it's own
method to allow the context parameter to be parsed by
retry_if_session_inactive.
NeutronLibImpact
Change-Id: I0af951f4346f9290c2eba6aad17f772fbf27bb2f
Commit I88e1aa2acf22389f69cb7d5704c80a5eb72a9bbe rehomed the
DVRMacAddressNotFound to neutron-lib. This patch consumes it by using
lib's version and removing neutron's version.
NeutronLibImpact
Change-Id: I70d1244c3bbf7664e15534e90a9d9b47e9c6e2f1
The dvr extension's API definition was rehomed into neutron-lib with
commit I6d98e83287833882515240d7db1faa4a1deb7da1
This patch consumes it by removing the rehomed code and using the
APIExtensionDescriptor for the extensions parent class.
NeutronLibImpact
Change-Id: Id5f0be33b036370b7545258a321830fdaa4dc0da
Refactoring neutron db config opts to be in neutron/conf/db/ and
neutron/conf/agent/database/ so that all the configuration options
reside in a centralized location. This simplifies the process of
looking up the config opts and provides an easy way to import.
Change-Id: I4da9bb48d49b99e8c2b34a5c1b83e7eb95b70b82
Partial-Bug: #1563069
get_random_mac() was marked for deprecation in
Queens, remove it and use the one from neutron-lib.
Change-Id: Ibcd3831a9ed419355a3d7969bd256b4b4bab6173
neutron-lib contains the OVO exception module. This patch switches all
such reference's over to lib's version of the module and removes it
from neutron.
Change-Id: Ic3d435eef51f970598bf50486894838498b2d438
Note: We don't currently have any out-of-tree consumers of this module.
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
This patch introduces and integrates Oslo-Versioned Objects for the
DistributedVirtualRouter mac address model class.
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Co-Authored-By: Victor Morales<victor.morales@intel.com>
Change-Id: I3b8a213a7daf95d2492b48ae59d3ad534911e1bb
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
Neutron-lib 1.1.0 is now out and contains the portbindings
API definition (as per commit [1]). This patch moves neutron
references over to the neutron-lib version.
NeutronLibImpact
- Consumers using the public constants within neutron's
portbindings API extension must now use the values
from neutron-lib.
[1] 87e42f993c07ae320159d5123662ee9f3bd4d903
Change-Id: I669af9b4c712877772d91a03857ab108714001d4
Probably the most common format for documenting arguments is reST field
lists [1]. This change updates some docstrings to comply with the field
lists syntax.
[1] http://sphinx-doc.org/domains.html#info-field-lists
Change-Id: I8fb91bcb8ae4bfbfa68ddf7c38a4f1f5cf548b36
Neutron-lib 1.1.0 is now out and contains the get_random_mac
definition[1]. This patch moves neutron references over to
the neutron-lib version.
NeutronLibImpact
[1] ee0f5b2ab27c828cfedb771735d237a968423da2
Change-Id: I28a2a1d85a85461f7a4344b86d18da7f68066c95
Those were moved under neutron.db.models start of Ocata. We should be
able to clean them up now.
NeutronLibImpact
Change-Id: Iadbf44d52ee8e30712807384152a29ce1a8b8f72
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
As there would be issue of cyclic imports while implementation of
Oslo-Versioned Objects for DVR which has db models definition and
mixins in same file, this patch will relocate DVR models.
Change-Id: I4005570c7507ae9c28fd2910b368e4fdf5e603b0
Co-Authored-By: Victor Morales <victor.morales@intel.com>
Partial-Bug: #1597913
Adds the retry decorator to the DVR mac DB module and
leverages its ability to retry on DBDuplicate entries
to simplify the code.
We also add a transaction guard to make the previously
implicit assumption that _create_dvr_mac_address would
be called outside of a transaction explicit.
Partial-Bug: #1612798
Change-Id: I0c919a3f08e0be1aea4e28dc8f6d92b51de129e0
As discussed in this patch [1] this option didn't come from any actual
user-encountered problem and is now effectively obsolete for its
original purpose anyway. In the interest of cleaning up options that
aren't of any real use, this patch obsoletes this one immediately and
marks it deprecated for removal in Ocata.
TrivalFix
[1] https://review.openstack.org/#/c/327413
Change-Id: I58003d5b07e28c7254ff836dec937f699bc83112
Once all agents are deleted from a host, the DVR MAC generated
for that host should be deleted as well to prevent a buildup of
pointless flows generated in the OVS agent for hosts that don't
exist.
Closes-Bug: #1568206
Change-Id: I51e736aa0431980a595ecf810f148ca62d990d20
Make one db query to get all needed ports.
No need to fetch all ports on the subnet and iterate through
them.
Change-Id: I76a8a75905a7ec4c202583f01b62f2aefb433026
Fix filters to get the right gateway port for a subnet.
Wrong filters led to random port's mac being returned which may cause
side effects. See bug for details.
Closes-Bug: #1530179
Change-Id: I8368255f00ab3e9586c8ff28dfe6739541000810
This patch updates docstring for get_ports_on_host_by_subnet, to
tell that the method will only get dvr serviced ports, not all ports
for given subnet and host.
Change-Id: I85ea5a9e37b6c894f7d51e73c10143b208a29f0e
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
I looked backed to stable/kilo and stable/liberty,
I'm not sure this method was ever used, so there is
no upgrade concern either.
Change-Id: Id72ee9114a8c3f1e49314c03dbab0b15b327eb93
In DVR routers when a port is added to a router, then
the command succeeds but the l2 ovs agent raises an
error stating that it could not retrieve the gateway
port for the subnet.
The reason for this is there is mismatch in the
ip_address returned from the subnet for the gateway
port and the actual ip_address of the port that we
added to the router.
Since the subnet info was passed to "get_subnet_for_dvr"
this mismatch was seen.
Instead of passing the subnet we will be passing the
actual fixed_ips with all the details and the subnet
gateway port will be filtered based on the ip_address
in the fixed_ips passed.
Closes-Bug: #1404823
Change-Id: I87a3983951f814350e79f5e2274f4639bb6bc0f5
The help text for the option in the current static configuration file
contained additional information which would be lost when we switch to
generating the file. This patch copies it over to the oslo.config help
parameter.
Related-blueprint: autogen-neutron-conf-file
Change-Id: Id03925b4bbbee697bf5372392c2345802a1863f8
The rename from Quantum to Neutron left a few q_ strings
around, let's go ahead and clean them up.
Change-Id: I06e6bdbd0c2f3a25bb90b5fa291009b9ec2d471d
oslo.log now provides a logging helper that is similar to custom neutron
helper (actually, the helper in oslo.log started from neutron version).
Now switching to library implementation.
Deprecated neutron.common.log.log
Change-Id: I85d5fc570950ff18cfdb8db20ad20b166e195299
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
Mostly trivial import changes.
- oslo.i18n no longer provide install() method to inject _() into
globals(), so removed all calls to it;
- removed Babel from dependencies (it will now be grabbed by oslo.i18n);
- updated tox.ini to ignore import violations for oslo.i18n.
Change-Id: I6623d551f512fb7fe9bf35ee734ed6d4c6cbc287
Currently, DVR router namespaces are created only
when there is a valid VM port on the compute
node, or for the gateway-port on the service node.
But when an LBaaS VIP port is created the l3 agent
does not create a DVR namespace to service the VIP port.
This fix enables DVR namespaces to be created to
service the LBaaS VIP port.
Also, this fix enables L2 Agent running in DVR
mode, to add-in OVS rules to enable packets to
be routed to such LBaaS VIP Ports which are
resident on DVR routed interfaces.
Therefore, with this fix both East-West and
North-South traffic will be serviced by DVR
for LBaas VIP Ports.
DocImpact
Authored-by: Swaminathan Vasudevan <swaminathan.vasudevan@hp.com>
Co-Authored-By: Vivekanandan Narasimhan <vivekanandan.narasimhan@hp.com>
Change-Id: I698b971d50721fb0512a11569f7d3139d0d456f3
Closes-Bug: #1356464
This patch introduces the RPC contract changes
required for both the server (plugin) and agent
to propagate and retrieve additional information
about Distributed Routers, like MAC addresses
and Port Bindings.
Partially-implements: blueprint neutron-ovs-dvr
Change-Id: I04a2ee5fceea79d2786c799178f8dd1675925a39
Authored-by: Vivekanandan Narasimhan <vivekanandan.narasimhan@hp.com>
Co-Authored-By: Armando Migliaccio <armamig@gmail.com>
This patch introduces the models, the DB migrations
and the config options required by the L2 layer to
support DVR east/west traffic.
These changes will be used by the control-plane made
of ML2, L2pop and L2 agent.
Two new configuration options have been introduced:
'dvr_base_mac' is used to set DVR MAC addresses apart
from tenant ones (every distributed router will have
ports being created on compute hosts) and
'enable_distributed_routing' is used to enable dvr
support in the L2 agent. This gives the capability of
rolling out the dvr functionality in stages.
Partially-implements: blueprint neutron-ovs-dvr
DocImpact
Change-Id: Iab6505f239d2c4c9bcbf4e32a292d7b4b5320c8e
Authored-by: Vivekanandan Narasimhan <vivekanandan.narasimhan@hp.com>
Co-Authored-By: Armando Migliaccio <armamig@gmail.com>
Rodolfo Alonso Hernandez