Delete the DVR port record on the node when the router is deleted from
the node. This patch fixes an issue where the DB actions have no context
writer which will actually update nothing duing the following
`delete_distributed_port_binding_if_stale`.
As well as fixing that when the dvr router was deleted from a node,
only the ml2_distributed_port_bindings were cleaned up and not the
ml2_port_binding_levels records for the dvr port.
Remove the last VM under one router in one host, the
ml2_distributed_port_bindings and ml2_port_binding_levels will remain
the record. So if VMs under one router had spread on many hosts,
and router (router ports) still exists, even there is only one VM under it,
the binding entries will still be equal to amount of hosts where this
router resided before. The result is a large amount of redundant data
in the database, even if some nodes are no longer in the cluster,
there are still records for that node.
Closes-Bug: #1976439
Change-Id: I320ac2306e0f25ff933d8271203e192486062d61
When two segments are concurrently created this could have resulted in
both threads creating a segment, thus resulting in two segments with
different segmentation ids. To prevent this we now introduce a new
unique constraint onto the networksegments table, which requires
(network_id, network_type, physical_network) to be unique, which allows
only a single segment with a single segmentation id to exist per
combination of these three values.
With the constraint in place a DB error will be thrown, which will cause
allocate_dynamic_segment() to be executed again and this time it will
find the already existing segment. To make sure that no additional DB
objects are created when segment creation failed we need to put all of
the allocation code into a DB transaction.
Change-Id: I407ae88d69ed971bf8d9a9b79120366f33bb56fd
Closes-Bug: #1791233
SR-IOV agent can handle ports with same MAC address (located in
different networks). The agent can retrieve, from the system, the
MAC address and the PCI slot; because the PCI slot is unique per
port in the same host, this parameter is used to match with the
Neutron port ID stored in the database (published via RPC).
RPC API bumped to version 1.9.
Closes-Bug: #1791159
Change-Id: Id8c3e0485bebc55c778ecaadaabca1c28ec56205
Subnet delete triggers dhcp port deletion but asynchronously,
therefore in the condition described in the bug report we may
get a conflict when deleting the segment too fast after the subnet.
Here we follow the example of how we auto-delete ports in net delete.
Please also find a fullstack test in Related-Change below.
Change-Id: Iba02f5a2211b18c2deb9097daad6be5e7d21faf8
Closes-Bug: #1878632
Related-Change: https://review.opendev.org/728904
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.
Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.
Unit tests only, removing from tests/base.py affects
functional tests which need additional work.
Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
The bulk port creation scenario requires the ability to generate
multiple MAC addresses for the bulk added ports. This change leverages
the code added in [1] to make bulk MAC creation available.
[1] https://review.openstack.org/510830
Implements: blueprint speed-up-neutron-bulk-creation
Depends-On: https://review.openstack.org/613149
Change-Id: Ia769dadf69781ba511a19c52998949b668963a19
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
With the adoption of multiple port bindings, get_port_binding_host is
updated to select only the active binding
Change-Id: I57ee16aa970527de2c2d787ebf0df71241f01e74
Partial-Bug: #1580880
Migrate to using Port OVO in the ML2 database testing, specifically in
Ml2DvrDBTestCase.
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: Ic7d31f4853776a641283ecb94d4357bb71cd2419
This reverts commit febeaf5d40.
This patch broke postgres tempest jobs, as well as introduced potential
race conditions in database layer because of mixed usage of old and new
engine facades.
Related-Bug: #1744829
Change-Id: Ic142ae7faf4e5f10cbdf761d7e6f3d442e94a3eb
When the test case fails, it's useful to see which warnings triggered
the failure.
Change-Id: Ia4150a06989ba253ffed0c0995e1023ca1ed42a0
Related-Bug: #1744394
This patch integrates Port Binding OVO in /plugins/ml2/db.py
and /plugins/ml2/plugin.py.
Co-Authored-By: Artur Korzeniewski <artur.korzeniewski@intel.com>
Change-Id: Idb76c0cb2a4d66690c9aca5ba338d5df814cd21e
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
The method was marked deprecated in commit
ffa7695038. This is not used
anywhere in the neutron code base.
The only place that this method was used is in the GBP code
base and they have patched the method.
TrivialFix
Change-Id: I709c701a0904c0e42266345d339f286a8a89b24d
The ml2 MechanismDriver is now in neutron-lib along with its associated
constants. This patch switches over to the lib versions of those, but
leaves a shim of the MechanismDriver that just ref's the driver from
lib. This shim allows our broad consumer base of the driver to switch
over at their leisure.
NeutronLibImpact
Change-Id: I99e3de6d933a1bb341394f85415fb07306a82a01
Neutron-lib 1.1.0 is now out and contains the portbindings
API definition (as per commit [1]). This patch moves neutron
references over to the neutron-lib version.
NeutronLibImpact
- Consumers using the public constants within neutron's
portbindings API extension must now use the values
from neutron-lib.
[1] 87e42f993c07ae320159d5123662ee9f3bd4d903
Change-Id: I669af9b4c712877772d91a03857ab108714001d4
Usage reader and writer for db operations.
In Neutron code we have common situation like:
with context.session.begin():
context.session.add(obj)
self._make_obj_dict(obj)
With new enginefacade we change context.session.begin() for
db.context_manager.writer(reader).using(context).
When object leaves this with-block, its reference to session is
cleared because session is discarded. To use this object later to
load some data from its dependencies, we have to provide different
session to work in. To solve this obj either can be moved under
with-block or we have to do context.session.add(obj) one more time
to be able to load relations.
This change also switches to usage of new enginefacade for some db
operations with ports, in order to pass unit and functional tests.
Partially-Implements blueprint: enginefacade-switch
Change-Id: Ia15c63f94d2c67791da3b65546e59f6929c8c685
This patch integrates Oslo-Versioned Object for Network model class
in neutron/tests/unit/plugins/ml2
Change-Id: Ic73398bc81202460c37af59824ec076d65816715
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
This relationship doesn't serve a purpose and it results
in SQLAlchemy trying to set the network_id column on ports
to NULL when attempting to delete a network with ports in it.
This results in the following exception:
IntegrityError: (1048, u"Column 'network_id' cannot be null")
This exception is not treated as a DBReferenceError by olso.db
which makes detecting races between ports being created on a
network being deleted very noisy in the logs since oslo.db treats
this as a generic DBError.
By removing the relationship, deleting a network will now just
result in a boring standard foreign key constraint failure which
will propagate from oslo.db as a DBReferenceError that is easy
to catch.
The patch that depends on this one adjusts the delete network logic
to capture these ReferenceErrors.
Change-Id: I4166a9bbaded57a77576400929a150123b8f1b2e
Port object is merged, this patch uses port object
in plugins/ml2/db.py and unit/plugins/ml2/test_db.py.
Change-Id: Ic1b617a789ea9c7e0ad369f34478d33e17837ba9
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
We have a number of functions that expect to get session as one
of the arguments. Passing session is not correct and this prevents
using new enginefacade which expects context to be passed and
session will be injected in current context.
NeutronLibImpact
Partially-Implements blueprint: enginefacade-switch
Change-Id: Ie1c54138ceaf9ac6f0674ad2786d4aaea9c80f73
We have a number of functions that expect to get session as one of
the arguments. Passing session is not correct and this prevents
the usage of new enginefacade which expect context to be passed
and session will be injected in current context.
NeutronLibImpact
Change-Id: If600033de801e1a3759919b272ac52f0850bf49c
This patch adds the name and description column to the
networksegments table.
Change-Id: I7edc224e25e604dfd7613b945a4ca16d9e385760
Partially-Implements: blueprint routed-networks
As part of making DVR portbinding implementation generic, we rename
dvr portbinding functions as distributed portbinding functions.
In next patch we make dvr logic for port binding generic,
to be useful for all distributed router ports(for example, HA).
Partial-Bug: #1595043
Partial-Bug: #1522980
Change-Id: I402df76c64299156d4ed48ac92ede1e8e9f28f23
This adds logic to add segment to host mappings for agents after a
network with one or more segments is created. The previous logic was
doing this only after an agent initially reported after the server
started. This meant that any segmented networks created after the
agent was running would not get mappings to the agent's host.
Change-Id: Ie265500639b1119f0448ed272d235fb709db36f6
Partially-Implements: blueprint routed-networks
Distributed port binding need to be implemented for HA router ports
to fix bug 1522980. HA ports can use existing DVR implementation for
multiple port binding. So we have to make current DVR port binding
implementation generic, so that all distributed ports(like DVR, HA)
can use it.
As part of making it generic, we rename 'ml2_dvr_port_bindings' table
to 'ml2_distributed_port_bindings', so that all distributed ports
(DVR, HA ..) can use this table.
Partial-Bug: #1595043
Partial-Bug: #1522980
Change-Id: I24650b7dee6305f801b457c4f21c8b16fb0eb6e0
This is a clean up for patch [1], the functions map should be removed
to make code easy to read. Start a deprecation cycle for these functions
in case external projects will use them.
[1] https://review.openstack.org/#/c/242393
Change-Id: I77c83bd7ee0c8ef92d8aaaa8e968479b848532fe
Partially-Implements: blueprint routed-networks
uselist=False is used for one to one relationship, but for
DVRPortBinding, a router_interface_distributed port can have
multiple bindings. That causes SAWarning:
Multiple rows returned with uselist=False for lazily-loaded attribute
'Port.dvr_port_binding'.
"uselist=False" is misused in DVRPortBinding port relationship,
this patch will fix this.
Change-Id: I2b00d96aaa445e0977bc3d4957d35a28d44dd953
Closes-Bug: #1540259
Fix params order to correspond to real signature:
assertEqual(expected, actual)
Change-Id: I722b998f6eae47076f3d10213073296a0a9a2081
Closes-Bug: #1277104
This method is dead, not been used in upstream code, nor
in stable/liberty, nor in stable/kilo, remove it now.
Change-Id: I072cd548bf56c3f08764571e8c20888427ad801f
A net-list operation was calling extend_network_dict_provider for
each network individually which would result in a database call for
each network.
This adds a new call in the manager to extend multiple networks at
once and then it adds a bulk version of get_network_segments that
it calls.
Now 1 net list of any number of networks will only result in 1
segment DB call.
Change-Id: I2543b3bdbb178ee4bb8d1288e9a27af1c5c8c8b4
Closes-Bug: #1525423
Partial-Bug: #1513782
This patch changes the get_port_from_device_mac() and
get_sg_ids_grouped_by_port() methods in ML2 db.py module so that
they do not create a new database session (via get_session()), but
instead reuse the session associated with the caller's context.
In order to make the session that is associated with the caller's
context available to these ML2 DB methods, the
get_ports_from_devices plugin API in securitygroups_rps_base.py
needs to be modified so that the context can be passed down to the
ML2 plugin. (A similar change is made to the get_port_from_device
plugin API for consistency.)
Change-Id: I3f990895887e156de929bd7ac3732df114dd4a4b
Closes-Bug: 1441205
This patch modifies ML2 DB get_port_binding_host method so that it
reuses the existing context session to do the database query
rather than creating a new database session.
Note that there are other methods in ML2 DB that do not re-use
the caller's session (get_port_from_device_mac() and
get_sg_ids_grouped_by_port()). These will be modified using
a separate bug (https://bugs.launchpad.net/neutron/+bug/1441205).
Change-Id: I8aafb0a70f40f9306ccc366e5db6860c92c48cce
Closes-Bug: #1440183
This change moves plugin test modules to conform to the new rules on
unit test tree structure (see TESTING.rst).
Vendor plugin paths continue to be ignored, and unit test modules that
test features instead of modules are also ignored pending their
removal to the functional test tree.
Change-Id: I482c377ca72ffd58692ad84bd9692356513e4c98
Closes-Bug: #1440834
liujinxin