SR-IOV agent can handle ports with same MAC address (located in
different networks). The agent can retrieve, from the system, the
MAC address and the PCI slot; because the PCI slot is unique per
port in the same host, this parameter is used to match with the
Neutron port ID stored in the database (published via RPC).
RPC API bumped to version 1.9.
Closes-Bug: #1791159
Change-Id: Id8c3e0485bebc55c778ecaadaabca1c28ec56205
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.
Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.
Unit tests only, removing from tests/base.py affects
functional tests which need additional work.
Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
Currently the ovs agent calls update_device_list with the
agent_restarted flag set only on the first loop iteration. Then the
server knows to send the l2pop flooding entries for the network to
the agent. But when a compute node with many instances on many
networks reboots, it takes time to readd all the active devices and
some may be readded after the first loop iteration. Then the server
can fail to send the flooding entries which means there will be no
flood_to_tuns flow and broadcasts like dhcp will fail.
This patch fixes that by renaming the agent_restarted flag to
refresh_tunnels and setting it if the agent has not received the
flooding entries for the network.
Change-Id: I607aa8fa399e72b037fd068ad4f02b6210e57e91
Closes-Bug: #1853613
Although notify_nova_on_port_status_changes defaults to true, it
could be to false, making the nova_notifier attribute unsafe to
use without checking.
This patch checks both the config option and that the attribute
exists, since the config could be changed after the plugin is
already initialized without the nova_notifier attribute being set.
Change-Id: Ide0f93275e60dffda10b7da59f6d81c5582c3849
Closes-bug: #1843269
test_rpc.py:348:15: E1123: Unexpected keyword argument 'tenant'
in constructor call (unexpected-keyword-arg)
'project_id' is the preferred name now.
Change-Id: I8daf498fa5ad29c088b90ece0e8971febad1877d
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
In case of Smart NIC vNIC type neutron should mimic nova-compute
that plug the port to the ovs bridge.
Extend the Neutron OVS mechanism driver and Neutron OVS Agent to bind
the Neutron port for the baremetal host with Smart NIC. This will allow
the Neutron OVS Agent to configure the pipeline of the OVS running on
the Smart NIC and leverage the pipeline features such as: VXLAN,
Security Groups and ARP Responder.
Story: #2003346
Closes-Bug: #1785608
Change-Id: I6d520d3bac2e9ceb30b5b6197c6eb0f958cc3659
Added the ability to change the segmentation ID of a network
with ports bound to OVS agent. The rules, both in the integration
bridge and the physical bridge, to convert the internal VLAN tag
and the external segmentation ID (external VLAN tag) are deleted
and created again with the new value. The traffic from the tenant
networks will be tagged then with the new segmentation ID.
Added get network details agent RPC call to retrieve the information
of the updated network.
Partial-Bug: #1806052
Change-Id: I69f6f3ef717c3ed40218099b1f389afd3d39bd62
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
Ovs-agent can process the ports in large sets, then all
of these ports will have to update DB status or attributes.
But neutron server is centralized. It may have to do
something else, or the database processing can be also
time-consuming. Because of these, it sometimes returns
the RPC timeout exception to ovs-agent. And a fullsync
will be triggered in next rpc loop. The restart time is
becoming longer and longer.
Adds a default step to update the port to reduce
the probability of RPC timeout.
Related-Bug: #1813703
Related-Bug: #1813704
Related-Bug: #1813706
Related-Bug: #1813707
Change-Id: Ie37f4a4869969e235ce16b73cdfcbdc98626823e
Ovs-agent can be very time-consuming in handling a large number
of ports. At this point, the ovs-agent status report may have
exceeded the set timeout value. Some flows updating operations
will not be triggerred. This results in flows loss during agent
restart, especially for hosts to hosts of vxlan tunnel flow.
This fix will let the ovs-agent explicitly, in the first rpc loop,
indicate that the status is restarted. Then l2pop will be required
to update fdb entries.
Closes-Bug: #1813703
Closes-Bug: #1813714
Closes-Bug: #1813715
Closes-Bug: #1794991
Closes-Bug: #1799178
Change-Id: I8edc2deb509216add1fb21e1893f1c17dda80961
As part of the implementation of multiple port bindings [1], add binding
activation support to the linux bridge agent. This will enable the
execution with linux bridge agents of the complete sequence of steps
outlined in [1] during an instance migration:
1) Create inactive port bindings for destination host
2) Migrate the instance to the destination host and plug its VIFs
3) Activate the port bindings in the destination host
4) Delete the port bindings for the source host
[1] https://review.openstack.org/#/c/309416/
Change-Id: I2c937cc0a551e5ce0e8534c4dd4384ec2ca92da1
Partial-Bug: #1580880
DeprecationWarning: Using the 'user' argument is deprecated
in version '2.18' and will be removed in version '3.0'
Change-Id: I96061cf624c84145c4ec7031414aecb58c2ced79
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
NeutronLibImpact
Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
neutron-lib contains the QoS constants. This patch removes them from
neutron and replaces all such uses to lib's QoS constants.
NeutronLibImpact
Change-Id: I6ed379b178a2b79fd14385a1d1e87d87eb04bfb9
In case of a live migration and with linux bridge the events are not
sent to Nova, because the port UUID returned by _device_to_port_id may
be a truncated UUID and the current plugin._get_port() can't find it.
Related-Bug: #1414559
Change-Id: Icb039ae2d465e3822ab07ae4f9bc405c1362afba
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>
Refactoring neutron ml2 config opts to be in neutron/conf/plugins/ml2.
This would allow centralization of all configuration options and
provides an easy way to import.
NeutronLibImpact
Change-Id: Ibc5a9ab268578c243ef13f7e0041bacd6c0c410b
Partial-Bug: #1563069
Needed-By: Id0a97dda7718f06e33b2d30ce01cdcb3e9a46f7d
The well known service type constants are in
neutron_lib.plugins.constants, but for legacy reasons a few still exist
and are referenced from neutron_lib.constants that we'd like to remove.
This patch switches references over to neutron_lib's plugin constants.
Change-Id: I1861448cec303725b30cef8f42029f467f9e03a3
This is a small refactor to separate updating the port status
from the method retrieving device details. This is in preparation
for patch I99c2b77b35e6eabb6e4f633c4e8e2533594c6b55 which will allow
updating ports' status in bulk.
Change-Id: Ifa78f6911cfbbd3e982807122bddc898613f323f
Partial-Bug: #1665215
This patch calls update_port_up and update_port_down inside
of the l2pop driver directly from update_device_up and
update_device_down in the ML2 RPC layer. This allows the L2pop
driver to setup forwarding entries completely independent of
the port status update events.
This will allow L2pop to function without requiring the ports to
transition from ACTIVE->BUILD->ACTIVE every time the agent requests
device details.
This will unblock the push notifications work and will additionally
enable us to remove the update to BUILD status as part of a performance
improvement backport for bug #1665215.
Partial-Bug: #1665215
Partially-Implements: blueprint push-notifications
Change-Id: Icd4cd4e3f735e88299e86468380c5f786e7628fe
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
With the switch to subquery relationships, individual get_port calls
can get expensive with large numbers of ports
(100ms per port in my dev environment). This patch bulks up the
retrieval of the port contexts so one set of queries covers all
of the devices in an RPC call.
Partial-Bug: #1665215
Change-Id: I63757e143b23c24c349be98dc5a09115b8709a25
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
This patch makes L3 HA failover not depended on neutron components
(during failover).
All HA agents(active and backup) call update_device_up/down after wiring
the ports. But l2pop driver is called for only active agent as port
binding in DB reflects active agent. Then l2pop creates unicast and
multicast flows for active agent.
On failover, flows to new active agent is created. For this to happen -
all of database, messaging server, neutron-server and destination L3
agent should be active during failover. This creates two issues -
1) When any of the above resources(i.e neutron-server, .. ) are dead,
flows between new master and other agents won't be created and
L3 Ha failover is not working. In same scenario, L3 Ha failover will
work if l2pop is disabled.
2) Packet loss during failover is higher as above neutron resources
interact multiple times, so will take time to create l2 flows.
In this change, we allow plugin to notify l2pop when update_device_up/down
is called by backup agents also. Then l2pop will create flood flows to
all HA agents(both active and slave). L2pop won't create unicast flow for
this port, instead unicast flow is created by learning action of table 10
when keepalived sends GARP after assigning ip address to master router's
qr-xx port. As flood flows are already created and unicast flow is
dynamically added, L3 HA failover is not depended on l2pop.
This solves two isses
1) with L3 HA + l2pop, failover will work even if any of above agents
or processes dead.
2) Reduce failover time as we are not depending on neutron to create
flows during failover.
We use L3HARouterAgentPortBinding table for getting all HA agents of a
router port. HA router port on slave agent is also considered for l2pop
distributed_active_network_ports and agent_network_active_port_count
Closes-bug: #1522980
Closes-bug: #1602614
Change-Id: Ie1f5289390b3ff3f7f3ed7ffc8f6a8258ee8662e
Remove deprecation warnings for various constants
and exceptions that have moved to neutron_lib.
Fix miscellaneous other deprecations.
Uses constants instead of l3_constants when importing
neutron-lib constants.
Co-Authored By: Henry Gessau <gessau@gmail.com>
Co-Authored By: Gary Kotton <gkotton@vmware.com>
Change-Id: Ib0e8ff5c3e23677c1009241a1818cbc8a3430c38
- during live migration on pre migration step nova plugs instance
vif device on the destination compute node;
- L2 agent on destination host detects new device and requests device
info from server;
- server does not change port status since port is bound to another
host (source host);
- L2 agent processes device and sends update_device_up to server;
- again server does not update status as port is bound to another host;
Nova notifications are sent only in case port status change so in this case
no notifications are sent.
The fix is to explicitly notify nova if agent reports device up from a host
other than port's current host.
This is the fix on neutron side, the actual fix of the bug is on nova side:
change-id Ib1cb9c2f6eb2f5ce6280c685ae44a691665b4e98
Closes-Bug: #1414559
Change-Id: Ifa919a9076a3cc2696688af3feadf8d7fa9e6fc2
Sometimes an object requires multiple disjoint actors to complete
a set of tasks before the status of the object should be transitioned
to ACTIVE. The main example of this is when a port is being created.
The L2 agent has to do its business to wire up the VIF, but at the same
time the DHCP agent has to setup the DHCP reservation. This led to
Nova booting the VM when the L2 agent was done even though the DHCP
agent may have been nowhere near ready.
This patch introduces a provisioning blocks mechansim that allows the
entities to be tracked that need to be involved to make a transition
to ACTIVE happen. See the devref in the dependent patch for a high-level
view of how this works.
The ML2 code is updated to use this new mechanism to prevent updating
the port status to ACTIVE without both the DHCP agent and L2 agent
reporting that the port is ready.
The DHCP RPC API required a version bump to allow the port ready
notification.
This also adds a devref doc for the provisioning_blocks
module with a high-level overview of how it works in addition
to a detailed description of how it is used specifically with
ML2, the L2 agents, and the DHCP agents.
Closes-Bug: #1453350
Change-Id: Id85ff6de1a14a550ab50baf4f79d3130af3680c8
Neutron tests should use:
self.assertIn(value, list)
self.assertNotIn(value, list)
instead of:
self.assertTrue(value in list)
self.assertFalse(value in list)
because assertIn and assertNotIn raise more meaningful errors:
self.assertIn(3, [1, 2]
>>> MismatchError: 3 not in [1, 2]
self.assertTrue(3 in [1, 2])
>>> AssertionError: False is not true
Closes-Bug: #1218713
Change-Id: Ic8492a88935bf005feb9dae726a4bee604a8bd09
Network devices, like internal router legs, or dhcp ports
should not be affected by bandwidth limiting rules.
This patch disables application of network attached policies
to network/neutron owned ports.
Closes-bug: #1486039
DocImpact
Change-Id: I75d80227f1e6c4b3f5fa7762b8dc3b0c0f1abd46
This is needed to make l2 agent qos extension determine which rules to
apply to the port, if any.
Partially-Implements: blueprint quantum-qos-api
Change-Id: Idefa819f9a21cf53762b1fb923dafb63f2b256e0
This patch introduces two new RPC calls:
* update_device_list
* get_devices_details_list_and_failed_devices
update_device_up and update_device_down are the RPC calls
used by the L2 agent to notify the plugin that
a device is up or down. These calls accept only one device.
update_device_list accepts a list of devices that will
be put up or down and returns a list of devices for which
the operation failed.
get_devices_details_list_and_failed_devices is a new RPC
call that returns a list of devices details and a list of
devices for which it was not possible to retrieve the
details.
Thanks to these new calls, the agent will be able to update
the status of several devices using one RPC call
and to retry the operation only for the devices that failed
instead of performing a full resync.
This will be implemented in another patch.
This patch introduces the new calls only for the OVS agent.
A following patch will take care of the other L2 agents.
DocImpact
Partially-Implements: blueprint restructure-l2-agent
Change-Id: Idb4e81f4399cf66025ec7edfcb78801c15e493af
`contextlib.nested` is deprecated since Python 2.7 and incompatible with
Python 3. This patch removes all its occurences by using the helper
script at [1].
This is a necessary step to allow us running all unit tests with
Python 3 (not just a small subset as it is done now).
[1]: https://github.com/adrienverge/context_unnester
Change-Id: I8d1de09ff38ed0af9fb56f423a2c43476408e0fb
Blueprint: neutron-python3
Closes-Bug: #1428424
Add various initializations and imports so that unit tests can be run
independently.
This change fixes the following test cases which could not be run
independently, that is running any individual unit test case by going
in to the py27 venv and running
"unit2 neutron.tests.unit.module.Class.test_case":
neutron.tests.unit.plugins.ml2.drivers.arista.test_mechanism_arista.*
neutron.tests.unit.plugins.ml2.drivers.cisco.apic.*
neutron.tests.unit.plugins.ml2.test_rpc.RpcCallbacksTestCase.*
neutron.tests.unit.services.l3_router.test_l3_apic.*
neutron.tests.unit.agent.dhcp.test_agent.TestDhcpAgentEventHandler.*
(Note that these issues are not seen when running tox because the
initializations occur when all test modules are imported for test
discovery.)
Closes-bug: 1438463
Closes-bug: 1454640
Change-Id: I681caa66b51ce9a7bfbee5dfc43d534ba0d51947
This patch changes the get_port_from_device_mac() and
get_sg_ids_grouped_by_port() methods in ML2 db.py module so that
they do not create a new database session (via get_session()), but
instead reuse the session associated with the caller's context.
In order to make the session that is associated with the caller's
context available to these ML2 DB methods, the
get_ports_from_devices plugin API in securitygroups_rps_base.py
needs to be modified so that the context can be passed down to the
ML2 plugin. (A similar change is made to the get_port_from_device
plugin API for consistency.)
Change-Id: I3f990895887e156de929bd7ac3732df114dd4a4b
Closes-Bug: 1441205
Currently, nothing prevents the port status to be changed to BUILD
state when get_device_details() is sent by a host that doesn't own
the port.
In some cases the port might stay in BUILD state.
This could happen during a live-migration, or for multi-hosted ports
such as HA ports.
This commit allows the port status modification only if the port
is bound to the host that is asking for it.
Closes-Bug: #1439857
Closes-Bug: #1438040
Closes-Bug: #1416933
Change-Id: I9b3673f453abbafaaa4f78542fcfebe8dc93f2bb
This change moves plugin test modules to conform to the new rules on
unit test tree structure (see TESTING.rst).
Vendor plugin paths continue to be ignored, and unit test modules that
test features instead of modules are also ignored pending their
removal to the functional test tree.
Change-Id: I482c377ca72ffd58692ad84bd9692356513e4c98
Closes-Bug: #1440834
Rodolfo Alonso Hernandez