The optional network argument was deprecated in
update_port_status() in Pike. Let's remove it as there
are are only in-tree callers passing it anymore.
TrivialFix
Change-Id: Iab8d3dada2e147da009e195700e64b072e5bfabb
rpc_workers can be set < 1 with 'ovn' backend when no
other agent is running apart from ovn agents to
consume these rpc notifications.
Add and apply disable_notifications decorator on
methods which do rpc cast calls to agents, the
decorator makes the caller method execute only
when rpc_workers >=1. This patch not changing
default behavior and utilizes the rpc_workers config option
to enable rpc notification on resources updates only when
rpc_workers >= 1.
Also set rpc_workers=0 in ovn jobs to cover this scenario.
Closes-Bug: #1889737
Closes-Bug: #1992352
Change-Id: I700fe2cd422bc1eb8b5144ec116e7f0a60238419
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/plugins.
Trivialfix
Change-Id: Id9138652f5f07ef12fa682e182fe210019e8f975
SR-IOV agent can handle ports with same MAC address (located in
different networks). The agent can retrieve, from the system, the
MAC address and the PCI slot; because the PCI slot is unique per
port in the same host, this parameter is used to match with the
Neutron port ID stored in the database (published via RPC).
RPC API bumped to version 1.9.
Closes-Bug: #1791159
Change-Id: Id8c3e0485bebc55c778ecaadaabca1c28ec56205
Currently the ovs agent calls update_device_list with the
agent_restarted flag set only on the first loop iteration. Then the
server knows to send the l2pop flooding entries for the network to
the agent. But when a compute node with many instances on many
networks reboots, it takes time to readd all the active devices and
some may be readded after the first loop iteration. Then the server
can fail to send the flooding entries which means there will be no
flood_to_tuns flow and broadcasts like dhcp will fail.
This patch fixes that by renaming the agent_restarted flag to
refresh_tunnels and setting it if the agent has not received the
flooding entries for the network.
Change-Id: I607aa8fa399e72b037fd068ad4f02b6210e57e91
Closes-Bug: #1853613
Although notify_nova_on_port_status_changes defaults to true, it
could be to false, making the nova_notifier attribute unsafe to
use without checking.
This patch checks both the config option and that the attribute
exists, since the config could be changed after the plugin is
already initialized without the nova_notifier attribute being set.
Change-Id: Ide0f93275e60dffda10b7da59f6d81c5582c3849
Closes-bug: #1843269
Generate an osprofiler trace_point every time neutron-server receives
an update_device_* rpc message with trace_info in its context. We assume
this marks the time when ovs-agent finished processing a port.
Change-Id: I554bc45ad44931691029e0066d5b8483522a7dd2
Partial-Bug: #1833674
In case of Smart NIC vNIC type neutron should mimic nova-compute
that plug the port to the ovs bridge.
Extend the Neutron OVS mechanism driver and Neutron OVS Agent to bind
the Neutron port for the baremetal host with Smart NIC. This will allow
the Neutron OVS Agent to configure the pipeline of the OVS running on
the Smart NIC and leverage the pipeline features such as: VXLAN,
Security Groups and ARP Responder.
Story: #2003346
Closes-Bug: #1785608
Change-Id: I6d520d3bac2e9ceb30b5b6197c6eb0f958cc3659
Added the ability to change the segmentation ID of a network
with ports bound to OVS agent. The rules, both in the integration
bridge and the physical bridge, to convert the internal VLAN tag
and the external segmentation ID (external VLAN tag) are deleted
and created again with the new value. The traffic from the tenant
networks will be tagged then with the new segmentation ID.
Added get network details agent RPC call to retrieve the information
of the updated network.
Partial-Bug: #1806052
Change-Id: I69f6f3ef717c3ed40218099b1f389afd3d39bd62
There are a few places in neutron that are not yet using the agent
topics module from neutron-lib. This patch address them by:
- Removing neutron.agent._topics
- Switching all uses of topics over to use neutron-lib
NeutronLibImpact
Change-Id: Ic9699ca93cd6fe59ed575ab444c4a4f9456323b2
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
Ovs-agent can be very time-consuming in handling a large number
of ports. At this point, the ovs-agent status report may have
exceeded the set timeout value. Some flows updating operations
will not be triggerred. This results in flows loss during agent
restart, especially for hosts to hosts of vxlan tunnel flow.
This fix will let the ovs-agent explicitly, in the first rpc loop,
indicate that the status is restarted. Then l2pop will be required
to update fdb entries.
Closes-Bug: #1813703
Closes-Bug: #1813714
Closes-Bug: #1813715
Closes-Bug: #1794991
Closes-Bug: #1799178
Change-Id: I8edc2deb509216add1fb21e1893f1c17dda80961
The neutron.common.rpc module has been in neutron-lib for awhile now and
neutron is shimmed to use neutron-lib already.
This patch removes neutron.common.rpc and switches the code over to use
neutron-lib's implementation where needed.
NeutronLibImpact
Change-Id: I733f07a8c4a2af071b3467bd710290eee11a4f4c
As part of the implementation of multiple port bindings [1], add binding
activation support to the linux bridge agent. This will enable the
execution with linux bridge agents of the complete sequence of steps
outlined in [1] during an instance migration:
1) Create inactive port bindings for destination host
2) Migrate the instance to the destination host and plug its VIFs
3) Activate the port bindings in the destination host
4) Delete the port bindings for the source host
[1] https://review.openstack.org/#/c/309416/
Change-Id: I2c937cc0a551e5ce0e8534c4dd4384ec2ca92da1
Partial-Bug: #1580880
This commit adds a binding_deactivate method to the Linux bridge agent
to receive messages from the ML2 plugin when a binding is de-activated
for a port. After receiving that message, the agent un-plugs the
corresponding tap interface from the port's network bridge.
To support this, a binding_deactivate method is also added to the agents
notifier. Finally, the activate method in the ML2 plugin is updated to
use the binding_deactivate method in the agents notifier.
Change-Id: I3f4e34766791c472a2c81842190094f697baa05c
Partial-Bug: #1580880
neutron-openvswitch-agent will refresh flows when it's restarted.
But the port's binding status is not changed, update_port_postcommit
will be skipped at function '_update_individual_port_db_status' in
'neutron/plugins/ml2/plugin.py', l2pop don't handle DVR ports, the
fdb entries about DVR port will not be added.
So, we can't skip DVR port at notify_l2pop_port_wiring when agent
is restared.
Closes-Bug: #1773286
Change-Id: I54e3db4822830a0c83daf7b5150575f8d6e2497b
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
NeutronLibImpact
Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
neutron-lib contains the QoS constants. This patch removes them from
neutron and replaces all such uses to lib's QoS constants.
NeutronLibImpact
Change-Id: I6ed379b178a2b79fd14385a1d1e87d87eb04bfb9
In case of a live migration and with linux bridge the events are not
sent to Nova, because the port UUID returned by _device_to_port_id may
be a truncated UUID and the current plugin._get_port() can't find it.
Related-Bug: #1414559
Change-Id: Icb039ae2d465e3822ab07ae4f9bc405c1362afba
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
I15cc2d6ae48e505c2da121880e27481dedf36d3b catches exceptions for
specific RPC endpoints affected by a recent change related to
push-notifications. There may be more changes like that in the future,
so instead of fixing them one by one, this patch consistently catches
exceptions from all cast calls for all RPC clients.
Change-Id: Ia7e6cd717758a9d5b18fe9cb07c55938f52040ce
Partial-Bug: #1705351
Recently we have been seeing an error in neutron associated
with DVR routers, that says 'Binding info for DVR port not
found'.
This error is thrown when the 'get_bound_port_context' is
called when trying to notify_l2pop_port_wiring.
notify_l2pop_port_wiring is intended for router 'HA' ports, so
the get_bound_port_context should be called here only for 'HA'
ports.
This was introduced by a recent refactor in neutron
Icd4cd4e3f735e88299e86468380c5f786e7628fe
Change-Id: I1c636344068518aa26be6c96c598c61b7f0f3563
Closes-Bug: #1702769
After an upgrade we can get NotFound exceptions from AMQP after there
are no longer any agents interested in listening to that exchange.
These should be ignored and not propagated up since they don't indicate
any kind of issue.
However, oslo messaging is not translating the exceptions into an
oslo.messaging exception so there is no way to catch them without
importing amqp directly.
As a workround to unblock Ironic, this patch just proposes a generic
catch of all exceptions during a cast. These casts occur after work
has been done so raising them back to the user doesn't do anything
useful anyway.
Once we have an oslo.messaging exception to catch we can narrow it to
that.
Change-Id: I15cc2d6ae48e505c2da121880e27481dedf36d3b
Partial-Bug: #1705351
The ml2 MechanismDriver is now in neutron-lib along with its associated
constants. This patch switches over to the lib versions of those, but
leaves a shim of the MechanismDriver that just ref's the driver from
lib. This shim allows our broad consumer base of the driver to switch
over at their leisure.
NeutronLibImpact
Change-Id: I99e3de6d933a1bb341394f85415fb07306a82a01
This eliminates the last of the bottlenecks in
get_devices_details_list_and_failed_devices by making the status
updates use a bulk data retrieval as well.
The last remaining thing that will impact performance is the status
update back to ACTIVE on removal of the provisioning blocks. However,
that will require a much larger refactor since it is callback driven
at the individual port level.
Elimination of the L2pop driver will ultimately solve this completely
since we won't need to cycle the port status anymore on every single
agent restart.
Closes-Bug: #1665215
Change-Id: I99c2b77b35e6eabb6e4f633c4e8e2533594c6b55
This is a small refactor to separate updating the port status
from the method retrieving device details. This is in preparation
for patch I99c2b77b35e6eabb6e4f633c4e8e2533594c6b55 which will allow
updating ports' status in bulk.
Change-Id: Ifa78f6911cfbbd3e982807122bddc898613f323f
Partial-Bug: #1665215
This patch calls update_port_up and update_port_down inside
of the l2pop driver directly from update_device_up and
update_device_down in the ML2 RPC layer. This allows the L2pop
driver to setup forwarding entries completely independent of
the port status update events.
This will allow L2pop to function without requiring the ports to
transition from ACTIVE->BUILD->ACTIVE every time the agent requests
device details.
This will unblock the push notifications work and will additionally
enable us to remove the update to BUILD status as part of a performance
improvement backport for bug #1665215.
Partial-Bug: #1665215
Partially-Implements: blueprint push-notifications
Change-Id: Icd4cd4e3f735e88299e86468380c5f786e7628fe
The port security API definition has been in neutron-lib for awhile.
This patch consumes the definition from neutron-lib.
NeutronLibImpact
- Consumers using the public definitions from
neutron.extensions.portsecurity must now switch over to the api-def in
neutron-lib. See the changes herein for additional details.
Change-Id: If43b65861efc536d01c43dc0d2bbcbcf062c1271
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
With the switch to subquery relationships, individual get_port calls
can get expensive with large numbers of ports
(100ms per port in my dev environment). This patch bulks up the
retrieval of the port contexts so one set of queries covers all
of the devices in an RPC call.
Partial-Bug: #1665215
Change-Id: I63757e143b23c24c349be98dc5a09115b8709a25
Libvirt does not set the MTU of the tap device it creates when creating
a bridge interface. It also does not set the MTU of the bridge itself.
This cannot be fixed on the Nova side since libvirt doesn't appear to
have support for setting MTUs on bridges until version 3x.
This results in a situation where the first VM tap interface attached to
a bridge will always have an MTU of 1500. The Neutron agent will then
add in VLAN/VXLAN interfaces with the correct MTU; however, the bridge
MTU will still be pinned to the smallest interface MTU attached to it.
This breaks jumbo frames until all small MTU tap devices are removed
from the bridge.
This patch explicitly sets the MTU on tap devices to match the network
MTU when processing the device.
We will have to carry this workaround until we stop Nova from
plugging taps into bridges[1] or until we drop support for older
libvirts on the Nova side and have it set the MTU.
This bug was introduced by change
I53c0eb57da956b36f09731d25db989719e9bc9dc which reverted automatic
setting of tap MTUs to match those of the physical device.
1. I23c5faaeab69aede1fd038a36f4a0b8f928498ce
Closes-Bug: #1684038
Change-Id: Ia245a3e22339fce026901e24a82e836c8b27cc28
Neutron-lib 1.1.0 is now out and contains the portbindings
API definition (as per commit [1]). This patch moves neutron
references over to the neutron-lib version.
NeutronLibImpact
- Consumers using the public constants within neutron's
portbindings API extension must now use the values
from neutron-lib.
[1] 87e42f993c07ae320159d5123662ee9f3bd4d903
Change-Id: I669af9b4c712877772d91a03857ab108714001d4
This reverts commit 75edc1ff28.
Ports with port security disabled require firewall entries in
neutron-openvswi-FORWARD chain to work properly.
Ports created with no security groups will not get skipped with current
code.
With fixed security groups check, these ports' security groups can not
be updated after creation.
Change-Id: I95ddbe38d8ac8a927a860a98f54e41e17fb71d43
Closes-Bug: #1549443
We have a number of functions that expect to get session as one of
the arguments. Passing session is not correct and this prevents
the usage of new enginefacade which expect context to be passed
and session will be injected in current context.
NeutronLibImpact
Change-Id: If600033de801e1a3759919b272ac52f0850bf49c
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
This patch introduces context in methods where it is needed for
object operations done during the integration of Router Extra
Attributes OVO.
Used in integration patch: https://review.openstack.org/#/c/381209/
Change-Id: I4dea67207220a19abf5d3cc754f02150b3621550
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
This patch makes L3 HA failover not depended on neutron components
(during failover).
All HA agents(active and backup) call update_device_up/down after wiring
the ports. But l2pop driver is called for only active agent as port
binding in DB reflects active agent. Then l2pop creates unicast and
multicast flows for active agent.
On failover, flows to new active agent is created. For this to happen -
all of database, messaging server, neutron-server and destination L3
agent should be active during failover. This creates two issues -
1) When any of the above resources(i.e neutron-server, .. ) are dead,
flows between new master and other agents won't be created and
L3 Ha failover is not working. In same scenario, L3 Ha failover will
work if l2pop is disabled.
2) Packet loss during failover is higher as above neutron resources
interact multiple times, so will take time to create l2 flows.
In this change, we allow plugin to notify l2pop when update_device_up/down
is called by backup agents also. Then l2pop will create flood flows to
all HA agents(both active and slave). L2pop won't create unicast flow for
this port, instead unicast flow is created by learning action of table 10
when keepalived sends GARP after assigning ip address to master router's
qr-xx port. As flood flows are already created and unicast flow is
dynamically added, L3 HA failover is not depended on l2pop.
This solves two isses
1) with L3 HA + l2pop, failover will work even if any of above agents
or processes dead.
2) Reduce failover time as we are not depending on neutron to create
flows during failover.
We use L3HARouterAgentPortBinding table for getting all HA agents of a
router port. HA router port on slave agent is also considered for l2pop
distributed_active_network_ports and agent_network_active_port_count
Closes-bug: #1522980
Closes-bug: #1602614
Change-Id: Ie1f5289390b3ff3f7f3ed7ffc8f6a8258ee8662e
- during live migration on pre migration step nova plugs instance
vif device on the destination compute node;
- L2 agent on destination host detects new device and requests device
info from server;
- server does not change port status since port is bound to another
host (source host);
- L2 agent processes device and sends update_device_up to server;
- again server does not update status as port is bound to another host;
Nova notifications are sent only in case port status change so in this case
no notifications are sent.
The fix is to explicitly notify nova if agent reports device up from a host
other than port's current host.
This is the fix on neutron side, the actual fix of the bug is on nova side:
change-id Ib1cb9c2f6eb2f5ce6280c685ae44a691665b4e98
Closes-Bug: #1414559
Change-Id: Ifa919a9076a3cc2696688af3feadf8d7fa9e6fc2
Sometimes an object requires multiple disjoint actors to complete
a set of tasks before the status of the object should be transitioned
to ACTIVE. The main example of this is when a port is being created.
The L2 agent has to do its business to wire up the VIF, but at the same
time the DHCP agent has to setup the DHCP reservation. This led to
Nova booting the VM when the L2 agent was done even though the DHCP
agent may have been nowhere near ready.
This patch introduces a provisioning blocks mechansim that allows the
entities to be tracked that need to be involved to make a transition
to ACTIVE happen. See the devref in the dependent patch for a high-level
view of how this works.
The ML2 code is updated to use this new mechanism to prevent updating
the port status to ACTIVE without both the DHCP agent and L2 agent
reporting that the port is ready.
The DHCP RPC API required a version bump to allow the port ready
notification.
This also adds a devref doc for the provisioning_blocks
module with a high-level overview of how it works in addition
to a detailed description of how it is used specifically with
ML2, the L2 agents, and the DHCP agents.
Closes-Bug: #1453350
Change-Id: Id85ff6de1a14a550ab50baf4f79d3130af3680c8
Brian Haley
yatinkarel