Prior to this patch, ML2/OVS and ML2/OVN had inconsistent IGMP
configurations. Neutron only exposed one configuration option for IGMP:
igmp_snooping_enabled.
Other features such as IGMP flood, IGMP flood reports and IGMP flood
unregistered were hardcoded differently on each driver (see LP#2044272
for a more details).
These hardcoded values has led to many changes over the years tweaking
them to work on different scenarios but they were never final because
the fix for one case would break the other.
This patch introduces 3 new configuration options for these other IGMP
features that can be enabled or disabled on both backends. Operators
can now fine tune their deployments in the way that will work for them.
As a consequence of the hardcoded values for each driver we had to break
some defaults and, in the case of ML2/OVS, if operators want to keep
things as they were before this patch they will need to enable the new
mcast_flood and mcast_flood_unregistered configuration options.
That said, the for ML2/OVS there was also an inconsistency with the help
string of igmp_snooping_enabled configuration option as it mentioned
that enabling snooping would disable flooding to unregistered ports but
that was not true anymore after the fix [0].
[0] https://bugs.launchpad.net/neutron/+bug/1884723
Closes-Bug: #2044272
Change-Id: Ic4dde46aa0ea2b03362329c87341c83b24d32176
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
The tunneling bridge uses different openflow rules depending if the
agent is running in DVR mode or not. With DVR enabled initial rule was
installed that caused traffic coming from the integration bridge to be
flooded to all tunnels. After a few miliseconds this flow was replaced
by a DVR specific flow, correctly dropping the traffic. This small time
window caused a network loop on the compute node with restarted agent.
This patch skips installing the non-dvr specific flow in case OVS agent
is working in DVR mode. Hence the traffic is never flooded to the
tunnels.
Closes-bug: #2028795
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
Change-Id: I3ce026054286c8e28ec1500f1a4aa607fe73f337
This change is updating the vlanmanager data structure to handle for a
given network more than one vlan mapping. This is a prerequisite work
needed to progress on accepting several segments per network per
host.
The work done here is trying to avoid changing logic in the
current implementation. Unit test should not have value updated,
but probably signatures changed.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: Ic3c147136549b17aea0fe78e930a41a5b33ab9d8
During some ml2 ovs agent port processing performance test, we noticed
that some ports are missing tag before it really done processing. While
ovs treats those ports without tag as trunk port, so some packets will
be flooded to it. In large scale cloud, if too many port added to the
bridge, the ovs-vswitchd will consume a huge amount of CPU cores if
ports are not bound in a short time.
So, in the port_bound function of ovs-agent, we set the port tag to
it after a local_vlan id is allocated. Because after that, setup
security groups (setup_port_filters) and bind devices in DB
(update_device_list) are really time-consuming.
And also fix a potential bug, port is processed as created first,
but no tag in ovsdb, so openflow security group will not be processed
successfully [1]. It must be done in a update event during next loop,
after port bound and ovsdb set the required value.
This patch can also fix some upstream test failures of waiting too
long time to ping some cases.
[1] https://github.com/openstack/neutron/blob/master/neutron/agent/linux/openvswitch_firewall/firewall.py#L112
Closes-Bug: #1952567
Change-Id: I3533f0d416d32f8d0888ad58f975960d89a985d9
The [agent] veth_mtu parameter has had unused since the [ovs]
use_veth_interconnection parameter was removed by [1] during Wallaby.
[1] https://review.opendev.org/c/openstack/neutron/+/759947
This change formally deprecate the parameter so that we can remove it
in a next cycle.
Change-Id: Ib85959fbc06928a49df7ea104eae3aca3f04e091
Closes-Bug: #1957180
Add a new ovs agent extension to support distributed DHCP for
VMs in compute nodes directly. For large scale deployment, this
can be used to reduce the number of neutron agents. Large scale
cloud can benefit from it.
From the perspective of virtual machine, this will reduce the
probability of DHCP request failure. The VMs will get a higher
level availability for DHCP R/R, no single point of failure
permanently. If one host goes down, VMs in other hosts will not
be influnced by it.
For the perspective of network performance, after using this
extension, the DHCP broadcasting packages will be limited
to the host locally.
Partially-Implements: bp/distributed-dhcp-for-ml2-ovs
Closes-Bug: #1900934
Change-Id: Id8a4c501daad7c2185e6d69441182666ef987e61
In the ML2/OVS when igmp_snooping is enabled but there is no
external querier multicast traffic will stop working after few minutes
as packets will not be flooded to tunnel/external bridges.
So this patch sets "mcast-snooping-disable-flood-unregistered" option
of the br-int to False (default value) even when igmp_snooping is
enabled in the neutron-ovs-agent's config file.
Additionally it configures "mcast-snooping-flood-reports" and
"mcast-snooping-flood" on patch ports in br-int to True.
That way we can provide best effort snooping: multicast isolation where
IGMP queriers are available and flood everywhere else?
Closes-Bug: #1884723
Change-Id: Iefa0044dba9e92592295a79448e5d57d9e14a40b
Using veth to interconnect openvswitch bridges was deprecated
in Victoria cycle. Now it's time to remove it from the code.
In neutron-ovs-agent code, there is still kept piece of code which
migrates from the veth to the patch ports for bridges interconnection.
We will be able to remove that piece of code in X release.
Change-Id: I94545c3c3d9be46ac2062691f69663e5e59cd648
Closes-Bug: #1587296
When there is no openflow firewall, aka the ovs agent security group
is disabled or Noop/HybridIptable, this patch will introduce a different
ingress pipeline for bridge ports which will avoid ingress flood:
(1) table=0, in_port=patch_bridge,dl_vlan=physical_vlan action=mod_vlan:local_vlan,goto:60 (original)
(2) table=60, in_port=patch_bridge action=goto:61 (new)
(3) table=61, dl_dst=local_port_mac,dl_vlan=local_vlan, action=strip_vlan,output:<ofport> (changes)
And changes the local ports pipeline:
(1) table=0, in_port=local_ofport action=goto:25 (original)
(2) table=25, in_port=local_ofport,dl_src=local_port_mac action=goto:60 (original)
(3) table=60, in_port=local_ofport,dl_src=local_port_mac action=local_vlan->reg6,goto:61 (changes)
(4) table=61, dl_dst=local_port_mac,reg6=local_vlan, action=output:<ofport> (changes)
Closes-Bug: #1884708
Closes-Bug: #1881070
Related-Bug: #1732067
Related-Bug: #1866445
Related-Bug: #1883321
Change-Id: Iecf9cffaf02616342f1727ad7db85545d8adbec2
This change removes the "_check_ofport" function and its use form
the ovs_lib.py file.
By skipping ports without a unique ofport in the "get_vifs_by_ids"
and "get_vifs_by_id" functions, the OVS agent incorrectly treated
newly added port with an ofport of -1 as removed ports in the
"treat_devices_added_or_updated" function.
Co-Authored-By: Rodolfo Alonso Hernandez <ralonsoh@redhat.com>
Change-Id: I79158baafbb99bee99a1d687039313eb454d3a9b
Partial-Bug: #1734320
Partial-Bug: #1815989
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.
Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.
Unit tests only, removing from tests/base.py affects
functional tests which need additional work.
Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
The unit tests in test_ovs_tunnel.py were verifying that
port_exists() was calling bool(), which fails when using
unittest.mock. Since it doesn't really gain anything,
just remove the check for that exact call.
Trivialfix
Change-Id: Id7712330a24f51f0cfee8d7b3916c05d3501ee3f
From the comments, this code existed to have API compatibility
between the native openflow and ovs-ofctl of_interface drivers,
but since the latter was removed, this code is no longer
necessary.
Change-Id: Icf346e58904412a97e5e22155166821faed19fc2
Do not flood the packets to bridge, since we have the
bridge port list, we can add a simple direct flow to
the right port only.
Closes-Bug: #1732067
Related-Bug: #1841622
Change-Id: I14fefe289a19b718b247bf0740ca9bc47f8903f4
Neutron-ovs-agent can now enable IGMP snooping in integration bridge
if config option "igmp_snooping_enable" in OVS section in config will
be set to True.
It will also set mcast-snooping-disable-flood-unregistered=true
so flooding of multicast packets to all unregistered ports will be
disabled also.
Both changes are applied on integration bridge.
Change-Id: I12f4030a35d10d1715d3b4bfb3ed5efb9aa28f2b
Closes-Bug: #1840136
Default value for "of_interface" config option was switched
to "native" in Pike release.
In the same release this option was deprecated to removal.
Now it's time to remove it and force use of "native" driver to
manage openflows.
Change-Id: Ic900209868acfbe3bbb56fabbbf5c4472857e412
Co-Authored-By: Ihar Hrachyshka <ihrachys@redhat.com>
Co-Authored-By: Slawek Kaplonski <skaplons@redhat.com>
The dump-flows action will get a very large sets of flow information
if there are enormous ports or openflow security group rules. For now
we can meet some known exception during such action, for instance,
memory issue, timeout issue.
So after this patch, the cleanup action of the bridge stale flows
will be done one table by one table. But note, this only supports
for 'native' OpenFlow interface driver.
Related-Bug: #1813703
Related-Bug: #1813712
Related-Bug: #1813709
Related-Bug: #1813708
Change-Id: Ie06d1bebe83ffeaf7130dcbb8ca21e5e59a220fb
In the OVS agent, when setting up the ancillary bridges, the parameter
external_id:bridge-id is retrieved. If this parameter is not defined
(e.g.: manually created bridges), ovsdbapp writes an error in the logs.
This information is irrelevant and can cause confusion during debugging time.
Change-Id: Ic85db65f651eb67fcb56b937ebe5850ec1e8f29f
Closes-Bug: #1815912
The native OVS/ofctl controllers talk to the bridges using a
datapath-id, instead of the bridge name. The datapath ID is
auto-generated based on the MAC address of the bridge's NIC.
In the case where bridges are on VLAN interfaces, they would
have the same MACs, therefore the same datapath-id, causing
flows for one physical bridge to be programmed on each other.
The datapath-id is a 64-bit field, with lower 48 bits being
the MAC. We set the upper 12 unused bits to identify each
unique physical bridge
This could also be fixed manually using ovs-vsctl set, but
it might be beneficial to automate this in the code.
ovs-vsctl set bridge <mybr> other-config:datapath-id=<datapathid>
You can change this yourself using above command.
You can view/verify current datapath-id via
ovs-vsctl get Bridge br-vlan datapath-id
"00006ea5a4b38a4a"
(please note that other-config is needed in the set, but not get)
Closes-Bug: #1697243
Co-Authored-By: Rodolfo Alonso Hernandez <ralonsoh@redhat.com>
Change-Id: I575ddf0a66e2cfe745af3874728809cf54e37745
The ovs agent will install some basic drop flows first for the
physical bridge mappings during the init procedure. If message
queue is not connected, or neutron-servers are all down, real
traffic flows will not be refreshed anymore. This will cause
the data plane down if tenant network and provider network are
sharing the physical NICs.
This patch adds a RPC check during init L2 agent. When restart
the ovs-agent, if the MQ is OK and we have available neutron-server,
go next step. Otherwise, a rpc timeout will be raised. L2 agent
will start fail, physical bridge mapping drop flows will not be
installed. The original flows will not be replaced, so the traffic
can still work properly.
Closes-Bug: #1803919
Change-Id: Ie15cf625b3710eaf290d6aafecb3f65df664b9df
This patch implements an OVS bridge monitor based in the OVSDB
native implementation (OVSDB IDL, Open vSwitch Database Interface
Definition Language). This new implementation supersedes the CLI
OVSDB monitor.
Partial-Bug: #1789592
Change-Id: I9c512d4cbd4cebf94c339231f83bbe89b37650ba
This incorporates flake8 2.6.x and pycodestyle will be used
instead of older pep8. This ensures future python3 compatibility
and a bit better code styling.
Change-Id: Ia7c7c5a44727f615a151e1e68dd94c7ed42f974f
Adding ability to set DSCP field in OVS tunnels outer header, or
inherit it from the inner header's DSCP value for OVS and linuxbridge.
Change-Id: Ia59753ded73cd23019605668e60cfbc8841e803d
Closes-Bug: #1692951
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.
NeutronLibImpact
Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
Otherwise we don't see some of them for the agent, for example,
AGENT.root_helper is missing.
To make sure the logging is as early as possible, and to make sure that
options that may be registered by extensions are also logged, some
refactoring was applied to the code to move the extension manager
loading as early as possible, even before agent's __init__ is called.
Related-Bug: #1718767
Change-Id: I823150cf6406f709d1e4ffa74897d598e80f5329
Replace the calls to the OVSPluginAPI info retrieval functions
with reads directly from the push notification cache.
Since we now depend on the cache for the source of truth, the
'port_update'/'port_delete'/'network_update' handlers are configured
to be called whenever the cache receives a corresponding resource update.
The OVS agent will no longer subscribe to topic notifications for ports
or networks from the legacy notification API.
Partially-Implements: blueprint push-notifications
Change-Id: Ib2234ec1f5d328649c6bb1c3fe07799d3e351f48
idl_factory was removed in favor of just passing in an Idl instance
as an Idl doesn't start a connection until its .run() is called.
The try/excepts will be removed when the ovsdbapp 0.4.0 constraint
changes are merged.
Change-Id: Id22faa1f6179c2fdf8a136972d65f10749c9fc2e
The other_config columns in OVSDB are defined as maps with string
keys and string values. The OVS agent was passing an integer
segmentation id and could pass None as the physical_network.
Unfortunately, the upstream Python OVS library does not pass the
exceptions through to us.
Change-Id: Iafa6be3749b1ee863f5fa71150c708fc46951510
Closes-Bug: #1630920
The use_veth_interconnection config doesn't work fine because
IPDevice is passed into OVSBridge's add_port() although the method
expects port_name. This patch fixes the wrong argument.
Change-Id: I6ea3e37d857f34228c41118709b91f4407555a33
Closes-Bug: #1622850
Physical bridges can cause network disruption when ofctl controller becomes
inaccessible due to heavy load or when the traffic to controller is blocked.
By setting secure fail mode, the openflow rules remain untouched on such
an event, while with the default setting, the flows are cleared.
Co-Authored-By: Jakub Libosvar <libosvar@redhat.com>
Closes-Bug: 1607787
Change-Id: I1dffe0a248664d2a675fd1ca58530c233e335d2d
UpgradeImpact
Previously, local mapping from tunnel ids or vlan ids to internal vlans
was held in ovs agent itself not exposing this mapping outside. This
patch itroduces a singleton object in memory providing needed interface
for handling local vlan mappings.
Partially-implements: blueprint vlan-aware-vms
Partially-implements: blueprint l2-api-extensions
Change-Id: I514c7632c1c26d6cfeb706fc5d829a46dcce3782
get_vif_ports returns ports with INVALID and UNASSIGNED
ofports and get_vif_port_set does not. The main scan_ports
loop uses the latter so any INVALID ofports (i.e. ofport == -1)
will be treated as removed and have their local VLANs reclaimed.
So an INVALID ofport could have the same local VLAN as a new
port that was added after it had been reclaimed.
This was causing an error in the _restore_local_vlan_map function
since it was using get_vif_ports which would cause it to process
INVALID ports as well so it could get two network UUIDs using the
same VLAN.
This fixes it by skipping INVALID and UNASSIGNED ofports in the
vlan restoration so it matches the behavior of scan_ports
(which is responsible for deciding which ports are added/removed
for VLAN allocation).
Closes-Bug: #1526974
Change-Id: I9d722fa4fabd467ded44d9cd291a3fa4d1af90f6
- unit tests were fixed mainly by mocking
Connection class of native implementation.
- some ovs-lib tests rely on direct ovs-vsctl
output. Temporarily decorated with @vsctl_only.
UpgradeImpact
Change-Id: I2632b0e21edd61536867a9fc830a45d9899091e4
When agent starts up, it checks whether patch ports exists
before adding them. But the routine used to query the
patch port's existence is get_port_ofport() which retries
the opertation because of the @_ofport_retry decoration.
This creates an unwanted delay in the startup of the
agent, when the port do not exist.
The port's existence can be checked with port_exists()
call on the bridge with no retries.
Change-Id: I9fac0066d6c03491536a6e2718d6340acd275d9d
Closes-Bug: #1579769
When starting up, we don't want to delete the patch port between br-int
and the physical bridges. In liberty the br-int bridge was changed to
not tear down flows on startup, and change
I9801b76829021c9a0e6358982e1136637634a521 will change the physical
bridges to not tear down flows also.
Without this patch the patch port is torn down and not reinstalled until
after the initial flows are set back up.
Partial-Bug: #1514056
Change-Id: I05bf5105a6f3acf6a313ce6799648a095cf8ec96
Perform deletion of the stale flows in physical bridges consistently with
br-int and br-tun, respecting drop_flows_on_start configuration option.
Added tests for auxiliary bridge and functional tests for the physical
bridge using VLAN/flat external network. Fixes part of the bug 1514056;
together with [1] and [2], the bug should be considered fixed.
The commit also fixes inconsistency between netmask of allocated IP
addresses assigned in _create_test_port_dict and ip_len in _plug_ports
of base.py.
[1] https://review.openstack.org/#/c/297211/
[2] https://review.openstack.org/#/c/297818/
Co-Authored-By: Jian Wen <wenjianhn@gmail.com>
Partial-Bug: 1514056
Change-Id: I9801b76829021c9a0e6358982e1136637634a521
On agent start, we check the interface type to cleanup ports of
wrong type if any. We should not log error on not finding db entry
for interface type, because the interface and hence the db entry
may not exist yet.
Change-Id: Ie619a7fd141fbaa92d39b73f77e5c8c1efc8ec48
Closes-Bug: #1545058
When starting up, we don't want to delete the patch port between br-int
and br-tun unless we're also dropping the flows.. In liberty both of
these bridges were switched to not dump flows on startup and to put the
bridges in secure mode so that default flood flows are not installed
when the bridge is created.
Without this patch the patch port is torn down and not reinstalled until
br-tun is setup again.
Partial-Bug: #1514056
Change-Id: Ia518a99a2de5d1bda467fde57892c43970f88bcd
OVSDB implementation refuses to set options:peer column value
as there is no such column in the Interface table. The correct way
is to set 'options' column value to a map containing key 'peer', as
already used in ovs_lib.
Change-Id: Ib5e956f425b36f54cda017c91ac71d9d7ee9747c
Closes-Bug: 1528894
This patch makes sure that existing connection breaks once security
group rule that allowed such connection is removed. Due to correctly
track connections on the same hypervisor, zones were changed from
per-port to per-network (based on port's vlan tag). This information is
now stored in register 6. Also there was added a test for RELATED
connections to avoid marking such connection as invalid by REPLY rules.
Closes-Bug: 1549370
Change-Id: Ibb5942a980ddd8f2dd7ac328e9559a80c05789bb
Lucas Alvares Gomes