This patch adds support for QoS minimum bandwidth rules in tunnelled
networks. Now the ML2/OVS and ML2/OVN mechanism drivers can represent
in the Placement API the available bandwidth of the tunnelled networks
in each compute host.
Both mechanism drivers represent the compute VTEP (VXLAN) or TEP
(Geneve) interface as an IP address. This new resource provider
(by default called "rp_tunnelled") represents the available bandwidth
of this interface. Any new port created in a compute node that belongs
to a tunnelled network, will request to the Placement API the
corresponding bandwidth from the resource provider inventory.
This patch does not provide backend enforcement support for minimum
bandwidth rules.
RFE spec: https://review.opendev.org/c/openstack/neutron-specs/+/860859
What is missing and will be added in next patches:
* Tempest tests, that will be pushed to the corresponding repository.
Depends-On: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/863880
Partial-Bug: #1991965
Related-Bug: #1578989
Change-Id: I3bfc2c0f9566bcc6861ca91339e32257ea92c7e9
This patch also some helper methods used in the
minimum bandwidth qos methods as it seems that we had things almost
duplicated in methods like _find/update/delete_{qos,queue} and
find/update/delete_{qos,queue}.
It also moves functional tests for the ingress bandwidth limit rules
methods to more appropriate module.
Related-bug: 1959567
Change-Id: I848af01c8fe3a08b26d05e37d225c944ea080f03
https://review.opendev.org/c/openstack/neutron/+/820897 added
a dead vlan flow that pushes the dead vlan tag onto frames
belonging to dead ports before these ports are reassigned to
their proper vlans. However add_flow and delete_flows race and
delete_flows may run before add_flow, in this case deleting 0 flows
but not giving us a chance to detect this: neither does it throw
an error nor does it return the number of deleted flows.
This leads to port staying inaccessible forever and hence
breaks corresponding DHCP or router.
Current patch suggests another approach to make sure no packets are
leaked from newly plugged ports: setting their "vlan_mode" attribute
to "trunk" and "trunks"=[4095] (along with assigning dead VLAN tag).
With this OVS normal pipeline will allow only packets tagged with 4095
from such ports [1], which normally not happens, but even if it does -
default rule in br-int will drop them anyway.
Thus untagged packets from such ports will also be dropped until
ovs agent sets proper VLAN tag and clears vlan_mode to default
("access").
This approach avoids the race between dhcp/l3 and ovs agents because
dhcp/l3 agents no longer modify flow table.
This partially reverts commit 7aae31c9f9
[1] https://docs.openvswitch.org/en/latest/ref/ovs-actions.7/?highlight=ovs-actions#the-ovs-normal-pipeline
Closes-Bug: #1930414
Closes-Bug: #1959564
Change-Id: I0391dd24224f8656a09ddb002e7dae8783ba37a4
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This completes removal of all references to "import mock",
changing to "from unittest import mock" in fullstack and
functional tests.
Added a hacking check to enforce it in future patches.
Change-Id: Ifcaf1c21bea0ec3c35278e49cecc90a101a82113
Remove this duplicated option and rely only in OVS.integration_bridge.
NOTE: other projects are still using it; first we need to deprecate it
in those projects.
Change-Id: I4e826c8b9fa764b1820adacc3427934dc393c0bc
Related-Bug: #1856152
Currently the ovs agent calls update_device_list with the
agent_restarted flag set only on the first loop iteration. Then the
server knows to send the l2pop flooding entries for the network to
the agent. But when a compute node with many instances on many
networks reboots, it takes time to readd all the active devices and
some may be readded after the first loop iteration. Then the server
can fail to send the flooding entries which means there will be no
flood_to_tuns flow and broadcasts like dhcp will fail.
This patch fixes that by renaming the agent_restarted flag to
refresh_tunnels and setting it if the agent has not received the
flooding entries for the network.
Change-Id: I607aa8fa399e72b037fd068ad4f02b6210e57e91
Closes-Bug: #1853613
Neutron-ovs-agent can now enable IGMP snooping in integration bridge
if config option "igmp_snooping_enable" in OVS section in config will
be set to True.
It will also set mcast-snooping-disable-flood-unregistered=true
so flooding of multicast packets to all unregistered ports will be
disabled also.
Both changes are applied on integration bridge.
Change-Id: I12f4030a35d10d1715d3b4bfb3ed5efb9aa28f2b
Closes-Bug: #1840136
This patch ensures that the ovsdb monitor propagates the events
received for modified ports.
We'll use a new list for the modified ports, which the neutron ovs
agent can handle.
In particular, this will cover the situation in which the ofport
changes. When using recent OVS Windows versions, VM ofports change
to -1 (invalid) when the VMs are shut down, receiving a valid
ofport when the VMs are powered back on (different than the initial
one). With this patch applied, "modify" events will be propagated
to the ovs agent, which will then update the OpenFlow rules.
The old rules are cleaned up by "update_stale_ofport_rules"
once the invalid ofport is detected.
Closes-Bug: #1843870
Co-authored-by: Alin Serdean <aserdean@cloudbasesolutions.com>
Change-Id: I0c3a570cbb3fbb03b4224744b32e034e9e255f8e
Ovs-agent will scan and process the ports during the
first rpc_loop, and a local port update notification
will be sent out. This will cause these ports to
be processed again in the ovs-agent next (second)
rpc_loop.
This patch passes the restart flag (iteration num 0)
to the local port_update call trace. After this patch,
the local port_update notification will be ignored in
the first RPC loop.
Related-Bug: #1813703
Change-Id: Ic5bf718cfd056f805741892a91a8d45f7a6e0db3
Default value for "of_interface" config option was switched
to "native" in Pike release.
In the same release this option was deprecated to removal.
Now it's time to remove it and force use of "native" driver to
manage openflows.
Change-Id: Ic900209868acfbe3bbb56fabbbf5c4472857e412
Co-Authored-By: Ihar Hrachyshka <ihrachys@redhat.com>
Co-Authored-By: Slawek Kaplonski <skaplons@redhat.com>
Ovs-agent can be very time-consuming in handling a large number
of ports. At this point, the ovs-agent status report may have
exceeded the set timeout value. Some flows updating operations
will not be triggerred. This results in flows loss during agent
restart, especially for hosts to hosts of vxlan tunnel flow.
This fix will let the ovs-agent explicitly, in the first rpc loop,
indicate that the status is restarted. Then l2pop will be required
to update fdb entries.
Closes-Bug: #1813703
Closes-Bug: #1813714
Closes-Bug: #1813715
Closes-Bug: #1794991
Closes-Bug: #1799178
Change-Id: I8edc2deb509216add1fb21e1893f1c17dda80961
Sometimes, when the OVSDB is too loaded (that could happen during the
functional tests), there is a delay between the OVSDB post transaction
end and when the register (new or updated) can be read. Although this is
something that should not happen (considering the OVSDB is transactional),
tests should deal with this inconvenience and provide a robust method to
retrieve a value and at the same time check the value. This new method
should provide a retrieving mechanism to read again the value in case of
discordance.
In order to solve the gate problem ASAP, another bug is fixed in this
patch: to skip the QoS removal when OVS agent is initialized during
funtional tests
When executing functional tests, several OVS QoS policies specific for
minimum bandwidth rules [1]. Because during the functional tests
execution several threads can create more than one minimum bandwidth
QoS policy (something in a production environment cannot happen), the
OVS QoS driver must skip the execution of [2] to avoid removing other
QoS created in parellel in other tests.
This patch is marking as unstable "test_min_bw_qos_policy_rule_lifecycle"
and "test_bw_limit_qos_port_removed". Those tests will be investigated
once the CI gates are stable.
[1] Those QoS policies are created only to hold minimum bandwidth rules.
Those policies are marked with:
external_ids: {'_type'='minimum_bandwidth'}
[2] d6fba30781/neutron/plugins/ml2/drivers/openvswitch/agent/extension_drivers/qos_driver.py (L43)
Closes-Bug: #1818613
Closes-Bug: #1818859
Related-Bug: #1819125
Change-Id: Ia725cc1b36bc3630d2891f86f76b13c16f6cc37c
Now, that the ovsdb monitor is also available on Windows, we can
use it on Windows as well, minimizing polling.
We're simply moving the bits to the common polling module.
Change-Id: Ia12ad970085f2cff8c8ac5b7d3d69e7dc9214c40
Agent OVS interface code adds ports without a vlan tag,
if neutron-openvswitch-agent fails to set the tag, or takes
too long, the port will be a trunk port, receiving
traffic from the external network or any other port
sending traffic on br-int.
Also, those kinds of ports are triggering a code path
on the ovs-vswitchd revalidator thread which can eventually
hog the CPU of the host (that's a bug under investigation [1])
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1558336
Co-Authored-By: Slawek Kaplonski <skaplons@redhat.com>
Change-Id: I024bbbdf7059835b2f23c264b48478c71633a43c
Closes-Bug: 1767422
Adding ability to set DSCP field in OVS tunnels outer header, or
inherit it from the inner header's DSCP value for OVS and linuxbridge.
Change-Id: Ia59753ded73cd23019605668e60cfbc8841e803d
Closes-Bug: #1692951
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.
NeutronLibImpact
Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
Otherwise we don't see some of them for the agent, for example,
AGENT.root_helper is missing.
To make sure the logging is as early as possible, and to make sure that
options that may be registered by extensions are also logged, some
refactoring was applied to the code to move the extension manager
loading as early as possible, even before agent's __init__ is called.
Related-Bug: #1718767
Change-Id: I823150cf6406f709d1e4ffa74897d598e80f5329
Refactoring neutron agent linux and ovsdb config opts
to be in neutron/conf/agent so that all the config options
reside in a centralized location. This simplifies the
process of looking up the config opts and provides an easy
way to import.
NeutronLibImpact
Change-Id: Ib1e0e63dec2985c417412d1ecc68e2a74ef87182
Partial-Bug: #1563069
This was deprecated over a year ago in [1] so let's
get rid of it to clean up some code.
1. Ib63ba8ae7050465a0786ea3d50c65f413f4ebe38
Change-Id: I6039fb7e743c5d9a1a313e3c174ada36c9874c70
Refactoring Neutron configuration options for agent common config to be
in neutron/conf/agent/common. This will allow centralization of all
configuration options and provide an easy way to import.
Partial-Bug: #1563069
Change-Id: Iebac0cdd3bcfd0135349128921b7ad7a1a939ab8
Needed-By: Ib676003bbe909b5a9013a3178b12dbe291d936af
Neutron-lib 1.1.0 is now out and contains the get_random_mac
definition[1]. This patch moves neutron references over to
the neutron-lib version.
NeutronLibImpact
[1] ee0f5b2ab27c828cfedb771735d237a968423da2
Change-Id: I28a2a1d85a85461f7a4344b86d18da7f68066c95
This reverts commit 75edc1ff28.
Ports with port security disabled require firewall entries in
neutron-openvswi-FORWARD chain to work properly.
Ports created with no security groups will not get skipped with current
code.
With fixed security groups check, these ports' security groups can not
be updated after creation.
Change-Id: I95ddbe38d8ac8a927a860a98f54e41e17fb71d43
Closes-Bug: #1549443
Refactoring ml2 plugin openvswitch driver configuration options to be
in neutron/conf/plugins/ml2/drivers. This would allow centralization
of all configuration options and provides an easy way to import.
Change-Id: Ie8c6023b2d012eae7ecdb99d5d413956608f4294
Partial-Bug: #1563069
Upon restart the agent reconciles the logical with the physical
state by removing/adding physical subports that are used to
be/are current present in the logical view.
This patch adds a functional test to demonstrate that there's
no need to handle the resync in the trunk driver, since the ovs
agent already takes care of it.
Change-Id: I164153c79313f2ae7a1fca0414736d5987656185
Partially-implements: blueprint vlan-aware-vms
Partial-bug: #1623708
neutron-sanity-check tool was importing neutron.tests.base module, which
may be not present on some systems (f.e. RDO splits neutron/tests/
subtree in a separate python-neutron-tests package). It made the tool
not usable in some setups.
https://bugzilla.redhat.com/show_bug.cgi?id=1374282
This is not the first time when we by mistake import from
neutron.tests.* and break distributions. It's time to stop it by
proactively forbidding that pattern via a new hacking check.
Some functions were moved from neutron.tests.base to
neutron.common.utils to fulfill the need requirement. They were moved
using debtcollector, no current consumers should be affected.
Closes-Bug: #1621782
Change-Id: I790777ddcbd1b02218b3db54ae3d5c931d72d4fa
Refactoring neutron common config opts to be in neutron/conf/common so
that all the configuration options reside in a centralized location.
This simplifies the process of looking up the config opts and provides
an easy way to import.
Moved conf/common/config.py to conf/common.py as per review comments.
Partial-Bug: #1563069
Change-Id: Ib5fa294906549237630f87b9c848eebe0644088c
This change generalizes agent extension code so that all agents can take
advantage of a common mechanism.
Co-Authored-By: Margaret Frances <margaret_frances@cable.comcast.com>
Partially-Implements: blueprint l3-agent-extensions
Change-Id: I9380343c09d28eec67077c9e6d77c33a195e516b
We need to be able to re-use wait_until_true in tempest scenario tests.
There is tempest bug https://bugs.launchpad.net/tempest/+bug/1592345
that prevents us to do so.
Also wait_until_true is not linux specific so it makes more sense to
have it in common package.
Change-Id: Ib8b0e51dbd9edaa58391774d428a737836dfdf77
Remove IPv4 restriction for local_ip configuration statement.
Check for IP version mismatch of local_ip and remote_ip before creating
tunnel.
Create hash of remote IPv6 address for OVS interface/port name with least
posibility for collissions.
Fix existing tests that fail because of the added check for IP version
and subsequently valid IP addresses in _setup_tunnel_port.
DocImpact
Change-Id: I9ec137ef8c688b678a0c61f07e9a01382acbeb13
Closes-Bug: #1525895
Perform deletion of the stale flows in physical bridges consistently with
br-int and br-tun, respecting drop_flows_on_start configuration option.
Added tests for auxiliary bridge and functional tests for the physical
bridge using VLAN/flat external network. Fixes part of the bug 1514056;
together with [1] and [2], the bug should be considered fixed.
The commit also fixes inconsistency between netmask of allocated IP
addresses assigned in _create_test_port_dict and ip_len in _plug_ports
of base.py.
[1] https://review.openstack.org/#/c/297211/
[2] https://review.openstack.org/#/c/297818/
Co-Authored-By: Jian Wen <wenjianhn@gmail.com>
Partial-Bug: 1514056
Change-Id: I9801b76829021c9a0e6358982e1136637634a521
With the new RPC calls get_devices_details_and_failed_devices
and update_device_list the agent gets a list of devices
for which some operation failed. The agent can now make use
of this information and instead of syncing all the devices
can sync only those which failed.
With the current change if a device keeps failing, the agent
will try to sync it forever. In a following patch I will limit
the number of retrials.
Partially-Implements: blueprint restructure-l2-agent
Change-Id: I295dc79031a0547f8687c5835c7ba7bbc43df36d
When changing datapath_type in the config, physical and tunnel bridges
do not have their datapath_type updated. Calling create() on already
created bridges should be safe as it passes '--may-exist' when adding
the bridge, which will do nothing if the bridge already exists, but
the second part of the transaction will still update things like
datapath_type.
It should be noted that ancillary bridges (like br-ex) are not
modified by this patch as datapath_type was never applied to them to
begin with.
Incidentally, the native and vsctl versions behaved slightly
differently when handling datapath_type: vsctl builds the multi-cmd
transaction with add-br ... -- set ..., so that the second cmd would
actually complete. The native just bailed if may_exist and the bridge
existed. This is fixed as part of this patch.
Change-Id: Ib8bc817c7bc724d80193d0ca7af480a7ea103f77
Closes-Bug: 1532273
The original change had to be reverted because it caused
tests failing in the gate. The failures were due to the
fact that when a port was not ready, an exception was
thrown to trigger a full resync of the agent. This
behavior was meant to be temporary and was fixed in a
dependent patch that was not merged though. This revert
moves the handling of not ready ports in this patch.
It also refactors the unit tests a bit.
This reverts commit e7270d9505.
Change-Id: I6574cef3c95525ace6a98cf968ee159190681394
Removed create_agent_config_map method which creates specific
configurations dictionary according to cfg.CONF and call the OVS-agent
with that structure. Passing oslo_config directly to init-method
of OVS-agent is more straightforward and makes it more testable.
Also refactored unit and functional tests of OVS-agent in accordance
with the changes in init-method.
Closes-bug: #1464394
Change-Id: I88742e4d454709e35481f2c505c9d64995497cac
This might be associated to manifestation of bug #1514935
This reverts commit 1992d52d63.
Closes-Bug: #1514935
Change-Id: If01cc87b6735e1bc039f99c4c6121e7c5ce547d0
OVSDB monitor generates the events that the OVS agent
needs to process (device added or updated). Instead of
polling the agent processes the queue of events.
Change-Id: I168a3cc3aa96a809153a30635ad7bda29e8ee47c
Partially-Implements: blueprint restructure-l2-agent
This change will not force a resync in the case where a virtual machine is
deleted, and therefore its OVS port deleted, in between the time an RPC
call was made to get the devices and where we make the call to correlate
those devices to vif ports.
Change-Id: Ie55eb69ad7ee177f0cf8ee8fc7fc585fbd0d4a22
Closes-Bug: #1499488
When agent is restarted it drops all existing flows. This
breaks all networking until the flows are re-created.
This change adds an ability to drop only old flows.
Agent_uuid_stamp is added for agents. This agent_uuid_stamp is set as
cookie for flows and then flows with stale cookies are deleted during
cleanup.
Co-Authored-By: Ann Kamyshnikova<akamyshnikova@mirantis.com>
Closes-bug: #1383674
DocImpact
Change-Id: I95070d8218859d4fff1d572c1792cdf6019dd7ea
We can skip trying to setup firewall filters for ports which are
having port_security_enabled as False or which are not associated
to any security group.
Closes-Bug: #1482554
Change-Id: Ie65201308d93c746fe4ef38f402ec300227b7d27
Rodolfo Alonso Hernandez