The semantics changed since I2e3aa7c400d7bb17cc117b65faaa160b41013dde
but the code was not updated to reflect it explicitly.
This patch modifies _update_flows_for_vlan_subr.
Change-Id: Id5e0d8bcc379f19dc85b23b4602be4e0e82f3d00
After updating pylint, it started emitting additional "R"
warnings in some cases, fix some of them.
use-a-generator,
unnecessary-lambda-assignment,
consider-using-max-builtin,
consider-using-generator,
consider-using-in,
use-list-literal,
consider-using-from-import
Trivialfix
Change-Id: Ife6565cefcc30b4e8a0df9121c9454cf744225df
Parse and validate the "permitted_ethertypes" configuration variable at
the OVS firewall initialization. Any error in the configuration will be
logged only once.
Closes-Bug: #2009728
Change-Id: Ib5439791d7bc0a8457bb10f887b1077fd6438619
This patch is a partial revert of [1], reinstantiating the code merged
in [2]. This patch is the complementary to [1]: the traffic with
custom ethertypes is allowed in the ingress processing tables, same
as [1] is allowing all traffic from the virtual machine ports in this
host to leave the node. Both, this patch and [1], are bypassing the
OVS firewall just for the traffic with the configured allowed
ethertypes and just for/to the local ports and MAC addresses.
Any other traffic not coming from a local port or with destination
a local port, will be blocked as is now.
[1]https://review.opendev.org/c/openstack/neutron/+/678021
[2]https://review.opendev.org/c/openstack/neutron/+/668224/
Closes-Bug: #2009221
Related-Bug: #1832758
Change-Id: Ib8340d9430b946a446edf80886c49fbac729073c
When vm live-migration is done, at the vm start moment, the
vm will send rarp packet, but the table 71 of openflow will
drop these packets, because they cannot match any flow in
high priority(65 70 80 95), only the priority 10 flow can
match them, and its aciton is drop, so all the rarp packets
will be dropped, and the connectivity will be broken after
live-migration.
Change-Id: Id1cc34f0f544b2cb8cc89af2cb81bb15e4605a0e
Partial-Bug: #1831404
Depends-On: https://review.openstack.org/661938
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/agent and neutron/privileged/agent.
Trivialfix
Change-Id: Ib94257481e62b99b3d7648ae5137af5411b4867a
Reduces the deletion time of conjunction flows on hypervisors
where virtual machines reside which are part of a security
group that has remote security groups as target which contain
thousands of ports.
Without deferred deletion the agent will call ovs-ofctl several
hundred times in succession, during this time the agent will
block any new vm creation or neutron port modifications on this
hypervisor.
This patch has been tested using a single network with a single
vm with a security group that points to a remote security group
with 2000 ports.
During testing without the patch, the iteration time for deletion
was at around 500 seconds. After adding the patch to the l2 agent
on the test environment the same deletion time went down to
4 seconds.
Closes-Bug: #1975674
Change-Id: I46b1fe94b2e358f7f4b2cd4943a74ebaf84f51b8
This patch adds a new configuration variable to control the OVS
OpenFlow rule processing operations:
* ``openflow_processed_per_port``: by default "False". If enabled,
all OpenFlow rules associated to a port will be processed at once,
in one single transaction. If disabled, the flows will be processed
in batches of "AGENT_RES_PROCESSING_STEP=100" number of OpenFlow
rules.
With ``openflow_processed_per_port`` enabled, all Firewall
OpenFlow rules related to a port are processed in one transaction
(executed in one single command). That ensures the rules are written
atomically and apply all of them at the same time.
That means all needed rules to handle the ingress and egress traffic
of a port using the Open vSwitch Firewall, are committed in the OVS
DB at the same time. That will prevent from partially applied OpenFlow
sets in the Firewall and inconsistencies when applying new SG rules or
during the OVS agent restart.
That will override, if needed, the hard limit of
"AGENT_RES_PROCESSING_STEP=100" OpenFlow rules that could be
processed in OVS at once.
If the default configuration values are not modified, the behaviour of
the OVS library does not change.
Closes-Bug: #1934917
Change-Id: If4984dece266a789d607725f8497f1aac3d73d23
"ICMPV6_TYPE_RA" was removed from "ICMPV6_ALLOWED_INGRESS_TYPES"
because of a bug in the iptables firewall (described in the LP
bug). This rule was added in "_add_ingress_ra_rule" to
port["security_group_rules"]. However, the OVS firewall does not
use this rule list but builds the default rules from scratch.
Closes-Bug: #1958643
Change-Id: I53ee3c87ab2a6306b31fc3387b706d8296031a14
There are two types of OF rules in TRANSIENT_TABLE:
- With priority 100: these rules match by "in_port", that is a
unique identifier.
- With priority 90: these rules match by MAC address and VLAN ID.
This combination (MAC, VLAN) is unique.
That means when a deleting an OF rule in TRANSIENT_TABLE, it is
enough to specify the "in_port" or the (MAC, VLAN) tuple. The
"--strict" parameter, added to also define the priority, is not
needed.
By removing the "--strict" parameter, these deletion commands can
be executed synchronously at the end of the OVS deferred context,
when all the OF rule commands (addition or deletion), are executed
at the same time. That removes the small window, detected in the
related bug, when the OF rule set for a port is not complete.
Closes-Bug: #1952770
Change-Id: I9f5bd8a1404dde3a0aa163ce72aef2961f537676
The OVS Firewall has a singleton class that manages the conjuction IDs
to used in the OF rules. Those conjuntions are used to group rules
related to remote security group IDs.
Now each time the OVS agent is started, the OVS Firewall initial
conjunction ID is calculated based on the present OF rules. This value
and the next one used won't clash with any present rule in the
integration bridge during the initial transient period.
Related-Bug: #1934917
Change-Id: Ie2e4441f766947a2164dec2d1555c7049428903f
Neighbor Advertisments are used to inform other machines of the MAC
address to use to reach an IPv6. This commits prevents VMs from
pretending they are assigned IPv6 they should not use.
It also prevents sending UDP packets with spoofed IP or MAC even using
DHCP(v6) request ports.
Co-authored-by: David Sinquin <david.sinquin@gandi.net>
Closes-bug: #1902917
Change-Id: Iffb6643359562487414460f5a7e19a7fae9f935c
Before that patch it was possible to send ICMPv6 packets like e.g.
neutron_lib.constants.ICMPV6_TYPE_MLD_QUERY,
neutron_lib.constants.ICMPV6_TYPE_RS,
neutron_lib.constants.ICMPV6_TYPE_NS,
neutron_lib.constants.ICMPV6_TYPE_NA
And that could cause some security issues as instance could advertise
that it owns IPv6 address which really don't belong to it.
Now rules in table=71 which allows that traffic are "per mac/ipaddress"
and are allowed only for fixed ips allocated to port and port's
allowed_address_pairs.
Closes-Bug: #1902917
Change-Id: I4749fdc6a6cabd253b971bf4010ff76f5593c59c
In case when security group is removed from port, connections
established to/from port which were hitting some of the removed SG
rules will be marked with CT_MARK_INVALID in the conntrack table.
Due to that such existing connections will be stopped.
But then if SG will be added again to the port, those conntrack entries
marked as invalid will not be removed and those hang connections will
not be revived.
To fix that, this patch adds deletion of the conntrack entries marked
with CT_MARK_INVALID and related to the port of which SG are
updated/added.
Closes-Bug: #1915530
Change-Id: I84718b8ac4d8d6e39426e1c4485c2d7fe28185dd
Support security group rules with remote_address_group_id in openvswitch
firewall. This change reuses most of the firewall functions handling remote
security groups to also process remote address groups. The conjunctive flows
for a rule with remote_adress_group_id are similar to others with
remote_group_id but have different conj_ids.
Change-Id: I8c69e62ba56b0d3204e9c12df3133126071b92f7
Implements: blueprint address-groups-in-sg-rules
Currently when deleting a remote-group's member IPs, the deleted IPs'
conjunctive flows are not cleaned up in OF tables. This is because
the conjunctive flows' cookies don't match with the OVSBridge default
cookie used by the delete flow method. This patch fixed the issue by
using an ANY cookie that can always match with the cookies of the
conjunctive flows.
Change-Id: I74916acf8311989dca267f23261ec4cf449a6abf
Closes-Bug: 1907491
This reverts commit b8be1a05fa.
As was reported in bug [1] this patch broke multicast traffic send
from ports with disabled port security. And that broke L3HA routers
as keepalived processes couldn't talk to each other.
During attempt to fix that issue with keepalived we found out another
corner cases which we may break and in fact to fix them, we would
effectively revert this change and allow multicast traffic for all
ports in e.g. networks with ports which have port security and ports
which don't have port security and are on same node.
As we also don't really know what other corner cases we may hit going
further with that, lets revert this patch.
As a follow up patch I will propose new patch which will document
differences in handling multicast traffic between iptables and
openvswitch based firewall drivers.
[1] https://bugs.launchpad.net/neutron/+bug/1899967
Change-Id: I37a8b33cf8e16d5bb5dc1966fc2dca6bb619026c
Closes-Bug: #1899967
In RULES_INGRESS_TABLE table 82 there is a rule for allow established and
related connections. The current rule sends the packet directly to the dest
port without doing a mac learning. This is causing ovs to age out the dest mac
of the remote VM and causing the rule to be changed in flood rule. For the normal
case it fine as they try to avoid high cpu. ovs hardware offload reduce cpu usage
by moving some of the packet processing to nic and flood rule is not offloaded,
therefore it prefre to use the NORMAL action to avoid the flood rule.
We also keep the same logic as today when using explicitly_egress_direct=True
which avoid NORMAL action in the entire pipeline.
Closes-Bug: #1897637
Change-Id: I9b611d62be5d0529e8b35e3d8280baa5be54bc2b
By default, if any multicast traffic sent to 224.0.0.X is allowed
in the OVS firewall (that means there is a specific egress rule),
this traffic is sent, in table 73 (ACCEPT_OR_INGRESS_TABLE), to
a rule with action NORMAL.
As commented in the related bug, https://tools.ietf.org/html/rfc4541,
chapter 2.1.2, section (2):
"Packets with a destination IP (DIP) address in the 224.0.0.X range
which are not IGMP must be forwarded on all ports."
That means those packets will be forwarded to all ports regardless of
any ingress rule. This patch process this traffic separately, sending
those packets to table 102 (MCAST_RULES_INGRESS_TABLE). In this table
the ingress rules that have a defined protocol, will have an Open Flow
rule to output the traffic directly to those ports associated to this
rule.
For example, in the problem reported in the related bug, the VRRP
protocol (112), will be sent only to those ports that have this
ingress rule.
Change-Id: Ie271de144f78e364d938731ec9f5297e1a9d73f9
Closes-Bug: #1889631
Since [1], "get_vif_port_by_id" is also returning ports with an
invalid ofport. OVS firewall cannot set an OpenFlow rule for a port
without a valid ofport. "get_ovs_port" should filter out those ports.
Related-Bug: #1815989
Related-Bug: #1734320
[1]https://review.opendev.org/#/c/640258/
Change-Id: Id12486b3127ab4ac8ad9ef2b3641da1b79a25a50
When any port in the OVS agent is using a security groups (SG) and
this SG is removed, is marked to be deleted. This deletion process
is done in [1].
The SG deletion process consists on removing any reference of this SG
from the firewall and the SG port map. The firewall removes this SG in
[2].
The information of a SG is stored in:
* ConjIPFlowManager.conj_id_map = ConjIdMap(). This class stores the
conjunction IDS (conj_ids) in a dictionary using the following keys:
ConjIdMap.id_map[(sg_id, remote_sg_id, direction, ethertype,
conj_ids)] = conj_id_XXX
* ConjIPFlowManager.conj_ids is a nested dictionary, built in the
following way:
self.conj_ids[vlan_tag][(direction, ethertype)][remote_sg_id] = \
set([conj_id_1, conj_id_2, ...])
This patch stores all conjuntion IDs generated and assigned to the
tuple (sg_id, remote_sg_id, direction, ethertype). When a SG is
removed, the deletion method will look for this SG in the new storage
variable created, ConjIdMap.id_map_group, and will mark all the
conjuntion IDs related to be removed. That will cleanup those rules
left in the OVS matching:
action=conjunction(conj_id, 1/2)
[1]118930f03d/neutron/agent/linux/openvswitch_firewall/firewall.py (L731)
[2]118930f03d/neutron/agent/linux/openvswitch_firewall/firewall.py (L399)
Change-Id: I63e446a30cf10e7bcd34a6f0d6ba1711301efcbe
Related-Bug: #1881157
Currently codes only support assocate tunnel network and vlan network
to DVR router. This patch add codes that make the flat network assocate
to DVR router and make it work fine.
The patch also remove two unused constant entries: 'FLAT_VLAN_ID' and
'LOCAL_VLAN_ID'
Change-Id: I7d792ce288d96548298f169748565266a130bd86
Closes-Bug: #1876092
Security group can have a state of empty ports but non-empty members. So
we need skip the flow update only when members dict is empty.
Change-Id: I429edb3d2dea5fa97441909b4d2c776f97f0516f
Closes-Bug: #1862703
Related-Bug: #1854131
Do not flood the packets to bridge, since we have the
bridge port list, we can add a simple direct flow to
the right port only.
Closes-Bug: #1732067
Related-Bug: #1841622
Change-Id: I14fefe289a19b718b247bf0740ca9bc47f8903f4
During e.g. migration or shelve of VM it may happend that
port update event will be send to the ovs agent and in the almost
the same time, port will be removed from br-int.
In such case during update_port_filter method openvswitch firewall
driver will not find port in br-int, and it will do nothing with it.
That will lead to leftover rules for this port in br-int.
So this patch adds calling remove_port_filter() method if port was
not found in br-int. Just to be sure that there is no any leftovers
from the port in br-int anymore.
Change-Id: I06036ce5fe15d91aa440dc340a70dd27ae078c53
Closes-Bug: #1850557
Only check sg object is not enough, we should also
check sg'ports is {} or not. Otherwise the old conjunction
will still exist.
Change-Id: I10588e73a9da7fdd43677f9247c176811dd68c62
Closes-Bug: #1854131
This will prevent ovs agent from endless fail loop when dealing
with unbound port: like when port was created in neutron before
agent become alive, then agent gets online and and starts processing
devices.
This patch adds exception handling to prepare_port_filter() -
same as done in update_port_filter().
Change-Id: I1137eb18efaf51c67fab145e645f58cbd3772e40
Closes-Bug: #1849098
In order to assist debugging of OVS flows involving conjunctions, log
the conjunction ID and other pertinent details. Without this, there is
no good way to verify the port was added to the correct conjunction.
Change-Id: Ie9c3eaa9c828ef5a0a68a286bc0465f2bcd00a4f
For vlan type network, we add a segment match flow
to the openflow security group ingress table. Then
the packets will be recorded in conntrack table, and
the reply packets can be processed properly.
Change-Id: Ieded0654d0ad16235ec923b822dcd842bd7735e5
Closes-Bug: #1831534
Some traffic does not work if the OVS flows to permit custom ethertypes
are not set on the base egress table. If the rule is added to the base
egress table then both ingress and egress work properly. Also move
initialization code to the function to initialize egress.
Related-Bug: #1832758
Change-Id: Ia312fe75df58723bf41804eec4bd918d223bd60c
FirewallDriver.process_trusted_ports" is called with many ports,
"_initialize_egress_no_port_security" retrieves the VIF ports
("Interface" registers in OVS DB), one per iteration, based in the
port_id. Instead of this procedure, if the DB is called only once to
retrieve all the VIF ports, the performance increase is noticeable.
E.g.: bridge with 1000 ports and interfaces.
Retrieving 100 ports:
- Bulk operation: 0.08 secs
- Loop operation: 5.6 secs
Retrieving 1000 ports:
- Bulk operation: 0.08 secs
- Loop operation: 59 secs
Closes-Bug: #1836095
Related-Bug: #1836023
Change-Id: I5b259717c0fdb8991f1df86b1ef4fb8ad0f18e70
process_trusted_ports() appeared to be greenthread unfriendly, so
if there are many trusted ports on a node, openvswitch agent may
"hang" for a significant time.
This patch adds explicit yield.
Change-Id: I7c00812f877e2fc966bbac3060e1187ce1b809ca
Closes-Bug: #1836023
The OVS Firewall blocks traffic that does not have either the IPv4 or
IPv6 ethertypes at present. This is a behavior change compared to the
iptables_hybrid firewall, which only operates on IP packets and thus
does not address other ethertypes.
This is a lightweight change that sets a configuration option in the
neutron openvswitch agent configuration file for permitted ethertypes
and then ensures that the requested ethertypes are permitted on
initialization. This addresses the security and usability concerns on
both master and stable branches while a full-fledged extension to the
security groups API is considered.
Change-Id: Ide78b0b90cf6d6069ce3787fc60766be52062da0
Related-Bug: #1832758
The current code will remove the port from sg_port_map, but then it
won't be added into the map, when we resize/migrate this instance,
the related openflow won't be deleted, this will cause vm connectivity
problem.
Closes-Bug: #1825295
Change-Id: I94ddddda3c1960d43893c7a367a81279d429e469
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
This patch switches the code over to the payload style of callbacks [1]
for AGENT OVS_RESTARTED events. In addition it updates the unit tests to
also use payloads for OVS_RESTARTED based events as needed.
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html
Change-Id: I7b774735fb875a808b048a17da1e0487e5bbcb44
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
In neutron.agent.linux.openvswitch_firewall.firewall make the method
update_port_filter catch OVSFWTagNotFound and log it to avoid
traceback in log files.
Change-Id: I584d867f0e1c47447cb8790fd715fa01ec902438
Closes-Bug: #1811405
When ovs-vswitchd process is restarted on host neutron-ovs-agent
can properly handle that and recover openflows and bridges config.
But when ovs firewall driver is used, it wasn't reinitialized
so there were missing some OF rules in br-int and connectivity
to/from instances was broken.
This patch adds reinitialization of firewall in such case so
all OF rules are properly reconfigured and connectivity is
restored.
Change-Id: I8d525bbe48216fef4da890582b8e198933a27833
Closes-Bug: 1805808
Ihar Hrachyshka