Default SG rules created as template in the Neutron DB are now used to
create security group rules for each new default and non-default SG
created in Neutron.
Closes-bug: #1983053
Change-Id: Iaf27deb955c3844409fcd36239511478e9607a82
When deployed with k8s, k8s service types like NodePort
or ExternalIP will affect vm traffic on nat table's
PREROUTING chain. This PS try to mitigate the effect
by allowing vm traffic to bypass those rules.
Change-Id: Iae12d9c2f37bc0fca9c3d5e85e46c642263e4a77
Closes-Bug: #1908957
Removed unneeded database context in
``_create_port_security_group_binding``. This method is called always
from inside an active database transaction, when the port is udpated
or created.
The retry context only makes sense when a method is called outside
an active database transaction. It doesn't retry the command if the
context passed has an active transaction.
Trivial-Fix
Change-Id: I8f84c1bce0d1ce2538789e3511fd8f7b64fcd41a
Support security group rules with remote_address_group_id in openvswitch
firewall. This change reuses most of the firewall functions handling remote
security groups to also process remote address groups. The conjunctive flows
for a rule with remote_adress_group_id are similar to others with
remote_group_id but have different conj_ids.
Change-Id: I8c69e62ba56b0d3204e9c12df3133126071b92f7
Implements: blueprint address-groups-in-sg-rules
Replace rootwrap execution with privsep context execution.
This series of patches will progressively replace any
rootwrap call.
This patch migrates any "iptables" and "ipset" command related
to privsep.
Change-Id: I4a1e137b2b414067504ad7c799d68f482bf3d36c
Story: #2007686
Task: #41558
A partial upgrading of neutron cluster, neutron-server
has a newer version while neutron-agents not, does not
run well after a RPC data structure upgrading. This
patch upgrades the security group related RPC version
between neutron-server and agents. A partial upgrading
neutron cluster will explicitly raise error. The RPC
version should be aligned.
Closes-bug: #1903531
Related-bug: #1867119
Change-Id: I6cb2ba05fa3337be46eb01f2d9f869efa41e4db6
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.
Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.
Unit tests only, removing from tests/base.py affects
functional tests which need additional work.
Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
When add allowed-address-pair 0.0.0.0/0 to one port, it will
unexpectedly open all others' protocol under same security
group. IPv6 has the same problem.
The root cause is the openflow rules calculation of the
security group, it will unexpectedly allow all IP(4&6)
traffic to get through.
For openvswitch openflow firewall, this patch adds a source
mac address match for the allowed-address-pair which has
prefix lenght 0, that means all ethernet packets from this
mac will be accepted. It exactly will meet the request of
accepting any IP address from the configured VM.
Test result shows that the remote security group and
allowed address pair works:
1. Port has 0.0.0.0/0 allowed-address-pair clould send any
IP (src) packet out.
2. Port has x.x.x.x/y allowed-address-pair could be accepted
for those VMs under same security group.
3. Ports under same network can reach each other (remote
security group).
4. Protocol port number could be accessed only when there
has related rule.
Closes-bug: #1867119
Change-Id: I2e3aa7c400d7bb17cc117b65faaa160b41013dde
This param is set in the functions, but actually no use. It
was never set to True. And it reduces the code readability.
So better remove it. From the code search [1], only neutron
and networking-vsphere have such global_refresh_firewall now.
[1] http://codesearch.openstack.org/?q=global_refresh_firewall
Change-Id: I87e1463d2cc1ee855f0cf6631863d89a6bd8ee2e
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
In one specific compute node, the security group rules
can be enormous quantity. This patch adds a step-by-step
processing method to deal with the large number of the
security group rules. And also changes or adds some LOG.
Related-Bug: #1813703
Related-Bug: #1813704
Related-Bug: #1813707
Change-Id: I57bf27ec75cf848271c5a28b22beee12b8bd5faa
The test tool class UnorderedList lives in neutron-lib.
This patch uses it from neutron-lib rather than neutron and removes
the class from neutron.
Depends-On: https://review.openstack.org/#/c/641662/
Change-Id: Ie92c2629b6e7d7af46287ef9e822786e71123439
The neutron.common.rpc module has been in neutron-lib for awhile now and
neutron is shimmed to use neutron-lib already.
This patch removes neutron.common.rpc and switches the code over to use
neutron-lib's implementation where needed.
NeutronLibImpact
Change-Id: I733f07a8c4a2af071b3467bd710290eee11a4f4c
This incorporates flake8 2.6.x and pycodestyle will be used
instead of older pep8. This ensures future python3 compatibility
and a bit better code styling.
Change-Id: Ia7c7c5a44727f615a151e1e68dd94c7ed42f974f
Fixed all pep8 E265 errors and changed tox.ini to no longer
ignore them. Also removed an N536 comment missed from a
previous change.
Change-Id: Ie6db8406c3b884c95b2a54a7598ea83476b8dba1
Iptables firewall driver now adds rules for trusted ports to FORWARD
chain in EGRESS and INGRESS directions.
Unfiltered and trusted port rules are too wide. We have to match
traffic against security groups first.
Change-Id: I61e4dc92669e33a207adfb72a1692184884143e1
Closes-Bug: #1762736
In corner cases, the firewall code could try and remove
non-existent conntrack zone jump rules if a zone has never
been allocated. This could happen on an agent restart
when there are no longer ports in the zone on the
compute node. Skip the removal since it will just generate
an iptables warning complaining the existing rule does
not exist.
Change-Id: Ie32733b4a06b6d75cf1eb78915a510a4bb78f619
Closes-bug: #1765208
Iptables firewall driver can now add process trusted ports and
adds rules for them to FORWARD chain.
Change-Id: I67d0f17b4b56671fc2e2dd6e2fc4518dc42cd131
Closes-Bug: #1720205
SG with hybrid-iptables driver uses per port conntrack zones.
FWaaS port security uses per network conntrack zones based on
local vlans assigned by ovs l2 agent.
In case both SG iptables-hybrid driver and FWaaS port security is enabled,
there is a posibility of iptables-hybrid and OVS based FWaaS driver
allocating overlapping zone and creating security holes.
This patch changes the zone allocation range for iptables and
hybrid_iptables driver to 4097 - 65535. While OVS based
port security driver can use zones based on local vlan range 1 - 4096
Closes-Bug: #1745642
Change-Id: I4d51637ed1de8fe85b4982a03410d4a3f637ea3f
The allowed address pairs extension's API definition was rehomed into
neutron-lib with I7958a2d6f470f088ca2cb8ad638c075788f22851. This patch
consumes the API definition in neutron.
NeutronLibImpact
Change-Id: I46cfeee9711973ec15881c8dc2bd0bf763bc2226
In Pike, the agent side of security_groups_provider_updated()
RPC code was changed to a NOOP when the provider rules were
changed to be static, https://review.openstack.org/#/c/432506
Now that we're in Queens we can deprecate it.
Change-Id: Ie018ff653633d3524f0e80c5e172a5d01bdad437
This will drop Pecan in place of the old APIRouter for all
unit tests. This will significantly increase the UT coverage
of pecan and will allow us to drop the old API routing logic
completely.
The rest of the inline test changes are due to slight plugin loading
differences.
Implements: blueprint wsgi-pecan-switch
Change-Id: I76dc23fb7b96d82b0da50285bd0aac76142e81e5
Change Ibfbf011284cbde396f74db9d982993f994082731 moves
generation of ICMPv6 RA rule from being hardcoded
in iptables_firewall to being generated on server
side and passed to agent.
Unfortunatelly it wasn't removed from iptables_firewall
and it was still added to rules which should be applied
by firewall driver.
That caused issue with warning message about duplicated rule.
detected
This patch removes this hardcoded rule to stop logging messages
about duplicated rules.
Change-Id: Ic5e95405d4dd8ffbe8ec5b053aed257aec91b1c8
Closes-Bug: #1708465
Currently trusted ports (DHCP & router ports) are considered trusted
ports based on theirs owner. Trusted ports are not passed to the
firewall driver and hence the driver doesn't have any way how to deal
with such ports.
This change introduces two new methods to firewall driver API:
- process_trusted_ports
- remove_trusted_ports
These methods will give firewall driver a chance to process traffic
coming from the trusted ports in case its needed.
For specific case, see patch depending on this one.
Change-Id: I0be64483515e45f98ffffce8346a6bff06bc0fd1
Related-bug: #1626010
In [1] the setup of security group rules for DHCP was refactored.
However it assumed that DHCPv6 responses would go to the target IP,
instead the destination will be the link-local address of the instance,
so we will need to add this to the list of allowed destinations.
[1] Ibfbf011284cbde396f74db9d982993f994082731
Change-Id: I6bd24a98d7c72a1bac28999eb3c9a831ab2ab9bf
Closes-Bug: 1693771
This is something really old, like... Havana old. This code is also
triggered by neutron-server when loading extension for SGs, which
doesn't make sense at all, because server is not supposed to configure
firewall_driver (it's agent job). So in setups that (correctly) leave
firewall_driver unset in neutron.conf, we get a warning log message.
It's not the only guilt of the check: it also compares firewall_driver
against full import path to noop driver instead of using stevedore
aliases.
This patch just kills the whole check, from both agent as well as server
sides.
Change-Id: Iec9893cb2e30967a77f4f7151d5c31d45cd8e690
The regular IPTablesFirewall needs zones to support safely
clearly conntrack entries.
In order to support the single bridge use case, the conntrack
manager had to be refactored slightly to allow zones to be
either unique to ports or unique to networks.
Since all ports in a network share a bridge in the IPTablesDriver
use case, a zone per port cannot be used since there is no way
to distinguish which zone traffic should be checked against when
traffic enters the bridge from outside the system.
A zone per network is adequate for the single bridge per network
solution since it implicitly does not suffer from the double-bridge
cross in a single network that led to per port usage in OVS.[1]
This had to adjust the functional firewall tests to use the correct
bridge name now that it's relevant in the non hybrid IPTables case.
1. Ibe9e49653b2a280ea72cb95c2da64cd94c7739da
Closes-Bug: #1668958
Closes-Bug: #1657260
Change-Id: Ie88237d3fe4807b712a7ec61eb932748c38952cc
Setting up rules to allow DHCPv6, DHCP, and RAs from specific
IP addresses based on Neutron resources has a few issues:
1. It violates separation of concerns. We are implementing logic to
calculate where an IPv6 RA advertisement or DHCP advertisement
should be coming from in the security group code. This code should
not be trying to guess IPv6 LLAs, know about subnet modes, DHCP server
implementations, or the type of L3 plugin being used. Currently all
of these assumptions are baked into code that should only be
filtering, which makes it very rigid and brittle when it comes to
other implementations for DHCP and/or RAs.
2. It has scaling issues on large networks. Every time one of these
provider rules is updated, it triggers every L2 agent to refresh
all of the security group rules for ports in that network, which puts
significant load on the server.
3. It's main purpose: preventing spoofing of RA[1,2] and DHCP packets,
has long been superceded by preventing VMs from acting as DHCP/RA
servers[3][4].
This patch completely removes all of this logic and just returns
static provider rules to the agents that allow all DHCP server
and RA traffic ingress to the client. This addresses the issues
highlighted above since the code is significantly simplified and
the provider rules don't require refreshes on the agents.
Now that the provider rules never change, the RPC notification
listener on the agent-side for 'notify_provider_updated' is now
just a NOOP that doesn't trigger any refreshes. The notification
was left in place on the server side for older version agents
that have stale IP-specific provider rules. The entire notification
can be removed in the future.
The one open concern with this approach is that VMs will now be
able to receive DHCP offers from other DHCP servers on the same
network that aren't being filtered (e.g. a VM with port security
disabled or another device on a provider network). In order to
address this for DHCP, this patch adds two rules that only allow
DHCP offers targeted to either the broadcast or the correct client
IP. This prevents incorrect offers from ever reaching the client.
For RAs, this patch just allows all RAs so we may pick up
advertisements from other v6 routers attached to a network;
however, the instance won't actually be allowed to use bad addresses.
1. https://bugs.launchpad.net/neutron/+bug/1262759
2. I1d5c7aaa8e4cf057204eb746c0faab2c70409a94
3. Ice1c9dd349864da28806c5053e38ef86f43b7771
4. https://git.openstack.org/cgit/openstack/neutron/tree/
neutron/agent/linux/iptables_firewall.py
?h=521b1074f17574a5234843bce68f3810995e0e1d#n475
Closes-Bug: #1653830
Closes-Bug: #1663077
Change-Id: Ibfbf011284cbde396f74db9d982993f994082731
There is no need to add iptables nat and mangle rules for security
group, these rules will slow down network performance especially
when using 6wind Virtual Accelerator.
Change-Id: I1d5748394665535d114e8d942a68d5bd43927058
Closes-Bug: #1650466
Through [1] ipset members are updated in update_security_group_members
instead of updating during firewall apply. In the same way, we will
delete conntrack entries immediately after deleting remote ipset
members(in update_security_group_members) instead of deleting them after
firewall apply.
As explained in [2], this change partially fixes bug #1580377 i.e it
deletes conntrack entries on remote hosts for a removed port.
[1] https://review.openstack.org/#/c/347068/
[2] https://bugs.launchpad.net/neutron/+bug/1580377/comments/13
Co-Authored-By:shihanzhang <shihanzhang@huawei.com>
Partial-Bug: #1580377
Change-Id: Iea3344a24e2a068b794c44796b4c945432379c13
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
Zuul