Created new add_ip_addresses privileged function
which takes an iterable of cidrs and adds them
in one privileged call. This is so we dont have to
take on additional priv overhead when calling
add_ip_address in a loop.
For parity, performed the same change on the
delete_ip_address function.
Closes-Bug: #1987281
Partial-Bug: #1981113
Change-Id: Ib1278af20c3b3b057712453cb249aba34b684a21
To improve the execution isolation of "IpMonitorTestCase", the test
cases will be executed inside a network namespace only.
Closes-Bug: #1991798
Change-Id: I69200f8236e65ea50d1ad3ac322441554634c9c3
Now it is mandatory, at least for IPv6 addresses, to define a table
when an IP rule is added. The default table selected is "default"
(table=253). In any case, all commands calling this method right now are
specifying the table in the kwargs.
Partial-Bug: #1981963
Change-Id: Ia44ac34ca9b91719a86f4d573c9777a4708d69a4
Check if group and/or local addresses passed to ip_lib / add_vxlan()
are IPv4 or IPv6. In case of IPv4 fill 'vxlan_group' and 'vxlan_local'
arguments and in case of IPv6 fill 'vxlan_group6' and 'vxlan_local6'
arguments to be passed down to privileged create_interface() method.
In case of an invalid address format raise an AddrFormatError exception.
Closes-Bug: #1952897
Change-Id: I2e3b0c1635627edb2c86c6120b0410ab3c4678b2
"IPWrapper.add_vxlan" method must have "dev" parameter as possitional
argument. A VXLAN interface must be always created on top of an existing
network device:
https://www.kernel.org/doc/Documentation/networking/vxlan.txt
Closes-Bug: #1954316
Change-Id: Ia082f8531ffcc1599206124774599dcdb500274a
When an interface is moved to a new namespace, specially with OVS
internal ports, the interface first dissapears from any network
namespace and then is added again. ovs-vswitchd service detects
this interface change as reported in [1]. This delay is the cause
of the related bug, where some interfaces are not present when
the L3 agent needs to manipulate them.
[1]https://bugs.launchpad.net/neutron/+bug/1948832/comments/3
Closes-Bug: #1948832
Change-Id: I3af4d0afa784899689ccb595ce6ba64495431eb9
To check the existance of a namespace, instead of listing the
namespaces directory (by default "/var/run/netns"), this patch
directly checks the existence of the namespace directory, using
"os.path.exists".
This check is faster than listing the whole directory and avoids
timeout problems as reported in the related bug.
Closes-Bug: #1947974
Change-Id: I558d50d28378beb3710d98a2113ff9549c82ae17
"get_routing_table" uses "pyroute2.IPDB" that has been deprecated.
"list_ip_routes" has been improved to be able to read multipath
routes.
Closes-Bug: #1926476
Change-Id: I0299fa11a7afefbd2999f81cd4ed3beed572009c
This is a leftover of the "ip route" command migration to Pyroute2.
A new paremeter, "proto", is added to the IP route add and list
commands. The default protocol used is "static".
Story: #2007686
Task: #41284
Related-Bug: #1492714
Change-Id: I319fd0611d3e8a3a09d6d4e077a17a622f74f51c
As reported in LP#1896734, there is a limit in the size of information
that can be transmitted in one single message between an application
and the privsep daemon. The read socket buffer is limited in size;
a message exceeding this size will generate an exception.
In order to limit the amount of information to be sent, this patch
improves the performance of "get_devices_with_ip". In the previous
implementation, the whole list of network devices from a namespace
was retrieved. In some environments, the list of devices could be
so big that the list returned by "privileged.get_link_devices" can
exceed the read buffer size (as reported in the LP bug when the
OVS agent tries to retrieve the list of IP addresses in the system).
Now the function calls "privileged.get_ip_addresses", that returns
a much smaller list. This patch is also reducing the number of system
calls to just one; the previous implementation was retrieving first
the devices link information list (that method was returning a much
bigger blob) and then, per device, retrieving the IP address
information.
Change-Id: I97ada62484023b9833ed12afd68eb4c8d337fd1f
Related-Bug: #1896734
When reading the JSON blob from the file written by "ip_monitor",
do not fail if the JSON blob is decoded incorrectly.
Change-Id: I0b3a3cf94b1d0ae721d5446c8abcf74bd29883c7
Closes-Bug: #1911927
assertItemsEqual was removed from Python's unittest.TestCase in
Python 3.3 [1][2]. We have been able to use them since then, because
testtools required unittest2, which still included it. With testtools
removing Python 2.7 support [3][4], we will lose support for
assertItemsEqual, so we should switch to use assertCountEqual.
NOTE(dmllr): added hacking check
[1] - https://bugs.python.org/issue17866
[2] - https://hg.python.org/cpython/rev/d9921cb6e3cd
[3] - testing-cabal/testtools#286
[4] - testing-cabal/testtools#277
Change-Id: I7c20fec08e5dc9f67b34100c925ea6724bbd25f0
In "IpAddrCommand.list" method, the "scope" parameter is a string
("link", "site", "global" or "host"). This method will retrieve all
devices with an IP address calling "ip_lib.get_devices_with_ip".
Since [1], "ip_lib.get_devices_with_ip" makes the conversion of
"scope" string parameter to pyroute2 format (see
"pyroute2.netlink.rtnl.rtscopes"). The list command should skip then
the previous conversion.
Closes-Bug: #1899141
[1]https://review.opendev.org/#/c/747406/
Change-Id: I55a0f4341b328af52ea3bd758a72f633fbe3abcb
By default, if no metric is defined, the kernel interprets the
highest value (0).
The current implementation, using pyroute2, is a translation from
the CLI command "ip route". This command uses the netlink API to
communicate with the kernel. In IPv6, when the metric value is not
set is translated as 1024 as default [1].
[1]https://access.redhat.com/solutions/3659171
Change-Id: I0c5f9e320bbbf314a2d6a22c515bf903de84cdaf
Related-Bug: #1855759
The gateway IP address in the gateway dictionary returned by
"ip_lib.list_ip_routes" is stored in "via".
"priority" parameter is changed to "metric", to match input and
output parameters.
Change-Id: I67ae473dca8d706f963c3b55b9410f9a79d7f32b
Closes-Bug: #1855759
Since it's no longer supported past Train, lets stop
running the tests.
Updated docs and made some pep8 code tweaks as well.
Change-Id: I1c171ab906a3b4c66558163ad26947ebf710a276
IP monitor is a method that is going to be executed in a separate
process, to monitor the IP addresses changes in a namespace.
This method spawns a thread to read from a socket opened by Pyroute2.
The read function is a blocking method that will end only when the
socket is closed. To avoid thread starvation that can happen using
greenthreads, IP monitor will use kernel threads.
This will increase the resources used but will ensure that no message
is lost when reading the monitor socket.
Reduced the number of IPs generated in "test_add_and_remove_multiple_ips"
to shrink the testing time used.
Change-Id: I3fbba2854d40ab0f683443aa30c2a95752345d2e
Closes-Bug: #1849547
In order to capture all IP address changes, the method reading the
netlink socket will be executed in a parallel thread. Once the
"ip_monitor" method is stopped, this blocking thread will be killed.
A new functional test, "test_add_multiple_ips", is added in order to
stress test this method.
Change-Id: I8f1de4a31f97bab734a33f94c3069444defd870f
Closes-Bug: #1832307
This method allows to track any IP address change in a
namespace. In future patches, this method will replace
the current IP monitor used in the keepalived_state_change
daemon. The current implementation relays in a spawned shell,
executed in root mode, and the output of this shell,
conveniently parsed.
If the passed namespace is not None, this new method must
be executed in privileged mode (root user), but cannot use
privsep because is a blocking function and can exhaust the
number of working threads.
This function should be executed in a parallel thread, returning
the data using the eventlet queue. Pyroute does not implement yet
a non blocking method to retrieve the command output or to know if
the buffer has data. This method, spawned in a greenthread, must be
stopped by killing this thread.
An example of how to use it can be found in the functional tests
implemented in this patch.
Change-Id: I86e4487035d60e1b52e951dd3cd50d6bb54f388b
Related-Bug: #1680183
Now ip_lib.get_devices_info function is implemented using pyroute2,
"vlan_in_use" and "vxlan_in_use" can make use of it.
Change-Id: I82a2c3ea76195b10880cf37bf2229341b995b0ae
Closes-Bug: #1815498
In case when MTU value configured for VXLAN network is
higher than physical device's MTU - VXLAN overhead
then Linuxbridge agent will not be able to create properly
vxlan interface with desired MTU value.
In patch [1] there was introduced validation if MTU of
physical device is big enough to create MTU with required
MTU value but it was working only for IPv4 tunnels as
for IPv6 overhead is bigger.
This patch changes this validation a bit and now it will
works properly for both IPv4 and IPv6 VXLAN tunnels.
[1] https://review.openstack.org/#/c/546291/
Change-Id: Ib707312adb45dae7295884aba4ece6538d330d56
Related-Bug: #1744101
Adding IP address on device is now done with pyroute2 lib.
This commit handle the case when pyroute2 function raise
exception when same IP address is already configured on
device.
In such case it will now raise exception which inherits from
RuntimeError what is consistent with eariler code when it
was done by running "ip addr" command to configue IP.
Change-Id: I89f22310782f2f0baf0ce6b20d2ab0e1d68654a0
Related-Bug: #1492714
This commit adds functional tests for add/delete/flush/get
gateway IP methods from ip_lib.IpRouteCommand class.
Change-Id: I59d2361a849660a6a67ed56a5be52ee0e88d6e4a
This patch switches IpLinkCommand methods from ip_lib
module to use pyroute2 library.
This commit adds also additional link attribute "allmulticast"
thus allowing to make easy functional test for
set_allmulticast_on method from IpLinkCommand class.
Related-Bug: #1492714
Change-Id: I9ffd23c240c607ffd5f10beff5c8c8d5f01441e9
When privileged delete_ip_address function is called to delete
IP address which is already not configured on device, it should
not fail with any error.
Change-Id: I9247ac899a76e5d9a2962d2cb81279f2d6f16c0b
Closes-Bug: #1757259
IpAddrCommand.get_devices_with_ip() method now returns also broadcast
address configured for IP address on given network device.
In case when there is no broadcast address configured None value
will be returned.
This patch adds also additional functional checks for
IpAddrCommand.add() method to test if broadcast IP address is
properly configured on network device.
Change-Id: Iae99f0f122988008187cf72c71123364ac6460e4
This patch adds functional tests for add/delete/flush IP addresses
for neutron.agent.linux.ip_lib.IpAddrCommand class.
Change-Id: Ic6e775c300e0ac9351f0f6f06ad200f88593e7db
This patch adds functional tests for operations like:
* set link mtu
* set link namespace
* set link name
* set link alias
Change-Id: Ie623792e9a9f2bf2885019fdde05acfe02bc4831
If callers of get_devices_with_ip(), or
device.addr.list(to=address) pass an ip_cidr, it
could match any ip_cidr in that range on the interface.
Callers need to pass the IP without the prefix portion in
order to match it exactly. Added a helper utility to
strip the cidr part from a ip_cidr.
Determined the unit test for this can't actually check
this case since we are mocking the return value from
/sbin/ip, so modified it to just make sure the dict
is correct.
Added a functional test that adds two IP addresses in
the same IP range to verify that we actually filter
correctly when a 'to=IP' is specified.
Change-Id: I3a95b3bb72a43f322ad23892d8959398aac22a1c
Closes-bug: #1728080
This reverts commit fd1403fd9a.
It didn't solve problem with race condition during creation
of namespace through "ensure_namespace()" method.
Change-Id: I6f7a1cb7b685d0c1d9c6b165cfbb6e85e68faf61
In some cases if ip_lib.IPWrapper.ensure_namespace() method
is called more than once for same namespace in very short
period of time it could raise error that "File already exists"
for second call of this method.
It happens often e.g. in fullstack tests.
Reason of such problem is in Netlink protocol which is used
by iproute2 to communicate with kernel. This protocol, according
to http://man7.org/linux/man-pages/man7/netlink.7.html is not
reliable so it is not guaranteed when the message will be
delivered to kernel and when action will be really executed.
Because of that if on quite loaded host ensure_namespace() method
would be executed twice it can lead to error described above.
This patch is changing way how ensure_namespace() method works
to avoid raising ProcessExecutionError exception with this
error message.
Closes-Bug: #1717582
Change-Id: I1898426789c85ce1faa97665bfd47f1fa38ef727
Refactoring neutron agent linux and ovsdb config opts
to be in neutron/conf/agent so that all the config options
reside in a centralized location. This simplifies the
process of looking up the config opts and provides an easy
way to import.
NeutronLibImpact
Change-Id: Ib1e0e63dec2985c417412d1ecc68e2a74ef87182
Partial-Bug: #1563069
Rodolfo Alonso Hernandez