This commit is a squash of 3 different commits required to fix
DragonFlow gate (See below).
Upgrade redis driver
Redis python package API was changed over time, while we did not
change our handling of the code.
In the gate the driver was updated, and the gate tests were broken.
Updated the code in a way that supports both python2 (soon to be
obsoleted) and python3.
Updated the requirements to install new versions of the packages.
Remove testing revision number in test_df_qos_driver
This code tests Neutron behaviour of increasing the revision number.
This is not code that is controlled by Dragonflow. If (when) it fails,
Dragonflow cannot fix it generically.
use payloads for ROUTER_GATEWAY events
This patch switches the code over to the payload style of callbacks [1]
for ROUTER_GATEWAY events for those that are not using them yet.
The unit tests are also updated where needed to account for the
payload style callbacks and publish() method. Finally the patch
normalizes the passing of gateway IPs which are currently referred to
as 'gw_ips' and 'gateway_ips' depending on the event; now all events use
'gateway_ips'.
Depends-On: https://review.opendev.org/#/c/638274/
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html
Co-Authored-By: Omer Anson <omer.anson@toganetworks.com>
Co-Authored-By: boden <bodenvmw@gmail.com>
Change-Id: I5cd456028ed8af86c6c512e514eb91cec56ea77c
In the devstack gate rc files, the link to the dragonflow CI configuration
file was wrong (actually for quite a long time now).
Updated the link to the correct location.
Change-Id: I8d8f507f57ccb72a73f8d2f620995809c23c4d8e
We no longer use OVS new features, and the minimal version we require is 2.9.1.
Also, on ubuntu, the version that is supplied by apt is 2.9.2, so it
should be sufficient, and we can save hassle and time by NOT recompiling
the ovs ourselves...
Change-Id: I453dc1d38395015c2d24477b592d1a2ec212e43c
This commit is composed of 4 (four) commits that fix various aspects
that were broken in the gate.
Co-Authored-By: Omer Anson <omer.anson@toganetworks.com>
Bump up the version of libzmq for Ubuntu and Suse
This is required as libzmq1 is no longer supported under these
platforms.
Change-Id: I17c73025856c006926ea6b2444c4fde86726db9d
Install ruby via the operating system
As opposed to installing it from source, success of which is a matter of
chance.
Change-Id: I39f13525d9b1d814c90ac898148b9543ed64cd7e
Cleanup registered apps with os-ken
Otherwise, os-ken complains that the apps are already
instantiated. This might be causing the get failure.
Change-Id: Iad8f76aafe98be443b0efa34e2f9cef8f40a9588
Disable test TestNeighborAdvertiser
It isn't working. Reported in bug 1820977.
Change-Id: Ie1f9c05c3847320ba1892c52182d3e7d856f4957
Related-Bug: #1820977
The current cirros image version is 0.4.0.
As we have the image name hard-coded, it is required that we update it manually.
See [1] for reference.
[1] https://review.openstack.org/#/c/521825/
Change-Id: I99a49975c5674991d84932ebd2c7ab6599d343ea
Migrate Dragonflow from Ryu to os-ken
This change causes Dragonflow to stop consuming Ryu in favor of os-ken.
os-ken becomes the reference implementation for BGP services and
Dragonflow no longer has a dependency on Ryu.
See [1] for reference in Neutron.
[1] https://review.openstack.org/607008/
Change-Id: Ifbf294f2f586cf765d20cfa60774c45aa735ca62
Force delete of VM on close
There are tests that fail because we are waiting for the VM to complete shutdown
In tests, we do not mind destroying the machine forcefully as there is no reason
to wait for it.
Change-Id: I5f8e4928ceb7b467af891835600cd8096ec88d2b
use payloads for SECURITY_GROUP BEFORE_DELETE events
This patch switches over to the payload style callbacks for
BEFORE_DELETE events of SECURITY_GROUP resources.
Co-Authored-By: Boden R <bodenvmw@gmail.com>
Depends-On: https://review.openstack.org/#/c/597599/
Change-Id: Ie9163ad7baecf85e234e4253160f30015b43166b
In patch [1] there were some lines that were missed out. Specifically
an iptables rule and a cleanup line.
This patch re-adds the rule and cleanup code.
[1] https://review.openstack.org/#/c/583878/
Change-Id: I32045f60e205d59382ef3d98f845c4c1c8bea37f
Partial-Bug: #1781376
The deployment should be done via deployment scripts and not in the
code (nor in plugin.sh directly, as to allow production deployment).
Moved the deployment to a script located in the tools directory, which
is now called from the plugin.sh, and may be called from any production
deployment mechanism.
Partial-Bug: #1781376
Change-Id: I90a7e8a87764354ab9e2a8ab149f00f2eaea94f5
To work around the ZMQ issues (publisher binds to a port, thus only one
publisher can work on a given IP), we had a lot of junk code in our
repo.
Removed all redundant mechanisms (use_multiproc, is_neutron_server) and
modified to have 2 types of configurable ZMQ drivers:
1. zmq_remote_pubsub_driver - Has TCP publisher and IPC subscriber, This
should be used in the publisher service.
2. zmq_pubsub_driver - Has IPC publisher and TCP subscriberi, This
should be used in all other uses.
The way to set the one to use is via configuration, thus it is up to the
deployment to make sure this is configured correctly.
Change-Id: Ibf7894e608187e87bdeb7774749bfa0cc15eae56
The function marked as common was called from one driver only, while the
other code was duplicated.
Change-Id: I7760f6ec974fd7caea5caed38a16ddc731cdeee5
Add a new script, tools/add_table_names, that adds to the output of
ovs-ofctl dump-flows the table names, including those allocated by the
new datapath. It relies on the new datapath output file.
Change-Id: I82069d1d084145ea0a55fd8c7ea2573ba5a86c4f
Previously, we installed openvswitch 2.7.1 from git. This is problematic
in new kernels due to symbol name collision[1]. A fix was commited and
tagged as 2.9.1.
[1] 138df3e563
Change-Id: Id24b6caaeabfa22b10308b9bfbc48b8c1c7a6b9a
Publisher service to update Publisher table
When there are no events for a long time, the publisher
service does not update the publisher table. This may
make it look like the publisher is inactive. This will
cause some pub/sub messages to be lost.
This change makes the publisher automatically update
the timestamp in the database every once in a while
(half the timeout time).
Change-Id: Ie02120f2e31552e8bd93fc02101294500c959ac2
Add initial scaffolding to support applications with non-hard-coded
table names and registers.
Includes:
* 'wiring' configuration (vertices and edges)
* Decorator for new applications, with States, Entrypoints, Exitpoints,
and public and private variables.
Includes code for reading and parsing this information, and connecting
the 'external' tables, i.e. the exitpoints of one app to the entrypoint
if the next app.
Change-Id: I81ee47b0c6e741888b3446602cca5e1835c9ae2f
Related-Bug: #1738986
Co-Authored-By: Lihi Wishnitzer <lihiwish@gmail.com>
To enable skydive, one should now enable the service in the local.conf
file using the following line:
enable_service df_skydive
If you would like to install the SkyDive analyzer on the same node, the
SkyDive install lines MUST be before the DragonFlow ones!
Please see doc/source/single-node-conf/etcd_skydive_local_controller.conf
file for usage example
Change-Id: I66c3eaf18904ae1c109d0ca8007269c41261d791
Depends-On: c5921051d7bc05d2c29494be842329e2ffcc008a
Related-Bug: #1749429
Use the ACTION and STEP variables instead of $1 and $2.
Move code into functions for better readability.
Change-Id: If096c5536657fbe29e05d2bdb82c3c1347fe9bcc
The zookeeper devstack installation had few issues:
1. The configured port was ignored, and DF could not
connect to the DB
2. The systemd service file is already created by
the installation itself
Co-Authored-By: Shachar Snapiri <shachar.snapiri@huawei.com>
Change-Id: I82bc200181768a7065adce7a7fc5999b4ae97cb3
The add_table_names script is to be used with the output of the
ovs-ofctl dump-flows command, and adds the names of the tables being
used to their IDs. This is done for better readability and easier
debugging.
Change-Id: I52907e8d3b81f3f23eff2b0e062160141285bfed
Related-Bug: #1740867
df-bgp service was not enabled for tempest, and the DF_PUBSUB was not
set for zmq fullstack. The latter does not seem to cause errors in the
build, but I believe it should be there, and only removed after thorough
consideration.
Change-Id: I6a4498325f9cff58367876533e1367bcd27258ca
Related-Bug: #1737889
Patch https://review.openstack.org/529515/ had a problem in the tempest
configuration, it was missing the enabling of the tempest service, thus
causing the tempest builds to fail even more.
Change-Id: If4b5d7e9488e7676c281198107701409a1d457a5
Related-Bug: #1737889
Added ability to call ovs-appctl, this is to allow calling the
SimulateAndSend for a packet for debug purposes.
Co-Authored-By: Omer Anson <omer.anson@toganetworks.com>
Change-Id: I8cb2ed5ae19d7e285e6db395aeedecc21c77b215
All common configuration was set to common files for the specific ones
to consume.
This allows us to put only the differing configuration, and keep the
common stuff together, and in this way avoid mistakes (e.g. having
selective-proactive on only for etcd/zmq).
Change-Id: Iace7b9116aefbf169245d42ef5ad04525cd72a33
Related-Bug: #1737889
By default, when loadbalancer VIP ports are created (either via Octavia
or via neutron-lbaas (v2)), then the port is created with
admin_state_up=False (i.e. the port is disabled). Dragonflow respects
this attribute, and does not 'wire' the port in this case.
Rather than change the default in Octavia and neutron-LBaaS, this patch
detects this case, and enables the VIP ports.
Change-Id: Ibbe927b2672f327349bdd9cff46349d4d01c990a
Partial-Bug: #1720734
The devstac-etcd-zmq gate still had the selective topo distribution set
even though it was supposed to be removed. this caused issues in the
gate (e.g. network creation messages did not arrive until the port was
created, thus causing an exception).
Change-Id: I8310806ae9373c6ca3e47257c329648d2b285000
Related-Bug: #1737889
Add support to infer MACVLAN ports (and IPVLAN ports) from Neutron port
and allowed address pair specification.
Partial bug, since tests are still missing.
Partial-Bug: #1720734
Change-Id: I4e07f074b3db999d72bfa60bae1622ce9547c3e5
Rename the fullstack gates in order to emphasize the difference
between the gates, and what is tested in each gate.
Change-Id: I56200b4eedac14a0b3f3f795d6b952f799a36de1
Add a flag (DF_REINSTALL_OVS) - when set to false, the neutron ovs will not
be uninistalled and will be used. When set to True, the current behavior
of removing the OVS and reinstalling it will remain.
This will allow us to use some preconfigurations done by neurton in some
specific cases.
Change-Id: I2b7a9597033155fe4584559bebb6adda25198667
Closes-Bug: #1718115
Until now, DNAT app maintained its own patch ports towards the external
network and classified the relevant traffic in table 0. This falls into
the responsibility of provider/tunneling apps now.
This patch removes a lot of the functionality done by provider / L2 apps.
DNAT apps marks the binding of floating port and emits a relevant event,
making it remote or local to other apps, and any traffic directed to
floating ports is forwarded to EGRESS table with relevant reg7 set. This lets
DNAT apps to focus on just the translation part.
Closes-Bug: #1636829
Change-Id: I8fb9e0e85510c0814e47cc7bdfd27648fd4b88ba
* Move OVS related defaults from override-defaults to settings
* Add default value to Q_ENABLE_DRAGONFLOW_LOCAL_CONTROLLER
* Removed PUBLIC_BRIDGE as our defualt matches devstack
* Removed Q_USE_PROVIDERNET_FOR_PUBLIC which will now be true by
default. This will make `public` network flat rather than VxLAN so we
can reach its ports through br-ex
Change-Id: I2cef4e6b565d83860ce700aae2b498a9aa129ce3
Sean McGinnis