From 678e889fc8f2b34c85f5652a96e72b9eeeb48208 Mon Sep 17 00:00:00 2001 From: James Page Date: Wed, 11 Sep 2019 10:11:16 +0100 Subject: [PATCH] Add spec for Mellanox hardware offload support New spec for delivery of support for hardware offload of networking to supported Mellanox Connect-X 5+ networking cards using Open vSwitch. Included drive-by fixes for pep8 failures with Py3.8. Change-Id: I89c668cb44fd6308d8819afba19db42a52e8dbd0 --- specs/ussuri/approved/mellanox-hw-offload.rst | 159 ++++++++++++++++++ test-requirements.txt | 2 +- tests/test_titles.py | 2 +- 3 files changed, 161 insertions(+), 2 deletions(-) create mode 100644 specs/ussuri/approved/mellanox-hw-offload.rst diff --git a/specs/ussuri/approved/mellanox-hw-offload.rst b/specs/ussuri/approved/mellanox-hw-offload.rst new file mode 100644 index 0000000..e1fde09 --- /dev/null +++ b/specs/ussuri/approved/mellanox-hw-offload.rst @@ -0,0 +1,159 @@ +.. + Copyright 2019, Canonical Ltd + + This work is licensed under a Creative Commons Attribution 3.0 + Unported License. + http://creativecommons.org/licenses/by/3.0/legalcode + +.. + This template should be in ReSTructured text. Please do not delete + any of the sections in this template. If you have nothing to say + for a whole section, just write: "None". For help with syntax, see + http://sphinx-doc.org/rest.html To test out your formatting, see + http://www.tele3.cz/jbar/rest/rest.html + +================================= +ML2/OVS Mellanox Hardware Offload +================================= + +High end networking adapters such as the Mellanox Connect-X series support +up to 200Gbps networking; instances using traditional software virtualized +networking via OVS and virtio-net cannot achieve networking throughput of +more than 10Gbps without a high CPU load; it’s possible to use SR-IOV to +increase performance to nearer 100Gbps but that has limitations in terms +of the flexibility of networking options (no support for overlay networks) +and no support for security groups. + +The Mellanox Connect-X 5+ supports hardware offload via a feature called +ASAP2; integrated with Open vSwitch (OVS), this pushes the network data +path from the kernel directly onto a high performance embedded switch +on the network card. This will allow instances to achieve much higher network +performance with very low CPU overheads as well as utilizing as much +of the overall network capacity to the hypervisor as possible. + +Problem Description +=================== + +Hardware offload support is not currently enabled in the OpenStack charms. + +Proposed Change +=============== + +The neutron-openvswitch charm will be updated to support configuration of +Mellanox Connect-X 5+ network adapters to support hardware offloading +(card needs to be placed in switchdev mode on boot); OVS will +be configured to make use of hardware offloading where possible. + +A new tool (mlnx-switchdev-boot) will be developed to configure the +Connect-X card in the right mode on boot prior to it being used by OVS +for networking. This may be superseded at some future date by features +in networkd and/or netplan. + +This feature does make use of a number of leading edge features across the +hypervisor software stack: + +- Linux Kernel >= 5.2 +- OVS >= 2.11.0 + +This limits the scope of support to OpenStack Stein or later in-conjunction +with the Ubuntu LTS hardware enablement kernel from Ubuntu 19.10. + +The Connect-X series of cards also supports Virtual Function (VF) Link +Aggregation (LAG) providing upstream switch/cable/port resilience for +hardware offloaded ports which are supported using a VF on the +underlying network card - this is driven by the normal configuration +process for bonding network devices via Linux - in this case the +Physical Functions (PF’s) for the network cards. + +The existing SR-IOV VF configuration code and scripts in the +neutron-openvswitch charm is a little inconsistent and not very reliable +on reboot; this change will cover refactoring the SR-IOV VF code into +a new tool (sriov-netplan-shim) dealing with configuration of VF +functions via the charm and on reboot. This tool will be superceeded +by SR-IOV support in netplan at some future date. + +.. note:: + + The most recent kernel and OVS versions don’t yet support connection + tracking offload which means that security groups cannot be offloaded + to the network card switch; this is under development and will land in + a later release of OVS and Linux. + +.. note:: + + The total number of hardware offloaded ports a hypervisor can support + is limited by the total number of VF’s that the underlying network + cards can support. + +Alternatives +------------ + +It’s worth noting that for VLAN or flat networking requirements, it’s +possible to achieve high throughput networking to instances by using +SR-IOV which is already supported by the neutron-openvswitch charm. + +Implementation +============== + +Assignee(s) +----------- + +Primary assignees: + wgrant + james-page + +Gerrit Topic +------------ + +Use Gerrit topic "mlnx-hardware-offload" for all patches related to this spec. + +.. code-block:: bash + + git-review -t mlnx-hardware-offload + +Work Items +---------- + +Develop and test tool for configuration of Connect-X adapters for hardware +offload support. + +Update neutron-openvswitch to support configuration of OVS in hardware +offload mode. + +Add appendix to charm deployment guide to detail configuration and use +of OpenStack with Mellanox Connect-X 5+ cards in hardware offload mode. + +Repositories +------------ + +A new repository (mlnx-switchdev-mode) will be required for tooling to +configure the Mellanox Connect-X networking cards on boot. This tool +has scope outside of the OpenStack charms project so can be developed +on github. + +A new repository (sriov-netplan-shim) will be required for tooling to +configure SR-IOV VF functions; this tool will superceed the existing +SR-IOV VF configuration code in the neutron-openvswitch charm. + +Documentation +------------- + +Documentation of this feature will be provided in the charm deployment guide. + +Security +-------- + +No additional security risks are introduced by this feature. + +Testing +------- + +Testing of this feature requires specific hardware in the form of Mellanox +Connect-X 5+ networking cards. + +Dependencies +============ + +- Linux >= 5.2 +- OVS >= 2.11.0 +- Mellanox Firmware >= 16.26.1040 diff --git a/test-requirements.txt b/test-requirements.txt index 86f503a..85331d1 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -2,4 +2,4 @@ # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. -hacking>=0.10.0,<0.11 +hacking>=0.10.0 diff --git a/tests/test_titles.py b/tests/test_titles.py index acb0f23..46a95a0 100644 --- a/tests/test_titles.py +++ b/tests/test_titles.py @@ -82,7 +82,7 @@ class TestTitles(testtools.TestCase): if "http://" in line or "https://" in line: continue # Allow lines which do not contain any whitespace - if re.match("\s*[^\s]+$", line): + if re.match(r"\s*[^\s]+$", line): continue self.assertTrue( len(line) < 80,