openstack
/
openstack-ansible-os_neutron
Code Issues Proposed changes

Separate OVN gateway functions from ovn-controllers 

This patch aims for the following:

- Update docs for OVN to expand on supported scenarios
- Split out the OVN gateway chassis from the OVN controller group.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/862924
Change-Id: I14859a19f386676fd687ea601f2d31298cf35e84
This commit is contained in:
James Denton 2022-10-27 22:06:29 -05:00
parent c79d3c2e4c
commit 3d9a20b69b
2 changed files with 229 additions and 224 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

450
doc/source/app-ovn.rst
View File

@ -3,30 +3,15 @@ Scenario - Open Virtual Network (OVN)
========================================
Overview
~~~~~~~~
--------
Operators can choose to utilize the Open Virtual Network (OVN) mechanism
driver instead of Linux bridges or plain Open vSwitch for the Neutron ML2
plugin. This offers the possibility to deploy virtual networks and routers
using OVN with Open vSwitch, which replaces the agent-based model used by
the aforementioned architectures. This document outlines how to set it up in
your environment.
driver (ML2/OVN) instead of ML2/LXB or ML2/OVS. This offers the possibility
of deploying virtual networks and routers using OVN with Open vSwitch, which
replaces the agent-based models used by the legacy ML2/LXB and ML2/OVS
architectures. This document outlines how to set it up in your environment.
.. warning::
The current implementation of OVN in OpenStack-Ansible is experimental
and not production ready.
The following architectural assumptions have been made:
* Each compute node will act as an OVN controller
* Each compute node is eligible to serve as an OVN gateway node
.. note::
Physical VTEP integration is not supported.
Recommended reading
Recommended Reading
~~~~~~~~~~~~~~~~~~~
Since this is an extension of the basic Open vSwitch scenario, it is worth
@ -34,160 +19,154 @@ reading that scenario to get some background. It is also recommended to be
familiar with OVN and networking-ovn projects and their configuration.
* `Scenario: Open vSwitch <app-openvswitch.html>`_
* `OVN Architecture <http://www.openvswitch.org/support/dist-docs/ovn-architecture.7.html>`_
* `Networking-ovn <https://github.com/openstack/networking-ovn>`_
* `OVN Architecture Docs <https://www.ovn.org/en/architecture/>`_
* `OpenStack Integration with OVN <https://docs.openstack.org/networking-ovn/latest/>`_
* `OVN OpenStack Tutorial <https://docs.ovn.org/en/stable/tutorials/ovn-openstack.html>`_
Prerequisites
~~~~~~~~~~~~~
* Open vSwitch >= 2.10.0
* Open vSwitch >= 2.17.0
* A successful deployment of OVN requires a dedicated network
interface be attached to the OVS provider bridge. This is not
handled automatially and may require changes to the network
interface configuration file.
Inventory Architecture
----------------------
While OVN itself supports many different configurations, Neutron and ``networking-ovn`` leverage
specific functionality to provide virtual routing capabilities to an OpenStack-based Cloud.
OpenStack-Ansible separates OVN-related services and functions into three groups:
- neutron_ovn_northd
- neutron_ovn_controller
- neutron_ovn_gateway
The ``neutron_ovn_northd`` group is used to specify which host(s) will contain
the OVN northd daemon responsible for translating the high-level OVN
configuration into logical configuration consumable by daemons such as
``ovn-controller``. In addition, these nodes host the OVN Northbound and
OVN Southbound databases the ``ovsdb-server`` services. Members of this group
are typically the controller nodes hosting the Neutron APIs (``neutron-server``).
The ``neutron_ovn_controller`` group is used to specify which host(s) will run
the local ``ovn-controller`` daemon for OVN, which registers the local chassis
and VIFs to the OVN Southbound database and converts logical flows to
physical flows on the local hypervisor node. Members of this group are
typically the compute nodes hosting virtual machine instances (``nova-compute``).
The ``neutron_ovn_gateway`` group is used to specify which hosts are eligible to
act as an OVN Gateway Chassis, which is a node running ``ovn-controller`` that
is capable of providing external (north/south) connectivity to the tenant traffic.
This is essentially a node(s) capable of hosting the logical router performing
SNAT and DNAT (Floating IP) translations. East/West traffic flow is not limited
to a gateway chassis and is performed between an OVN chassis nodes.
When planning out your architecture, it is important to determine early if you
want to centralize OVN gateway chassis functions to a subset of nodes or
across all compute nodes. Centralizing north/south routing to a set of dedicated
network or gateway nodes is reminiscent of the legacy network node model. Enabling
all compute nodes as gateway chassis will narrow the failure domain and potential
bottlenecks at the cost of ensuring the computes can connect to the provider
networks.
The following section will describe how to configure your inventory to meet certain
deployment scenarios.
Deployment Scenarios
--------------------
OpenStack-Ansible supports the following common deployment scenarios:
- Collapsed Network/Gateway Nodes
- Collapsed Compute/Gateway Nodes
- Standalone Gateway Nodes
In an OpenStack-Ansible deployment, members of the ``network_hosts`` group are
considered members of the ``neutron_ovn_northd`` group and run OVN northd-related
services, while members of the ``compute_hosts`` group are considered members of
the ``neutron_ovn_controller`` group and run OVN controller-related services. It is
up to the deployer to dictate which nodes are considered "OVN Gateway Chassis" nodes
by using the ``network-gateway_hosts`` inventory group in ``openstack_user_config.yml``.
In ``openstack_user_config.yml``, specify the hosts or aliases that will run the
``ovn-controller`` service and act as an OVN Gateway Chassis, like so:
.. code-block:: yaml
network-gateway_hosts:
network-node1:
ip: 172.25.1.21
network-node2:
ip: 172.25.1.22
network-node3:
ip: 172.25.1.23
Existing inventory aliases can also be used. In the following example, members of
the ``infrastructure_hosts`` group are also network hosts and will serve as
OVN Gateway Chassis nodes:
.. code-block:: yaml
network-gateway_hosts: *infrastructure_hosts
In the following example, members of the ``compute_hosts`` group running the
``ovn-controller`` service will also serve as OVN Gateway Chassis nodes:
.. code-block:: yaml
network-gateway_hosts: *compute_hosts
Lastly, specific hosts can also be targeted:
.. code-block:: yaml
network-gateway_hosts:
compute5:
ip: 172.25.1.55
compute10:
ip: 172.25.1.60
compute15:
ip: 172.25.1.65
compute20:
ip: 172.25.1.70
compute25:
ip: 172.25.1.75
OpenStack-Ansible user variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Create a group var file for your network hosts
``/etc/openstack_deploy/group_vars/network_hosts``. It has to include:
.. code-block:: yaml
# Ensure the openvswitch kernel module is loaded
openstack_host_specific_kernel_modules:
- name: "openvswitch"
pattern: "CONFIG_OPENVSWITCH"
Copy the neutron environment overrides to
``/etc/openstack_deploy/env.d/neutron.yml`` to disable the creation of the
neutron agents container and implement the ``neutron_ovn_northd_container``
hosts group containing all network nodes:
.. code-block:: yaml
component_skel:
neutron_ovn_controller:
belongs_to:
- neutron_all
neutron_ovn_northd:
belongs_to:
- neutron_all
container_skel:
neutron_agents_container:
contains: {}
neutron_ovn_northd_container:
belongs_to:
- network_containers
contains:
- neutron_ovn_northd
- neutron_sriov_nic_agent
Copy the nova environment overrides to
``/etc/openstack_deploy/env.d/nova.yml`` to implement the
``neutron_ovn_controller`` hosts group containing all compute nodes:
.. code-block:: yaml
container_skel:
nova_compute_container:
belongs_to:
- compute_containers
- kvm-compute_containers
- lxd-compute_containers
- qemu-compute_containers
contains:
- neutron_ovn_controller
- nova_compute
- neutron_sriov_nic_agent
properties:
is_metal: true
Specify provider network definitions in your
``/etc/openstack_deploy/openstack_user_config.yml`` that define
one or more Neutron provider bridges and related configuration:
.. note::
Bridges specified here will be created automatically. If
``network_interface`` is defined, the interface will be placed into
the bridge automatically. Only VLAN network types are supported at
this time.
.. code-block:: yaml
- network:
container_bridge: "br-privatenet"
container_type: "veth"
type: "vlan"
range: "101:200,301:400"
net_name: "private"
network_interface: "bond2"
group_binds:
- neutron_ovn_controller
- network:
container_bridge: "br-publicnet"
container_type: "veth"
type: "vlan"
range: "203:203,467:500"
net_name: "public"
network_interface: "bond1"
group_binds:
- neutron_ovn_controller
Specify an overlay network definition in your
``/etc/openstack_deploy/openstack_user_config.yml`` that defines
overlay network-related configuration:
.. note::
The bridge name should correspond to a pre-created Linux bridge.
Only GENEVE overlay network types are supported at this time.
.. code-block:: yaml
- network:
container_bridge: "br-vxlan"
container_type: "veth"
container_interface: "eth10"
ip_from_q: "tunnel"
type: "geneve"
range: "1:1000"
net_name: "geneve"
group_binds:
- neutron_ovn_controller
Set the following user variables in your
``/etc/openstack_deploy/user_variables.yml``:
To deploy OpenStack-Ansible using the ML2/OVN mechanism driver, set the
following user variables in the``/etc/openstack_deploy/user_variables.yml``
file:
.. code-block:: yaml
neutron_plugin_type: ml2.ovn
neutron_plugin_base:
- neutron.services.ovn_l3.plugin.OVNL3RouterPlugin
- ovn-router
neutron_ml2_drivers_type: "vlan,local,geneve"
neutron_ml2_drivers_type: "vlan,local,geneve,flat"
The overrides are instructing Ansible to deploy the OVN mechanism driver and
associated OVN components. This is done by setting ``neutron_plugin_type``
to ``ml2.ovn``.
The ``neutron_plugin_base`` override instructions Neutron to use OVN for
routing functions rather than the standard L3 agent model.
The ``neutron_plugin_base`` override enables Neutron to use OVN for
routing functions.
The ``neutron_ml2_drivers_type`` override provides support for all type
drivers supported by OVN.
If provider network overrides are needed on a global or per-host basis,
the following format can be used in ``user_variables.yml`` or per-host
in ``openstack_user_config.yml``.
Provider network overrides can be specified on a global or per-host basis,
and the following format can be used in ``user_variables.yml`` or per-host
in ``openstack_user_config.yml`` or host vars.
.. note::
These overrides are not normally required.
When ``network_interface_mappings`` are defined, the playbooks will attempt
to connect the mapped interface to the respective OVS bridge. Omitting
``network_interface_mappings`` will require the operator to connect the
interface to the bridge manually using the ``ovs-vsctl add-port`` command.
.. code-block:: yaml
@ -231,8 +210,8 @@ Create a group var file for neutron server
ovn:
enable_distributed_floating_ip: True
Open Virtual Network (OVN) commands
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Useful Open Virtual Network (OVN) Commands
------------------------------------------
The following commands can be used to provide useful information about the
state of Open vSwitch networking and configurations.
@ -242,22 +221,25 @@ The ``ovs-vsctl list open_vswitch`` command provides information about the
.. code-block:: console
root@aio1:~# ovs-vsctl list open_vswitch
_uuid : 855c820b-c082-4d8f-9828-8cab01c6c9a0
bridges : [37d3bd82-d436-474e-89b7-705aea634d7d, a393b2f6-5c3d-4ccd-a2f9-e9817391612a]
cur_cfg : 14
root@mnaio-controller1:~# ovs-vsctl list open_vswitch
_uuid : 7f96baf2-d75e-4a99-bb19-ca7138fc14c2
bridges : []
cur_cfg : 1
datapath_types : [netdev, system]
db_version : "7.15.1"
external_ids : {hostname="aio1", ovn-bridge-mappings="vlan:br-provider", ovn-encap-ip="172.29.240.100", ovn-encap-type="geneve,vxlan", ovn-remote="tcp:172.29.236.100:6642", rundir="/var/run/openvswitch", system-id="11af26c6-9ec1-4cf7-bf41-2af45bd59b03"}
iface_types : [geneve, gre, internal, lisp, patch, stt, system, tap, vxlan]
datapaths : {}
db_version : "8.3.0"
dpdk_initialized : false
dpdk_version : none
external_ids : {hostname=mnaio-controller1, rundir="/var/run/openvswitch", system-id="a67926f2-9543-419a-903d-23e2aa308368"}
iface_types : [bareudp, erspan, geneve, gre, gtpu, internal, ip6erspan, ip6gre, lisp, patch, stt, system, tap, vxlan]
manager_options : []
next_cfg : 14
next_cfg : 1
other_config : {}
ovs_version : "2.9.0"
ovs_version : "2.17.2"
ssl : []
statistics : {}
system_type : ubuntu
system_version : "16.04"
system_version : "20.04"
The ``ovn-sbctl show`` command provides information related to southbound
connections. If used outside the ovn_northd container, specify the
@ -265,83 +247,107 @@ connection details:
.. code-block:: console
root@aio1-neutron-ovn-northd-container-57a6f1a9:~# ovn-sbctl show
Chassis "11af26c6-9ec1-4cf7-bf41-2af45bd59b03"
hostname: "aio1"
root@mnaio-controller1:~# ovn-sbctl show
Chassis "5335c34d-9233-47bd-92f1-fc7503270783"
hostname: mnaio-compute1
Encap geneve
ip: "172.25.1.31"
options: {csum="true"}
Encap vxlan
ip: "172.29.240.100"
ip: "172.25.1.31"
options: {csum="true"}
Port_Binding "852530b5-1247-4ec2-9c39-8ae0752d2144"
Chassis "ff66288c-5a7c-41fb-ba54-6c781f95a81e"
hostname: mnaio-compute2
Encap vxlan
ip: "172.25.1.32"
options: {csum="true"}
Encap geneve
ip: "172.29.240.100"
options: {csum="true"}
root@aio1:~# ovn-sbctl --db=tcp:172.29.236.100:6642 show
Chassis "11af26c6-9ec1-4cf7-bf41-2af45bd59b03"
hostname: "aio1"
Encap vxlan
ip: "172.29.240.100"
ip: "172.25.1.32"
options: {csum="true"}
Chassis "cb6761f4-c14c-41f8-9654-16f3fc7cc7e6"
hostname: mnaio-compute3
Encap geneve
ip: "172.29.240.100"
ip: "172.25.1.33"
options: {csum="true"}
Encap vxlan
ip: "172.25.1.33"
options: {csum="true"}
Port_Binding cr-lrp-022933b6-fb12-4f40-897f-745761f03186
The ``ovn-nbctl show`` command provides information about networks known
to OVN and demonstrates connectivity between the northbound database
and neutron-server.
The ``ovn-nbctl show`` command provides information about networks, ports,
and other objects known to OVN and demonstrates connectivity between the
northbound database and neutron-server.
.. code-block:: console
root@aio1-neutron-ovn-northd-container-57a6f1a9:~# ovn-nbctl show
switch 5e77f29e-5dd3-4875-984f-94bd30a12dc3 (neutron-87ec5a05-9abe-4c93-89bd-c6d40320db87) (aka testnet)
port 65785045-69ec-49e7-82e3-b9989f718a9c
root@mnaio-controller1:~# ovn-nbctl show
switch 03dc4558-f83e-4531-b854-156292f1dbad (neutron-a6e65821-93e2-4521-9e31-37c35d52d953) (aka project-tenant-network)
port 852530b5-1247-4ec2-9c39-8ae0752d2144
addresses: ["fa:16:3e:d2:af:bf 10.3.3.49"]
port 624de478-7e75-472f-b867-e6f514790a81
addresses: ["fa:16:3e:bf:c0:c3 10.3.3.3", "unknown"]
port 1cca8ef3-d3c9-4307-a779-13348db5e647
addresses: ["fa:16:3e:4a:67:ed 10.3.3.4", "unknown"]
port 05e20b32-2933-414a-ba31-eac683d09ac2
addresses: ["fa:16:3e:bd:5d:e8 10.3.3.5", "unknown"]
port 5a2e35cb-178b-443b-9f15-4c6ec4db4ac7
type: router
router-port: lrp-5a2e35cb-178b-443b-9f15-4c6ec4db4ac7
port 2d52a2bf-ab37-4a18-87bd-8808a99c67d3
type: localport
addresses: ["fa:16:3e:68:a3:c8"]
addresses: ["fa:16:3e:30:b4:a0 10.3.3.2"]
switch 3e03d5f1-4cfe-4c61-bd4c-8a661634d77b (neutron-b0b4017f-a9d1-4923-af35-944b88b7a393) (aka flat-external-provider-network)
port 022933b6-fb12-4f40-897f-745761f03186
type: router
router-port: lrp-022933b6-fb12-4f40-897f-745761f03186
port 347a7d8d-fd0f-48be-be02-d603258f0a08
addresses: ["fa:16:3e:f4:6a:17 192.168.25.5", "unknown"]
port 29c83838-329d-4839-bddb-818c7e2e9bc7
addresses: ["fa:16:3e:a3:48:a8 192.168.25.3", "unknown"]
port 173c9ceb-4dd3-4268-aaa3-c7b0f693a557
type: localport
addresses: ["fa:16:3e:0c:37:ed 192.168.25.2"]
port 7a0175fd-ac09-4466-b3d0-26f696e3769c
addresses: ["fa:16:3e:ad:19:c2 192.168.25.4", "unknown"]
port provnet-525d3402-d582-49b4-b946-f28de8bbc615
type: localnet
addresses: ["unknown"]
router 5ebb0cdb-2026-4454-a32e-eb5425ae7296 (neutron-b0d6ca32-fda3-4fdc-b648-82c8bee303dc) (aka project-router)
port lrp-5a2e35cb-178b-443b-9f15-4c6ec4db4ac7
mac: "fa:16:3e:3a:1c:bb"
networks: ["10.3.3.1/24"]
port lrp-022933b6-fb12-4f40-897f-745761f03186
mac: "fa:16:3e:1f:cd:e9"
networks: ["192.168.25.242/24"]
gateway chassis: [cb6761f4-c14c-41f8-9654-16f3fc7cc7e6 ff66288c-5a7c-41fb-ba54-6c781f95a81e 5335c34d-9233-47bd-92f1-fc7503270783]
nat 79d8486c-8b5e-4d6c-a56f-9f0df115f77f
external ip: "192.168.25.242"
logical ip: "10.3.3.0/24"
type: "snat"
nat d338ccdf-d3c4-404e-b2a5-938d0c212e0d
external ip: "192.168.25.246"
logical ip: "10.3.3.49"
type: "dnat_and_snat"
The ``ovn-nbctl list Address_Set`` command provides information related to
security groups. If used outside the ovn_northd container, specify the
connection details:
.. code-block:: console
root@aio1-neutron-ovn-northd-container-57a6f1a9:~# ovn-nbctl list Address_Set
_uuid : 575b3015-f83f-4bd6-a698-3fe67e43bec6
addresses : []
external_ids : {"neutron:security_group_id"="199997c1-6f06-4765-89af-6fd064365c6a"}
name : "as_ip4_199997c1_6f06_4765_89af_6fd064365c6a"
_uuid : b6e211af-e52e-4c59-93ce-adf75ec14f46
addresses : []
external_ids : {"neutron:security_group_id"="199997c1-6f06-4765-89af-6fd064365c6a"}
name : "as_ip6_199997c1_6f06_4765_89af_6fd064365c6a"
root@aio1:~# ovn-nbctl --db=tcp:172.29.236.100:6641 list Address_Set
_uuid : 575b3015-f83f-4bd6-a698-3fe67e43bec6
addresses : []
external_ids : {"neutron:security_group_id"="199997c1-6f06-4765-89af-6fd064365c6a"}
name : "as_ip4_199997c1_6f06_4765_89af_6fd064365c6a"
_uuid : b6e211af-e52e-4c59-93ce-adf75ec14f46
addresses : []
external_ids : {"neutron:security_group_id"="199997c1-6f06-4765-89af-6fd064365c6a"}
name : "as_ip6_199997c1_6f06_4765_89af_6fd064365c6a"
Additional commands can be found in upstream OVN documentation.
Additional commands can be found in upstream OVN documentation and other
resources listed on this page.
Notes
~~~~~
-----
The ``ovn-controller`` service on compute nodes will check in as an agent
and can be observed using the ``openstack network agent list`` command:
The ``ovn-controller`` service will check in as an agent and can be observed
using the ``openstack network agent list`` command:
.. code-block:: console
root@aio1-utility-container-35bebd2a:~# openstack network agent list
+--------------------------------------+------------------------------+------+-------------------+-------+-------+----------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+------------------------------+------+-------------------+-------+-------+----------------+
| 4db288a6-8f8a-4153-b4b7-7eaf44f9e881 | OVN Controller Gateway agent | aio1 | n/a | :-) | UP | ovn-controller |
+--------------------------------------+------------------------------+------+-------------------+-------+-------+----------------+
The HAproxy implementation in use may not properly handle active/backup
failover for ovsdb-server with OVN. Work may be done to implement
pacemaker/corosync or wait for upstream active/active support.
+--------------------------------------+------------------------------+-------------------+-------------------+-------+-------+----------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+------------------------------+-------------------+-------------------+-------+-------+----------------------------+
| 5335c34d-9233-47bd-92f1-fc7503270783 | OVN Controller Gateway agent | mnaio-compute1 | | :-) | UP | ovn-controller |
| ff66288c-5a7c-41fb-ba54-6c781f95a81e | OVN Controller Gateway agent | mnaio-compute2 | | :-) | UP | ovn-controller |
| cb6761f4-c14c-41f8-9654-16f3fc7cc7e6 | OVN Controller Gateway agent | mnaio-compute3 | | :-) | UP | ovn-controller |
| 38206799-af64-589b-81b2-405f0cfcd198 | OVN Metadata agent | mnaio-compute1 | | :-) | UP | neutron-ovn-metadata-agent |
| 9e9b49c7-dd00-5f58-a3f5-22dd01f562c4 | OVN Metadata agent | mnaio-compute2 | | :-) | UP | neutron-ovn-metadata-agent |
| 72b1a6e2-4cca-570f-83a4-c05dcbbcc11f | OVN Metadata agent | mnaio-compute3 | | :-) | UP | neutron-ovn-metadata-agent |
+--------------------------------------+------------------------------+-------------------+-------------------+-------+-------+----------------------------+

3
tasks/providers/setup_ovs_ovn.yml
View File

@ -19,11 +19,10 @@
when:
- (neutron_services['neutron-ovn-controller']['group'] in group_names)
# (NOTE) This makes all computes eligible to be gateway nodes
- name: Set CMS Options for Gateway Scheduling
command: "ovs-vsctl set open . external-ids:ovn-cms-options=enable-chassis-as-gw"
when:
- neutron_services['neutron-ovn-controller']['group'] in group_names
- '"neutron_ovn_gateway" in group_names'
- name: Configure OVN Southbound Connection
command: "ovs-vsctl set open . external-ids:ovn-remote={{ neutron_ovn_sb_connection }}"