From 42a2a1833e7525703cc6964fdd155020043e404f Mon Sep 17 00:00:00 2001 From: Vladyslav Drok Date: Fri, 11 Nov 2016 14:25:23 +0200 Subject: [PATCH] Add bonding configuration fields to portgroups This change: * adds mode and properties fields to the portgroup objects and describes the changes to nova needed to use them; * makes portgroup.address field not mandatory; * brings some updates to CLI & OSC CLI; * makes this spec dependent on attach/detach interfaces work; * contains some minor cosmetic changes for more readability. Related-bug: #1618754 Change-Id: I7d1c72e80826d077e36e60443aa7573a1c81646a --- specs/approved/ironic-ml2-integration.rst | 464 ++++++++++++---------- 1 file changed, 260 insertions(+), 204 deletions(-) diff --git a/specs/approved/ironic-ml2-integration.rst b/specs/approved/ironic-ml2-integration.rst index e72644d5..a7f55e8d 100644 --- a/specs/approved/ironic-ml2-integration.rst +++ b/specs/approved/ironic-ml2-integration.rst @@ -66,8 +66,7 @@ port. In this way, each portgroup can represent a LAG for which all member switch ports will belong to the same network and to the same network segment. The node should first be placed on a provisioning network and then onto the -tenant network as specified in the `network-provider spec -`_. To help +tenant network as specified in the network-provider spec [1]_. To help facilitate this there will be a variable stored in the binding profile to record whether a port binding has been requested. This will allow Ironic to defer binding until Ironic is able to populate the extra information that is @@ -120,16 +119,41 @@ addition to the base object it will have the following fields: * name * node_id * address +* mode +* properties * extra * internal_info * standalone_ports_supported -The 'address' field represents the MAC address for bonded NICs of the bare -metal server. The 'extra' field can be used to hold any additional information +The ``address`` field represents the MAC address for bonded NICs of the bare +metal server. It is optional, as its value is highly dependent on the instance +OS. In case of using ironic through nova, the ``address`` gets populated by +whatever value is generated by neutron for the VIF's address representing this +portgroup, if there is one. This happens during every VIF attach call. +In case of standalone mode, it can be always ``None``, as it should be +configured on the instance through the configdrive. + +The ``mode`` field is used to specify the bond mode. If not provided in the +portgroup creation request, its default value is determined by the +``[DEFAULT]default_portgroup_mode`` configuration option, which in turn has a +default value of ``active-backup``. For a portgroup that was created in an +older API version, this configuration value will be used as the value for that +portgroup's mode. ``active-backup`` is chosen as the default because this mode +does not require any additional configuration on the switch side. + +The ``properties`` field is a dictionary that can contain any parameters +needed to configure the portgroup. + +For additional information about ``mode`` and ``properties`` fields' contents, +see `linux kernel bonding documentation `_. + +The ``extra`` field can be used to hold any additional information that operators or developers want to store in the portgroup. -The 'internal_info' field is used to store internal metadata. This field is + +The ``internal_info`` field is used to store internal metadata. This field is read-only. -The 'standalone_ports_supported' indicates whether ports that are members of + +The ``standalone_ports_supported`` indicates whether ports that are members of this portgroup can be used as stand-alone ports. The Ironic port object will then have the following fields added to support @@ -161,47 +185,31 @@ The following port binding related information needs to be passed to Neutron: | host_id | This should be set to the Ironic node uuid. | +------------------------+--------------------------------------------------+ -A JSON example to describe the structure is: +A JSON example to describe the structure is:: -{"port": - { + { + "port": { , - "vnic_type": "baremetal", - "host_id": , - "binding:profile": { - - "local_link_information": [ - { - "switch_id": xxx, - - "port_id": xxx, - - "switch_info": zzz, - - - - }, - - { - "switch_id": xxx, - - "port_id": yyy, - - "switch_info": zzz, - - - - } ] - - - + "local_link_information": [ + { + "switch_id": xxx, + "port_id": xxx, + "switch_info": zzz, + + }, + { + "switch_id": xxx, + "port_id": yyy, + "switch_info": zzz, + + } + ] + } - } - } @@ -249,7 +257,11 @@ The portgroup object is proposed with the following fields and data types: +----------------------------+-------------------------+ | node_id | int_or_none | +----------------------------+-------------------------+ -| address | str | +| address | str_or_none | ++----------------------------+-------------------------+ +| mode | str_or_none | ++----------------------------+-------------------------+ +| properties | dict_or_none | +----------------------------+-------------------------+ | extra | dict_or_none | +----------------------------+-------------------------+ @@ -262,6 +274,13 @@ The portgroup object is proposed with the following fields and data types: | updated_at | datetime_or_str_or_none | +----------------------------+-------------------------+ +.. note:: + While ``mode`` attribute of the portgroup object has type str_or_none, its + value can not be ``None``, unless the database was changed manually. It + gets populated either during the database migration, or during portgroup + creation (if not specified explicitly). In both cases it is set to the + ``[DEFAULT]default_portgroup_mode`` configuration option value. + State Machine Impact -------------------- @@ -736,11 +755,28 @@ model: "rel": "bookmark" } ], + "mode": "802.3ad", + "name": "node1_portgroup1", "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", + "ports": [ + { + "href": "http://127.0.0.1:6385/v1/portgroups/ + 6eb02b44-18a3-4659-8c0b-8d2802581ae4/ports", + "rel": "self" + }, + { + "href": "http://127.0.0.1:6385/portgroups/ + 6eb02b44-18a3-4659-8c0b-8d2802581ae4/ports", + "rel": "bookmark" + } + ], + "properties": { + "bond_xmit_hash_policy": "layer3+4", + "bond_miimon": 100 + }, "standalone_ports_supported": true, "updated_at": "2015-05-15T09:04:12.011844+00:00", - "uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4", - "name": "node1_portgroup1" + "uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4" } * JSON schema definition of PortgroupCollection: @@ -792,6 +828,7 @@ model: "rel": "bookmark" } ], + "mode": "802.3ad", "name": "node1_portgroup1", "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", "ports": [ @@ -806,6 +843,10 @@ model: "rel": "bookmark" } ], + "properties": { + "bond_xmit_hash_policy": "layer3+4", + "bond_miimon": 100 + }, "standalone_ports_supported": true, "updated_at": "2016-11-04T17:46:09+00:00", "uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4" @@ -831,14 +872,14 @@ by the caller and the response. :: { - "address": "fe:54:00:77:07:d9", - "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", - "local_link_connection": { - "switch_id": "0a:1b:2c:3d:4e:5f", - "port_id": "Ethernet3/1", - "switch_info": "switch1", - }, - "pxe_enabled": true + "address": "fe:54:00:77:07:d9", + "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", + "local_link_connection": { + "switch_id": "0a:1b:2c:3d:4e:5f", + "port_id": "Ethernet3/1", + "switch_info": "switch1", + }, + "pxe_enabled": true } * Response 201 with body: @@ -846,32 +887,32 @@ by the caller and the response. :: { - "address": "fe:54:00:77:07:d9", - "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", - "local_link_connection": { - "switch_id": "0a:1b:2c:3d:4e:5f", - "port_id": "Ethernet3/1", - "switch_info": "switch1", + "address": "fe:54:00:77:07:d9", + "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", + "local_link_connection": { + "switch_id": "0a:1b:2c:3d:4e:5f", + "port_id": "Ethernet3/1", + "switch_info": "switch1", + }, + "pxe_enabled": true + "created_at": "2015-05-12T10:00:00.529243+00:00", + "extra": { }, - "pxe_enabled": true - "created_at": "2015-05-12T10:00:00.529243+00:00", - "extra": { - }, - "links": [ - { - "href": "http://localhost:6385/v1/ports/ - 1004e542-2f9f-4d9b-b8b9-5b719fa6613f", - "rel": "self" - }, - { - "href": "http://localhost:6385/ports/ - 1004e542-2f9f-4d9b-b8b9-5b719fa6613f", - "rel": "bookmark" - } - ], - "updated_at": null, - "uuid": "1004e542-2f9f-4d9b-b8b9-5b719fa6613f", - "portgroup_uuid": null, + "links": [ + { + "href": "http://localhost:6385/v1/ports/ + 1004e542-2f9f-4d9b-b8b9-5b719fa6613f", + "rel": "self" + }, + { + "href": "http://localhost:6385/ports/ + 1004e542-2f9f-4d9b-b8b9-5b719fa6613f", + "rel": "bookmark" + } + ], + "updated_at": null, + "uuid": "1004e542-2f9f-4d9b-b8b9-5b719fa6613f", + "portgroup_uuid": null, } * Example of portgroup create. @@ -881,10 +922,10 @@ by the caller and the response. :: { - "address": "fe:54:00:77:07:d9", - "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", - "standalone_ports_supported": true, - "name": "node1_portgroup1" + "address": "fe:54:00:77:07:d9", + "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", + "standalone_ports_supported": true, + "name": "node1_portgroup1" } * Response 201 with body: @@ -892,28 +933,42 @@ by the caller and the response. :: { - "address": "fe:54:00:77:07:d9", - "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", - "name": "node1_portgroup1" - "created_at": "2015-05-12T10:10:00.529243+00:00", - "extra": { - }, - "internal_info": {}, - "links": [ - { - "href": "http://localhost:6385/v1/portgroups/ - 6eb02b44-18a3-4659-8c0b-8d2802581ae4", - "rel": "self" + "address": "fe:54:00:77:07:d9", + "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", + "name": "node1_portgroup1" + "created_at": "2015-05-12T10:10:00.529243+00:00", + "extra": { }, - { - "href": "http://localhost:6385/portgroups/ - 6eb02b44-18a3-4659-8c0b-8d2802581ae4", - "rel": "bookmark" - } - ], - "standalone_ports_supported": true, - "updated_at": null, - "uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4", + "internal_info": {}, + "links": [ + { + "href": "http://localhost:6385/v1/portgroups/ + 6eb02b44-18a3-4659-8c0b-8d2802581ae4", + "rel": "self" + }, + { + "href": "http://localhost:6385/portgroups/ + 6eb02b44-18a3-4659-8c0b-8d2802581ae4", + "rel": "bookmark" + } + ], + "mode": null, + "ports": [ + { + "href": "http://127.0.0.1:6385/v1/portgroups/ + 6eb02b44-18a3-4659-8c0b-8d2802581ae4/ports", + "rel": "self" + }, + { + "href": "http://127.0.0.1:6385/portgroups/ + 6eb02b44-18a3-4659-8c0b-8d2802581ae4/ports", + "rel": "bookmark" + } + ], + "properties": {}, + "standalone_ports_supported": true, + "updated_at": null, + "uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4", } * Example of port update. @@ -932,32 +987,32 @@ by the caller and the response. :: { - "address": "fe:54:00:77:07:d9", - "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", - "local_link_connection": { - "switch_id": "0a:1b:2c:3d:4e:5f", - "port_id": "Ethernet3/1", - "switch_info": "switch1", + "address": "fe:54:00:77:07:d9", + "node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd", + "local_link_connection": { + "switch_id": "0a:1b:2c:3d:4e:5f", + "port_id": "Ethernet3/1", + "switch_info": "switch1", + }, + "pxe_enabled": true + "created_at": "2015-05-12T10:00:00.529243+00:00", + "extra": { }, - "pxe_enabled": true - "created_at": "2015-05-12T10:00:00.529243+00:00", - "extra": { - }, - "links": [ - { - "href": "http://localhost:6385/v1/ports/ - 1004e542-2f9f-4d9b-b8b9-5b719fa6613f", - "rel": "self" - }, - { - "href": "http://localhost:6385/ports/ - 1004e542-2f9f-4d9b-b8b9-5b719fa6613f", - "rel": "bookmark" - } - ], - "updated_at": "2015-05-12T10:20:00.529243+00:00", - "uuid": "1004e542-2f9f-4d9b-b8b9-5b719fa6613f", - "portgroup_uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4", + "links": [ + { + "href": "http://localhost:6385/v1/ports/ + 1004e542-2f9f-4d9b-b8b9-5b719fa6613f", + "rel": "self" + }, + { + "href": "http://localhost:6385/ports/ + 1004e542-2f9f-4d9b-b8b9-5b719fa6613f", + "rel": "bookmark" + } + ], + "updated_at": "2015-05-12T10:20:00.529243+00:00", + "uuid": "1004e542-2f9f-4d9b-b8b9-5b719fa6613f", + "portgroup_uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4", } * Note that the port update API should support updating the portgroup_id @@ -1021,6 +1076,10 @@ Client (CLI) impact The python-ironicclient and OSC would need updated to support the new portgroups APIs. +In the commands below, ```` means that this placeholder can contain +both portgroup UUID or name. ```` can contain only portgroup +UUID. + Example usage of the new methods: * For ports, the CLI would support port creation with new optional @@ -1031,50 +1090,54 @@ Example usage of the new methods: "ironic" CLI: * ironic port-create -a
-n [-e ] - [--local-link-connection ] - [--portgroup-uuid ] [--pxe-enabled ] - - * ironic port-update port_uuid replace portgroup_uuid= - - * ironic port-list [--detail] [--address ] - [--portgroup-uuid ] + [--local-link-connection ] + [--portgroup ] [--pxe-enabled ] + * ironic port-update add portgroup_uuid= + --local-link-connection --pxe-enabled "openstack baremetal" CLI: * openstack baremetal port create --node - [--local-link-connection ] - [--portgroup-uuid ] - [--pxe-enabled ] -
+ [--local-link-connection ] [--port-group ] + [--pxe-enabled ]
- * openstack baremetal port set [--portgroup-uuid ] + * openstack baremetal port set [--port-group ] + [--local-link-connection ] [--pxe-enabled ] - * openstack baremetal port list --address ] - [--node ] [--portgroup-uuid ] + * openstack baremetal port list [--address ] + [--node | --port-group ] * For portgroups, the CLI would support the following new methods: "ironic" CLI: - * ironic portgroup-create --node [--name ] - [--address ] [-e ] + * ironic portgroup-create --node [--address ] + [--name ] [-e ] + [--standalone-ports-supported ] [-m ] + [-p ] - * ironic portgroup-delete + * ironic portgroup-delete [ ...] - * ironic portgroup-list [--detail] [--node ] - [--address ] - [--limit ] [--marker ] + * ironic portgroup-list [--detail | --fields [ ...]] + [--node ] [--address ] [--limit ] + [--marker ] [--sort-key ] [--sort-dir ] - * ironic portgroup-show [--address] + * ironic portgroup-port-list + [--detail | --fields [ ...]] + [--limit ] [--marker ] [--sort-key ] + [--sort-dir ] - * is the UUID of the portgroup (or MAC address if --address is - specified) + * ironic portgroup-show [--address] [--fields [ ...]] + - * ironic portgroup-update + * is the UUID or name of the portgroup (or MAC address if + --address is specified) + + * ironic portgroup-update [ ... ] * is add, remove or replace. @@ -1089,30 +1152,33 @@ Example usage of the new methods: "openstack baremetal" CLI: - * openstack baremetal portgroup create --node [--name NAME] + * openstack baremetal port group create --node [--name ] [--extra ] [--support-standalone-ports | --unsupport-standalone-ports] -
+ [--mode ] [--properties ] + [--address ] - * openstack baremetal portgroup delete [ ...] + * openstack baremetal port group delete [ ...] - * openstack baremetal portgroup list [--marker ] + * openstack baremetal port group list [--marker ] [--address ] [--node ] [--sort [:]] [--long | --fields [ ...]] - * openstack baremetal portgroup show [--address] - [--fields [ ...]] - + * openstack baremetal port group show [--address] + [--fields [ ...]] - * openstack baremetal portgroup set [--address] [--name NAME] - [--node ] [--extra ] + * is the UUID or name of the portgroup (or MAC address if + --address is specified) + + * openstack baremetal port group set [--address ] + [--name ] [--node ] [--extra ] [--support-standalone-ports | --unsupport-standalone-ports] - [--fields [ ...]] + [--mode ] [--properties ] - * openstack baremetal portgroup unset [--name] [--extra ] - [--node ] + * openstack baremetal port group unset [--address] [--name] + [--extra ] [--properties key] * To add ports to a portgroup, the portgroup should first @@ -1149,8 +1215,7 @@ extracting ``vif_port_id`` from the ``extra`` attributes of Ironic ports. This method should be updated if vifs are bound to portgroups as well as ports. -The complementary `network-provider spec -`_ provides +The complementary network-provider spec [1]_ provides details regarding the workflow of the network flip and the point at which the binding profile will be passed to Neutron to bind the port. @@ -1159,31 +1224,17 @@ the binding profile will be passed to Neutron to bind the port. Nova driver impact ------------------ -There will be changes necessary to the Nova driver. Proposed changes are: +As this work depends on the attach/detach interface work [2]_, the only thing +that needs to be changed to fully support portgroups is configdrive +generation. -* To enable the mapping between Neutron ports and Ironic ports and - portgroups. - - The Ironic Nova driver has methods ``macs_for_instance``, - ``dhcp_options_for_instance``, ``extra_options_for_instance`` and - ``plug_vifs``. Currently Nova puts a network on one port at random - see - `ports cannot be mapped to networks - `_. This bug has high - priority and the issue is being addressed. Once addressed, these methods - should determine the number of Neutron ports that are - created as well as the mapping between Neutron and Ironic ports. These - methods should be updated to not only account for Ironic ports but also - Ironic portgroups. The selection process would be: - - * Select all Ironic ports that do not belong to Ironic portgroups - (possible if the Ironic port list API returns portgroup_uuid as - standard, as suggested in the above section) - - * Select all Ironic portgroups - - This modified functionality could be implemented using a new config flag in - Nova to allow toggling between the old and the new methods. The flag could - help de-couple the upgrading of Nova and of Ironic. +Nova will call into ironic to get the list of ports of each portgroup that has +a VIF associated with it, along with portgroup ``mode`` and ``properties`` +fields (see `Data model impact`_ section), and update the network metadata +with the needed information. When the contents of the ``properties`` +dictionary gets passed to the config drive builder in nova, we will ensure +that ``bond_`` prefix is prepended to all key names, so that these keys are +not ignored by cloud-init when reading the config drive. Ramdisk impact -------------- @@ -1210,8 +1261,7 @@ Using the binding profile to enable flipping between provisioning and tenant networks means there will be no support for PXE booting after deploy (i.e. local disk installation only). How to allow operators to deploy instances using either net-boot or local boot using the same Ironic conductor should be -discussed in the complementary `network-provider spec -`_. +discussed in the complementary network-provider spec [1]_. Scalability impact ------------------ @@ -1236,9 +1286,10 @@ Deployers will need to deploy an ML2 mechanism driver that supports connecting baremetal resources to Neutron networks. If using Nova, deployers will need to deploy a version of Nova that supports -this feature. Deployers will need to set a flag in the Nova config file to -turn this new feature on or off, which would be important when upgrading -Nova and Ironic. +this feature. + +Deployers may want to set the ``[DEFAULT]default_portgroup_mode`` configuration +option to match their environment. Its default value is ``active-backup``. Deployers should be aware that automated upgrade or migration for already-provisioned nodes is not supported. Deployers should follow this @@ -1247,12 +1298,9 @@ new feature: * Upgrade the OpenStack services. -* Update the flag in the Nova config file to turn this feature on. - * Move node into the MANAGEABLE state. -* Update node driver field (see `network-provider spec - `_). +* Update node driver field (see the network-provider spec [1]_). * Create Ironic portgroups. @@ -1286,6 +1334,10 @@ Assignee(s) * sukhdev-8 +* vsaienko + +* vdrok + Work Items ---------- @@ -1314,8 +1366,10 @@ Work Items Dependencies ============ -Network flip is dependent on `network-provider spec -`_. +Network flip is dependent on the network-provider spec [1]_. + +Nova `changes `_ +are dependent on attach/detach interfaces work [2]_. VLAN provisioning on switch(es) is dependent on ML2 driver functionality being developed to support this feature. @@ -1330,8 +1384,7 @@ New tests will need to be written to test the new APIs and database updates. Simulation of connecting real hardware to real switches for testing -purposes is described in `network-provider spec -`_. +purposes is described in the network-provider spec [1]_. Upgrades and Backwards Compatibility @@ -1350,13 +1403,16 @@ This feature will be fully documented. References ========== +.. [1] http://specs.openstack.org/openstack/ironic-specs/specs/6.1/network-provider.html + +.. [2] http://specs.openstack.org/openstack/ironic-specs/specs/approved/interface-attach-detach-api.html + Discussions on the topic include: * https://etherpad.openstack.org/p/YVR-neutron-ironic * https://etherpad.openstack.org/p/liberty-ironic-network-isolation -* Logs from https://wiki.openstack.org/wiki/Meetings/Ironic-neutron +* https://etherpad.openstack.org/p/network-interface-vifs-configdrive -* The network provider spec enabling the network flip between provisioning - and tenant network: https://review.openstack.org/#/c/187829 +* Logs from https://wiki.openstack.org/wiki/Meetings/Ironic-neutron