Revise deploy process documentation
Add more details about the node deployment process, when an instance is booted by nova. This change covers iscsi and direct deploy methods with PXE boot. This change also updates the sequence diagrams for deploy process. Co-Authored-By: Galyna Zholtkevych <galynazholtkevych1991@gmail.com> Change-Id: Icaec14d10e702627bab0763e40efa77131564dec
This commit is contained in:
Vladyslav Drok
parent
5a5d485c1d
commit
9280560752
|
|
@ -11,12 +11,22 @@
|
|||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import eventlet
|
||||
|
||||
# NOTE(dims): monkey patch subprocess to prevent failures in latest eventlet
|
||||
# See https://github.com/eventlet/eventlet/issues/398
|
||||
try:
|
||||
eventlet.monkey_patch(subprocess=True)
|
||||
except TypeError:
|
||||
pass
|
||||
|
||||
# -- General configuration ----------------------------------------------------
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
|
||||
extensions = ['sphinx.ext.autodoc',
|
||||
'sphinx.ext.viewcode',
|
||||
'sphinx.ext.graphviz',
|
||||
'sphinxcontrib.httpdomain',
|
||||
'sphinxcontrib.pecanwsme.rest',
|
||||
'sphinxcontrib.seqdiag',
|
||||
|
|
@ -95,7 +105,7 @@ exclude_patterns = ['api/ironic_tempest_plugin.*']
|
|||
# Ignore the following warning: WARNING: while setting up extension
|
||||
# wsmeext.sphinxext: directive 'autoattribute' is already registered,
|
||||
# it will be overridden.
|
||||
suppress_warnings = [ 'app.add_directive']
|
||||
suppress_warnings = ['app.add_directive']
|
||||
|
||||
# -- Options for HTML output --------------------------------------------------
|
||||
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 76 KiB |
File diff suppressed because it is too large
Load Diff
|
Before Width: | Height: | Size: 155 KiB |
|
|
@ -183,111 +183,199 @@ These pre-requisites must be met before the deployment process:
|
|||
+ user-image-initrd
|
||||
- Hardware to be enrolled via Ironic RESTful API service.
|
||||
|
||||
.. figure:: ../images/deployment_steps.png
|
||||
:alt: Deployment Steps
|
||||
|
||||
Deploy Process
|
||||
-----------------
|
||||
--------------
|
||||
|
||||
This describes a typical ironic node deployment using PXE and the Ironic
|
||||
Python Agent (IPA). Depending on the ironic driver interfaces used, some of the
|
||||
steps might be marginally different, however the majority of them will remain
|
||||
the same.
|
||||
|
||||
#. A boot instance request comes in via the Nova API, through the message
|
||||
queue to the Nova scheduler.
|
||||
|
||||
#. Nova scheduler applies filter and finds the eligible compute node. Nova
|
||||
scheduler uses flavor extra_specs detail such as 'cpu_arch',
|
||||
'baremetal:deploy_kernel_id', 'baremetal:deploy_ramdisk_id' etc to match
|
||||
the target physical node.
|
||||
#. Nova scheduler applies filters and finds the eligible hypervisor. The nova
|
||||
scheduler also uses the flavor's ``extra_specs``, such as ``cpu_arch``, to
|
||||
match the target physical node.
|
||||
|
||||
#. A spawn task is placed by the driver which contains all information such
|
||||
as which image to boot from etc. It invokes the driver.spawn from the
|
||||
virt layer of Nova compute.
|
||||
#. Nova compute manager claims the resources of the selected hypervisor.
|
||||
|
||||
#. Information about the bare metal node is retrieved from the bare metal
|
||||
database and the node is reserved.
|
||||
#. Nova compute manager creates (unbound) tenant virtual interfaces (VIFs) in
|
||||
the Networking service according to the network interfaces requested in the
|
||||
nova boot request. A caveat here is, the MACs of the ports are going to be
|
||||
randomly generated, and will be updated when the VIF is attached to some
|
||||
node to correspond to the node network interface card's (or bond's) MAC.
|
||||
|
||||
#. Images from Glance are pulled down to the local disk of the Ironic
|
||||
conductor servicing the bare metal node.
|
||||
#. A spawn task is created by the nova compute which contains all
|
||||
the information such as which image to boot from etc. It invokes the
|
||||
``driver.spawn`` from the virt layer of Nova compute. During the spawn
|
||||
process, the virt driver does the following:
|
||||
|
||||
#. For pxe_* drivers these include all images: both the deploy ramdisk and
|
||||
user instance images.
|
||||
#. Updates the target ironic node with the information about deploy image,
|
||||
instance UUID, requested capabilities and various flavor properties.
|
||||
|
||||
#. For agent_* drivers only the deploy ramdisk is stored locally. Temporary
|
||||
URLs in OpenStack's Object Storage service are created for user instance
|
||||
images.
|
||||
#. Validates node's power and deploy interfaces, by calling the ironic API.
|
||||
|
||||
#. Virtual interfaces are plugged in and Neutron API updates DHCP port to
|
||||
support PXE/TFTP options.
|
||||
#. Attaches the previously created VIFs to the node. Each neutron port can
|
||||
be attached to any ironic port or port group, with port groups having
|
||||
higher priority than ports. On ironic side, this work is done by the
|
||||
network interface. Attachment here means saving the VIF identifier
|
||||
into ironic port or port group and updating VIF MAC to match the port's
|
||||
or port group's MAC, as described in bullet point 4.
|
||||
|
||||
#. Nova's ironic driver issues a deploy request via the Ironic API to the
|
||||
#. Generates config drive, if requested.
|
||||
|
||||
#. Nova's ironic virt driver issues a deploy request via the Ironic API to the
|
||||
Ironic conductor servicing the bare metal node.
|
||||
|
||||
#. PXE driver prepares tftp bootloader.
|
||||
#. Virtual interfaces are plugged in and Neutron API updates DHCP port to
|
||||
set PXE/TFTP options. In case of using ``neutron`` network interface,
|
||||
ironic creates separate provisioning ports in the Networking service, while
|
||||
in case of ``flat`` network interface, the ports created by nova are used
|
||||
both for provisioning and for deployed instance networking.
|
||||
|
||||
#. The IPMI driver issues command to enable network boot of a node and power
|
||||
it on.
|
||||
#. The ironic node's boot interface prepares (i)PXE configuration and caches
|
||||
deploy kernel and ramdisk.
|
||||
|
||||
#. The DHCP boots the deploy ramdisk. Next, depending on the exact driver
|
||||
used, either the conductor copies the image over iSCSI to the physical node
|
||||
(pxe_* group of drivers) or the deploy ramdisk downloads the image from
|
||||
a temporary URL (agent_* group of drivers), which can be generated by
|
||||
a variety of object stores, e.g. *swift*, *radosgw*, etc, and uploaded
|
||||
to OpenStack's Object Storage service. In the former case, the conductor
|
||||
connects to the iSCSI end point, partitions volume, "dd" the image and
|
||||
closes the iSCSI connection.
|
||||
#. The ironic node's management interface issues commands to enable network
|
||||
boot of a node.
|
||||
|
||||
The deployment is done. The Ironic conductor will switch pxe config to service
|
||||
mode and notify ramdisk agent on the successful deployment.
|
||||
#. The ironic node's deploy interface caches the instance image (in case of
|
||||
``iscsi`` deploy interface or most ``pxe_*`` classic drivers), and kernel
|
||||
and ramdisk if needed (it is needed in case of netboot for example).
|
||||
|
||||
#. The IPMI driver reboots the bare metal node. Note that there are 2 power
|
||||
cycles during bare metal deployment; the first time when powered-on, the
|
||||
images get deployed as mentioned in step 9. The second time as in this case,
|
||||
after the images are deployed, the node is powered up.
|
||||
#. The ironic node's power interface instructs the node to power on.
|
||||
|
||||
#. The bare metal node status is updated and the node instance is made
|
||||
available.
|
||||
#. The node boots the deploy ramdisk.
|
||||
|
||||
#. Depending on the exact driver used, either the conductor copies the image
|
||||
over iSCSI to the physical node (``iscsi`` deploy interface or most
|
||||
``pxe_*`` classic drivers) or the deploy ramdisk downloads the image from a
|
||||
temporary URL (``direct`` deploy interface or ``agent_*`` classic drivers).
|
||||
The temporary URL can be generated by Swift API-compatible object stores,
|
||||
for example Swift itself or RadosGW.
|
||||
|
||||
The image deployment is done.
|
||||
|
||||
#. The node's boot interface switches pxe config to refer to instance images
|
||||
(or, in case of local boot, sets boot device to disk), and asks the ramdisk
|
||||
agent to soft power off the node. If the soft power off by the ramdisk agent
|
||||
fails, the bare metal node is powered off via IPMI/BMC call.
|
||||
|
||||
#. The deploy interface triggers the network interface to remove provisioning
|
||||
ports if they were created, and binds the tenant ports to the node if not
|
||||
already bound. Then the node is powered on.
|
||||
|
||||
.. note:: There are 2 power cycles during bare metal deployment; the
|
||||
first time the node is powered-on when ramdisk is booted, the
|
||||
second time after the image is deployed.
|
||||
|
||||
#. The bare metal node's provisioning state is updated to ``active``.
|
||||
|
||||
Below is the diagram that describes the above process.
|
||||
|
||||
.. graphviz::
|
||||
|
||||
digraph "Deployment Steps" {
|
||||
|
||||
node [shape=box, style=rounded, fontsize=10];
|
||||
edge [fontsize=10];
|
||||
|
||||
/* cylinder shape works only in graphviz 2.39+ */
|
||||
{ rank=same; node [shape=cylinder]; "Nova DB"; "Ironic DB"; }
|
||||
{ rank=same; "Nova API"; "Ironic API"; }
|
||||
{ rank=same; "Nova Message Queue"; "Ironic Message Queue"; }
|
||||
{ rank=same; "Ironic Conductor"; "TFTP Server"; }
|
||||
{ rank=same; "Deploy Interface"; "Boot Interface"; "Power Interface";
|
||||
"Management Interface"; }
|
||||
{ rank=same; "Glance"; "Neutron"; }
|
||||
"Bare Metal Nodes" [shape=box3d];
|
||||
|
||||
"Nova API" -> "Nova Message Queue" [label=" 1"];
|
||||
"Nova Message Queue" -> "Nova Conductor" [dir=both];
|
||||
"Nova Message Queue" -> "Nova Scheduler" [label=" 2"];
|
||||
"Nova Conductor" -> "Nova DB" [dir=both, label=" 3"];
|
||||
"Nova Message Queue" -> "Nova Compute" [dir=both];
|
||||
"Nova Compute" -> "Neutron" [label=" 4"];
|
||||
"Nova Compute" -> "Nova Ironic Virt Driver" [label=5];
|
||||
"Nova Ironic Virt Driver" -> "Ironic API" [label=6];
|
||||
"Ironic API" -> "Ironic Message Queue";
|
||||
"Ironic Message Queue" -> "Ironic Conductor" [dir=both];
|
||||
"Ironic API" -> "Ironic DB" [dir=both];
|
||||
"Ironic Conductor" -> "Ironic DB" [dir=both, label=16];
|
||||
"Ironic Conductor" -> "Boot Interface" [label="8, 14"];
|
||||
"Ironic Conductor" -> "Management Interface" [label=" 9"];
|
||||
"Ironic Conductor" -> "Deploy Interface" [label=10];
|
||||
"Deploy Interface" -> "Network Interface" [label="7, 15"];
|
||||
"Ironic Conductor" -> "Power Interface" [label=11];
|
||||
"Ironic Conductor" -> "Glance";
|
||||
"Network Interface" -> "Neutron";
|
||||
"Power Interface" -> "Bare Metal Nodes";
|
||||
"Management Interface" -> "Bare Metal Nodes";
|
||||
"TFTP Server" -> "Bare Metal Nodes" [label=12];
|
||||
"Ironic Conductor" -> "Bare Metal Nodes" [style=dotted, label=13];
|
||||
"Boot Interface" -> "TFTP Server";
|
||||
|
||||
}
|
||||
|
||||
The following two examples describe what ironic is doing in more detail,
|
||||
leaving out the actions performed by nova and some of the more advanced
|
||||
options.
|
||||
|
||||
Example 1: PXE Boot and iSCSI Deploy Process
|
||||
--------------------------------------------
|
||||
|
||||
This process is used with pxe_* family of drivers.
|
||||
This process is used with ``pxe_*`` family of drivers (the only exception
|
||||
is ``pxe_agent_cimc`` driver).
|
||||
|
||||
.. seqdiag::
|
||||
:scale: 80
|
||||
:scale: 75
|
||||
:alt: pxe_ipmi
|
||||
|
||||
diagram {
|
||||
Nova; API; Conductor; Neutron; "TFTP/HTTPd"; Node;
|
||||
Nova; API; Conductor; Neutron; HTTPStore; "TFTP/HTTPd"; Node;
|
||||
activation = none;
|
||||
span_height = 1;
|
||||
edge_length = 250;
|
||||
default_note_color = white;
|
||||
default_fontsize = 14;
|
||||
|
||||
Nova -> API [label = "Set instance_info", note = "image_source\n,root_gb,etc."];
|
||||
Nova -> API [label = "Set provision_state"];
|
||||
Nova -> API [label = "Set instance_info\n(image_source,\nroot_gb, etc.)"];
|
||||
Nova -> API [label = "Validate power and deploy\ninterfaces"];
|
||||
Nova -> API [label = "Plug VIFs to the node"];
|
||||
Nova -> API [label = "Set provision_state,\noptionally pass configdrive"];
|
||||
API -> Conductor [label = "do_node_deploy()"];
|
||||
Conductor -> Conductor [label = "Cache images"];
|
||||
Conductor -> Conductor [label = "Build TFTP config"];
|
||||
Conductor -> Neutron [label = "Update DHCPBOOT"];
|
||||
Conductor -> Node [label = "IPMI power-on"];
|
||||
Conductor -> Conductor [label = "Validate power and deploy interfaces"];
|
||||
Conductor -> HTTPStore [label = "Store configdrive if configdrive_use_swift \noption is set"];
|
||||
Conductor -> Node [label = "POWER OFF"];
|
||||
Conductor -> Neutron [label = "Attach provisioning network to port(s)"];
|
||||
Conductor -> Neutron [label = "Update DHCP boot options"];
|
||||
Conductor -> Conductor [label = "Prepare PXE\nenvironment for\ndeployment"];
|
||||
Conductor -> Node [label = "Set PXE boot device \nthrough the BMC"];
|
||||
Conductor -> Conductor [label = "Cache deploy\nkernel, ramdisk,\ninstance images"];
|
||||
Conductor -> Node [label = "REBOOT"];
|
||||
Node -> Neutron [label = "DHCP request"];
|
||||
Neutron -> Node [label = "next-server = Conductor"];
|
||||
Node -> Conductor [label = "Attempts to tftpboot from Conductor"];
|
||||
"TFTP/HTTPd" -> Node [label = "Send deploy kernel, ramdisk and config"];
|
||||
Node -> Node [label = "Runs agent\nramdisk"];
|
||||
Node -> API [label = "lookup()"];
|
||||
API -> Conductor [label = "..."];
|
||||
Conductor -> Node [label = "Pass UUID"];
|
||||
API -> Node [label = "Pass UUID"];
|
||||
Node -> API [label = "Heartbeat (UUID)"];
|
||||
API -> Conductor [label = "Heartbeat"];
|
||||
Conductor -> Node [label = "Continue deploy: Pass image, disk info"];
|
||||
Node -> Node [label = "Exposes disks\nvia iSCSI"];
|
||||
Conductor -> Node [label = "Send IPA a command to expose disks via iSCSI"];
|
||||
Conductor -> Node [label = "iSCSI attach"];
|
||||
Conductor -> Node [label = "Copies user image"];
|
||||
Conductor -> Node [label = "Copies user image and configdrive, if present"];
|
||||
Conductor -> Node [label = "iSCSI detach"];
|
||||
Conductor -> Conductor [label = "Delete instance\nimage from cache"];
|
||||
Conductor -> Node [label = "Install boot loader, if requested"];
|
||||
Conductor -> Neutron [label = "Update DHCP boot options"];
|
||||
Conductor -> Conductor [label = "Prepare PXE\nenvironment for\ninstance image"];
|
||||
Conductor -> Node [label = "Set boot device either to PXE or to disk"];
|
||||
Conductor -> Node [label = "Collect ramdisk logs"];
|
||||
Conductor -> Node [label = "POWER OFF"];
|
||||
Conductor -> Neutron [label = "Detach provisioning network\nfrom port(s)"];
|
||||
Conductor -> Neutron [label = "Bind tenant port"];
|
||||
Conductor -> Node [label = "POWER ON"];
|
||||
Conductor -> Conductor [label = "Mark node as\nACTIVE"];
|
||||
Conductor -> Neutron [label = "Clear DHCPBOOT"];
|
||||
Conductor -> Node [label = "Reboot"];
|
||||
Node -> Node [label = "Reboots into\nuser instance"];
|
||||
}
|
||||
|
||||
(From a `talk`_ and `slides`_)
|
||||
|
|
@ -295,48 +383,58 @@ This process is used with pxe_* family of drivers.
|
|||
Example 2: PXE Boot and Direct Deploy Process
|
||||
---------------------------------------------
|
||||
|
||||
This process is used with agent_* family of drivers.
|
||||
This process is used with ``agent_*`` family of drivers.
|
||||
|
||||
.. seqdiag::
|
||||
:scale: 80
|
||||
:scale: 75
|
||||
:alt: pxe_ipmi_agent
|
||||
|
||||
diagram {
|
||||
Nova; API; Conductor; Neutron; "TFTP/HTTPd"; Node;
|
||||
Nova; API; Conductor; Neutron; HTTPStore; "TFTP/HTTPd"; Node;
|
||||
activation = none;
|
||||
edge_length = 250;
|
||||
span_height = 1;
|
||||
default_note_color = white;
|
||||
default_fontsize = 14;
|
||||
|
||||
Nova -> API [label = "Set instance_info", note = "image_source\n,root_gb,etc."];
|
||||
Nova -> API [label = "Set provision_state"];
|
||||
Nova -> API [label = "Set instance_info\n(image_source,\nroot_gb, etc.)"];
|
||||
Nova -> API [label = "Validate power and deploy\ninterfaces"];
|
||||
Nova -> API [label = "Plug VIFs to the node"];
|
||||
Nova -> API [label = "Set provision_state,\noptionally pass configdrive"];
|
||||
API -> Conductor [label = "do_node_deploy()"];
|
||||
Conductor -> Conductor [label = "Cache images"];
|
||||
Conductor -> Conductor [label = "Update pxe,\ntftp configs"];
|
||||
Conductor -> Neutron [label = "Update DHCPBOOT"];
|
||||
Conductor -> Node [label = "power on"];
|
||||
Conductor -> Conductor [label = "Validate power and deploy interfaces"];
|
||||
Conductor -> HTTPStore [label = "Store configdrive if configdrive_use_swift \noption is set"];
|
||||
Conductor -> Node [label = "POWER OFF"];
|
||||
Conductor -> Neutron [label = "Attach provisioning network to port(s)"];
|
||||
Conductor -> Neutron [label = "Update DHCP boot options"];
|
||||
Conductor -> Conductor [label = "Prepare PXE\nenvironment for\ndeployment"];
|
||||
Conductor -> Node [label = "Set PXE boot device \nthrough the BMC"];
|
||||
Conductor -> Conductor [label = "Cache deploy\nand instance\nkernel and ramdisk"];
|
||||
Conductor -> Node [label = "REBOOT"];
|
||||
Node -> Neutron [label = "DHCP request"];
|
||||
Neutron -> Node [label = "next-server = Conductor"];
|
||||
Node -> Conductor [label = "Attempts tftpboot"];
|
||||
"TFTP/HTTPd" -> Node [label = "Send deploy kernel, ramdisk and config"];
|
||||
Node -> Node [label = "Runs agent\nramdisk"];
|
||||
Node -> API [label = "lookup()"];
|
||||
API -> Conductor [label = "..."];
|
||||
Conductor -> Node [label = "Pass UUID"];
|
||||
API -> Node [label = "Pass UUID"];
|
||||
Node -> API [label = "Heartbeat (UUID)"];
|
||||
API -> Conductor [label = "Heartbeat"];
|
||||
Conductor -> Node [label = "Continue deploy: Pass image, disk info"];
|
||||
=== Node downloads image, writes to disk ===
|
||||
Node -> API [label = "Heartbeat periodically"];
|
||||
API -> Conductor [label = "..."];
|
||||
Conductor -> Node [label = "Is deploy done yet?"];
|
||||
Conductor -> Node [label = "Continue deploy asynchronously: Pass image, disk info"];
|
||||
Node -> HTTPStore [label = "Downloads image, writes to disk, \nwrites configdrive if present"];
|
||||
=== Heartbeat periodically ===
|
||||
Conductor -> Node [label = "Is deploy done?"];
|
||||
Node -> Conductor [label = "Still working..."];
|
||||
=== When deploy is done ===
|
||||
Conductor -> Neutron [label = "Clear DHCPBOOT"];
|
||||
Conductor -> Node [label = "Set bootdev HDD"];
|
||||
Conductor -> Node [label = "Reboot"];
|
||||
Node -> Node [label = "Reboots into\nuser instance"];
|
||||
=== ... ===
|
||||
Node -> Conductor [label = "Deploy is done"];
|
||||
Conductor -> Node [label = "Install boot loader, if requested"];
|
||||
Conductor -> Neutron [label = "Update DHCP boot options"];
|
||||
Conductor -> Conductor [label = "Prepare PXE\nenvironment for\ninstance image\nif needed"];
|
||||
Conductor -> Node [label = "Set boot device either to PXE or to disk"];
|
||||
Conductor -> Node [label = "Collect ramdisk logs"];
|
||||
Conductor -> Node [label = "POWER OFF"];
|
||||
Conductor -> Neutron [label = "Detach provisioning network\nfrom port(s)"];
|
||||
Conductor -> Neutron [label = "Bind tenant port"];
|
||||
Conductor -> Node [label = "POWER ON"];
|
||||
Conductor -> Conductor [label = "Mark node as\nACTIVE"];
|
||||
}
|
||||
|
||||
(From a `talk`_ and `slides`_)
|
||||
|
|
|
|||
Loading…
Reference in New Issue