Add a virt driver for Ironic
This specification proposes to add a virt driver to enable
Nova to deploy images to bare metal resources by using the
OpenStack Bare Metal Provisioning Service ("Ironic").
Change-Id: I585754dd5c11a97a4eef01819387ce37b9a6a8b3
This commit is contained in:
Devananda van der Veen
parent
6b9b856e40
commit
e82a9c4b85
|
|
@ -0,0 +1,285 @@
|
|||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
============================
|
||||
Add a virt driver for Ironic
|
||||
============================
|
||||
|
||||
https://blueprints.launchpad.net/nova/+spec/add-ironic-driver
|
||||
|
||||
This specification proposes to add a virt driver to enable Nova to deploy
|
||||
images to bare metal resources by using the OpenStack Bare Metal Provisioning
|
||||
Service ("Ironic").
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
The community has split out the functionality of provisioning bare metal
|
||||
servers into a separate program, which includes the ironic and
|
||||
python-ironicclient projects. The original intent of the
|
||||
nova.virt.baremetal driver was two-fold:
|
||||
|
||||
- to provide physical machines more suitable to HPC-style workloads,
|
||||
eg where virtualization overhead is too high;
|
||||
- to be an experimental proof-of-concept for enabling the TripleO project.
|
||||
|
||||
In order to address scalability and architectural concerns affecting both
|
||||
use-cases, this driver was split out into a separate OpenStack Program,
|
||||
and developed over the last year as such.
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
This proposal aims to enable Nova to use Ironic to perform the same functions
|
||||
which it is currently able to perform via the nova.virt.baremetal driver.
|
||||
This abstracts the details of physical hardware within Ironic, such that the
|
||||
user interacts with Nova in the same way when deploying instances to virtual or
|
||||
physical machines. The hardware-specific details are only exposed to the cloud
|
||||
operators.
|
||||
|
||||
Specifically, this will:
|
||||
|
||||
* add the nova.virt.ironic driver, which will use the python-ironicclient
|
||||
library to interact with Ironic's REST API for the purpose of provisioning
|
||||
physical machines.
|
||||
|
||||
* add new IronicHostManager class, similar to BaremetalHostManager, which
|
||||
fills the same purpose but is specific to Ironic. Namely, this provides
|
||||
several customizations to Nova's HostManager, tailoring it to consuming
|
||||
discrete and non-subdivisible physical resources.
|
||||
|
||||
* add exact-match scheduler filters, to facilitate users who wish to match
|
||||
nova flavor to hardware specifications exactly. The best matching possible
|
||||
today is greater-than-or-equal, which is often undesirable (eg. because
|
||||
a machine with 128GB of RAM could be selected to fulfil a request for an
|
||||
instance with 16GB of RAM).
|
||||
|
||||
This driver will initially implement a subset of the Nova virt driver API
|
||||
sufficient to support the same functionality that the nova.virt.baremetal
|
||||
driver supported. Over time, additional functionality will be added, as
|
||||
appropriate and possible for physical hardware. It is expected that some
|
||||
operations may never be added to this driver, eg when the operation is not
|
||||
possible where there is no local hypervisor.
|
||||
|
||||
This driver will expose the complete resources of the ironic service it is
|
||||
connected to. Therefor, running multiple nova-compute processes within a
|
||||
single cell or region will not be possible, and HA for the nova-compute
|
||||
service must be achieved externally, eg. via pacemaker+corosync. Scale-out
|
||||
may be achieved by running multiple ironic clusters, with a single n-cpu
|
||||
connected to each ironic end-point. This is not optimal, and is a result
|
||||
of a current limitation within Nova. See the Alternatives section below
|
||||
for a summary of the discussion which has occurred around this limitation.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
One alternative would be for users to directly interact with Ironic's API,
|
||||
circumventing Nova when deploying instances to bare metal. This would require
|
||||
Ironic to duplicate a significant amount of functionality present in Nova, and
|
||||
violate the abstraction layer. Note that giving end-users direct access to
|
||||
Ironic's API may present security concerns for some operators; see the
|
||||
Security Impact section below for a discussion of this.
|
||||
|
||||
Instead of creating a new IronicHostManager class, the existing
|
||||
BaremetalHostManager class could be refactored to support both drivers.
|
||||
|
||||
Instead of adding exact-match scheduler filters, we could create a new
|
||||
scheduler that is specifically geared towards non-divisible resources.
|
||||
However, this approach is sufficient for many use cases, and does not prevent
|
||||
the later creation of another scheduler.
|
||||
|
||||
An alternative was proposed which would allow multiple nova-compute processes
|
||||
to proxy for the same Ironic service end-point at the same time. This could be
|
||||
done by setting the same 'host' property on each nova-compute service, such
|
||||
that they expose the same set of resources. Therefor, certain operations would
|
||||
need to be skipped when starting the nova-compute process (eg, so it doesn't
|
||||
trample over an ongoing operation on another compute host). This would be
|
||||
accomplished by creating a new ClusteredComputeManager class (subclassed from
|
||||
ComputeManager) which would override init_host() to avoid the call to
|
||||
InstanceList.get_by_host(), self._destroy_evacuated_instances() and
|
||||
self._init_instance(). This proposal was denied due to architectural concerns
|
||||
within Nova, specifically around @utils.synchronize(instance['uuid']) calls,
|
||||
and event callbacks that could be routed to a host other than the one waiting
|
||||
for the callback.
|
||||
|
||||
Instead of overriding ComputeManager.init_host(), a significant rewrite of
|
||||
Nova's internal resource model could be undertaken -- eg, to remove the (host,
|
||||
hypervisor_hostname) tuple from all places within the code and make the
|
||||
nova-conductor process handle resource locking for clustered hypervisors, such
|
||||
as Ironic. This would be a significant undertaking, and while merited, it was
|
||||
agreed that this work would not block the Ironic driver.
|
||||
|
||||
|
||||
Data model impact
|
||||
-----------------
|
||||
|
||||
Adding the nova.virt.ironic driver will not impact the db model.
|
||||
|
||||
Some additional extra_specs may be leveraged in faciliating better scheduling
|
||||
in the future. There is precedent in the way that the nova.virt.baremetal
|
||||
driver leverages extra_specs:baremetal:cpu_arch. Ironic may extend this to
|
||||
support additional hardware metadata in the future.
|
||||
|
||||
REST API impact
|
||||
---------------
|
||||
|
||||
The nova.virt.ironic driver will not add any REST API extensions or require
|
||||
changes to any of Nova's APIs.
|
||||
|
||||
Security impact
|
||||
---------------
|
||||
|
||||
Allowing Nova to provision physical hardware has significant security
|
||||
implications. The nova.virt.baremetal driver required direct access to the OOB
|
||||
management (IPMI) network of the hardware it managed. A compromise of
|
||||
nova-compute would expose that hardware's management interface.
|
||||
|
||||
In a properly-secured OpenStack deployment, security will be improved by moving
|
||||
this functionality out of nova-compute and into ironic-conductor, because there
|
||||
is a strict API between the two services.
|
||||
|
||||
The ironic-api should not be reachable or discoverable by end-users, and only
|
||||
the ironic-conductor service should have access to the hardware management
|
||||
interface. The user of Nova who requests an instance on bare metal will thus
|
||||
have no direct access to the services managing that bare metal host. Should
|
||||
nova-compute be compromised, the malicious user would still need to gain access
|
||||
to the ironic-conductor host before having any access to the hardware
|
||||
management interface.
|
||||
|
||||
Considering how often IPMI is not properly secured, and that in many cases it
|
||||
can not be secured, the OOB management network should be as isolated from users
|
||||
as possible.
|
||||
|
||||
Notifications impact
|
||||
--------------------
|
||||
|
||||
None
|
||||
|
||||
Other end user impact
|
||||
---------------------
|
||||
|
||||
None
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
|
||||
No impact on Nova itself.
|
||||
|
||||
The performance profile of the nova.virt.ironic driver will be different than
|
||||
other virt drivers due to the nature of managing physical machines. For
|
||||
example, power cycling bare metal often takes more than five minutes as the
|
||||
hardware must complete a POST cycle. Thus, a deploy may be expected to take a
|
||||
minimum of ten minutes, though depending on the hardware, it may be more or
|
||||
less.
|
||||
|
||||
Other deployer impact
|
||||
---------------------
|
||||
|
||||
Deploying Nova with the nova.virt.ironic driver will be considerably different
|
||||
to deploying Nova with other virt drivers, and also different from the
|
||||
nova.virt.baremetal driver. Main areas of difference are:
|
||||
|
||||
* different system libraries will be required. No local hypervisor needs be
|
||||
installed, and none of the system libraries to enable baremetal need to be
|
||||
installed on the compute host itself.
|
||||
|
||||
* the OpenStack Ironic services must be properly set up and discoverable
|
||||
via Keystone in order for the nova.virt.ironic driver to function properly.
|
||||
|
||||
* Nova must be supplied with admin credentials capable of interacting
|
||||
with Ironic.
|
||||
|
||||
An upgrade path from the nova.virt.baremetal driver to the nova.virt.ironic
|
||||
driver will be provided. The details of that are proposed in another document:
|
||||
|
||||
https://blueprints.launchpad.net/nova/+spec/deprecate-baremetal-driver
|
||||
|
||||
Developer impact
|
||||
----------------
|
||||
|
||||
None
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
|
||||
Primary assignee:
|
||||
devananda
|
||||
|
||||
Other contributors:
|
||||
lucasagomes
|
||||
nobodycam
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
* Merge auxiliary components: HostManager and exact-match scheduler filters
|
||||
|
||||
* Delete auxiliary components from Ironic's tree
|
||||
|
||||
* Split the nova.virt.ironic driver into a series of patches, the sum of
|
||||
which will pass unit and functional tests.
|
||||
|
||||
* Delete driver from Ironic's tree after it has merged in Nova.
|
||||
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
There is already tempest testing being done upstream against changes in
|
||||
ironic, nova, devstack, and tempest. However, it is non-voting today.
|
||||
The following paragraph describes how it works.
|
||||
|
||||
Devstack creates a "mock" bare metal node, enrolls it with Ironic, and
|
||||
configures Nova appropriately to use the nova.virt.ironic driver. A tempest
|
||||
scenario test is then run against that devstack instance, which allows tempest
|
||||
to test functionality appropriate for this driver. Certain tests may be
|
||||
excluded when the functionality does not apply to bare metal (eg,
|
||||
live migrate). The current test is fairly simple: validate the boot process,
|
||||
network connectivity of the instance, and validate destroy. Additional tests
|
||||
have been proposed for more coverage, eg. "rebuild --preserve-ephemeral".
|
||||
|
||||
Testing of functionality not exposed via the nova virt driver interface is done
|
||||
directly in Tempest via the Ironic API (eg, management operations) and is
|
||||
mentioned here only for completeness.
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
Documentation should be added to Nova stating the existence of the new driver,
|
||||
and should include links to the Ironic project's developer and deployer
|
||||
documentation.
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
Current code, in Ironic's git tree::
|
||||
http://git.openstack.org/cgit/openstack/ironic/tree/ironic/nova
|
||||
|
||||
Devstack support for testing this driver::
|
||||
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/ironic
|
||||
http://git.openstack.org/cgit/openstack-dev/devstack/tree/tools/ironic
|
||||
|
||||
Tempest test which deploys using the nova.virt.ironic driver::
|
||||
http://git.openstack.org/cgit/openstack/tempest/tree/tempest/scenario/test_baremetal_basic_ops.py
|
||||
|
||||
Juno summit etherpad discussing this::
|
||||
https://etherpad.openstack.org/p/juno-nova-deprecating-baremetal
|
||||
|
||||
Some best practices for IPMI sanity::
|
||||
http://fish2.com/ipmi/bp.pdf
|
||||
|
||||
Discussions of IPMI vulnerabilities::
|
||||
http://fish2.com/ipmi/itrain.pdf
|
||||
http://fish2.com/ipmi/river.pdf
|
||||
Loading…
Reference in New Issue