Add nodepool drivers spec
This adds a spec to describe how multiple backend drivers for Nodepool (such as that for static hosts and other cloud providers) should be implemented. Change-Id: Ie1166466b016e67a45f18d07c99fd9c44cde61f9
This commit is contained in:
James E. Blair
parent
4e974448d1
commit
9869a77008
|
|
@ -41,6 +41,7 @@ permits.
|
|||
specs/ethercalc
|
||||
specs/jenkins-job-builder_2.0.0-api-changes
|
||||
specs/neutral-governance-website
|
||||
specs/nodepool-drivers
|
||||
specs/pholio
|
||||
specs/publish-election-repo
|
||||
specs/puppet_4_prelim_testing
|
||||
|
|
|
|||
|
|
@ -0,0 +1,192 @@
|
|||
::
|
||||
|
||||
Copyright (c) 2017 Red Hat, Inc.
|
||||
|
||||
This work is licensed under a Creative Commons Attribution 3.0
|
||||
Unported License.
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
================
|
||||
Nodepool Drivers
|
||||
================
|
||||
|
||||
Storyboard: TBD
|
||||
|
||||
Support multiple provider drivers in Nodepool, including static
|
||||
hosts.
|
||||
|
||||
Problem Description
|
||||
===================
|
||||
|
||||
As part of the Zuul v3 effort, it was envisioned that Nodepool would
|
||||
be expanded to support supplying static nodes in addition to the
|
||||
current support for OpenStack instances, as well as potentially nodes
|
||||
from other sources. The work to move the data structures and queue
|
||||
processing to ZooKeeper was expected to facilitate this. This
|
||||
specification relates both efforts, envisioning supplying static nodes
|
||||
as an implementation of the first alternative driver for Nodepool.
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
|
||||
There are many internal classes which will need to be changed to
|
||||
accomodate the additional level of abstraction necessary to support
|
||||
multiple drivers. This specification is intentionally vague as to
|
||||
exactly which should change, but instead lays out a high-level
|
||||
overview of what should be shared and where drivers should diverge in
|
||||
order to help guide implementation.
|
||||
|
||||
Nodepool's current internal architecture is well suited to an
|
||||
extension to support multiple provider drivers. Because the queue
|
||||
processing, communication, and data storage all occur via ZooKeeper,
|
||||
it's possible to create a component which fulfills Nodepool requests
|
||||
that is completely external to the current Nodepool codebase. That
|
||||
may prove useful in the future in the case of more esoteric systems.
|
||||
However, it would be useful for a wide range of Nodepool users to have
|
||||
built in support for not only OpenStack, but other cloud systems as
|
||||
well as static nodes. The following describes a method of extending
|
||||
the internal processing structure of Nodepool to share as much code
|
||||
between multiple drivers as possible (to reduce the maintenance cost
|
||||
of multiple drivers as well as the operational cost for users).
|
||||
Operators may choose to run multiple providers in a single process for
|
||||
ease of deployment, or they can split providers across multiple
|
||||
processes or hosts as needed for scaling or locality needs.
|
||||
|
||||
The nodepool-launcher daemon is internally structured as a number of
|
||||
threads, each dedicated to a particular task. The main thread,
|
||||
implemented by the ``NodePool`` class starts a ``PoolWorker`` for each
|
||||
provider-pool entry in the config file. That ``PoolWorker`` is
|
||||
responsible for accepting and fulfilling requests, though the
|
||||
specifics of actually fulfilling those requests are handled by other
|
||||
classes such as ``NodeRequestHandler``.
|
||||
|
||||
We should extend the concept of a ``provider`` in Nodepool to also
|
||||
include a driver. Every provider should have a driver and also a
|
||||
``pools`` section, but the rest of the provider configuration (clouds,
|
||||
images, etc.) should be specific to a given driver. Nodepool should
|
||||
start an instance of the ``PoolWorker`` class for every provider-pool
|
||||
combination in every driver. However, the OpenStack-specific
|
||||
behavior currently encoded in the classes utilized by ``PoolWorker``
|
||||
should be abstracted so that a ``PoolWorker`` can be given a different
|
||||
driver as an argument and use that driver to supply nodes.
|
||||
|
||||
When nodes are returned to nodepool (their locks having been
|
||||
released), the ``CleanupWorker`` currently deletes those nodes. It
|
||||
similarly should be extended to recognize the driver which supplied
|
||||
the node, and perform an appropriate action on return (in the case of
|
||||
a static driver, the appropriate action may be to do nothing other
|
||||
than reset the node state to ``ready``).
|
||||
|
||||
The configuration syntax will need some minor changes::
|
||||
|
||||
providers:
|
||||
- name: openstack-public-cloud
|
||||
driver: openstack
|
||||
cloud: some-cloud-name
|
||||
diskimages:
|
||||
- name: fedora25
|
||||
pools:
|
||||
- name: main
|
||||
max-servers: 10
|
||||
labels:
|
||||
- name: fedora25-small
|
||||
min-ram: 1024
|
||||
diskimage: fedora25
|
||||
- name: static
|
||||
driver: static
|
||||
pools:
|
||||
- name: main
|
||||
nodes:
|
||||
- name: static01.example.com
|
||||
host-key: <SSH host key>
|
||||
labels: static
|
||||
- name: static02.example.com
|
||||
host-key: <SSH host key>
|
||||
labels: static
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
We could require that any further drivers be implemented as separate
|
||||
processes, however, due to the careful attention paid to the Zookeeper
|
||||
and Nodepool protocol interactions when implementing the current
|
||||
fulfillment algorithm, prudence suggests that we at least provide some
|
||||
level of shared implementation code to avoid rewriting the otherwise
|
||||
boilerplate node request algorithm handling. As long as we're doing
|
||||
that, it is only a small stretch to also facilitate multiple drivers
|
||||
within a single Nodepool launcher process so that running Nodepool
|
||||
does not become unecessarily complicated for an operator who wants to
|
||||
use a cloud and a handful of static servers.
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee: TBD
|
||||
|
||||
Gerrit Branch
|
||||
-------------
|
||||
|
||||
Nodepool and Zuul are both branched for development related to this
|
||||
spec. The "master" branches will continue to receive patches related
|
||||
to maintaining the current versions, and the "feature/zuulv3" branches
|
||||
will receive patches related to this spec. The .gitreview files have
|
||||
been be updated to submit to the correct branches by default.
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
* Abstract Nodepool request handling code to support multiple drivers
|
||||
* Abstract Nodepool provider management code to support multiple drivers
|
||||
* Collect current request handling implementation in an OpenStack driver
|
||||
* Extend Nodepool configuration syntax to support multiple drivers
|
||||
* Implement a static driver for Nodepool
|
||||
|
||||
Repositories
|
||||
------------
|
||||
|
||||
N/A
|
||||
|
||||
Servers
|
||||
-------
|
||||
|
||||
N/A
|
||||
|
||||
DNS Entries
|
||||
-----------
|
||||
|
||||
N/A
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
|
||||
The Nodepool documentation should be reorganized by driver.
|
||||
|
||||
Security
|
||||
--------
|
||||
|
||||
There is no access control to restrict under what conditions static
|
||||
nodes can be requested. It is unlikely that Nodepool is the right
|
||||
place for that kind of restriction, so Zuul may need to be updated to
|
||||
allow such specifications before it is safe to add sensitive static
|
||||
hosts to Nodepool. However, for the common case of supplying specific
|
||||
real hardware in a known test environment, no access control is
|
||||
required, so the feature is useful without it.
|
||||
|
||||
Testing
|
||||
-------
|
||||
|
||||
This should be unit tested in the way typical for Nodepool.
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
This is related to the ongoing `Zuul v3`_ work and builds on the
|
||||
completed `Zookeeper Workers`_ work in Nodepool.
|
||||
|
||||
.. _Zuul v3:
|
||||
http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html
|
||||
.. _Zookeeper Workers:
|
||||
http://specs.openstack.org/openstack-infra/infra-specs/specs/nodepool-zookeeper-workers.html
|
||||
Loading…
Reference in New Issue