From 9869a7700819ded5e5402fd6a5a650e49f1437ab Mon Sep 17 00:00:00 2001 From: "James E. Blair" Date: Mon, 1 May 2017 14:11:06 -0700 Subject: [PATCH] Add nodepool drivers spec This adds a spec to describe how multiple backend drivers for Nodepool (such as that for static hosts and other cloud providers) should be implemented. Change-Id: Ie1166466b016e67a45f18d07c99fd9c44cde61f9 --- doc/source/index.rst | 1 + specs/nodepool-drivers.rst | 192 +++++++++++++++++++++++++++++++++++++ 2 files changed, 193 insertions(+) create mode 100644 specs/nodepool-drivers.rst diff --git a/doc/source/index.rst b/doc/source/index.rst index bb978cc..f9f5541 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -41,6 +41,7 @@ permits. specs/ethercalc specs/jenkins-job-builder_2.0.0-api-changes specs/neutral-governance-website + specs/nodepool-drivers specs/pholio specs/publish-election-repo specs/puppet_4_prelim_testing diff --git a/specs/nodepool-drivers.rst b/specs/nodepool-drivers.rst new file mode 100644 index 0000000..024040e --- /dev/null +++ b/specs/nodepool-drivers.rst @@ -0,0 +1,192 @@ +:: + + Copyright (c) 2017 Red Hat, Inc. + + This work is licensed under a Creative Commons Attribution 3.0 + Unported License. + http://creativecommons.org/licenses/by/3.0/legalcode + +================ +Nodepool Drivers +================ + +Storyboard: TBD + +Support multiple provider drivers in Nodepool, including static +hosts. + +Problem Description +=================== + +As part of the Zuul v3 effort, it was envisioned that Nodepool would +be expanded to support supplying static nodes in addition to the +current support for OpenStack instances, as well as potentially nodes +from other sources. The work to move the data structures and queue +processing to ZooKeeper was expected to facilitate this. This +specification relates both efforts, envisioning supplying static nodes +as an implementation of the first alternative driver for Nodepool. + +Proposed Change +=============== + +There are many internal classes which will need to be changed to +accomodate the additional level of abstraction necessary to support +multiple drivers. This specification is intentionally vague as to +exactly which should change, but instead lays out a high-level +overview of what should be shared and where drivers should diverge in +order to help guide implementation. + +Nodepool's current internal architecture is well suited to an +extension to support multiple provider drivers. Because the queue +processing, communication, and data storage all occur via ZooKeeper, +it's possible to create a component which fulfills Nodepool requests +that is completely external to the current Nodepool codebase. That +may prove useful in the future in the case of more esoteric systems. +However, it would be useful for a wide range of Nodepool users to have +built in support for not only OpenStack, but other cloud systems as +well as static nodes. The following describes a method of extending +the internal processing structure of Nodepool to share as much code +between multiple drivers as possible (to reduce the maintenance cost +of multiple drivers as well as the operational cost for users). +Operators may choose to run multiple providers in a single process for +ease of deployment, or they can split providers across multiple +processes or hosts as needed for scaling or locality needs. + +The nodepool-launcher daemon is internally structured as a number of +threads, each dedicated to a particular task. The main thread, +implemented by the ``NodePool`` class starts a ``PoolWorker`` for each +provider-pool entry in the config file. That ``PoolWorker`` is +responsible for accepting and fulfilling requests, though the +specifics of actually fulfilling those requests are handled by other +classes such as ``NodeRequestHandler``. + +We should extend the concept of a ``provider`` in Nodepool to also +include a driver. Every provider should have a driver and also a +``pools`` section, but the rest of the provider configuration (clouds, +images, etc.) should be specific to a given driver. Nodepool should +start an instance of the ``PoolWorker`` class for every provider-pool +combination in every driver. However, the OpenStack-specific +behavior currently encoded in the classes utilized by ``PoolWorker`` +should be abstracted so that a ``PoolWorker`` can be given a different +driver as an argument and use that driver to supply nodes. + +When nodes are returned to nodepool (their locks having been +released), the ``CleanupWorker`` currently deletes those nodes. It +similarly should be extended to recognize the driver which supplied +the node, and perform an appropriate action on return (in the case of +a static driver, the appropriate action may be to do nothing other +than reset the node state to ``ready``). + +The configuration syntax will need some minor changes:: + + providers: + - name: openstack-public-cloud + driver: openstack + cloud: some-cloud-name + diskimages: + - name: fedora25 + pools: + - name: main + max-servers: 10 + labels: + - name: fedora25-small + min-ram: 1024 + diskimage: fedora25 + - name: static + driver: static + pools: + - name: main + nodes: + - name: static01.example.com + host-key: + labels: static + - name: static02.example.com + host-key: + labels: static + +Alternatives +------------ + +We could require that any further drivers be implemented as separate +processes, however, due to the careful attention paid to the Zookeeper +and Nodepool protocol interactions when implementing the current +fulfillment algorithm, prudence suggests that we at least provide some +level of shared implementation code to avoid rewriting the otherwise +boilerplate node request algorithm handling. As long as we're doing +that, it is only a small stretch to also facilitate multiple drivers +within a single Nodepool launcher process so that running Nodepool +does not become unecessarily complicated for an operator who wants to +use a cloud and a handful of static servers. + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: TBD + +Gerrit Branch +------------- + +Nodepool and Zuul are both branched for development related to this +spec. The "master" branches will continue to receive patches related +to maintaining the current versions, and the "feature/zuulv3" branches +will receive patches related to this spec. The .gitreview files have +been be updated to submit to the correct branches by default. + +Work Items +---------- + +* Abstract Nodepool request handling code to support multiple drivers +* Abstract Nodepool provider management code to support multiple drivers +* Collect current request handling implementation in an OpenStack driver +* Extend Nodepool configuration syntax to support multiple drivers +* Implement a static driver for Nodepool + +Repositories +------------ + +N/A + +Servers +------- + +N/A + +DNS Entries +----------- + +N/A + +Documentation +------------- + +The Nodepool documentation should be reorganized by driver. + +Security +-------- + +There is no access control to restrict under what conditions static +nodes can be requested. It is unlikely that Nodepool is the right +place for that kind of restriction, so Zuul may need to be updated to +allow such specifications before it is safe to add sensitive static +hosts to Nodepool. However, for the common case of supplying specific +real hardware in a known test environment, no access control is +required, so the feature is useful without it. + +Testing +------- + +This should be unit tested in the way typical for Nodepool. + +Dependencies +============ + +This is related to the ongoing `Zuul v3`_ work and builds on the +completed `Zookeeper Workers`_ work in Nodepool. + +.. _Zuul v3: + http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html +.. _Zookeeper Workers: + http://specs.openstack.org/openstack-infra/infra-specs/specs/nodepool-zookeeper-workers.html