Add Federation API specification

Add a new spec covering the impact and implementation details for a new
Magnum API managing federation and federated clusters.

Implements: blueprint federation-api
Co-Authored-By: Clenimar Filemon <clenimar.filemon@gmail.com>

Change-Id: I1b58f44520c6d968dd0c930a54dbd0ef36c2cb4e
This commit is contained in:
Ricardo Rocha 2017-08-01 15:42:33 +02:00
parent 8bf99f093d
commit 1315420afe
2 changed files with 304 additions and 0 deletions

View File

@ -6,6 +6,14 @@
magnum-specs Design Specifications
==================================================
Queens approved specs:
.. toctree::
:glob:
:maxdepth: 2
specs/queens/*
Ocata approved specs:
.. toctree::

View File

@ -0,0 +1,296 @@
Magnum Federation API
========================
Launchpad blueprint:
https://blueprints.launchpad.net/magnum/+spec/federation-api
This is a proposal to extend the Magnum API adding support for
federating existing clusters.
Problem Description
-------------------
Magnum provides the means to deploy individual container clusters, supporting
multiple container orchestration engines (COE) - currently Swarm, Kubernetes
and Mesos/DCOS. Each of these clusters is independent from all others, with
its master node(s), worker node(s) and certificate authorities from which client
certificates are issued.
In the case of Kubernetes the COE has the ability to federate independent
clusters, joining them under a single API endpoint and distributing workloads
among all participants. We can envision that other COEs will add similar
functionality in the future.
This proposal adds a new API endpoint to Magnum to setup, persist and manage
federations of local or external clusters.
Use Cases
---------
Below are some of the use cases that this API addresses:
1. A set of Magnum clusters is created in independent projects so that each
is accounted for its own resources, but a single endpoint is desired where
policies on workload scheduling can be enforced.
2. A very large container infrastructure is desired, composed of thousands or
tens of thousands of nodes and growing regularly - as an example, version
1.7 of Kubernetes supports up to 5000 nodes [3]. A project administrator can
create a new cluster for each set of new resources and simply add this
new cluster to an existing federation, easing the management of the overall
resources and increasing scalability.
3. A set of Magnum clusters is created, each with different characteristics:
node flavor, storage setup, etc. Federating them together forms a
heterogeneous cluster.
4. An existing Magnum cluster in an OpenStack environment is to be extended
using external resources. An external cluster endpoint (deployed in AWS,
Azure, GKE, another OpenStack or cloud) can be added to an existing
Magnum federated cluster, including the complex setup and management of
cluster credentials.
5. A project has several existing clusters which it would like to expose to a
set of users in a single endpoint, without disrupting existing users of
each cluster.
Proposed Changes
----------------
The proposed change includes:
* Adding a new '/federation' REST API endpoint to Magnum providing management
of federated clusters. This includes federation creation, update and
deletion, as well as adding and removing members from a federation.
* Adding a new entity to the data model (federation) to represent a federated
cluster and its metadata.
Check sections 'Data Model Impact' and 'REST API Impact' for more details.
Alternatives
------------
A valid alternative would be to manage the setup of the federated control
plane on the client side only. This is possible but has a few drawbacks:
* The logic for the federation setup is in some cases contained in a tool
supported by the COE. Relying on it would mean adding a client side
dependency on each of the supported COE clients (hard to manage) or
replicating the full functionality in the Magnum client code (hard to keep
up with upstream COE developments).
* Managing the cluster client access context is COE specific and required to
setup the federation control plane. Asking the user to do this herself
would greatly reduce the added value of Magnum on managing federations, and
miss the opportunity to (re)use a lot of the cluster metadata information
that Magnum already stores.
Data Model Impact
-----------------
A new entity would be added (corresponding tables will be added):
* **federation**
* uuid
* name
* project_id
* hostcluster_id (the cluster hosting the federation control plane)
* member_ids (the clusters that are members of the federation)
* status (the current status of the federation)
* status_reason (a message with more info regarding the status)
* properties (additional metadata, coe specific in some cases)
We chose to add a new entity so that we:
* decouple the federation entity from the existing resources
* minimize the impact on the current interfaces
* more easily extend the federation functionality in the future
REST API Impact
---------------
This change leads to a minor version increase in the Magnum API, the
addition of a new REST endpoint and a new set of CLI commands.
Below is a description of the commands to manage federations:
* federation create: creates a new federation, in an existing cluster::
openstack coe federation create myfederation --hostcluster cluster1
* federation delete: deletes an existing federation::
openstack coe federation delete myfederation
* federation update: updates the metadata of an existing federation::
openstack coe federation update --property dns='cluster.local' myfederation
* federation list: list existing federations, with metadata and membership::
openstack coe federation list
+------+--------------+---------+-----------------+
| uuid | name | members | status |
+------+--------------+---------+-----------------+
| ... | myfederation | 3 | UPDATE_COMPLETE |
+------+--------------+---------+-----------------+
* federation show: show details of an existing federation::
openstack coe federation show myfederation
+---------------------+-------------------------------------------+
| Property | Value |
+---------------------+-------------------------------------------+
| uuid | 5b2ee3b5-2f85-4917-be7c-11a2c82031ad |
| name | myfederation |
| hostcluster | <uuid-cluster1> |
| members | ['<uuid-cluster2>', '<uuid-cluster3>'] |
| project_id | ae72a4f3-30cf-4406-83b4-40b16bb480d6 |
| properties | dns=cluster.local |
| status | UPDATE_COMPLETE |
| status_reason | Federation UPDATE completed successfully |
+---------------------+-------------------------------------------+
* federation join: adds an existing cluster to a federation::
openstack coe federation join myfederation cluster2 cluster3
* federation unjoin: parts a cluster from a federation::
openstack coe federation unjoin myfederation cluster3
Other Implementation Options
----------------------------
See Alternatives.
Security Impact
---------------
Management of federations implies the same level of security as for the
existing cluster management functionality - creation, deletion, update based
on a policy.
Joining/unjoining a federation requires special care and a special policy
evaluating permissions on both source and destination clusters - the host
and the additional members. An example policy is shown below:
Notifications Impact
--------------------
New notifications will be added for:
* federation creation
* federation deletion
* federation update (including metadata and members)
Other End User Impact
---------------------
Users will not be able to delete a cluster that is part of a federation, and
will have to unjoin first.
New subcommands will be added to the openstack client as described above.
Developer Impact
----------------
This functionality will be added via a separate endpoint and metadata stored
in a new entity. Impact in the existing Magnum code will be minimal.
Implementation
--------------
The implementation will be done in 5 phases.
1. Add the new API endpoint and data model entity, and the corresponding
controller implementation linked to each driver. At this point we will
have all drivers declaring this functionality as 'Not Implemented'.
2. Implement the federation functionality for the Kubernetes Atomic driver.
This includes all the described setup in [2] (dns, context, credentials).
The initial implementation will only support federating Magnum clusters
in the same OpenStack deployment.
3. Add the new command line tools to the openstack client.
4. Add support for including external Kubernetes clusters (deployed in GKE,
ACS, AWS, ...) in a Magnum federation. Given this is all set at the level
of Kubernetes the source environment of the cluster should not be
relevant, but a way for the user to provide the cluster context to Magnum
will be added to the command line tools and API.
5. Implement the Magnum federation notifications, for creation, deletion and
update.
Implementation of this functionality for drivers other than Kubernetes will be
left for future specs, as there is currently no support in other COEs.
Assignee(s)
-----------
See blueprint:
https://blueprints.launchpad.net/magnum/+spec/federation-api
Work Items
----------
See blueprint:
https://blueprints.launchpad.net/magnum/+spec/federation-api
Dependencies
------------
Setting up federations is often done by a COE specific tool - kubefed in the
case of Kubernetes. We will rely on these tools for the driver specific
implementation, thus adding new dependencies on them.
Testing
-------
A new set of tests covering creation, deletion, update of federations and
join/unjoin of clusters needs to be added to both unit and functional tests.
Documentation Impact
--------------------
New documentation will be added to describe the new API endpoint and its
functionality. The quickstart guide needs an update to include information
regarding cluster federation.
References
----------
[1] - Magnum Federation API Blueprint:
https://blueprints.launchpad.net/magnum/+spec/federation-api
[2] - Tutorial on Cluster Federation setup with Kubefed
https://kubernetes.io/docs/tasks/federation/set-up-cluster-federation-kubefed
[3] - Kubernetes - Building Large Cluster:
https://kubernetes.io/docs/admin/cluster-large/