summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorRicardo Rocha <rocha.porto@gmail.com>2017-08-01 15:42:33 +0200
committerRicardo Rocha <rocha.porto@gmail.com>2017-10-25 10:45:41 +0200
commit1315420afe5ea7667554f309f3459232530886e3 (patch)
tree2e3a5c66d025a8fff6610f8356b5ccf6ac4f7751
parent8bf99f093ded5aff8ee68eca01e85948ee9717a2 (diff)
Add Federation API specification
Add a new spec covering the impact and implementation details for a new Magnum API managing federation and federated clusters. Implements: blueprint federation-api Co-Authored-By: Clenimar Filemon <clenimar.filemon@gmail.com> Change-Id: I1b58f44520c6d968dd0c930a54dbd0ef36c2cb4e
Notes
Notes (review): Code-Review+2: Spyros Trigazis (strigazi) <strigazi@gmail.com> Code-Review+2: yatin <ykarel@redhat.com> Workflow+1: Spyros Trigazis (strigazi) <strigazi@gmail.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Thu, 02 Nov 2017 13:18:32 +0000 Reviewed-on: https://review.openstack.org/489609 Project: openstack/magnum-specs Branch: refs/heads/master
-rw-r--r--doc/source/index.rst8
-rw-r--r--specs/queens/federation-api.rst296
2 files changed, 304 insertions, 0 deletions
diff --git a/doc/source/index.rst b/doc/source/index.rst
index a292f3b..36493d0 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -6,6 +6,14 @@
6magnum-specs Design Specifications 6magnum-specs Design Specifications
7================================================== 7==================================================
8 8
9Queens approved specs:
10
11.. toctree::
12 :glob:
13 :maxdepth: 2
14
15 specs/queens/*
16
9Ocata approved specs: 17Ocata approved specs:
10 18
11.. toctree:: 19.. toctree::
diff --git a/specs/queens/federation-api.rst b/specs/queens/federation-api.rst
new file mode 100644
index 0000000..2d88d49
--- /dev/null
+++ b/specs/queens/federation-api.rst
@@ -0,0 +1,296 @@
1Magnum Federation API
2========================
3
4Launchpad blueprint:
5
6https://blueprints.launchpad.net/magnum/+spec/federation-api
7
8This is a proposal to extend the Magnum API adding support for
9federating existing clusters.
10
11
12Problem Description
13-------------------
14
15Magnum provides the means to deploy individual container clusters, supporting
16multiple container orchestration engines (COE) - currently Swarm, Kubernetes
17and Mesos/DCOS. Each of these clusters is independent from all others, with
18its master node(s), worker node(s) and certificate authorities from which client
19certificates are issued.
20
21In the case of Kubernetes the COE has the ability to federate independent
22clusters, joining them under a single API endpoint and distributing workloads
23among all participants. We can envision that other COEs will add similar
24functionality in the future.
25
26This proposal adds a new API endpoint to Magnum to setup, persist and manage
27federations of local or external clusters.
28
29
30Use Cases
31---------
32
33Below are some of the use cases that this API addresses:
34
351. A set of Magnum clusters is created in independent projects so that each
36 is accounted for its own resources, but a single endpoint is desired where
37 policies on workload scheduling can be enforced.
38
392. A very large container infrastructure is desired, composed of thousands or
40 tens of thousands of nodes and growing regularly - as an example, version
41 1.7 of Kubernetes supports up to 5000 nodes [3]. A project administrator can
42 create a new cluster for each set of new resources and simply add this
43 new cluster to an existing federation, easing the management of the overall
44 resources and increasing scalability.
45
463. A set of Magnum clusters is created, each with different characteristics:
47 node flavor, storage setup, etc. Federating them together forms a
48 heterogeneous cluster.
49
504. An existing Magnum cluster in an OpenStack environment is to be extended
51 using external resources. An external cluster endpoint (deployed in AWS,
52 Azure, GKE, another OpenStack or cloud) can be added to an existing
53 Magnum federated cluster, including the complex setup and management of
54 cluster credentials.
55
565. A project has several existing clusters which it would like to expose to a
57 set of users in a single endpoint, without disrupting existing users of
58 each cluster.
59
60
61Proposed Changes
62----------------
63
64The proposed change includes:
65
66* Adding a new '/federation' REST API endpoint to Magnum providing management
67 of federated clusters. This includes federation creation, update and
68 deletion, as well as adding and removing members from a federation.
69
70* Adding a new entity to the data model (federation) to represent a federated
71 cluster and its metadata.
72
73Check sections 'Data Model Impact' and 'REST API Impact' for more details.
74
75
76Alternatives
77------------
78
79A valid alternative would be to manage the setup of the federated control
80plane on the client side only. This is possible but has a few drawbacks:
81
82* The logic for the federation setup is in some cases contained in a tool
83 supported by the COE. Relying on it would mean adding a client side
84 dependency on each of the supported COE clients (hard to manage) or
85 replicating the full functionality in the Magnum client code (hard to keep
86 up with upstream COE developments).
87
88* Managing the cluster client access context is COE specific and required to
89 setup the federation control plane. Asking the user to do this herself
90 would greatly reduce the added value of Magnum on managing federations, and
91 miss the opportunity to (re)use a lot of the cluster metadata information
92 that Magnum already stores.
93
94
95Data Model Impact
96-----------------
97
98A new entity would be added (corresponding tables will be added):
99
100* **federation**
101
102 * uuid
103 * name
104 * project_id
105 * hostcluster_id (the cluster hosting the federation control plane)
106 * member_ids (the clusters that are members of the federation)
107 * status (the current status of the federation)
108 * status_reason (a message with more info regarding the status)
109 * properties (additional metadata, coe specific in some cases)
110
111We chose to add a new entity so that we:
112
113* decouple the federation entity from the existing resources
114* minimize the impact on the current interfaces
115* more easily extend the federation functionality in the future
116
117
118REST API Impact
119---------------
120
121This change leads to a minor version increase in the Magnum API, the
122addition of a new REST endpoint and a new set of CLI commands.
123
124Below is a description of the commands to manage federations:
125
126* federation create: creates a new federation, in an existing cluster::
127
128 openstack coe federation create myfederation --hostcluster cluster1
129
130* federation delete: deletes an existing federation::
131
132 openstack coe federation delete myfederation
133
134* federation update: updates the metadata of an existing federation::
135
136 openstack coe federation update --property dns='cluster.local' myfederation
137
138* federation list: list existing federations, with metadata and membership::
139
140 openstack coe federation list
141
142 +------+--------------+---------+-----------------+
143 | uuid | name | members | status |
144 +------+--------------+---------+-----------------+
145 | ... | myfederation | 3 | UPDATE_COMPLETE |
146 +------+--------------+---------+-----------------+
147
148
149* federation show: show details of an existing federation::
150
151 openstack coe federation show myfederation
152
153 +---------------------+-------------------------------------------+
154 | Property | Value |
155 +---------------------+-------------------------------------------+
156 | uuid | 5b2ee3b5-2f85-4917-be7c-11a2c82031ad |
157 | name | myfederation |
158 | hostcluster | <uuid-cluster1> |
159 | members | ['<uuid-cluster2>', '<uuid-cluster3>'] |
160 | project_id | ae72a4f3-30cf-4406-83b4-40b16bb480d6 |
161 | properties | dns=cluster.local |
162 | status | UPDATE_COMPLETE |
163 | status_reason | Federation UPDATE completed successfully |
164 +---------------------+-------------------------------------------+
165
166* federation join: adds an existing cluster to a federation::
167
168 openstack coe federation join myfederation cluster2 cluster3
169
170* federation unjoin: parts a cluster from a federation::
171
172 openstack coe federation unjoin myfederation cluster3
173
174
175Other Implementation Options
176----------------------------
177
178See Alternatives.
179
180
181Security Impact
182---------------
183
184Management of federations implies the same level of security as for the
185existing cluster management functionality - creation, deletion, update based
186on a policy.
187
188Joining/unjoining a federation requires special care and a special policy
189evaluating permissions on both source and destination clusters - the host
190and the additional members. An example policy is shown below:
191
192
193Notifications Impact
194--------------------
195
196New notifications will be added for:
197* federation creation
198* federation deletion
199* federation update (including metadata and members)
200
201
202Other End User Impact
203---------------------
204
205Users will not be able to delete a cluster that is part of a federation, and
206will have to unjoin first.
207
208New subcommands will be added to the openstack client as described above.
209
210
211Developer Impact
212----------------
213
214This functionality will be added via a separate endpoint and metadata stored
215in a new entity. Impact in the existing Magnum code will be minimal.
216
217
218Implementation
219--------------
220
221The implementation will be done in 5 phases.
222
2231. Add the new API endpoint and data model entity, and the corresponding
224 controller implementation linked to each driver. At this point we will
225 have all drivers declaring this functionality as 'Not Implemented'.
226
2272. Implement the federation functionality for the Kubernetes Atomic driver.
228 This includes all the described setup in [2] (dns, context, credentials).
229 The initial implementation will only support federating Magnum clusters
230 in the same OpenStack deployment.
231
2323. Add the new command line tools to the openstack client.
233
2344. Add support for including external Kubernetes clusters (deployed in GKE,
235 ACS, AWS, ...) in a Magnum federation. Given this is all set at the level
236 of Kubernetes the source environment of the cluster should not be
237 relevant, but a way for the user to provide the cluster context to Magnum
238 will be added to the command line tools and API.
239
2405. Implement the Magnum federation notifications, for creation, deletion and
241 update.
242
243Implementation of this functionality for drivers other than Kubernetes will be
244left for future specs, as there is currently no support in other COEs.
245
246
247Assignee(s)
248-----------
249
250See blueprint:
251
252https://blueprints.launchpad.net/magnum/+spec/federation-api
253
254
255Work Items
256----------
257
258See blueprint:
259
260https://blueprints.launchpad.net/magnum/+spec/federation-api
261
262
263Dependencies
264------------
265
266Setting up federations is often done by a COE specific tool - kubefed in the
267case of Kubernetes. We will rely on these tools for the driver specific
268implementation, thus adding new dependencies on them.
269
270
271Testing
272-------
273
274A new set of tests covering creation, deletion, update of federations and
275join/unjoin of clusters needs to be added to both unit and functional tests.
276
277
278Documentation Impact
279--------------------
280
281New documentation will be added to describe the new API endpoint and its
282functionality. The quickstart guide needs an update to include information
283regarding cluster federation.
284
285
286References
287----------
288
289[1] - Magnum Federation API Blueprint:
290https://blueprints.launchpad.net/magnum/+spec/federation-api
291
292[2] - Tutorial on Cluster Federation setup with Kubefed
293https://kubernetes.io/docs/tasks/federation/set-up-cluster-federation-kubefed
294
295[3] - Kubernetes - Building Large Cluster:
296https://kubernetes.io/docs/admin/cluster-large/