summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJohn Garbutt <john@johngarbutt.com>2017-09-18 15:15:35 +0100
committerMark Goddard <mark@stackhpc.com>2018-12-13 14:58:59 +0000
commit8e95563a681fbd901004eb1784453af8a58707c7 (patch)
treebfb018bf0619fc1be6dab1a466729de9526b34cf
parent8722aa582a908c87804f1894e947b34cbd748653 (diff)
Deploy Templates
This builds on the deployment steps framework spec (I2f68170e6741b0dbb6d5fbd5315a3be9fd7b28a7) such that it is possible to request a particular set of deployment steps by requesting a set of traits. Change-Id: I7a56db52873655b15efe83282df8f864d932ca79 Story: 1722275 Task: 10626
Notes
Notes (review): Code-Review+2: Dmitry Tantsur <divius.inside@gmail.com> Code-Review+2: Julia Kreger <juliaashleykreger@gmail.com> Workflow+1: Julia Kreger <juliaashleykreger@gmail.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Mon, 17 Dec 2018 16:43:27 +0000 Reviewed-on: https://review.openstack.org/504952 Project: openstack/ironic-specs Branch: refs/heads/master
-rw-r--r--specs/approved/deploy-templates.rst721
l---------specs/not-implemented/deploy-templates.rst1
2 files changed, 722 insertions, 0 deletions
diff --git a/specs/approved/deploy-templates.rst b/specs/approved/deploy-templates.rst
new file mode 100644
index 0000000..1e65245
--- /dev/null
+++ b/specs/approved/deploy-templates.rst
@@ -0,0 +1,721 @@
1..
2 This work is licensed under a Creative Commons Attribution 3.0 Unported
3 License.
4
5 http://creativecommons.org/licenses/by/3.0/legalcode
6
7================
8Deploy Templates
9================
10
11https://storyboard.openstack.org/#!/story/1722275
12
13This builds on the `deployment steps
14spec <http://specs.openstack.org/openstack/ironic-specs/specs/11.1/deployment-steps-framework.html>`__,
15such that it is possible to request a particular set of deployment steps by
16requesting a set of traits.
17
18When Ironic is used with Nova, this allows the end user to pick a Flavor, that
19requires a specific set of traits, that Ironic turns into a request for a
20specific set of deployment steps in Ironic.
21
22Problem description
23===================
24
25One node can be configured in many ways. Different user workloads require the
26node to be configured in different ways. As such, operators would like a single
27pool of hardware to be reconfigured to match each users requirements, rather
28than attempting to guess how many of each configurations are required in
29advance.
30
31In this spec, we are focusing on reconfiguring regular hardware to match the
32needs of each users workload. We are not considering composable hardware.
33
34When creating a Nova instance with Ironic as the backend, there is currently no
35way to influence the deployment steps used by Ironic to provision the node.
36
37Example Use Case
38----------------
39
40Consider a bare metal node that has three disks. Different workloads may
41require different RAID configurations. Operators want to test their hardware
42and determine the specific set of RAID configurations that work best, and offer
43that choice to users of Nova, so they can pick the configuration that best
44matches their workload.
45
46Proposed change
47===============
48
49Context: Deployment Steps Framework
50-----------------------------------
51
52This spec depends on the deployment steps framework spec. The deployment steps
53framework provides the concept of a deployment step that may be executed when a
54node is deployed.
55
56It is assumed that there is a default set of steps that an operator configures
57to happen by default on every deploy. Each step task has a priority defined, to
58determine the order that that step runs relative to other enabled deploy steps.
59Configuration options and hard-coded priorities define if a task runs by
60default or not. The priority says when a task runs if the task should run
61during deployment.
62
63Deploy Templates
64----------------
65
66This spec introduces the concept of a ``Deploy Template``. It is a mapping
67from a valid trait name for the node to one or more Deployment Steps and all
68the arguments that should be given to those deployment steps. There is a new
69API added to Ironic for Operators to specify these ``Deploy Templates``.
70
71To allow a ``Deploy Template`` for a given Ironic node, you add the
72trait name of the deploy template to the ``traits`` list on each Ironic
73node that needs the deploy template enabled. The validation of the node must
74fail if any of the enabled ``Deploy Templates`` are not supported on that node.
75
76It is worth noting that all traits set on the node are synchronised to Nova's
77placement API. In turn, should a user request a flavor that requires a trait
78(that may or may not map to a deploy template) only nodes that have the trait
79set will be offered as candidates by Nova's Placement API.
80
81To request the specified ``Deploy Template`` when you provision a particular
82Ironic node, the corresponding trait is added to a list in
83``node.instance_info`` under the key of ``traits``. This is what Nova's ironic
84virt driver already does for any required trait in the flavor's extra specs for
85that instance. Again, Ironic already validates that the traits in
86``node.instance_info`` are a subset of the traits that the Operator has set on
87the Ironic node, via the new node-traits API.
88
89During the provisioning process Ironic checks the list of traits set in
90``node.instance_info`` and checks if they match any ``Deploy Templates``. The
91list of matching ``Deploy Templates`` are then used to extend the list of
92deployment steps for this particular provision operation. As already
93defined in the deployment steps framework, this is then combined with the list
94of deployment steps from what is configured to happen by default for all
95builds.
96
97The order in which the steps are executed will be defined by the priority of
98each step. If the code for a deploy step defines a deploy priority of 0, and
99that is not changed by a configuration option, that deploy step does not
100get executed by default. If a deploy template specifies a priority
101(this is required if the code has a default priority of 0), this overrides both
102the code default and any configuration override.
103
104It is acceptable to request the same deploy step more than once. This could be
105done to execute a deploy step multiple times with different arguments, for
106example to partition multiple disks.
107
108Any deploy step in a requested deploy template will override the default
109arguments and priority for that deploy step. 0 is the only priority override
110that can be set for any of the core deploy steps, i.e. you can only disable the
111core step, you can't change the order of its execution.
112
113Trait names used in Deploy Templates should be unique - no two deploy templates
114should specify the same trait name.
115
116In summary, you can use traits to specify additional deploy steps, by the
117mapping between traits and deploy steps specified in the new
118``Deploy Templates`` API.
119
120Current Limitations
121-------------------
122
123When mapping this back to Nova integration, currently there would need to be
124a flavor for each of these combinations of traits and resource_class. Longer
125term Nova is expected to offer the option of a user specifying an override
126trait on a boot request, based on what the flavor says is possible. This spec
127has no impact on the Nova ironic virt driver beyond what is already implemented
128to support `node-traits
129<http://specs.openstack.org/openstack/ironic-specs/specs/approved/node-traits.html>`__.
130
131Example
132-------
133
134Lets now look at how we could request a particular deploy template via Nova.
135
136``FlavorVMXMirror`` has these extra specs:
137
138* ``resource:CUSTOM_COMPUTE_A = 1``
139* ``trait:CUSTOM_CLASS_A = required``
140* ``trait:CUSTOM_BM_CONFIG_BIOS_VMX_ON = required``
141* ``trait:CUSTOM_BM_CONFIG_RAID_DISK_MIRROR = required``
142
143``FlavorNoVMXStripe`` has these extra specs:
144
145* ``resource:CUSTOM_COMPUTE_A = 1``
146* ``trait:CUSTOM_CLASS_A = required``
147* ``trait:CUSTOM_BM_CONFIG_BIOS_VMX_OFF = required``
148* ``trait:CUSTOM_BM_CONFIG_RAID_DISK_STRIPE = required``
149
150It's possible the operator has set all of the Ironic nodes with ``COMPUTE_A``
151as the resource class to have all of these traits assigned:
152
153* ``CUSTOM_BM_CONFIG_BIOS_VMX_ON``
154* ``CUSTOM_BM_CONFIG_BIOS_VMX_OFF``
155* ``CUSTOM_OTHER_TRAIT_I_AM_USUALLY_IGNORED``
156* ``CUSTOM_BM_CONFIG_RAID_DISK_MIRROR``
157* ``CUSTOM_BM_CONFIG_RAID_DISK_STRIPE``
158
159The Operator has also defined the following deploy templates::
160
161 {
162 "deploy-templates": [
163 {
164 "name": "CUSTOM_BM_CONFIG_RAID_DISK_MIRROR",
165 "steps": [
166 {
167 "interface": "raid",
168 "step": "create_configuration",
169 "args": {
170 "logical_disks": [
171 {
172 "size_gb": "MAX",
173 "raid_level": "1",
174 "is_root_volume": true
175 }
176 ],
177 "delete_configuration": true
178 },
179 "priority": 10
180 }
181 ]
182 },
183 {
184 "name": "CUSTOM_BM_CONFIG_RAID_DISK_STRIPE",
185 "steps": [
186 {
187 "interface": "raid",
188 "step": "create_configuration",
189 "args": {
190 "logical_disks": [
191 {
192 "size_gb": "MAX",
193 "raid_level": "0",
194 "is_root_volume": true
195 }
196 ],
197 "delete_configuration": true
198 },
199 "priority": 10
200 }
201 ]
202 },
203 {
204 "name": "CUSTOM_BM_CONFIG_BIOS_VMX_ON",
205 "steps": [...]
206 },
207 {
208 "name": "CUSTOM_BM_CONFIG_BIOS_VMX_OFF",
209 "steps": [...]
210 }
211 ]
212 }
213
214When a Nova instance is created with ``FlavorVMXMirror``, the required traits
215for that flavor are set on ``node.instance_info['traits']`` such that Ironic
216adds the deploy steps defined in ``CUSTOM_BM_CONFIG_BIOS_VMX_ON`` and
217``CUSTOM_BM_CONFIG_RAID_DISK_MIRROR``, and the node is appropriately configured
218for workloads that want that specific flavor.
219
220Alternatives
221------------
222
223Alternative approach
224~~~~~~~~~~~~~~~~~~~~
225
226This design solves two problems:
227
2281. I want to request some custom configuration to be applied to my bare metal
229 server during provisioning.
2302. Ensure that my instance is scheduled to a bare metal node that supports
231 the requested configuration.
232
233As with capabilities, the proposed design uses a single field (traits) to
234encode configuration and scheduling information. An alternative approach could
235separate these two concerns.
236
237Deploy templates could be requested by a name (not necessarily a trait) or UUID
238as a nova flavor ``extra_spec``, and pushed to a ``deploy_templates`` field in
239the ironic node's ``instance_info`` field by the nova virt driver. Ironic would
240then apply the requested deploy templates during provisioning.
241
242If some influence in the scheduling process is required, this could be provided
243by traits, but this would be a separate concern.
244
245Adapting the earlier example:
246
247``FlavorVMXMirror`` has these extra specs:
248
249* ``resource:CUSTOM_COMPUTE_A = 1``
250* ``trait:CUSTOM_BM_CONFIG_BIOS_VMX_ON = required``
251* ``trait:CUSTOM_BM_CONFIG_RAID_DISK_MIRROR = required``
252* ``deploy_template:BIOS_VMX_ON=<?>``
253* ``deploy_template:BIOS_RAID_DISK_MIRROR=<?>``
254
255Only ironic nodes supporting the ``CUSTOM_BM_CONFIG_BIOS_VMX_ON`` and
256``CUSTOM_BM_CONFIG_RAID_DISK_MIRROR`` traits would be scheduled to, and the
257nova virt driver would set ``instance_info.deploy_templates`` to
258``BIOS_VMX_ON,BIOS_RAID_DISK_MIRROR``.
259
260There are some benefits to this alternative approach:
261
262* It would automatically support cases beyond the simple one trait mapping to
263 one deploy template case we have here. For example, to support deploy
264 template ``X``, features ``Y`` and ``Z`` must be supported by the node
265 (without combinatorial trait explosions).
266* In isolation, the configuration mechanism is conceptually simpler - the
267 flavor specifies a deploy template directly.
268* It would work in standalone ironic installs without introducing concepts from
269 placement.
270* We don't overload the concept of traits for carrying configuration
271 information.
272
273There are also some drawbacks:
274
275* Additional complexity for users and operators that now need to apply both
276 traits and deploy templates to flavors.
277* Less familiar for users of capabilities.
278* Having flavors that specify resources, traits and deploy templates in
279 ``extra_specs`` could leave operators and users scratching their heads.
280
281Extensions
282~~~~~~~~~~
283
284This spec attempts to specify the minimum viable feature that builds on top
285of the deployment steps framework specification. As such, there are many
286possible extensions to this concept that are not being included:
287
288* While you can use standard traits as names of the deploy templates, it is
289 likely that many operators will be forced into using custom traits for most
290 of their deploy templates. We could better support the users of standard
291 traits if we added a list of traits associated with each deploy template,
292 in addition to the trait based name. This list of traits will act as an alias
293 for the name of the deploy template, but this alias may also be used
294 by many other deploy templates. The node validate will fail if for any
295 individual node one of traits set maps to multiple deploy templates.
296 To disambiguate which deploy template is requested, you can look at what
297 deploy template names are in the chosen node's trait list. For each deploy
298 template you look at any other traits that can be used to trigger that
299 template, eventually building up a trait to deploy template mapping for each
300 trait set on the node (some traits will not map to any deploy template).
301 That can be used to detect if any of the traits on the node map to multiple
302 deploy templates, causing the node validate to fail.
303
304* For some operators, they will end up creating a crazy number of flavors to
305 cover all the possible combinations of hardware they want to offer. It is
306 hoped Nova will eventually allow operators to have flavors that list possible
307 traits, and a default set of traits, such that end users can request the
308 specific set of traits they require in addition to the chosen flavor.
309
310* While ironic inspector can be used to ensure each node is given an
311 appropriate set of traits, it feels error prone to add so many traits to each
312 Ironic node. It is hoped when a concept of node groups is added, traits could
313 be applied to a group of nodes instead of only applying traits to individual
314 nodes (possibly in a similar way to host aggregates in Nova). One suggestion
315 was to use the Resource Class as a possible grouping, but that is only a very
316 small part of the more general issue of groups nodes to physical networks,
317 routed network segments, power distribution groups, all mapping to different
318 ironic conductors, etc.
319
320* There were discussions about automatically detecting which Deploy
321 Templates each of the nodes supported. However most operators will want to
322 control what is available to only the things they wish to support.
323
324Data model impact
325-----------------
326
327Two new database tables will be added for deploy templates::
328
329 CREATE TABLE deploy_templates (
330 id INT(11) NOT NULL AUTO_INCREMENT,
331 name VARCHAR(255) CHARACTER SET utf8 NOT NULL,
332 uuid varchar(36) DEFAULT NULL,
333 PRIMARY KEY (id),
334 UNIQUE KEY `uniq_deploy_templaes0uuid` (`uuid`),
335 UNIQUE KEY `uniq_deploy_templaes0name` (`name`),
336 )
337
338 CREATE TABLE deploy_template_steps (
339 deploy_template_id INT(11) NOT NULL,
340 interface VARCHAR(255) NOT NULL,
341 step VARCHAR(255) NOT NULL,
342 args TEXT NOT NULL,
343 priority INT NOT NULL,
344 KEY `deploy_template_id` (`deploy_template_id`),
345 KEY `deploy_template_steps_interface_idx` (`interface`),
346 KEY `deploy_template_steps_step_idx` (`step`),
347 CONSTRAINT `deploy_template_steps_ibfk_1` FOREIGN KEY (`deploy_template_id`) REFERENCES `deploy_templates` (`id`),
348 )
349
350The ``deploy_template_steps.args`` column is a JSON-encoded object of step
351arguments, ``JsonEncodedDict``.
352
353New ``ironic.objects.deploy_template.DeployTemplate`` and
354``ironic.objects.deploy_template_step.DeployTemplateStep`` objects will be
355added to the object model. The deploy template object will provide support for
356looking up a list of deploy templates that match any of a list of trait names.
357
358State Machine Impact
359--------------------
360
361No impact beyond that already specified in the deploy steps specification.
362
363REST API impact
364---------------
365
366A new REST API endpoint will be added for deploy templates, hidden behind a new
367API microversion. The endpoint will support standard CRUD operations.
368
369In the following API, a UUID or trait name is accepted for a deploy template's
370identity.
371
372List all
373~~~~~~~~
374
375List all deploy templates::
376
377 GET /v1/deploy-templates
378
379Request: empty
380
381Response::
382
383 {
384 "deploy-templates": [
385 {
386 "name": "CUSTOM_BM_CONFIG_RAID_DISK_MIRROR",
387 "steps": [
388 {
389 "interface": "raid",
390 "step": "create_configuration",
391 "args": {
392 "logical_disks": [
393 {
394 "size_gb": "MAX",
395 "raid_level": "1",
396 "is_root_volume": true
397 }
398 ],
399 "delete_configuration": true
400 },
401 "priority": 10
402 }
403 ],
404 "uuid": "8221f906-208b-44a5-b575-f8e8a59c4a84"
405 },
406 {
407 ...
408 }
409 ]
410 }
411
412Response codes: 200, 400
413
414Policy: admin or observer.
415
416Show one
417~~~~~~~~
418
419Show a single deploy template::
420
421 GET /v1/deploy-templates/<deploy template ident>
422
423Request: empty
424
425Response::
426
427 {
428 "name": "CUSTOM_BM_CONFIG_RAID_DISK_MIRROR",
429 "steps": [
430 {
431 "interface": "raid",
432 "step": "create_configuration",
433 "args": {
434 "logical_disks": [
435 {
436 "size_gb": "MAX",
437 "raid_level": "1",
438 "is_root_volume": true
439 }
440 ],
441 "delete_configuration": true
442 },
443 "priority": 10
444 }
445 ],
446 "uuid": "8221f906-208b-44a5-b575-f8e8a59c4a84"
447 }
448
449Response codes: 200, 400, 404
450
451Policy: admin or observer.
452
453Create
454~~~~~~
455
456Create a deploy template::
457
458 POST /v1/deploy-templates
459
460Request::
461
462 {
463 "name": "CUSTOM_BM_CONFIG_RAID_DISK_MIRROR",
464 "steps": [
465 {
466 "interface": "raid",
467 "step": "create_configuration",
468 "args": {
469 "logical_disks": [
470 {
471 "size_gb": "MAX",
472 "raid_level": "1",
473 "is_root_volume": true
474 }
475 ],
476 "delete_configuration": true
477 },
478 "priority": 10
479 }
480 ],
481 }
482
483Response: as for show one.
484
485Response codes: 201, 400, 409
486
487Policy: admin.
488
489Update
490~~~~~~
491
492Update a deploy template::
493
494 PATCH /v1/deploy-templates/{deploy template ident}
495
496Request::
497
498 [
499 {
500 "op": "replace",
501 "path": "/name"
502 "value": "CUSTOM_BM_CONFIG_RAID_DISK_MIRROR"
503 },
504 {
505 "op": "replace",
506 "path": "/steps"
507 "value": [
508 {
509 "interface": "raid",
510 "step": "create_configuration",
511 "args": {
512 "logical_disks": [
513 {
514 "size_gb": "MAX",
515 "raid_level": "1",
516 "is_root_volume": true
517 }
518 ],
519 "delete_configuration": true
520 },
521 "priority": 10
522 }
523 ]
524 }
525 ]
526
527Response: as for show one.
528
529Response codes: 200, 400, 404, 409
530
531Policy: admin.
532
533The ``name`` and ``steps`` fields can be updated. The ``uuid`` field cannot.
534
535Delete
536~~~~~~
537
538Delete a deploy template::
539
540 DELETE /v1/deploy-templates/{deploy template ident}
541
542Request: empty
543
544Response: empty
545
546Response codes: 204, 400, 404
547
548Policy: admin.
549
550Client (CLI) impact
551-------------------
552
553"ironic" CLI
554~~~~~~~~~~~~
555
556None
557
558"openstack baremetal" CLI
559~~~~~~~~~~~~~~~~~~~~~~~~~
560
561In each of the following commands, a UUID or trait name is accepted for the
562deploy template's identity.
563
564For the ``--steps`` argument, either a path to a file containing the JSON data
565or ``-`` is required. If ``-`` is passed, the JSON data will be read from
566standard input.
567
568List deploy templates::
569
570 openstack baremetal deploy template list
571
572Show a single deploy template::
573
574 openstack baremetal deploy template show <deploy template ident>
575
576Create a deploy template::
577
578 openstack baremetal deploy template create --name <trait> --steps <deploy steps>
579
580Update a deploy template::
581
582 openstack baremetal deploy template set <deploy template ident> [--name <trait] [--steps <deploy steps>]
583
584Delete a deploy template::
585
586 openstack baremetal deploy template delete <deploy template ident>
587
588In these commands, ``<deploy steps>`` are in JSON format and support the same
589input methods as clean steps - string, file or standard input.
590
591RPC API impact
592--------------
593
594None
595
596Driver API impact
597-----------------
598
599None
600
601Nova driver impact
602------------------
603
604Existing traits integration is enough, only now the selected traits on boot
605become more important.
606
607Ramdisk impact
608--------------
609
610None
611
612Security impact
613---------------
614
615Allowing the deployment process to be customised via deploy templates could
616open up security holes. These risks are mitigated, as seen through the
617following observations:
618
619* Only admins can define the set of allowed traits for each node.
620* Only admins can define the set of requested traits for each Nova flavor, and
621 allow access to that flavor for other users.
622* Only admins can create or update deploy templates via the API.
623* Deploy steps referenced in deploy templates are defined in driver code.
624
625Other end user impact
626---------------------
627
628Users will need to be able to discover what each Nova flavor does in terms of
629deployment customisation. Beyond checking requested traits and
630cross-referencing with the ironic deploy templates API, this is deemed to be
631out of scope. Operators should provide sufficient documentation about the
632properties of each flavor. The ability to look up a deploy template by trait
633name should help here.
634
635Scalability impact
636------------------
637
638Increased activity during deployment could have a negative impact on the
639scalability of ironic.
640
641Performance Impact
642------------------
643
644Increased activity during deployment could have a negative impact on the
645performance of ironic, including increasing the time required to provision a
646node.
647
648Other deployer impact
649---------------------
650
651Deployers will need to ensure that Nova flavors have required traits set
652appropriately.
653
654Developer impact
655----------------
656
657None
658
659Implementation
660==============
661
662Assignee(s)
663-----------
664
665Primary assignee:
666 Mark Goddard (mgoddard)
667
668Other contributors:
669
670* Dmitry Tantsur (dtantsur)
671* Ruby Loo (rloo)
672
673Work Items
674----------
675
676* Add DB tables and objects for deploy templates
677* Write code to map traits to deploy templates
678* Extend node validation to check all deploy templates are valid
679* Add API to add deploy templates
680* Extend CLI to support above API
681* Write tests
682
683Dependencies
684============
685
686* Node traits spec
687 http://specs.openstack.org/openstack/ironic-specs/specs/approved/node-traits.html
688* Deploy steps spec
689 http://specs.openstack.org/openstack/ironic-specs/specs/11.1/deployment-steps-framework.html
690
691Testing
692=======
693
694Unit tests will be added to ironic. Tempest API tests will exercise the deploy
695templates CRUD API.
696
697Upgrades and Backwards Compatibility
698====================================
699
700The deploy steps API endpoint will be hidden behind a new API version.
701
702During normal operation when the ironic conductor is not pinned, deploy
703templates will be used to add deploy steps during node provisioning, even if
704the caller of the node state API uses a microversion that does not support
705deploy templates.
706
707During an upgrade when the ironic conductor is pinned, deploy templates will
708not be used to add deploy steps during node provisioning.
709
710Documentation Impact
711====================
712
713* Admin guide on how to configure Nova flavors and deploy templates
714* Update API ref
715* Update CLI docs
716
717References
718==========
719
720* http://specs.openstack.org/openstack/ironic-specs/specs/11.1/deployment-steps-framework.html
721* http://specs.openstack.org/openstack/ironic-specs/specs/approved/node-traits.html
diff --git a/specs/not-implemented/deploy-templates.rst b/specs/not-implemented/deploy-templates.rst
new file mode 120000
index 0000000..5dee813
--- /dev/null
+++ b/specs/not-implemented/deploy-templates.rst
@@ -0,0 +1 @@
../approved/deploy-templates.rst \ No newline at end of file