From 4b7dd6e512d412f3f321dfe38b169ca2448527e6 Mon Sep 17 00:00:00 2001 From: Ben Nemec Date: Wed, 1 Mar 2017 16:11:49 -0600 Subject: [PATCH] Machine Readable Sample Config Currently the sample configs generated by oslo.config are targeted for consumption by humans. However, many deployment tools would benefit from a sample config that is machine readable. Change-Id: Ia72980c1d7402af226d06147983be7d09721756a --- specs/pike/machine-readable-sample-config.rst | 232 ++++++++++++++++++ 1 file changed, 232 insertions(+) create mode 100644 specs/pike/machine-readable-sample-config.rst diff --git a/specs/pike/machine-readable-sample-config.rst b/specs/pike/machine-readable-sample-config.rst new file mode 100644 index 0000000..cc626fd --- /dev/null +++ b/specs/pike/machine-readable-sample-config.rst @@ -0,0 +1,232 @@ +================================ + Machine Readable Sample Config +================================ + +https://blueprints.launchpad.net/oslo?searchtext=machine-readable-sample-config + +Currently the sample configs generated by oslo.config are targeted for +consumption by humans. However, many deployment tools would benefit from a +sample config that is machine readable. + +Problem description +=================== + +All deployment tools have to solve a similar problem: how to generate the +config files for each service at deployment time. There are various approaches +to this today, including manually updated templates, semi-automatically +generated templates, and using an ini file tool to update the existing sample +config for each project. + +These approaches all have various drawbacks. Template approaches generally +require some level of tedious human editing to template all of the values +the deployment tool needs to set. Ini file tools are not aware of the sample +config format and thus are only able to set values in the correct section, +which usually separates them from the sample config documentation of the +option. None of these approaches make it easy for a deployment tool to +provide a user-friendly interface to the config options. + +The problem with the current ini format sample config in this case is that it +is not easily machine-readable. The content is all written in commented +blocks which would have to be parsed by a deployment tool. Since that +comment format is not guaranteed to remain the same, it would require ongoing +maintenance by each deployment tool using this approach. + +Proposed change +=============== + +The proposal is to write machine-readable sample config files that output the +same data as the existing ini files, but in a YAML or JSON format that can be +more easily consumed by deployment tools. + +Example of proposed output:: + + generator_options: + args: --config-file=etc/nova/nova-config-generator.conf + output_file: etc/nova/nova.conf.yaml + wrap_width: 80 + namespace: + - nova.conf + - oslo.log + ... + summarize: False + # minimal intentionally omitted as I don't think it makes sense in + # this context + [more generator details can be added as desired] + options: + DEFAULT: + image_service: + # The list of parameters is for example only. The actual output + # can include anything deployment tools would find useful. + default: nova.image.glance.GlanceImageService + documentation: Service used to search for and retrieve images. + type: string + required: True + deprecated: False + deprecated_reason: blabla + deprecated_for_removal: False + deprecated_since: XXX + deprecated_name: old_parameter + [all other parameters passed to the opt constructor] + new_opt: + deprecated_name: old_opt + ... + ... + conductor: + workers: (...) + ... + ... + deprecated_options: + DEFAULT: + old_opt: + default: foo + documentation: Example deprecated opt + type: string + replacement_name: new_opt + replacement_group: DEFAULT + ... + +This structure would contain all of the details needed to write a +well-formatted config file, and it would be easier to inject values in the +appropriate place because we would be starting from the actual opt data +instead of an ini file full of comments. It should require little to no +manual maintenance because it would be generated from sample config data +already provided by each project. + +Another suggested change in this format from the existing generated sample +config is to write deprecated opts as first-class citizens rather than just +as a deprecated_name/group on the new opt. This way the structure will still +align with the previous config, which should make it easier to transition +from release to release. These opts would, of course, be marked deprecated so +it is clear that they should no longer be used. An example of this is included +above. + +Note on the implementation: In order to easily support both YAML and JSON +formats, I think the generator should work with nested dicts that can simply +be output using the appropriate module. Since YAML and JSON formatters +already exist we shouldn't need to reinvent those wheels. + +Alternatives +------------ + +Some of the existing methods for doing this and their drawbacks are discussed +above in the problems section. These methods work, but there is a lot of +duplication of effort across all deployment projects so it is not desirable +to continue this way. + +Each project could automate their config file generation, similar to the +OpenStack Helm tool in the References section, but there again we have a +duplication problem that oslo.config can solve. + +Impact on Existing APIs +----------------------- + +We would need to add a --format parameter to the oslo-config-generator so +users can generate sample configs with the new format(s). This would +default to the existing ini-style format for backwards compatibility. + +Security impact +--------------- + +Should be none. For the most part this is just a new format for data that +was already being generated. One possible exception would be capturing the +parameters to the config generator, but I don't see anything in there that +should be sensitive. + +Performance Impact +------------------ + +Generating the new file formats will take additional time, but by default +only the existing format will be created so there should be no change. + +Configuration Impact +-------------------- + +Outside of the --format CLI param, no new config opts are proposed. + +Developer Impact +---------------- + +None. Developers are already exposing config opts to the config generator, +and this is simply adding another use for that. + +Testing Impact +-------------- + +This functionality would be unit tested in the same way as the existing config +generator. We would need to add YAML/JSON scenarios to the unit tests. + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + bnemec + +Other contributors: + emacchi + +Milestones +---------- + +Target Milestone for completion: We would like to complete this work in Pike, +probably for consumption by deployment tools in the Queens cycle. + +Work Items +---------- + +* Implement YAML output +* Implement JSON output +* Add test scenarios for new formats + +Incubation +========== + +Not being incubated. + +Adoption +-------- + +NA + +Library +------- + +NA + +Anticipated API Stabilization +----------------------------- + +NA + +Documentation Impact +==================== + +This work is targeted at deployment tools, so there should be little to no +user-facing documentation changes required. We would want to reference the +existence of these new sample formats in the oslo.config documentation. + +The format and structure of the output files will need to be documented for +deployment tool consumption. + +Dependencies +============ + +None + +References +========== + +`Cross-Deployment Tool PTG Session +`_, specifically the +Problem #1 section. + +https://github.com/alanmeadows/gen-oslo-openstack-helm + +.. note:: + + This work is licensed under a Creative Commons Attribution 3.0 + Unported License. + http://creativecommons.org/licenses/by/3.0/legalcode +