Rework the 'oslo-config-generator' documentation

There's a lot of shuffling docs around, but the main point is to
document the application parameters and the support for multiple output
formats recently added.

Change-Id: I26827801df917619d4256ef4c718051f5c395a29
Implements: bp machine-readable-sample-config

doc/source/generator.rst

@@ -2,61 +2,136 @@
  oslo-config-generator
 =======================
 
-oslo-config-generator is a utility for generating sample config files. For
+`oslo-config-generator` is a utility for generating sample config files in a
-example, to generate a sample config file for oslo.messaging you would run::
+variety of formats. Sample config files list all of the available options,
+along with their help string, type, deprecated aliases and defaults.  These
+sample files can be used as config files for `oslo.config` itself (``ini``) or
+by configuration management tools (``json``, ``yaml``).
 
-  $> oslo-config-generator --namespace oslo.messaging > oslo.messaging.conf
+.. versionadded:: 1.4.0
 
-This generated sample lists all of the available options, along with their help
+.. versionchanged:: 4.3.0
-string, type, deprecated aliases and defaults.
 
-To generate a sample config file for an application ``myapp`` that has
+   The :option:`oslo-config-generator --format` parameter was added, which
-its own options and uses oslo.messaging, you can list both
+   allows outputting in additional formats.
-namespaces::
 
-  $> oslo-config-generator --namespace myapp --namespace oslo.messaging > myapp.conf
+Usage
+-----
 
-.. versionadded:: 1.4
+.. program:: oslo-config-generator
+
+.. code-block:: shell
+
+   oslo-config-generator
+      --namespace <namespace> [--namespace <namespace> ...]
+      [--output-file <output-file>]
+      [--wrap-width <wrap-width>]
+      [--format <format>]
+      [--minimal]
+      [--summarize]
+
+.. option:: --namespace <namespace>
+
+   Option namespace under ``oslo.config.opts`` in which to query for options.
+
+.. option:: --output-file <output-file>
+
+   Path of the file to write to.
+
+   :Default: stdout
+
+.. option:: --wrap-width <wrap-width>
+
+   The maximum length of help lines.
+
+   :Default: 70
+
+.. option:: --format <format>
+
+   Desired format for the output. ``ini`` is the only format that can be used
+   directly with `oslo.config`. ``json`` and ``yaml`` are intended for
+   third-party tools that want to write config files based on the sample config
+   data. For more information, refer to :ref:`machine-readable-configs`.
+
+   :Choices: ini, json, yaml
+
+.. option:: --minimal
+
+   Generate a minimal required configuration.
+
+.. option:: --summarize
+
+   Only output summaries of help text to config files. Retain longer help text
+   for Sphinx documents.
+
+For example, to generate a sample config file for `oslo.messaging` you would
+run:
+
+.. code-block:: shell
+
+   $ oslo-config-generator --namespace oslo.messaging > oslo.messaging.conf
+
+To generate a sample config file for an application ``myapp`` that has its own
+options and uses `oslo.messaging`, you would list both namespaces:
+
+.. code-block:: shell
+
+   $ oslo-config-generator --namespace myapp \
+       --namespace oslo.messaging > myapp.conf
+
+To generate a sample config file for `oslo.messaging` in `JSON` format, you
+would run:
+
+.. code-block:: shell
+
+   $ oslo-config-generator --namespace oslo.messaging \
+       --format json > oslo.messaging.conf
 
 Defining Option Discovery Entry Points
 --------------------------------------
 
-The ``--namespace`` option specifies an entry point name registered under the
+The :option:`oslo-config-generator --namespace` option specifies an entry point
-'oslo.config.opts' entry point namespace. For example, in oslo.messaging's
+name registered under the ``oslo.config.opts`` entry point namespace. For
-setup.cfg we have::
+example, in the `oslo.messaging` ``setup.cfg`` we have:
+
+.. code-block:: ini
 
   [entry_points]
   oslo.config.opts =
       oslo.messaging = oslo.messaging.opts:list_opts
 
 The callable referenced by the entry point should take no arguments and return
-a list of (``group``, [opt_1, opt_2]) tuples, where ``group`` is either a
+a list of ``(group, [opt_1, opt_2])`` tuples, where ``group`` is either a group
-group name as a string or an ``OptGroup`` object. Passing the ``OptGroup``
+name as a string or an ``OptGroup`` object. Passing the ``OptGroup`` object
-object allows the consumer of the ``list_opts`` method to access and publish
+allows the consumer of the ``list_opts`` method to access and publish group
-group help. An example, using both styles::
+help. An example, using both styles:
 
-  opts1 = [
+.. code-block:: python
-      cfg.StrOpt('foo'),
-      cfg.StrOpt('bar'),
-  ]
 
-  opts2 = [
+   from oslo_config import cfg
-      cfg.StrOpt('baz'),
-  ]
 
-  baz_group = cfg.OptGroup(name='baz_group'
+   opts1 = [
-                           title='Baz group options',
+       cfg.StrOpt('foo'),
-                           help='Baz group help text')
+       cfg.StrOpt('bar'),
-  cfg.CONF.register_group(baz_group)
+   ]
 
-  cfg.CONF.register_opts(opts1, group='blaa')
+   opts2 = [
-  cfg.CONF.register_opts(opts2, group=baz_group)
+       cfg.StrOpt('baz'),
+   ]
 
-  def list_opts():
+   baz_group = cfg.OptGroup(name='baz_group'
-      # Allows the generation of the help text for
+                            title='Baz group options',
-      # the baz_group OptGroup object. No help
+                            help='Baz group help text')
-      # text is generated for the 'blaa' group.
+   cfg.CONF.register_group(baz_group)
-      return [('blaa', opts1), (baz_group, opts2)]
+
+   cfg.CONF.register_opts(opts1, group='blaa')
+   cfg.CONF.register_opts(opts2, group=baz_group)
+
+   def list_opts():
+       # Allows the generation of the help text for
+       # the baz_group OptGroup object. No help
+       # text is generated for the 'blaa' group.
+       return [('blaa', opts1), (baz_group, opts2)]
 
 .. note::
 
@@ -89,11 +164,11 @@ The hooks are registered in a separate entry point namespace
 (``oslo.config.opts.defaults``), using the same entry point name as
 **the application's** ``list_opts()`` function.
 
-::
+.. code-block:: ini
 
-  [entry_points]
+   [entry_points]
-  oslo.config.opts.defaults =
+   oslo.config.opts.defaults =
-      keystone = keystone.common.config:update_opt_defaults
+       keystone = keystone.common.config:update_opt_defaults
 
 .. warning::
 
@@ -121,7 +196,7 @@ public :func:`set_defaults` functions in any libraries for which it
 has option defaults to override, just as the application does during
 its normal startup process.
 
-::
+.. code-block:: python
 
    from oslo_log import log
 
@@ -130,12 +205,105 @@ its normal startup process.
            default_log_levels=log.get_default_log_levels() + ['noisy=WARN'],
        )
 
+.. _machine-readable-configs:
+
+Generating Machine Readable Configs
+-----------------------------------
+
+All deployment tools have to solve a similar problem: how to generate the
+config files for each service at deployment time. To help with this problem,
+`oslo-config-generator` can generate machine-readable sample config files that
+output the same data as the INI files used by `oslo.config` itself, but in a
+YAML or JSON format that can be more easily consumed by deployment tools.
+
+.. important::
+
+   The YAML and JSON-formatted files generated by `oslo-config-generator`
+   cannot be used by `oslo.config` itself - they are only for use by other
+   tools.
+
+For example, some YAML-formatted output might look like so:
+
+.. code-block:: yaml
+
+   generator_options:
+     config_dir: []
+     config_file: []
+     format_: yaml
+     minimal: false
+     namespace:
+     - keystone
+     output_file: null
+     summarize: false
+     wrap_width: 70
+   options:
+     DEFAULT:
+       help: ''
+       opts:
+       - advanced: false
+         choices: []
+         default: null
+         deprecated_for_removal: false
+         deprecated_opts: []
+         deprecated_reason: null
+         deprecated_since: null
+         dest: admin_token
+         help: Using this feature is *NOT* recommended. Instead, use the `keystone-manage
+           bootstrap` command. The value of this option is treated as a "shared secret"
+           that can be used to bootstrap Keystone through the API. This "token" does
+           not represent a user (it has no identity), and carries no explicit authorization
+           (it effectively bypasses most authorization checks). If set to `None`, the
+           value is ignored and the `admin_token` middleware is effectively disabled.
+           However, to completely disable `admin_token` in production (highly recommended,
+           as it presents a security risk), remove `AdminTokenAuthMiddleware` (the `admin_token_auth`
+           filter) from your paste application pipelines (for example, in `keystone-paste.ini`).
+         max: null
+         metavar: null
+         min: null
+         mutable: false
+         name: admin_token
+         namespace: keystone
+         positional: false
+         required: false
+         sample_default: null
+         secret: true
+         short: null
+         type: string value
+       - ...
+     ...
+   deprecated_options:
+     DEFAULT:
+     - name: bind_host
+       replacement_group: eventlet_server
+       replacement_name: public_bind_host
+
+where the top-level keys are:
+
+``generator_options``
+
+  The options passed to the :program:`oslo-config-generator` tool itself
+
+``options``
+
+  All options registered in the provided namespace(s). These are grouped under
+  the ``OptGroup`` they are assigned to which defaults to ``DEFAULT`` if unset.
+
+  For information on the various attributes of each option, refer to
+  :doc:`opts`.
+
+``deprecated_options``
+
+  All **deprecated** options registered in the provided namespace(s). Like
+  ``options``, these options are grouped by ``OptGroup``.
+
 Generating Multiple Sample Configs
 ----------------------------------
 
 A single codebase might have multiple programs, each of which use a subset of
 the total set of options registered by the codebase. In that case, you can
-register multiple entry points::
+register multiple entry points:
+
+.. code-block:: ini
 
   [entry_points]
   oslo.config.opts =
@@ -145,32 +313,36 @@ register multiple entry points::
 
 and generate a config file specific to each program::
 
-  $> oslo-config-generator --namespace oslo.messaging \
+.. code-block:: shell
+
+   $ oslo-config-generator --namespace oslo.messaging \
                            --namespace nova.common \
                            --namespace nova.api > nova-api.conf
-  $> oslo-config-generator --namespace oslo.messaging \
+   $ oslo-config-generator --namespace oslo.messaging \
                            --namespace nova.common \
                            --namespace nova.compute > nova-compute.conf
 
 To make this more convenient, you can use config files to describe your config
-files::
+files:
 
-  $> cat > config-generator/api.conf <<EOF
+.. code-block:: shell
-  [DEFAULT]
+
-  output_file = etc/nova/nova-api.conf
+   $ cat > config-generator/api.conf <<EOF
-  namespace = oslo.messaging
+   [DEFAULT]
-  namespace = nova.common
+   output_file = etc/nova/nova-api.conf
-  namespace = nova.api
+   namespace = oslo.messaging
-  EOF
+   namespace = nova.common
-  $> cat > config-generator/compute.conf <<EOF
+   namespace = nova.api
-  [DEFAULT]
+   EOF
-  output_file = etc/nova/nova-compute.conf
+   $ cat > config-generator/compute.conf <<EOF
-  namespace = oslo.messaging
+   [DEFAULT]
-  namespace = nova.common
+   output_file = etc/nova/nova-compute.conf
-  namespace = nova.compute
+   namespace = oslo.messaging
-  EOF
+   namespace = nova.common
-  $> oslo-config-generator --config-file config-generator/api.conf
+   namespace = nova.compute
-  $> oslo-config-generator --config-file config-generator/compute.conf
+   EOF
+   $ oslo-config-generator --config-file config-generator/api.conf
+   $ oslo-config-generator --config-file config-generator/compute.conf
 
 Sample Default Values
 ---------------------
@@ -178,12 +350,14 @@ Sample Default Values
 The default runtime values of configuration options are not always the most
 suitable values to include in sample config files - for example, rather than
 including the IP address or hostname of the machine where the config file
-was generated, you might want to include something like '10.0.0.1'. To
+was generated, you might want to include something like ``10.0.0.1``. To
-facilitate this, options can be supplied with a 'sample_default' attribute::
+facilitate this, options can be supplied with a ``sample_default`` attribute:
 
-  cfg.StrOpt('base_dir'
+.. code-block:: python
-             default=os.getcwd(),
+
-             sample_default='/usr/lib/myapp')
+   cfg.StrOpt('base_dir'
+              default=os.getcwd(),
+              sample_default='/usr/lib/myapp')
 
 API
 ---