summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorStephen Finucane <sfinucan@redhat.com>2017-10-04 11:15:57 +0100
committerStephen Finucane <sfinucan@redhat.com>2017-10-04 11:47:56 +0100
commitd3c952064a4790d074a73b47ac30e2a35fe8b59d (patch)
treea9049001678509021966d6a3d7a5bb674d3dc344
parent3dd0834d7aa09096fecb8c0af86c78c75c4b856c (diff)
Remove 'autogenerate_config_docs'
This tool has been superseded by the 'oslo_config.sphinxext' Sphinx extension and can finally be put to rest. [1] https://docs.openstack.org/oslo.config/latest/reference/sphinxext.html Change-Id: Idae5ba656c3a0da8a26c6f1e63332365c8012c0b
Notes
Notes (review): Verified-1: Zuul Code-Review+2: Andreas Jaeger <jaegerandi@gmail.com> Code-Review+2: Alexandra Settle <a.settle@outlook.com> Workflow+1: Alexandra Settle <a.settle@outlook.com> Verified+2: Jenkins Submitted-by: Jenkins Submitted-at: Mon, 09 Oct 2017 10:28:21 +0000 Reviewed-on: https://review.openstack.org/509401 Project: openstack/openstack-doc-tools Branch: refs/heads/master
-rw-r--r--README.rst6
-rw-r--r--autogenerate_config_docs/README.rst121
-rw-r--r--autogenerate_config_docs/__init__.py18
-rwxr-xr-xautogenerate_config_docs/autohelp-wrapper273
-rwxr-xr-xautogenerate_config_docs/autohelp.py696
-rwxr-xr-xautogenerate_config_docs/diff_branches.py289
-rw-r--r--autogenerate_config_docs/requirements.txt5
-rw-r--r--autogenerate_config_docs/templates/autohelp-category.rst.j245
-rw-r--r--autogenerate_config_docs/templates/autohelp-group.rst.j240
-rw-r--r--autogenerate_config_docs/templates/changes.rst.j261
-rw-r--r--doc/source/autogenerate_config_docs.rst1
-rw-r--r--doc/source/index.rst1
-rw-r--r--releasenotes/notes/the-great-doc-tools-cleanup-1a79e2c200232489.yaml8
-rw-r--r--setup.cfg1
-rw-r--r--tox.ini5
15 files changed, 10 insertions, 1560 deletions
diff --git a/README.rst b/README.rst
index 08b8167..77844d2 100644
--- a/README.rst
+++ b/README.rst
@@ -59,9 +59,3 @@ On openSUSE::
59On Ubuntu:: 59On Ubuntu::
60 60
61 $ apt-get install libxml2-dev libxslt-dev 61 $ apt-get install libxml2-dev libxslt-dev
62
63
64Regenerating config option tables
65---------------------------------
66
67See :ref:`autogenerate_config_docs`.
diff --git a/autogenerate_config_docs/README.rst b/autogenerate_config_docs/README.rst
deleted file mode 100644
index 201d264..0000000
--- a/autogenerate_config_docs/README.rst
+++ /dev/null
@@ -1,121 +0,0 @@
1.. _autogenerate_config_docs:
2
3autogenerate_config_docs
4========================
5
6Automatically generate configuration tables to document OpenStack.
7
8Using the wrapper
9-----------------
10
11``autohelp-wrapper`` is the recommended tool to generate the configuration
12tables. Don't bother using ``autohelp.py`` manually.
13
14The ``autohelp-wrapper`` script installs a virtual environment and all the
15needed dependencies, clones or updates the projects and manuals repositories,
16then runs the ``autohelp.py`` script in the virtual environment.
17
18New and updated flagmappings are generated in the ``openstack-manuals``
19repository (``tools/autogenerate-config-flagmappings/`` directory).
20
21Prior to running the following commands, you need to install several development
22packages.
23
24On Ubuntu:
25
26.. code-block:: console
27
28 $ sudo apt-get install python-dev python-pip python-virtualenv \
29 libxml2-dev libxslt1-dev zlib1g-dev \
30 libmysqlclient-dev libpq-dev libffi-dev \
31 libsqlite3-dev libldap2-dev libsasl2-dev \
32 libjpeg-dev
33
34On RHEL 7 and CentOS 7:
35
36.. code-block:: console
37
38 $ sudo yum install https://www.rdoproject.org/repos/rdo-release.rpm
39 $ sudo yum update
40 $ sudo yum install python-devel python-pip python-virtualenv \
41 libxml2-devel libxslt-devel zlib-devel \
42 mariadb-devel postgresql-devel libffi-devel \
43 sqlite-devel openldap-devel cyrus-sasl-devel \
44 libjpeg-turbo-devel gcc git
45
46.. note::
47 * libjpeg is needed for ironic
48
49The workflow is:
50
51.. code-block:: console
52
53 $ pip install -rrequirements.txt
54 $ ./autohelp-wrapper update
55 $ $EDITOR sources/openstack-manuals/tools/autogenerate-config-flagmappings/*.flagmappings
56 $ ./autohelp-wrapper rst
57 $ # check the results in sources/openstack-manuals
58
59This will generate the tables for all the known projects.
60Note for neutron project: If the driver/plugin resides outside the neutron
61repository, then the driver/plugin has to be explicitly installed within the
62virtual environment to generate the configuration options.
63
64To generate the mappings and tables for a subset of projects, use the code
65names as arguments:
66
67.. code-block:: console
68
69 $ ./autohelp-wrapper update cinder heat
70 $ # edit the mappings files
71 $ ./autohelp-wrapper rst cinder heat
72
73
74Flagmappings files
75------------------
76
77The tool uses flagmapping files to map options to custom categories. Flag
78mapping files can be found in the ``tools/autogenerate-config-flagmappings``
79folder of the openstack-manuals project. Not all projects use flagmapping
80files, as those that do not will be disabled by the presence of a
81``$project.disable`` file in that folder. For those that do, however, the files
82use the following format::
83
84 OPTION_SECTION/OPTION_NAME group1 [group2, ...]
85
86Groups need to be defined manually to organize the configuration tables.
87
88The group values can only contain alphanumeric characters, _ and - (they will
89be used as document IDs).
90
91To make the table titles more user friendly, create or edit the PROJECT.headers
92file in the manuals repository. Each line of this file is of the form:
93
94::
95
96 GROUP A Nice Title
97
98Working with branches
99---------------------
100
101``autohelp-wrapper`` works on the master branch by default, but you can tell it
102to work on another branch:
103
104.. code-block:: console
105
106 $ ./autohelp-wrapper -b stable/liberty update
107
108.. note::
109 The ``-b`` switch doesn't apply to the ``openstack-manuals`` repository
110 which will be left untouched (no ``git branch``, no ``git update``).
111
112
113Generate configuration difference
114---------------------------------
115
116To generate "New, updated, and deprecated options" for each service,
117run ``diff_branches.py``. For example:
118
119.. code-block:: console
120
121 $ ./diff_branches.py stable/liberty stable/mitaka nova
diff --git a/autogenerate_config_docs/__init__.py b/autogenerate_config_docs/__init__.py
deleted file mode 100644
index 52601d6..0000000
--- a/autogenerate_config_docs/__init__.py
+++ /dev/null
@@ -1,18 +0,0 @@
1# Copyright 2012 OpenStack Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License"); you may
4# not use this file except in compliance with the License. You may obtain
5# a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
11# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
12# License for the specific language governing permissions and limitations
13# under the License.
14
15import pbr.version
16
17
18__version__ = pbr.version.VersionInfo('openstack-doc-tools').version_string()
diff --git a/autogenerate_config_docs/autohelp-wrapper b/autogenerate_config_docs/autohelp-wrapper
deleted file mode 100755
index da64300..0000000
--- a/autogenerate_config_docs/autohelp-wrapper
+++ /dev/null
@@ -1,273 +0,0 @@
1#!/bin/bash
2# Licensed under the Apache License, Version 2.0 (the "License"); you may
3# not use this file except in compliance with the License. You may obtain
4# a copy of the License at
5#
6# http://www.apache.org/licenses/LICENSE-2.0
7#
8# Unless required by applicable law or agreed to in writing, software
9# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
10# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
11# License for the specific language governing permissions and limitations
12# under the License.
13
14set -e
15
16HERE=$(pwd)
17VENVDIR=$HERE/venv
18SOURCESDIR=$HERE/sources
19MANUALSREPO=$SOURCESDIR/openstack-manuals
20MAPPINGS_DIR=$MANUALSREPO/tools/autogenerate-config-flagmappings
21AUTOHELP="python $HERE/autohelp.py"
22GITBASE=${GITBASE:-git://git.openstack.org/openstack}
23GITPROJ=${GITPROJ:-git://git.openstack.org/openstack}
24PROJECTS="aodh ceilometer cinder glance heat ironic keystone manila \
25 murano neutron nova sahara senlin trove zaqar"
26MANUALS_PROJECTS="openstack-manuals"
27BRANCH=master
28FAST=0
29QUIET=0
30CLONE_MANUALS=1
31
32usage() {
33 echo "Wrapper for autohelp.py"
34 echo "Usage:"
35 echo " $(basename $0) [ OPTIONS ] dump|update|rst|setup [ project... ]"
36 echo
37 echo "Subcommands:"
38 echo " dump: Dumps the list of options with their attributes"
39 echo " update: Update or create the flagmapping files"
40 echo " rst: Generate the options tables in RST format"
41 echo " setup: Install the environment only"
42 echo
43 echo "Options:"
44 echo " -b BRANCH: Work on this branch (defaults to master)"
45 echo " -g GITPROJ: Use this location for the project git repos "
46 echo " (defaults to git://git.openstack.org/openstack)"
47 echo " -c: Recreate the virtual environment"
48 echo " -f: Work offline: Do not change environment or sources"
49 echo " -e PATH: Create the virtualenv in PATH"
50 echo " -v LEVEL: Verbose message (1 or 2)"
51 echo " (check various python modules imported or not)"
52 echo " -o OUTDIR: Path to output openstack-manuals directory "
53 echo " (defaults to ./sources/openstack-manuals)"
54}
55
56setup_venv() {
57 project=$1
58
59 if [ ! -e $VENVDIR/$project/bin/activate ]; then
60 mkdir -p $VENVDIR/$project
61 virtualenv $VENVDIR/$project
62 fi
63 activate_venv $project
64}
65
66activate_venv() {
67 project=$1
68
69 . $VENVDIR/$project/bin/activate
70 pip install --upgrade pip setuptools
71}
72
73get_project() {
74 project=$1
75 git_url=$GITPROJ
76
77 if [ ! -e $SOURCESDIR/$project ]; then
78 if [[ $MANUALS_PROJECTS =~ (^| )$project($| ) ]]; then
79 git_url=$GITBASE
80 fi
81 git clone $git_url/$project $SOURCESDIR/$project
82
83 if [ -e $MAPPINGS_DIR/$project.extra_repos ]; then
84 while read extra; do
85 git clone $git_url/$extra $SOURCESDIR/$extra
86 done < $MAPPINGS_DIR/$project.extra_repos
87 fi
88
89 else
90 if [ $project != openstack-manuals ]; then
91 (cd $SOURCESDIR/$project && git pull)
92 fi
93
94 if [ -e $MAPPINGS_DIR/$project.extra_repos ]; then
95 while read extra; do
96 (cd $SOURCESDIR/$extra && git pull)
97 done < $MAPPINGS_DIR/$project.extra_repos
98 fi
99 fi
100}
101
102setup_tools() {
103 pip install -rrequirements.txt
104}
105
106while getopts :b:g:e:o:v:cfq opt; do
107 case $opt in
108 b)
109 BRANCH=$OPTARG
110 ;;
111 g)
112 GITPROJ=$OPTARG
113 ;;
114 c)
115 rm -rf $VENVDIR
116 ;;
117 e)
118 VENVDIR=$OPTARG
119 ;;
120 o)
121 MANUALSREPO=$OPTARG
122 MAPPINGS_DIR=$OPTARG/tools/autogenerate-config-flagmappings
123 CLONE_MANUALS=0
124 ;;
125 f)
126 FAST=1
127 ;;
128 q)
129 QUIET=1
130 ;;
131 v)
132 AUTOOPT="-v"
133 if [ $OPTARG = 2 ]; then
134 AUTOOPT="-vv"
135 fi
136 ;;
137 \?)
138 usage
139 exit 1
140 ;;
141 esac
142done
143shift $(($OPTIND - 1))
144
145if [ $# -lt 1 ]; then
146 usage
147 exit 1
148fi
149
150ACTION=$1
151shift
152
153if [ $QUIET -eq 1 ]; then
154 exec 3>&1 >/dev/null
155 exec 4>&2 2>/dev/null
156fi
157
158case $ACTION in
159 update|rst|dump|setup) ;;
160 *)
161 usage
162 exit 1
163 ;;
164esac
165
166if [ ! -e autohelp.py ]; then
167 echo "Execute this script in the autogenerate_config_docs directory."
168 exit 1
169fi
170
171[ $# != 0 ] && PROJECTS="$*"
172
173RELEASE=$(echo $BRANCH | sed 's,^stable.,,')
174
175if [ "$FAST" -eq 0 ] ; then
176 if [ "$CLONE_MANUALS" -eq 1 ] ; then
177 get_project openstack-manuals
178 fi
179
180 for project in $PROJECTS; do
181 setup_venv $project
182 setup_tools
183 if [ -e $MAPPINGS_DIR/$project.requirements ]; then
184 pip install -r $MAPPINGS_DIR/$project.requirements \
185 --allow-all-external
186 fi
187 get_project $project
188
189 (
190 pushd $SOURCESDIR/$project
191 module=$(echo $project | tr - _ )
192 find $module -name "*.pyc" -delete
193 GIT_CMD="git show-ref --verify --quiet refs/heads/$BRANCH"
194 if $GIT_CMD; then
195 git checkout $BRANCH
196 else
197 git checkout -b $BRANCH remotes/origin/$BRANCH
198 fi
199 pip install -rrequirements.txt
200 [ -e "test-requirements.txt" ] && \
201 pip install -rtest-requirements.txt
202 python setup.py install
203 popd
204
205 if [ -e $MAPPINGS_DIR/$project.extra_repos ]; then
206 while read extra; do
207 (
208 cd $SOURCESDIR/$extra
209 pip install -rrequirements.txt
210 [ -e "test-requirements.txt" ] && \
211 pip install -rtest-requirements.txt
212 python setup.py install
213 )
214 done < $MAPPINGS_DIR/$project.extra_repos
215 fi
216 )
217 done
218fi
219
220for project in $PROJECTS; do
221 echo "Working on $project..."
222 activate_venv $project
223 if [ "$ACTION" = "setup" ]; then
224 break
225 fi
226
227 if [ -e $MAPPINGS_DIR/$project.extra_repos ]; then
228 extra_flags=
229 while read extra; do
230 package=$(echo $extra | tr - _)
231 if [ $package = "networking_midonet" ]; then
232 package="midonet"
233 fi
234 if [ $package = "networking_hyperv" ]; then
235 package="hyperv"
236 fi
237 if [ $package = "networking_edge_vpn" ]; then
238 package="networking-edge-vpn"
239 fi
240 if [ $package = "networking_zvm" ]; then
241 package="neutron"
242 cp -r $SOURCESDIR/networking-zvm/neutron/plugins/zvm \
243 $SOURCESDIR/neutron/neutron/plugins
244 cp -r \
245 $SOURCESDIR/networking-zvm/neutron/plugins/ml2/drivers/zvm\
246 $SOURCESDIR/neutron/neutron/plugins/ml2/drivers
247 fi
248 extra_flags="$extra_flags -i $SOURCESDIR/$extra/$package"
249 done < $MAPPINGS_DIR/$project.extra_repos
250 fi
251
252 cd $MAPPINGS_DIR
253
254 case $ACTION in
255 update)
256 $AUTOHELP update $project -i $SOURCESDIR/$project/$project \
257 $extra_flags $AUTOOPT
258 mv $project.flagmappings.new $project.flagmappings
259 ;;
260 rst)
261 $AUTOHELP rst $project -i $SOURCESDIR/$project/$project \
262 $extra_flags $AUTOOPT
263 ;;
264 dump)
265 if [ $QUIET -eq 1 ]; then
266 exec 1>&3
267 exec 2>&4
268 fi
269 $AUTOHELP dump $project -i $SOURCESDIR/$project/$project \
270 $extra_flags $AUTOOPT
271 ;;
272 esac
273done
diff --git a/autogenerate_config_docs/autohelp.py b/autogenerate_config_docs/autohelp.py
deleted file mode 100755
index 66d7cfd..0000000
--- a/autogenerate_config_docs/autohelp.py
+++ /dev/null
@@ -1,696 +0,0 @@
1#!/usr/bin/env python
2# Licensed under the Apache License, Version 2.0 (the "License"); you may
3# not use this file except in compliance with the License. You may obtain
4# a copy of the License at
5#
6# http://www.apache.org/licenses/LICENSE-2.0
7#
8# Unless required by applicable law or agreed to in writing, software
9# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
10# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
11# License for the specific language governing permissions and limitations
12# under the License.
13#
14# A collection of tools for working with flags from OpenStack
15# packages and documentation.
16#
17# For an example of usage, run this program with the -h switch.
18#
19
20# Must import this before argparse
21from oslo_config import cfg
22
23import argparse
24import importlib
25import os
26import pickle
27import re
28import sys
29
30import jinja2
31import stevedore
32
33try:
34 from sqlalchemy import exc
35except Exception:
36 pass
37
38sys.path.insert(0, '.')
39try:
40 from hooks import HOOKS # noqa
41except ImportError:
42 pass
43
44
45EXTENSIONS = ['oslo.cache',
46 'oslo.concurrency',
47 'oslo.db',
48 'oslo.log',
49 'oslo.messaging',
50 'oslo.middleware',
51 'oslo.policy',
52 'oslo.service']
53
54_TYPE_DESCRIPTIONS = {
55 cfg.StrOpt: 'String',
56 cfg.BoolOpt: 'Boolean',
57 cfg.IntOpt: 'Integer',
58 cfg.FloatOpt: 'Floating point',
59 cfg.ListOpt: 'List',
60 cfg.DictOpt: 'Dict',
61 cfg.MultiStrOpt: 'Multi-valued',
62 cfg.IPOpt: 'IP address',
63 cfg.PortOpt: 'Port number',
64 cfg.HostnameOpt: 'Hostname',
65 cfg.URIOpt: 'URI',
66 cfg.HostAddressOpt: 'Host address',
67 cfg._ConfigFileOpt: 'List of filenames',
68 cfg._ConfigDirOpt: 'List of directory names',
69}
70
71register_re = re.compile(r'''^ +.*\.register_opts\((?P<opts>[^,)]+)'''
72 r'''(, (group=)?["'](?P<group>.*)["'])?\)''')
73
74
75def import_modules(repo_location, package_name, verbose=0):
76 """Import modules.
77
78 Loops through the repository, importing module by module to
79 populate the configuration object (cfg.CONF) created from Oslo.
80 """
81
82 with open('ignore.list') as fd:
83 ignore_list = [l for l in fd.read().split('\n')
84 if l and (l[0] != '#')]
85
86 pkg_location = os.path.join(repo_location, package_name)
87 for root, dirs, files in os.walk(pkg_location):
88 skipdir = False
89 for excludedir in ('tests', 'locale',
90 os.path.join('db', 'migration'), 'transfer'):
91 if ((os.path.sep + excludedir + os.path.sep) in root or (
92 root.endswith(os.path.sep + excludedir))):
93 skipdir = True
94 break
95 if skipdir:
96 continue
97 for pyfile in files:
98 if pyfile.endswith('.py'):
99 abs_path = os.path.join(root, pyfile)
100 modfile = abs_path.split(repo_location, 1)[1]
101 modname = os.path.splitext(modfile)[0].split(os.path.sep)
102 modname = [m for m in modname if m != '']
103 modname = '.'.join(modname)
104 if modname.endswith('.__init__'):
105 modname = modname[:modname.rfind(".")]
106 if modname in ignore_list:
107 continue
108 try:
109 module = importlib.import_module(modname)
110 if verbose >= 1:
111 print("imported %s" % modname)
112 except ImportError as e:
113 """
114 work around modules that don't like being imported in
115 this way FIXME This could probably be better, but does
116 not affect the configuration options found at this stage
117 """
118 if verbose >= 2:
119 print("Failed to import: %s (%s)" % (modname, e))
120 continue
121 except cfg.DuplicateOptError as e:
122 """
123 oslo.cfg doesn't allow redefinition of a config option, but
124 we don't mind. Don't fail if this happens.
125 """
126 if verbose >= 2:
127 print(e)
128 continue
129 except cfg.NoSuchGroupError as e:
130 """
131 If a group doesn't exist, we ignore the import.
132 """
133 if verbose >= 2:
134 print(e)
135 continue
136 except exc.InvalidRequestError as e:
137 if verbose >= 2:
138 print(e)
139 continue
140 except Exception as e:
141 print("Impossible to import module %s" % modname)
142 raise
143
144 _register_runtime_opts(module, abs_path, verbose)
145 _run_hook(modname)
146
147 # All the components provide keystone token authentication, usually using a
148 # pipeline. Since the auth_token options can only be discovered at runtime
149 # in this configuration, we force their discovery by importing the module.
150 try:
151 import keystonemiddleware.auth_token # noqa
152 except cfg.DuplicateOptError:
153 pass
154
155
156def _run_hook(modname):
157 try:
158 HOOKS[modname]()
159 except KeyError:
160 pass
161
162
163def _register_runtime_opts(module, abs_path, verbose):
164 """Handle options not registered on module import.
165
166 This function parses the .py files to discover calls to register_opts in
167 functions and methods. It then explicitly call cfg.register_opt on each
168 option to register (most of) them.
169 """
170
171 with open(abs_path) as fd:
172 for line in fd:
173 m = register_re.search(line)
174 if not m:
175 continue
176
177 opts_var = m.group('opts')
178 opts_group = m.group('group')
179
180 # Get the object (an options list) from the opts_var string.
181 # This requires parsing the string which can be of the form
182 # 'foo.bar'. We treat each element as an attribute of the previous.
183 register = True
184 obj = module
185 for item in opts_var.split('.'):
186 try:
187 obj = getattr(obj, item)
188 except AttributeError:
189 # FIXME(gpocentek): AttributeError is raised when a part of
190 # the opts_var string is not an actual attribute. This will
191 # need more parsing tricks.
192 register = False
193 if verbose >= 2:
194 print("Ignoring %(obj)s in %(module)s" %
195 {'obj': opts_var, 'module': module})
196 break
197
198 if register and isinstance(obj, list):
199 for opt in obj:
200 if not isinstance(opt, cfg.Opt):
201 continue
202 try:
203 cfg.CONF.register_opt(opt, opts_group)
204 except cfg.DuplicateOptError:
205 # ignore options that have already been registered
206 pass
207
208
209def _sanitize_default(opt):
210 """Adapts unrealistic default values."""
211
212 # If the Oslo version is recent enough, we can use the 'sample_default'
213 # attribute
214 if (hasattr(opt, 'sample_default') and opt.sample_default is not None):
215 return str(opt.sample_default)
216
217 if ((type(opt).__name__ == "ListOpt") and (type(opt.default) == list)):
218 return ", ".join(str(item) for item in opt.default)
219
220 default = str(opt.default)
221
222 if default == os.uname()[1]:
223 return 'localhost'
224
225 if opt.name == 'bindir':
226 return '/usr/local/bin'
227
228 if opt.name == 'my_ip':
229 return '10.0.0.1'
230
231 if isinstance(opt, cfg.StrOpt) and default.strip() != default:
232 return '"%s"' % default
233
234 for pathelm in sys.path:
235 if pathelm in ('.', ''):
236 continue
237 if pathelm.endswith('/'):
238 pathelm = pathelm[:-1]
239 if pathelm in default:
240 default = re.sub(r'%s(/sources)?' % pathelm,
241 '/usr/lib/python/site-packages', default)
242
243 return default
244
245
246def _get_overrides(package_name):
247 overrides_file = '%s.overrides' % package_name
248 if not os.path.exists(overrides_file):
249 return {}
250 overrides = {}
251 with open(overrides_file) as fd:
252 for line in fd:
253 if line == '#':
254 continue
255 try:
256 opt, sections = line.strip().split(' ', 1)
257 sections = [x.strip() for x in sections.split(' ')]
258 except ValueError:
259 continue
260
261 overrides[opt] = sections
262
263 return overrides
264
265
266class OptionsCache(object):
267 def __init__(self, overrides={}, verbose=0):
268 self._verbose = verbose
269 self._opts_by_name = {}
270 self._opts_by_group = {}
271 self._opt_names = []
272 self._overrides = overrides
273
274 for optname in cfg.CONF._opts:
275 opt = cfg.CONF._opts[optname]['opt']
276 # We ignore some CLI opts by excluding SubCommandOpt objects
277 if not isinstance(opt, cfg.SubCommandOpt):
278 self._add_opt(optname, 'DEFAULT', opt)
279
280 for group in cfg.CONF._groups:
281 for optname in cfg.CONF._groups[group]._opts:
282 self._add_opt(group + '/' + optname, group,
283 cfg.CONF._groups[group]._opts[optname]['opt'])
284
285 self._opt_names.sort(OptionsCache._cmpopts)
286
287 def _add_opt(self, optname, group, opt):
288 if optname in self._opts_by_name:
289 if self._verbose >= 2:
290 print("Duplicate option name %s" % optname)
291 return
292
293 opt.default = _sanitize_default(opt)
294
295 def fill(optname, group, opt):
296 if optname in self._opts_by_name:
297 return
298 self._opts_by_name[optname] = (group, opt)
299 self._opt_names.append(optname)
300
301 if group not in self._opts_by_group:
302 self._opts_by_group[group] = []
303
304 self._opts_by_group[group].append(opt)
305
306 if optname in self._overrides:
307 for new_group in self._overrides[optname]:
308 if new_group == 'DEFAULT':
309 new_optname = opt.name
310 else:
311 new_optname = new_group + '/' + opt.name
312 fill(new_optname, new_group, opt)
313
314 else:
315 fill(optname, group, opt)
316
317 def __len__(self):
318 return len(self._opt_names)
319
320 def load_extension_options(self, module):
321 # Note that options loaded this way aren't added to _opts_by_module
322 loader = stevedore.named.NamedExtensionManager(
323 'oslo.config.opts',
324 names=(module,),
325 invoke_on_load=False
326 )
327 for ext in loader:
328 for group, opts in ext.plugin():
329 for opt in opts:
330 if group is None:
331 self._add_opt(opt.dest, 'DEFAULT', opt)
332 else:
333 self._add_opt(group + '/' + opt.dest, group, opt)
334
335 self._opt_names.sort(OptionsCache._cmpopts)
336
337 def maybe_load_extensions(self, repositories):
338 # Use the requirements.txt of the project to guess if an oslo module
339 # needs to be imported
340 needed_exts = set()
341 for repo in repositories:
342 base_path = os.path.dirname(repo)
343 for ext in EXTENSIONS:
344 requirements = os.path.join(base_path, 'requirements.txt')
345 with open(requirements) as fd:
346 for line in fd:
347 if line.startswith(ext):
348 needed_exts.add(ext)
349
350 for ext in needed_exts:
351 self.load_extension_options(ext)
352
353 def get_group_names(self):
354 return self._opts_by_group.keys()
355
356 def get_option_names(self):
357 return self._opt_names
358
359 def get_group(self, name):
360 return self._opts_by_group[name]
361
362 def get_option(self, name):
363 return self._opts_by_name[name]
364
365 def dump(self):
366 """Dumps the list of options with their attributes.
367
368 This output is consumed by the diff_branches script.
369 """
370 for name, (group, option) in self._opts_by_name.items():
371 deprecated_opts = [{'group': deprecated.group,
372 'name': deprecated.name}
373 for deprecated in option.deprecated_opts]
374 help_str = option.help.strip() if option.help else "None"
375 new_option = {
376 'default': option.default,
377 'help': help_str,
378 'deprecated_opts': deprecated_opts,
379 'type': option.__class__.__name__.split('.')[-1]
380 }
381 self._opts_by_name[name] = (group, new_option)
382 print(pickle.dumps(self._opts_by_name))
383
384 @staticmethod
385 def _cmpopts(x, y):
386 if '/' in x and '/' in y:
387 prex = x[:x.find('/')]
388 prey = y[:y.find('/')]
389 if prex != prey:
390 return cmp(prex, prey)
391 return cmp(x, y)
392 elif '/' in x:
393 return 1
394 elif '/' in y:
395 return -1
396 else:
397 return cmp(x, y)
398
399
400def _use_categories(package_name):
401 return not os.path.isfile(package_name + '.disable')
402
403
404def _get_options_by_cat(package_name):
405 options_by_cat = {}
406
407 with open(package_name + '.flagmappings') as f:
408 for line in f:
409 if not line.strip() or line.startswith('#'):
410 continue
411 opt, categories = line.split(' ', 1)
412 for category in categories.split():
413 options_by_cat.setdefault(category, []).append(opt)
414
415 return options_by_cat
416
417
418def _get_category_names(package_name):
419 package_headers = package_name + '.headers'
420 category_names = {}
421 for headers_file in ('shared.headers', package_headers):
422 try:
423 with open(headers_file) as f:
424 for line in f:
425 if not line.strip() or line.startswith('#'):
426 continue
427 cat, nice_name = line.split(' ', 1)
428 category_names[cat] = nice_name.strip()
429 except IOError:
430 print("Cannot open %s (ignored)" % headers_file)
431
432 return category_names
433
434
435def _format_opt(option):
436
437 def _remove_prefix(text, prefix):
438 if text.startswith(prefix):
439 return text[len(prefix):].lstrip(':')
440 return text
441
442 def _reflow_text(text):
443 text = re.sub(r'\n+\s*\* ', '$sentinal$* ', text)
444 text = text.replace('\n\n', '$sentinal$')
445 text = text.replace('\n', ' ')
446 text = ' '.join(text.split())
447 return text.split('$sentinal$')
448
449 def _strip_indentation(text):
450 return ' '.join([x.strip() for x in text.split('\n')]).strip()
451
452 help_text = option.help or "No help text available for this option."
453 help_text = _remove_prefix(help_text.strip(), 'DEPRECATED')
454 help_text = _reflow_text(help_text)
455
456 deprecated_text = (option.deprecated_reason or
457 'No deprecation reason provided for this option.')
458 deprecated_text = _strip_indentation(deprecated_text)
459
460 opt_type = _TYPE_DESCRIPTIONS.get(type(option), 'Unknown')
461
462 flags = []
463
464 if (option.deprecated_for_removal or
465 (option.help and option.help.startswith('DEPRECATED'))):
466 flags.append(('Deprecated', deprecated_text))
467 if option.mutable:
468 flags.append(('Mutable', 'This option can be changed without'
469 ' restarting.'))
470
471 return {
472 'name': option.dest,
473 'type': opt_type,
474 'default': _sanitize_default(option),
475 'help': help_text,
476 'flags': flags
477 }
478
479
480def write_files(package_name, options, target):
481 """Write tables.
482
483 Some projects make use of flagmapping files, while others make use
484 of oslo.config's OptGroup to do the same in code. The function will
485 handle both, printing a list of options by either category or group.
486 """
487 target = target or '../../doc/config-reference/source/tables'
488 if not os.path.isdir(target):
489 os.makedirs(target)
490
491 if _use_categories(package_name):
492 _write_files_by_category(package_name, options, target)
493 else:
494 _write_files_by_group(package_name, options, target)
495
496
497def _write_files_by_category(package_name, options, target):
498 options_by_cat = _get_options_by_cat(package_name)
499 category_names = _get_category_names(package_name)
500
501 for cat in options_by_cat.keys():
502 env = {
503 'pkg': package_name,
504 'cat': cat,
505 'label': '-'.join([package_name, cat]),
506 'groups': [],
507 'items': [],
508 }
509
510 # Skip the options that is explicitly marked as disabled,
511 # which is used for common configuration options.
512 if cat == 'disable':
513 continue
514
515 if cat in category_names:
516 env['nice_cat'] = category_names[cat]
517 else:
518 env['nice_cat'] = cat
519 print("No nicename for %s" % cat)
520
521 curgroup = None
522 items = None
523 for optname in options_by_cat[cat]:
524 group, option = options.get_option(optname)
525
526 if group != curgroup:
527 if group is not None:
528 curgroup = group
529 env['groups'].append(group)
530 if items is not None:
531 env['items'].append(items)
532 items = []
533
534 items.append(_format_opt(option))
535
536 env['items'].append(items)
537
538 file_path = ("%(target)s/%(package_name)s-%(cat)s.rst" %
539 {'target': target, 'package_name': package_name,
540 'cat': cat})
541 tmpl_file = os.path.join(os.path.dirname(__file__),
542 'templates/autohelp-category.rst.j2')
543 _write_template(tmpl_file, file_path, env)
544
545
546def _write_files_by_group(package_name, options, target):
547 for group in options.get_group_names():
548 env = {
549 'pkg': package_name,
550 'group': group,
551 'label': '-'.join([package_name, group]),
552 'items': [],
553 }
554
555 for option in options.get_group(group):
556 env['items'].append(_format_opt(option))
557
558 file_path = ("%(target)s/%(package_name)s-%(group)s.rst" %
559 {'target': target, 'package_name': package_name,
560 'group': group})
561 tmpl_file = os.path.join(os.path.dirname(__file__),
562 'templates/autohelp-group.rst.j2')
563 _write_template(tmpl_file, file_path, env)
564
565
566def _write_template(template_path, output_path, env):
567 with open(template_path) as fd:
568 template = jinja2.Template(fd.read(), trim_blocks=True)
569 output = template.render(filename=output_path, **env)
570
571 with open(output_path, 'w') as fd:
572 fd.write(output)
573
574
575def update_flagmappings(package_name, options, verbose=0):
576 """Update flagmappings file.
577
578 Update a flagmappings file, adding or removing entries as needed.
579 This will create a new file $package_name.flagmappings.new with
580 category information merged from the existing $package_name.flagmappings.
581 """
582 if not _use_categories:
583 print("This project does not use flagmappings. Nothing to update.")
584 return
585
586 original_flags = {}
587 try:
588 with open(package_name + '.flagmappings') as f:
589 for line in f:
590 try:
591 flag, category = line.split(' ', 1)
592 except ValueError:
593 flag = line.strip()
594 category = "Unknown"
595 original_flags.setdefault(flag, []).append(category.strip())
596 except IOError:
597 # If the flags file doesn't exist we'll create it
598 pass
599
600 updated_flags = []
601 for opt in options.get_option_names():
602 if len(original_flags.get(opt, [])) == 1:
603 updated_flags.append((opt, original_flags[opt][0]))
604 continue
605
606 updated_flags.append((opt, 'Unknown'))
607
608 with open(package_name + '.flagmappings.new', 'w') as f:
609 for flag, category in updated_flags:
610 f.write(flag + ' ' + category + '\n')
611
612 if verbose >= 1:
613 removed_flags = (set(original_flags.keys()) -
614 set([x[0] for x in updated_flags]))
615 added_flags = (set([x[0] for x in updated_flags]) -
616 set(original_flags.keys()))
617
618 print("\nRemoved Flags\n")
619 for line in sorted(removed_flags, OptionsCache._cmpopts):
620 print(line)
621
622 print("\nAdded Flags\n")
623 for line in sorted(added_flags, OptionsCache._cmpopts):
624 print(line)
625
626
627def main():
628 parser = argparse.ArgumentParser(
629 description='Manage flag files, to aid in updating documentation.',
630 usage='%(prog)s <cmd> <package> [options]')
631 parser.add_argument('subcommand',
632 help='Action (update, rst, dump).',
633 choices=['update', 'rst', 'dump'])
634 parser.add_argument('package',
635 help='Name of the top-level package.')
636 parser.add_argument('-v', '--verbose',
637 action='count',
638 default=0,
639 dest='verbose',
640 required=False,)
641 parser.add_argument('-i', '--input',
642 dest='repos',
643 help='Path to a python package in which options '
644 'should be discoverd. Can be used multiple '
645 'times.',
646 required=False,
647 type=str,
648 action='append')
649 parser.add_argument('-o', '--output',
650 dest='target',
651 help='Directory or file in which data will be saved.\n'
652 'Defaults to ../../doc/common/tables/ '
653 'for "rst".\n'
654 'Defaults to stdout for "dump"',
655 required=False,
656 type=str,)
657 args = parser.parse_args()
658
659 if args.repos is None:
660 args.repos = ['./sources/%s/%s' % args.package]
661
662 for repository in args.repos:
663 package_name = os.path.basename(repository)
664 base_path = os.path.dirname(repository)
665
666 sys.path.insert(0, base_path)
667 try:
668 __import__(package_name)
669 except ImportError as e:
670 if args.verbose >= 1:
671 print(str(e))
672 print("Failed to import: %s (%s)" % (package_name, e))
673
674 import_modules(base_path, package_name, verbose=args.verbose)
675 sys.path.pop(0)
676
677 overrides = _get_overrides(package_name)
678 options = OptionsCache(overrides, verbose=args.verbose)
679 options.maybe_load_extensions(args.repos)
680
681 if args.verbose > 0:
682 print("%s options imported from package %s." % (len(options),
683 str(package_name)))
684
685 if args.subcommand == 'update':
686 update_flagmappings(args.package, options, verbose=args.verbose)
687
688 elif args.subcommand == 'rst':
689 write_files(args.package, options, args.target)
690
691 elif args.subcommand == 'dump':
692 options.dump()
693
694
695if __name__ == "__main__":
696 main()
diff --git a/autogenerate_config_docs/diff_branches.py b/autogenerate_config_docs/diff_branches.py
deleted file mode 100755
index 5b8ee08..0000000
--- a/autogenerate_config_docs/diff_branches.py
+++ /dev/null
@@ -1,289 +0,0 @@
1#!/usr/bin/env python
2# Licensed under the Apache License, Version 2.0 (the "License"); you may
3# not use this file except in compliance with the License. You may obtain
4# a copy of the License at
5#
6# http://www.apache.org/licenses/LICENSE-2.0
7#
8# Unless required by applicable law or agreed to in writing, software
9# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
10# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
11# License for the specific language governing permissions and limitations
12# under the License.
13#
14# A collection of tools for working with flags from OpenStack
15# packages and documentation.
16#
17# For an example of usage, run this program with the -h switch.
18#
19import argparse
20import os
21import pickle
22import subprocess
23import sys
24
25import jinja2
26
27
28PROJECTS = ['aodh', 'ceilometer', 'cinder', 'glance', 'heat', 'ironic',
29 'keystone', 'manila', 'neutron', 'nova', 'sahara', 'trove']
30MASTER_RELEASE = 'Ocata'
31CODENAME_TITLE = {'aodh': 'Alarming',
32 'ceilometer': 'Telemetry',
33 'cinder': 'Block Storage',
34 'glance': 'Image service',
35 'heat': 'Orchestration',
36 'ironic': 'Bare Metal service',
37 'keystone': 'Identity service',
38 'manila': 'Shared File Systems service',
39 'murano': 'Application Catalog service',
40 'neutron': 'Networking',
41 'nova': 'Compute',
42 'sahara': 'Data Processing service',
43 'senlin': 'Clustering service',
44 'trove': 'Database service',
45 'zaqar': 'Message service'}
46
47
48def setup_venv(projects, branch, novenvupdate):
49 """Setup a virtual environment for `branch`."""
50 dirname = os.path.join('venv', branch.replace('/', '_'))
51 if novenvupdate and os.path.exists(dirname):
52 return
53 if not os.path.exists('venv'):
54 os.mkdir('venv')
55 args = ["./autohelp-wrapper", "-b", branch, "-e", dirname, "setup"]
56 args.extend(projects)
57 if subprocess.call(args) != 0:
58 print("Impossible to create the %s environment." % branch)
59 sys.exit(1)
60
61
62def _get_packages(project, branch):
63 release = branch if '/' not in branch else branch.split('/')[1]
64 packages = [project]
65 try:
66 with open('extra_repos/%s-%s.txt' % (project, release)) as f:
67 packages.extend([p.strip() for p in f])
68 except IOError:
69 pass
70
71 return packages
72
73
74def get_options(project, branch):
75 """Get the list of known options for a project."""
76 print("Working on %(project)s (%(branch)s)" % {'project': project,
77 'branch': branch})
78 # And run autohelp script to get a serialized dict of the discovered
79 # options
80 dirname = os.path.join('venv', branch.replace('/', '_'))
81 args = ["./autohelp-wrapper", "-q", "-b", branch, "-e", dirname,
82 "dump", project]
83
84 path = os.environ.get("PATH")
85 bin_path = os.path.abspath(os.path.join(dirname, "bin"))
86 path = "%s:%s" % (bin_path, path)
87 serialized = subprocess.check_output(args,
88 env={'VIRTUAL_ENV': dirname,
89 'PATH': path})
90 ret = pickle.loads(serialized)
91 return ret
92
93
94def _cmpopts(x, y):
95 """Compare to option names.
96
97 The options can be of 2 forms: option_name or group/option_name. Options
98 without a group always comes first. Options are sorted alphabetically
99 inside a group.
100 """
101 if '/' in x and '/' in y:
102 prex = x[:x.find('/')]
103 prey = y[:y.find('/')]
104 if prex != prey:
105 return cmp(prex, prey)
106 return cmp(x, y)
107 elif '/' in x:
108 return 1
109 elif '/' in y:
110 return -1
111 else:
112 return cmp(x, y)
113
114
115def diff(old_list, new_list):
116 """Compare the old and new lists of options."""
117 new_opts = []
118 new_defaults = []
119 deprecated_opts = []
120 for name, (group, option) in new_list.items():
121 # Find the new options
122 if name not in old_list.viewkeys():
123 new_opts.append(name)
124
125 # Find the options for which the default value has changed
126 elif option['default'] != old_list[name][1]['default']:
127 new_defaults.append(name)
128
129 # Find options that have been deprecated in the new release.
130 # If an option name is a key in the old_list dict, it means that it
131 # wasn't deprecated.
132
133 # Some options are deprecated, but not replaced with a new option.
134 # These options usually contain 'DEPRECATED' in their help string.
135 if 'DEPRECATED' in option['help']:
136 deprecated_opts.append((name, None))
137
138 for deprecated in option['deprecated_opts']:
139 # deprecated_opts is a list which always holds at least 1 invalid
140 # dict. Forget it.
141 if deprecated['name'] is None:
142 continue
143
144 if deprecated['group'] in [None, 'DEFAULT']:
145 full_name = deprecated['name']
146 else:
147 full_name = deprecated['group'] + '/' + deprecated['name']
148
149 if full_name in old_list.viewkeys():
150 deprecated_opts.append((full_name, name))
151
152 return new_opts, new_defaults, deprecated_opts
153
154
155def format_option_name(name):
156 """Return a formatted string for the option path."""
157 if name is None:
158 return "None"
159
160 try:
161 section, name = name.split('/')
162 except ValueError:
163 # name without a section ('log_dir')
164 return "[DEFAULT] %s" % name
165
166 return "[%s] %s" % (section, name)
167
168
169def release_from_branch(branch):
170 if branch == 'master':
171 return MASTER_RELEASE
172 else:
173 return branch.replace('stable/', '').title()
174
175
176def get_env(project, new_branch, old_list, new_list):
177 """Generate the jinja2 environment for the defined branch and project."""
178 new_opts, new_defaults, deprecated_opts = diff(old_list, new_list)
179 release = release_from_branch(new_branch)
180
181 env = {
182 'release': release,
183 'project': project,
184 'codename': CODENAME_TITLE[project],
185 'new_opts': [],
186 'new_defaults': [],
187 'deprecated_opts': []
188 }
189
190 # New options
191 if new_opts:
192 for name in sorted(new_opts, _cmpopts):
193 opt = new_list[name][1]
194 name = format_option_name(name)
195 helptext = opt['help'].strip().replace('\n', ' ')
196 helptext = ' '.join(helptext.split())
197 cells = (("%(name)s = %(default)s" %
198 {'name': name,
199 'default': opt['default']}).strip(),
200 "(%(type)s) %(help)s" % {'type': opt['type'],
201 'help': helptext})
202 env['new_opts'].append(cells)
203
204 # New defaults
205 if new_defaults:
206 for name in sorted(new_defaults, _cmpopts):
207 old_default = old_list[name][1]['default']
208 new_default = new_list[name][1]['default']
209 if isinstance(old_default, list):
210 old_default = ", ".join(old_default)
211 if isinstance(new_default, list):
212 new_default = ", ".join(new_default)
213 name = format_option_name(name)
214 cells = (name, old_default, new_default)
215 env['new_defaults'].append(cells)
216
217 # Deprecated options
218 if deprecated_opts:
219 for deprecated, new in sorted(deprecated_opts, cmp=_cmpopts,
220 key=lambda tup: tup[0]):
221 deprecated = format_option_name(deprecated)
222 new = format_option_name(new)
223 env['deprecated_opts'].append((deprecated, new))
224
225 return env
226
227
228def main():
229 parser = argparse.ArgumentParser(
230 description='Generate a summary of configuration option changes.',
231 usage='%(prog)s [options] <old_branch> <new_branch> [projects]')
232 parser.add_argument('old_branch',
233 help='Name of the old branch.')
234 parser.add_argument('new_branch',
235 help='Name of the new branch.')
236 parser.add_argument('projects',
237 help='List of projects to work on.',
238 nargs='*',
239 default=PROJECTS)
240 parser.add_argument('-i', '--input',
241 dest='sources',
242 help='Path to a folder containing the git '
243 'repositories.',
244 required=False,
245 default='./sources',
246 type=str,)
247 parser.add_argument('-o', '--output',
248 dest='target',
249 help='Directory or file in which data will be saved.\n'
250 'Defaults to "."',
251 required=False,
252 default='.',
253 type=str,)
254 parser.add_argument('-n', '--no-venv-update',
255 dest='novenvupdate',
256 help='Don\'t update the virtual envs.',
257 required=False,
258 action='store_true',
259 default=False,)
260 args = parser.parse_args()
261
262 setup_venv(args.projects, args.old_branch, args.novenvupdate)
263 setup_venv(args.projects, args.new_branch, args.novenvupdate)
264
265 for project in args.projects:
266 old_list = get_options(project, args.old_branch)
267 new_list = get_options(project, args.new_branch)
268
269 release = args.new_branch.replace('stable/', '')
270 env = get_env(project, release, old_list, new_list)
271 filename = ("%(project)s-conf-changes.rst" %
272 {'project': project})
273 tmpl_file = 'templates/changes.rst.j2'
274 if not os.path.exists(args.target):
275 os.makedirs(args.target)
276 dest = os.path.join(args.target, filename)
277
278 with open(tmpl_file) as fd:
279 template = jinja2.Template(fd.read(), trim_blocks=True)
280 output = template.render(**env)
281
282 with open(dest, 'w') as fd:
283 fd.write(output)
284
285 return 0
286
287
288if __name__ == "__main__":
289 sys.exit(main())
diff --git a/autogenerate_config_docs/requirements.txt b/autogenerate_config_docs/requirements.txt
deleted file mode 100644
index 39abf12..0000000
--- a/autogenerate_config_docs/requirements.txt
+++ /dev/null
@@ -1,5 +0,0 @@
1docutils
2jinja2
3lxml
4oslo.config
5oslo.i18n
diff --git a/autogenerate_config_docs/templates/autohelp-category.rst.j2 b/autogenerate_config_docs/templates/autohelp-category.rst.j2
deleted file mode 100644
index ee8859c..0000000
--- a/autogenerate_config_docs/templates/autohelp-category.rst.j2
+++ /dev/null
@@ -1,45 +0,0 @@
1..
2 Warning: Do not edit this file. It is automatically generated from the
3 software project's code and your changes will be overwritten.
4
5 The tool to generate this file lives in openstack-doc-tools repository.
6
7 Please make any changes needed in the code, then run the
8 autogenerate-config-doc tool from the openstack-doc-tools repository, or
9 ask for help on the documentation mailing list, IRC channel or meeting.
10
11.. _{{ label }}:
12
13.. list-table:: Description of {{ nice_cat }} configuration options
14 :header-rows: 1
15 :class: config-ref-table
16
17 * - Configuration option = Default value
18 - Description
19{% for group in groups %}
20
21 * - **[{{ group }}]**
22 -
23{% for item in items[loop.index0] %}
24
25{% if item['default'] is equalto '' %}
26 * - ``{{ item['name'] }}`` =
27{% else %}
28 * - ``{{ item['name'] }}`` = ``{{ item['default'] }}``
29{% endif %}
30{% for paragraph in item['help'] %}
31
32{% if loop.first %}
33 - ({{ item['type'] }}) {{ paragraph }}
34{% else %}
35 {{ paragraph }}
36{% endif %}
37{% endfor %}
38{% for flagname, flagdesc in item['flags'] %}
39
40 - **{{ flagname }}**
41
42 {{ flagdesc }}
43{% endfor %}
44{% endfor %}
45{% endfor %}
diff --git a/autogenerate_config_docs/templates/autohelp-group.rst.j2 b/autogenerate_config_docs/templates/autohelp-group.rst.j2
deleted file mode 100644
index ea1504b..0000000
--- a/autogenerate_config_docs/templates/autohelp-group.rst.j2
+++ /dev/null
@@ -1,40 +0,0 @@
1..
2 Warning: Do not edit this file. It is automatically generated from the
3 software project's code and your changes will be overwritten.
4
5 The tool to generate this file lives in openstack-doc-tools repository.
6
7 Please make any changes needed in the code, then run the
8 autogenerate-config-doc tool from the openstack-doc-tools repository, or
9 ask for help on the documentation mailing list, IRC channel or meeting.
10
11.. _{{ label }}:
12
13.. list-table:: Description of {{ group }} configuration options
14 :header-rows: 1
15 :class: config-ref-table
16
17 * - Configuration option = Default value
18 - Description
19{% for item in items %}
20
21{% if item['default'] is equalto '' %}
22 * - ``{{ item['name'] }}`` =
23{% else %}
24 * - ``{{ item['name'] }}`` = ``{{ item['default'] }}``
25{% endif %}
26{% for paragraph in item['help'] %}
27
28{% if loop.first %}
29 - ({{ item['type'] }}) {{ paragraph }}
30{% else %}
31 {{ paragraph }}
32{% endif %}
33{% endfor %}
34{% for flagname, flagdesc in item['flags'] %}
35
36 - **{{ flagname }}**
37
38 {{ flagdesc }}
39{% endfor %}
40{% endfor %}
diff --git a/autogenerate_config_docs/templates/changes.rst.j2 b/autogenerate_config_docs/templates/changes.rst.j2
deleted file mode 100644
index 23b4f99..0000000
--- a/autogenerate_config_docs/templates/changes.rst.j2
+++ /dev/null
@@ -1,61 +0,0 @@
1New, updated, and deprecated options in {{ release }} for {{ codename }}
2~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{{ '~' * release|length }}~~~~~{{ '~' * codename|length }}
3
4..
5 Warning: Do not edit this file. It is automatically generated and your
6 changes will be overwritten. The tool to do so lives in the
7 openstack-doc-tools repository.
8
9{% if new_opts %}
10.. list-table:: New options
11 :header-rows: 1
12 :class: config-ref-table
13
14 * - Option = default value
15 - (Type) Help string
16{% for cells in new_opts %}
17 * - ``{{ cells[0] }}``
18 - {{ cells[1] }}
19{% endfor %}
20{% endif %}
21
22{% if new_defaults %}
23.. list-table:: New default values
24 :header-rows: 1
25 :class: config-ref-table
26
27 * - Option
28 - Previous default value
29 - New default value
30{% for cells in new_defaults %}
31 * - ``{{ cells[0] }}``
32{% if cells[1] is equalto '' %}
33 -
34{% else %}
35 - ``{{ cells[1] }}``
36{% endif %}
37{% if cells[2] is equalto '' %}
38 -
39{% else %}
40 - ``{{ cells[2] }}``
41{% endif %}
42{% endfor %}
43{% endif %}
44
45{% if deprecated_opts %}
46.. list-table:: Deprecated options
47 :header-rows: 1
48 :class: config-ref-table
49
50 * - Deprecated option
51 - New Option
52{% for cells in deprecated_opts %}
53 * - ``{{ cells[0] }}``
54 - ``{{ cells[1] }}``
55{% endfor %}
56{% endif %}
57
58{% if not new_opts and not new_defaults and not deprecated_opts %}
59There are no new, updated, and deprecated options
60in {{ release }} for {{ codename }}.
61{% endif %}
diff --git a/doc/source/autogenerate_config_docs.rst b/doc/source/autogenerate_config_docs.rst
deleted file mode 100644
index 0bc5eb9..0000000
--- a/doc/source/autogenerate_config_docs.rst
+++ /dev/null
@@ -1 +0,0 @@
1.. include:: ../../autogenerate_config_docs/README.rst
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 748a0ad..557eb0b 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -10,7 +10,6 @@ Contents:
10 doc-tools-readme 10 doc-tools-readme
11 installation 11 installation
12 usage 12 usage
13 autogenerate_config_docs
14 man/openstack-doc-test 13 man/openstack-doc-test
15 sitemap-readme 14 sitemap-readme
16 release_notes 15 release_notes
diff --git a/releasenotes/notes/the-great-doc-tools-cleanup-1a79e2c200232489.yaml b/releasenotes/notes/the-great-doc-tools-cleanup-1a79e2c200232489.yaml
new file mode 100644
index 0000000..f5b33ae
--- /dev/null
+++ b/releasenotes/notes/the-great-doc-tools-cleanup-1a79e2c200232489.yaml
@@ -0,0 +1,8 @@
1---
2upgrade:
3 - |
4 The `autogenerate_config_docs` set of tools has been removed. These were
5 always buggy and hard to maintain, and they have been superseded by the
6 ``oslo_config.sphinxext`` `Sphinx extension`__.
7
8 __ https://docs.openstack.org/oslo.config/latest/reference/sphinxext.html
diff --git a/setup.cfg b/setup.cfg
index 4a65ccb..86695c2 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -22,7 +22,6 @@ classifier =
22[files] 22[files]
23packages = 23packages =
24 os_doc_tools 24 os_doc_tools
25 autogenerate_config_docs
26data_files = 25data_files =
27 share/openstack-doc-tools/sitemap = sitemap/* 26 share/openstack-doc-tools/sitemap = sitemap/*
28 share/openstack-doc-tools/cleanup = cleanup/* 27 share/openstack-doc-tools/cleanup = cleanup/*
diff --git a/tox.ini b/tox.ini
index 706b94c..1316875 100644
--- a/tox.ini
+++ b/tox.ini
@@ -25,8 +25,7 @@ commands =
25 doc8 -e txt -e rst doc/source/ HACKING.rst 25 doc8 -e txt -e rst doc/source/ HACKING.rst
26 # Run bashate during pep8 runs to ensure violations are caught by 26 # Run bashate during pep8 runs to ensure violations are caught by
27 # the check and gate queues. 27 # the check and gate queues.
28 bashate autogenerate_config_docs/autohelp-wrapper \ 28 bashate bin/doc-tools-check-languages \
29 bin/doc-tools-check-languages \
30 cleanup/remove_trailing_whitespaces.sh 29 cleanup/remove_trailing_whitespaces.sh
31 30
32[testenv:pylint] 31[testenv:pylint]
@@ -58,7 +57,7 @@ usedevelop = False
58[flake8] 57[flake8]
59show-source = True 58show-source = True
60builtins = _ 59builtins = _
61exclude=.venv,.git,.tox,dist,*lib/python*,*egg,build,*autogenerate_config_docs/venv,*autogenerate_config_docs/sources,doc/source/conf.py 60exclude=.venv,.git,.tox,dist,*lib/python*,*egg,build,doc/source/conf.py
62# 28 is currently the most complex thing we have 61# 28 is currently the most complex thing we have
63max-complexity=29 62max-complexity=29
64ignore = H101 63ignore = H101