When troubleshooting, it's often interesting to see the deltas between
the default or sample configs and the running configuration. The
oslo_config.validator is a great tool to integrate this feature across
all projects. It can also be easily captured by data collection tools
like sosreports with the current deployment packages.
Change-Id: I172d82f19a81282093b0d5f7ae4c1817801cd887
Signed-off-by: David Vallee Delisle <dvd@redhat.com>
Introduced changes:
- pre-commit config and rules
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.
Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.
pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker (check-byte-order-marker);
- Checks that non-binary executables have a proper
shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)
For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks
Change-Id: I65a40bbff17d92b959764846801d5e0ddd3b97ce
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
oslo config code treat treats the '$' as a special character and it
needs to be delimited when using within a string. There are two ways of
doing this, '$$' or '\$'. We document the former but not the latter. Fix
this.
Change-Id: I37cdf31ba1c2924779a46b1a2eeabb66c46a2212
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Closes-Bug: #1872839
Add a section to the configuration guide to list all of the drivers
using stevedore's sphinx integration.
Change-Id: I6348f83efdd8a572821d2851ee4bd883f9e7e2b4
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.
[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045
Change-Id: Ied9f46640cf2f338a5784e1c72998f93b5c860c5
Some options are now automatically configured by the version 1.20:
- project
- html_last_updated_fmt
- latex_engine
- latex_elements
- version
- release.
Change-Id: Iac913e9dea8318ad8398ed7478cb66d845fe7f1e
Rather than have the API docs hand-curated and scattered throughout
the documentation, let's use the apidoc module like all of the other
Oslo projects so our API docs are consistent and easy to find.
The documents that were exclusively API docs are removed entirely,
and the ones where API docs were included with other content have
been changed to reference the generated API docs rather than include
them inline. The one exception is the drivers because they are in
private modules that don't show up in the API docs. Those are still
explicitly documented.
Change-Id: I00bdd963e0d4f270c0d4b50c05f420317a137fd5
The keystone_authtoken options are created (not just registered)
dynamically at runtime because of a need to not have a dependency on
oslo.config in that library. As a result, there is no sane way to
validate those options.
This change adds a list of known problematic groups that we should
ignore missing opts from. It also adds an info level log message that
we are ignoring them so users know we can't confirm or deny the
correctness of that piece of config.
Change-Id: Ic2b36fe8be872d298a106c3f64905a5b935ced61
Bandit 1.6.0 accidentally changed how the exclusion list option is
handled and breaks our use of it. Cap to the previous version until
Bandit has fixed the problem.
Sphinx 2.0 no longer works on python 2.7, so we need to start capping
it there as well.
Change-Id: Ib8da5b64084d5c9b7b7d896d6b7bb7844c0b9e90
Reference: https://github.com/PyCQA/bandit/pull/489
As a user completely new to oslo_config (well, I had only experienced it
in the already-implemented context of the nova project), I would like to
have had a reference document that provided a step-by-step guide to
writing a simple test script I could use to noodle around. It might
have gone something like this.
Change-Id: I02ff56f47ac8b93e0ee37eb55b53552059e4da27
Previously the precedence details for the various sources oslo.config
can consume were scattered around the developer documentation (for
example in [0]). Since this is relevant for users too, we need to
include it in the user-focused documentation.
This collects all of the precedence information I am aware of and puts
it on the documentation page for --config-file, --config-dir, and the
sample source driver options.
It also clarifies the file and dir precedence in the developer docs since
previously that was only available in the API docstring.
Change-Id: I41d8ac0f33ea60aea8ba5b150e53d7eafa323457
0: 0bc22839b0/oslo_config/cfg.py (L2095)
Closes-Bug: 1825859
The user-oriented docs for oslo.config were buried at the bottom of
the index below a pile of developer docs. This made them difficult to
find. Since we have far more users of oslo.config than we do developers
using it we should prioritize the user docs.
Change-Id: I602ee587e4acf1c1436b568906e316b78f104814
When dynamic groups are used, the sample config data may not know
about those group names. As a result, validation of such files will
always fail. This makes it hard to automate config checks since the
output would need to be inspected manually to verify that the missing
options are all from the dynamic group.
Ideally, we would provide some way to map sample config groups to
the dynamic group name used in the actual config, but that would be
more complicated and still might not work in every case if a project
doesn't include sample sections for a dynamic group (although they
_should_ be doing so).
Allowing certain group names to be excluded from validation lets
us to solve this problem in a simple way and maintain validation of
99%[0] of the config options and enables the validation to be scripted
since it won't need manual verification of the error output.
Change-Id: I352fd48f86ecb876fe26a5e50e9a2633af1fff3d
0: citation needed ;-)
CLI opts can also be read from file, so once an option is registered
for CLI use it doesn't need to be registered again to be usable in
a config file. Add a note in the Registering Options section that
explicitly explains this behavior.
Change-Id: Ib75b5c314b96ec4c46f94e2847016a953433f480
An _environment source is added that looks in os.environ for
values.
Using the environment is on by default, but can be shut down
by setting `use_env` to False when __call__ is called.
The enviroment is inspected before any other sources
of config data but the value is used after command line
arguments and before config file options.
This is done by checking both the command line and config
files and then inspecting the location of the result. If
it is command_line, we use it. If not, we use the environment
value (if any). If there's no environment value, the config
file value is used. If checking the command line and config
file results in a KeyError, the environment value is used,
if set.
The names of the environment variables follow the rules
described in oslo_config.sources._environment.
A new exception has been added: ConfigSourceValueError, this
is the superclass of the existing ConfigFileValueError. The
code in _do_get has been updated to only use
ConfigFileValueError when it is in fact a file from whence a
ValueError came.
Documentation has been updated and a rlease note created to
indicate the new functionality.
Change-Id: I3245c40ebdcc96f8e3b2dc0bab3b4aa71d07ad15
The cfg module is very large, and starts with a huge block of
documentation. This patch moves that information into separate files
in the reference section of the docs. A few formatting fixes need to
be made to have it build cleanly, but the content is not changed in a
substantive way.
Change-Id: I86aa90bbf180b5dc9acbcedb024e5361d49954c3
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
New documentation sections explaning how to use the backend
drivers for oslo.config, also include explanation on how to
use the remote_file driver.
Change-Id: I45fc2155f6fe2d8bee058dac880afba8bb9bfd53
Blueprint: oslo-config-drivers
We have a section in the reference guide showing the configuration
options built into oslo.config. Move that to the configuration guide
instead so it is linked from the Configuration Guide list at
https://docs.openstack.org/$series/configuration/ like the similar
documentation for other libraries.
Change-Id: I7da159cfaa643c2cc094f6fb1c0aeaa956f3eb77
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Fix an error in the example code by setting loc to the Location enum
value instead of the LocationInfo tuple.
Add a cross-reference from the ConfigOpts get_location() method to the
more extensive reference documentation about option locations.
Change-Id: Ic9850afd78a7b42f4e807b04e78729a674942e15
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
I'm not sure why this was here, but it doesn't make sense since
we're documenting StrOpts in this section.
Change-Id: I74c73c8088ec266d1f827bc51b4d78fc7be48a38
If OSLO_CONFIG_SHOW_CODE_LOCATIONS is set to any non-empty string it
will turn on the ability to see which file has the definition of an
Opt or where set_default() was invoked.
Change-Id: Ie705014dcf331e3c6b3367d2fefbfb9acc091799
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This has been deprecated for quite a while and some of the upcoming
work to be done in oslo.config will be easier if we don't have to
deal with this unused parser too.
The class is still currently in use in networking-cisco, but in
change https://review.openstack.org/554617 we have provided a fix
that removes their reliance on it. They are planning to remove it
entirely in Rocky and are only testing against a capped version in
Queens so we are not going to wait on them.
Change-Id: Id6ae40311f967e172e45bcb0e61812365a72618b
Replace the invalid links with the right ones for the document
according to the newest Openstack website.
Change-Id: Id94551a864710231a9d9d23d8632a86439cd61bd
".. code-block:: shell" appears in the generated doc html by mistake
because of the redundant colon in the doc source
Change-Id: I5c54fd48a7705cb7d84016707a4bb51e8dd79b2a
This information used to be in openstack-manuals/doc/config-reference.
Change-Id: I219dee590a706924978ce8d2a2c151a6ae51c8b6
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
html_last_updated_fmt option is interpreted as a
byte string in python3, causing Sphinx build to break.
This patch makes it utf-8 string.
Changing Popen to .check_output because of 3 reasons:
1. check_output() will raise CalledProcessError if
the called process returns a non-zero return code.
2. For consistency with keystone [1] and cinder [2]
3. It makes the code look much better.
[1] https://review.openstack.org/#/c/457142/
[2] https://review.openstack.org/#/c/433081
Change-Id: I8872866338d93bc8328391537198df6c7981859f
There's a lot of shuffling docs around, but the main point is to
document the application parameters and the support for multiple output
formats recently added.
Change-Id: I26827801df917619d4256ef4c718051f5c395a29
Implements: bp machine-readable-sample-config
Takashi Kajinami