The ``HostDomain`` config type have been added few months ago [1]
however the config option have been forgotten and this new type
isn't importable.
When we try to import this type without defining a new related cfg
option we get the following issue:
```
AttributeError: module 'oslo_config.cfg' has no attribute 'HostDomain'
```
These changes allow us to import this new type and allow us to use
it in our configs:
```
>>> from oslo_config import cfg
>>> foo = cfg.HostDomain('foo')
>>> foo.type.__call__("1")
...
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "~/oslo.config/oslo_config/types.py", line 893, in __call__
raise ValueError(
ValueError: 1 is not a valid host address
>>> foo.type.__call__("host_name")
'host_name'
```
Also properly initialize HostDomain because The HostDomain class wasn't
calling super in it's __init__() method, which resulted in the type_name not
being set properly for instances of that class.
[1] 6480356928
Change-Id: Ie947803f61ba0ef080018e0447de894a400d7975
Closes-Bug: 1924283
As discussed in bug 1841038, configuring a StrOpt with a regex parameter
and providing an invalid environment variable will result in an ugly
stacktrace due to an undefined 'alt_loc' variable. Resolve this by
creating said variable.
Change-Id: Id0fb6efdc435a2bd88d5dfea2b4720df453fbfec
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Closes-Bug: #1841038
We don't need this in a Python 3-only world. We can't remove mock yet
since there's an issue with the stdlib variant in python3.6, but that is
called out.
Change-Id: I9657b1fd4409be90d645175a6e177d7e1d2ebac2
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Currently, when you specify a positional argument with a hyphen in the
name, oslo.config is not able to retrieve a value back for that
argument, even if one is parsed by argparse, because there's a mismatch
between where oslo.config expects to find the value (in the dest, with
underscores, which positional arguments do not have), versus where
argparse puts it in the namespace (using the name, with hyphens).
Change-Id: Ibc44a880ffddfaeffca682ccf3b34525f3f0fe27
The 'positional' keyword specifically applies to oslo.config's argparse
support. Unlike oslo.config, argparse assumes that all positional
arguments are required by default, and you have to explicitly tell it
that a positional argument is optional if you'd like to opt into that
behavior.
This patch adopts that same behavior for oslo.config. When you define an
option to be non-positional (oslo.config's default, designed for config
files), then oslo.config makes that option optional:
However, when you define an option to be positional, oslo.config assumes
that the option is primarily going to be used on the CLI and thus sets
it as required, by default.
This change in behavior has the side effect of allowing argparse to
enforce required arguments on the CLI *while* parsing arguments, instead
of depending on oslo.config to detect the condition *after* argparse has
been allowed to parse "invalid" arguments. argparse correctly raises a
SystemExit in this case, and prints the actual command usage and a "hey,
you forgot this required argument", instead of allowing oslo.config to
dump a backtrace to the CLI with a context-less error message
("context-less" in that no useful CLI usage information is dumped along
with the crash to help you correct the condition).
Change-Id: Ifdc6918444fe72f7e1649483c237cce64b4c72d8
Partial-Bug: 1676989
oslo.config allows us to configure groups and options dynamically. This
can cause a race with our logging as we attempt to iterate through
option groups that are changing under our feet. This wouldn't be a huge
issue, since these are just logs are we can always log again, if needed,
but we store groups in a dictionary and Python doesn't like us changing
the size of a dict it's iterating through:
RuntimeError: dictionary changed size during iteration
Given that we're only reading through this option group and don't need
to worry about a group _disappearing_, the solution is pretty simple:
create a copy of our option group names ahead of time so we don't need
to worry about new ones coming and messing things up.
No tests are included since this is a race and the only way I see to
reproduce this would involve lots of ugly threading.
Change-Id: Id3b28465d645a24f0fcebff2dd68a9bd30e21594
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Closes-Bug: #1856312
These options are intended to be set from the command-line, as they
are used to find the config files we will parse. There's a
chicken-and-egg problem with setting them from those config files,
but since these appear in service sample configs it may not be
immediately obvious to users.
Change-Id: I7afc9ce979cebe5427a19cbf33c412ab9b23d3ae
The use of ABC classes directly from collections has been deprecated in
3.x versions of Python. The direction is to use the classes defined in
collections.abc. Python 2.7 does not have this, but Python 3.8 will be
dropping the backwards compatibility to use the old location.
Six also does not have support for this yet, so in the mean time to make
sure we don't run into issues as folks try to move to 3.8, and to get
rid of deprecation warnings in logs, this handles importing from the
preferred location and falls back if it not available.
Change-Id: I535640779da645e43ba6586308b07bf25dba7d89
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
get_location defines the type of its ``group`` parameter as a str,
but it then passes that directly into _do_get, which expects an
OptGroup for its parameter. When it tries to reference group.name
it blows up because a str doesn't have a .name attr.
In the interest of not changing either API, this change coerces the
group str into an OptGroup.
Closes-Bug: 1807230
Change-Id: Iae547cbd63059fa19691a7f7fe398b148effd177
An _environment source is added that looks in os.environ for
values.
Using the environment is on by default, but can be shut down
by setting `use_env` to False when __call__ is called.
The enviroment is inspected before any other sources
of config data but the value is used after command line
arguments and before config file options.
This is done by checking both the command line and config
files and then inspecting the location of the result. If
it is command_line, we use it. If not, we use the environment
value (if any). If there's no environment value, the config
file value is used. If checking the command line and config
file results in a KeyError, the environment value is used,
if set.
The names of the environment variables follow the rules
described in oslo_config.sources._environment.
A new exception has been added: ConfigSourceValueError, this
is the superclass of the existing ConfigFileValueError. The
code in _do_get has been updated to only use
ConfigFileValueError when it is in fact a file from whence a
ValueError came.
Documentation has been updated and a rlease note created to
indicate the new functionality.
Change-Id: I3245c40ebdcc96f8e3b2dc0bab3b4aa71d07ad15
d.keys() no longer returns a copy of the keys in Python 3.
This is part of PEP 3106.
Change-Id: Ie4897109c87d6dc2952c2eb35b83fe368fb31803
Closes-Bug: #1796163
The cfg module is very large, and starts with a huge block of
documentation. This patch moves that information into separate files
in the reference section of the docs. A few formatting fixes need to
be made to have it build cleanly, but the content is not changed in a
substantive way.
Change-Id: I86aa90bbf180b5dc9acbcedb024e5361d49954c3
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Add a separate cache for values coming from external sources,
this will protect us from having options from external sources
mutated when we reload the configuration files
Change-Id: Icf72f4e1745a0ddf53e661cc08d0fe7428cd9a41
Blueprint: oslo-config-drivers
Have the main _list_opts caller construct the driver option so
individual drivers do not need to repeat that.
Add choices with descriptions when emitting samples. We don't really
care about those for the runtime use, but they improve the output in
the config generator and documentation.
Use an OptGroup with the driver_option and dynamic_group_owner options
set instead of just a group name when describing the options.
Add sample_default values for some of the options in the URI driver.
Change-Id: I14c0a046e6c70a9108308db70a4efb70613d5bb3
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Ensure the sample generator emits an example group with
instructions on how to define extra URI sources.
Change-Id: Ica556771d8dd37cb02d9ca69c04e888d187041ee
Blueprint: oslo-config-drivers
Signed-off-by: Moises Guimaraes de Medeiros <moguimar@redhat.com>
Define a config_source option that can be used to specify the
alternative sources that require drivers to load configuration
settings.
Co-Authored-By: Moises Guimaraes de Medeiros <moguimar@redhat.com>
Change-Id: Ibd5a6d306bb98d30d973dfe3604dcc0691d2e369
Blueprint: oslo-config-drivers
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
While we can't add a hard dependency on oslo.log because it uses
oslo.config, in most cases oslo.log will be installed anyway. In
the interest of being able to make use of features like
fatal_deprecations in oslo.log, let's use it if it's available.
Change-Id: If9499aa6fc28a6b92447b3825d3ca1957cb2255a
Fix an error in the example code by setting loc to the Location enum
value instead of the LocationInfo tuple.
Add a cross-reference from the ConfigOpts get_location() method to the
more extensive reference documentation about option locations.
Change-Id: Ic9850afd78a7b42f4e807b04e78729a674942e15
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
If OSLO_CONFIG_SHOW_CODE_LOCATIONS is set to any non-empty string it
will turn on the ability to see which file has the definition of an
Opt or where set_default() was invoked.
Change-Id: Ie705014dcf331e3c6b3367d2fefbfb9acc091799
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Inspecting the stack every time we create an Opt or call
set_defaults() is too expensive. This patch disables it but leaves the
code in place to make it easier for someone else to come along and
work on the performance issue without having to revert the patch
completely.
Change-Id: I25bd71ffa49f4232972370a35bf9040e7c45bf70
Addresses-Bug: #1759689
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Before this change, two options that were identical except for the
location in the code where the Opt class was instantiated or how they
were initialized (the default, set_override, etc.) would compare as
not being equal. The result is that consumers of oslo.config that
include multiple definitions of the same option but in different files
would report an error for a duplicate option. The error may also be
triggered by using set_override() and re-registering an option under
some circumstances.
This patch ensures that the location where the option's value is set
does not factor into the equality check for the option.
Change-Id: I0c57f92c28f220009e01cfe641ec14039c51f671
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This has been deprecated for quite a while and some of the upcoming
work to be done in oslo.config will be easier if we don't have to
deal with this unused parser too.
The class is still currently in use in networking-cisco, but in
change https://review.openstack.org/554617 we have provided a fix
that removes their reliance on it. They are planning to remove it
entirely in Rocky and are only testing against a capped version in
Queens so we are not going to wait on them.
Change-Id: Id6ae40311f967e172e45bcb0e61812365a72618b
Keep track of which configuration file a group of settings is in to
try to report that as a detail for user settings.
Track settings from the command line separately from settings in the
configuration files.
It is possible that the configuration file details can be wrong if a
group appears in more than one file. We can be more accurate with this
after we have the backend driver feature implemented, but since most
deployments use a single file so this should be relatively useful.
Change-Id: I04ce72dbc5f676296acfeafc13169177ad815350
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Add a get_location() method to ConfigOpts to ask where the option
value was set. Update _do_get() to return this information based on
the search criteria.
The LocationInfo data structure has 2 fields. We are only using the
location for now, but the detail field will be filled in by changes
later in this series.
Change-Id: I3643c49b3de1850139913ce395199c238dbe6cf0
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Nova uses a common pattern for choices where a 'choices' parameter is
provided and the choices are later documented in the help text. This
frequently leads to code and docs getting out-of-sync and requires
authors to be consistent in how they write option descriptions.
Eliminate the need to do this by allowing users to describe the choices
in the same place as the choices are declared.
Change-Id: Ic084b04ebf232fb72c9c05bbea3a216391b15c83
Config option `StrOpt` accept choices parameter to indicate
valid values and include comments "# Allowed values:" when generating
sample config file, so it's unnecessary to add allowed values in hep
string of the config option. But it doesn't include choices information
when printing help for the config option. This commit makes consistency
for both of sample config file and help output of command line.
Closes-Bug:1727683
Change-Id: I962e49c81bdf44043a6e1233408a6ee5b883acde
It's no longer possible to assign a value of a different type in the
config via an override. This behavior was changed in the last release to
default to disallow overrides of different types, and in this release it
is no longer able to be overridden
Change-Id: I8a5e868e0adc2c7b6f46f56ee77a8129c34badb9
User provided configuration files in $SNAP_COMMON should always
be preferred over snap provided default config files in $SNAP.
Re-order the target location list in _get_config_dirs to ensure
that files in $SNAP_COMMON are selected over those in $SNAP.
Change-Id: I7cba87ab328edc05b1e6edd1ff512e8fd5192638
This log message has been around since change
I6e28c25f04273f7def486fadd541babc8cf423cb.
The OptGroup doesn't implement __str__ so the warning
message isn't helpful.
This change implements __str__ to return the group
name so the warning message actually helps people.
Change-Id: Ie6d218e7f6ea846b1f7d7f8452cc07f0be9e32a5
Closes-Bug: #1719032
The 'ignore_case' should be a boolean. Make it so and avoid the
momentary confusion where you figure out why this works.
TrivialFix
Change-Id: I66ce1b4f00cdeefb901a8bbdb30dcadc41e9b83f
In I6e28c25f04273f7def486fadd541babc8cf423cb we tried to add the
ability to look for a deprecated option using its old name but finding
the value associated with the new name, similar to how the lookup
works when reading values from config files. That patch did not
account for group values that are objects not strings. This fixes the
oversight.
Change-Id: I98e18e88626164b365466ff476125d41cc8641ec
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
In python 3.6, escape sequences that are not
recognized in string literals issue DeprecationWarnings.
Convert these to raw strings.
Change-Id: Ibfefa7467c29bcb30c65f3c5d296daa8cc1ff7be
Address some of the challenges the sample config generator has when
dealing with dynamically created groups and with options that may change
based on a driver or other plugin selection.
Change-Id: I66b195835a7db3e32b16ddac2166782ff8592806
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
With snap packaging (see snapcraft.io) the package is installed into a
read-only squashfs filesystem, which includes the default config. For
example, $SNAP/etc/nova/nova.conf. To override the defaults, a separate
writable directory is used, and this directory is also unique to the snap.
For example, either $SNAP_COMMON/etc/nova/nova.conf, or
$SNAP_COMMON/etc/nova/nova.conf.d/ can be used to override config.
This patch adds these snap directories to the default config paths where
oslo looks for config.
For more details on $SNAP and $SNAP_COMMON please refer to
https://snapcraft.io/docs/reference/env.
Change-Id: I83627e0f215382aedc7b32163e0303b39e8bccf8
Closes-Bug: 1696830
Philippe SERAPHIN