this is the inital patch of applying codespell to nova.
codespell is a programing focused spellchecker that
looks for common typos and corrects them.
i am breaking this into multiple commits to make it simpler
to read and will automate the execution of codespell
at the end of the series.
Change-Id: If24a6c0a890f713545faa2d44b069c352655274e
As discussed in PTG, we need to test the new RBAC in the
integrated gate and accordingly enable the new defaults
and scope check by default. A new integrated testing job
has been added and results show that the new defaults and
scope checks are working fine. During testing, we found a
few bugs in neutron policies but all are fixed now.
enforce_scope and enforce_new_defaults are oslo policy config
options but they are per service level and the default value
can be overridden. Oslo policy 3.11.0 version allows to override
the default value for these config options[1] so upgrading the
oslo policy version in requirements.txt
Depends-On: https://review.opendev.org/c/openstack/devstack/+/869781
Depends-On: https://review.opendev.org/c/openstack/placement/+/869525
[1] https://github.com/openstack/oslo.policy/blob/3.11.0/oslo_policy/opts.py#L125
Change-Id: I977b2daedf880229c8d364ca011f2ea965b86e3a
Default value of 'CONF.oslo_policy.policy_file' config option
has been changed from 'policy.json' to 'policy.yaml'. If new default
file 'policy.yaml' does not exist but old default 'policy.json' exist
then fallback to use old default file.
An upgrade checks is added to check the policy_file format and
fail upgrade checks if it is JSON formatted.
Added a warning in policy doc about JSON formatted file is deprecated,
also removed all the reference to policy.json file in doc as well as
in tests.
Related Blueprint: policy-json-to-yaml
Closes-Bug: #1875418
Change-Id: Ic4d3b998bb9701cb1e3ef12d9bb6f4d91cc19c18
There are only a few of these remaining in the code base. Remove them.
Change-Id: I33725e2439b0f39c1e9bec9e33a37bf3e24944fb
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We've been slowly removing these as we go. Remove the final few '_LE'
occurrences now.
Change-Id: I75ebd2e95a0c77585d7b4329ca01e4bacc1dd7c4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
oslo has new flag to disable the warning for default check_str
change and keep logging warning if policy name is changed or it is
marked for removal.
Policy defaults refresh BP work changed the defaults for all the
policy so logging their warning is dangerousa and fill up the log file.
Once we switch to new defaults then we can enable them if needed.
Logging warning for name change and marked for removal make sense
when old rules are override by operator so the new flag does not control
those. For policy defaults refresh BP work, we do not have many policies
in this category so logging their warning is ok.
This commits reset the testing flag 'suppress_deprecation_warnings'
default value back to False. Test policy fixture sets that to True to
avoid any warnig during tests run.
Bumping the oslo policy version where this new flag is present
- https://review.opendev.org/#/c/719547/
Partial implement blueprint policy-defaults-refresh
Change-Id: Iaa6baf6877890babb6205bdb3ee2e1a2b28ebd9c
Many policies are changing their defaults and
due to those lot of warning are added on log side.
Make suppress_deprecation_warnings default to True so
that we do not log the policy change warning. Once
we find the way to disable the warning for policy changing
the defaults then we can reset it to False.
Change-Id: I27703d671b06d9da9d131dc60109a89a14dff998
oslo.policy log the deprected policy warning while loading the
policy rule. Flag to suppress the warning was set after load_rule()
was called.
Moving the setting of suppress_deprecation_warnings flag before
load_rule method is called.
Closes-Bug: #1858652
Change-Id: I3f30df8f0fdf76d763ee4770af074a2f4f400ee8
In nova.policy.authorize() method, it convert the context
object to policy value by context.to_policy_values() and then
pass that into oslo.policy enforcer authorize() method.
This is fine till now and does not cause any issue but when
scope_type is set on policy and context then scope info is
not passed correctly.
In case of system scope, oslo.policy check for a key called 'system'
in creds. The oslo.context library uses `system_scope` instead[1],
and the compatibility between both oslo.policy and oslo.context
are handled when complete context is passed into oslo_policy[2].
If nova convert the context object to policy values then system scope
info is not passed into the oslo_policy. Better way is to pass the complete
context object to oslo_policy and let oslo_policy fetch the system scope
info in correct way.
Update the lower constraints for oslo.policy and oslo.context to have
system scope checks feature.
Partial implement blueprint policy-defaults-refresh
[1] f65408df5c/oslo_context/context.py (L321)
[2] b9fd10e261/oslo_policy/policy.py (L994)
Change-Id: I847fc44e62065e3d26e5595e178b83912ab5d19b
Move default target from context.can() into policy.authorize() so
it is easier to unit test the context behaviour.
This was not done originally due to this meaning placement avoided the
strange default target, but that is no longer required.
Change-Id: I23c433dcd459e7f930ac2eb8a3583c857836cae2
The target passed to Enforcer.authorize should be a dict,
similar to the target dict to the RequestContext.can method.
However, we were passing an instance of _DeprecatedPolicyValues
because that is ultimately what comes out of
RequestContext.to_policy_values(). As of change
I4642c57990b145c0e691140970574412682e66a5 in oslo.policy, that
incorrect type for the target parameter results in an error in
the debug logs for the policy check:
cannot format data, exception: Expected a dictionary, got
<class 'oslo_context.context._DeprecatedPolicyValues'> instead.
This resolves the issue by using the same default target dict
that RequestContext.can uses if a target is not supplied.
Note that we get here from NovaKeystoneContext via API middleware
before any request handler is invoked in the wsgi stack, so there
is no context from the request as to what to pass for the target
besides the user_id/project_id.
Change-Id: I4442a7b95d15233f76f7795d45b18ac440ddb831
Closes-Bug: #1807747
The get_enforcer method is used by oslopolicy CLI scripts to generate
policy files. The scripts will use configuration files to find policy
files so that overrides can be generated with default values
registered in code. The get_enforcer method used to parse the
arguments passed in from the system and remove the `namespace` and
`output-file` arguments because they wouldn't be recognized while
processing configuration values.
This commit simplifies the logic of get_enforcer. A related
discussion was held in review:
https://review.openstack.org/#/c/530828/3
Proposing this to nova since I attempted to use similar logic to
fix a problem in keystone, but figured the simplified logic might be
useful here, too.
Change-Id: I7cd27fe8c39ddfc6ec20f4cfe4d62912d4cebaa4
The same policy rule (os_compute_api:os-flavor-manage) is being used
for the create and delete actions of the flavors REST API. It is thus
impossible to provide different RBAC for the create and delete actions
based on roles. To address this, changes are made to have separate
policy rules for each action.
Most other places in nova (and OpenStack in general) have separate
policy rules for each action. This affords the ultimate flexibility
to deployers, who can obviously use the same rule if that is what they
want.
To address backwards compatibility, the new rules added to the
flavor_manage.py policy file, default to the existing rule
(os_compute_api:os-flavor-manage). That way across upgrades this
should ensure if an existing admin has customised the rule, it keeps
working, but folks that know about the new setting can override the
default rule. In addtion, a verify_deprecated_policy method is added
to see if the old policy action is being configured instead of the
new actions.
Closes-Bug: #1675147
Co-Authored-By: Felipe Monteiro <felipe.monteiro@att.com>
Change-Id: Ic67b52ebac3a47e9fb7e3c0d6c3ce8a6bc539e11
1.As mentioned in [1], we should avoid using
six.iteritems to achieve iterators. We can
use dict.items instead, as it will return
iterators in PY3 as well. And dict.items/keys
will more readable. 2.In py2, the performance
about list should be negligible, see the link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
The patch list:
1. cells.
2. compute api.
3. image.
4. network.
5. objects.
6. scheduler.
7. virt.
8. other resources.
Partial-Implements: blueprint replace-iteritems-with-items
Change-Id: Ic6e469eb80ee1774de1374bb36f38b5134b6b311
Now that the credentials dict is the return from
context.to_policy_values() and not the context object itself there's no
need to attempt to pop the 'auth_token' key out before logging.
Change-Id: I3b4eacf6c7d97a4243aff026aefbb53017d5e840
The base oslo.context defines to_policy_values with all the information
that it expects a service to require to enforce policy. Use that instead
of throwing everything in to_dict at policy enforcement.
Change-Id: I0a42b4425e9dd1bd062c48792c4d116dd370afe3
Closes-Bug: #1602081
User-based policy enforcement isn't supported by Nova. There still
exists support for some APIs to keep backwards compatiblity, but support
for them will be removed in the future. So this patch adds a warning
message when people use a rule that is user-based.
Partially implements blueprint user-id-based-policy-enforcement
Co-Authored-By: Ed Leafe <ed@leafe.com>
Change-Id: Iaa9a142ce93a8d13452f0c6318a3d0b54f6220ce
There are two helper scripts in oslo.policy to help deployers understand
their policy configuration better. With the setup.cfg entry these can be
called directly from oslo.policy.
Change-Id: I08dc33367401ec1f98e1795a52d4e981f09a07de
Implements: bp policy-in-code
_ENFORCER.enforce checks policy but does not require that the policy has
been registered beforehand. _ENFORCER.authorize does require that the
policy is registered. By not using enforce we can be sure that all used
policies are registered and will therefore show up in sample file
generation.
Change-Id: I9e986e2c42e8341b910cf8c3a10042fa24c8c8d8
Partially-Implements: bp policy-in-code
Registers in-code the last remaining policy rules.
Adds missing 'discoverable' rules. Without them,
the extension_info API can fail, as it tries to check the
os_compute_api:os_server_tags:discoverable rule. As it wasn't
previously registered, when listing the available extensions,
an exception of type PolicyNotRegistered is encountered.
In order to validate this, functional/api_sample_tests/test_extension_info.py
now runs without mocking policy.authorize.
Switches extension_info to context.can.
Switches nova.cells.filters to context.can.
Switches network.neutronv2.api to context.can.
Removes the rest of the entries in etc/policy.json.
Removes DefaultPolicyTestCase, as it tests the default
policy rule, which is not registered.
Removes rules from fake_policy.py that brings no value,
that are the same as the default values.
Removes extensions authorizer factories.
Removes nova.policy.enforce.
Change-Id: Ie7771768f4f3efe0edc787c12f297aa93d533d7e
Partially-Implements: bp policy-in-code
This adds the basic framework for registering and using default policy
rules. Rules should be defined and returned from a module in
nova/policies/, and then added to the list in nova/policies/__init__.py.
A new context.can() method has been added for policy enforcement of
registered rules. It has the same parameters as the enforce() method
currently being used.
To establish the full pattern for usage the policy checks in the servers
API module have been registered and converted to the new usage.
Now that some policy checks are registered they're being used properly
by tests. Some tests have been updated so that the instance project_id
matches the context project_id in order to pass the 'admin_or_owner'
check.
Change-Id: I71b3d1233255125cb280a000b990329f5b03fdfd
Partially-Implements: bp policy-in-code
When a policy check fails it is easy enough to recreate the target from
request information that's available, but the credentials used can not
easily be determined without asking the user. This logs the non
sensitive credentials when a policy check fails. Things like
project_id, user_id, and roles.
Change-Id: I599660386286529ec0914a18c44d5e2939b18ccd
The oslo has rewritten the common policy for a long time, using a
Enforer class to replace the old check function. In order to sync the
common policy to nova, We have to also changed the nova policy and
related unittests.
Sync policy and relate module from Oslo.
Some related change-id:Ife909bdf3277ef33c2fb1eae16ae261fa6374c63
change-id:Ife84189be4b86a3ee90da4539ff2dbed125be23d
Closes-Bug: #1288178
Change-Id: Ic106851a24cfdc9d847f7d0e0d8a6ecbff888e05
* Test live etc/nova/policy.json data
* Convert all actions in policy.json to admin-only actions
* Check that Policy auth exception is raised when all above actions are
called using non-admin context
Change-Id: I1b97987f8e9e3bacc606385a56daad189cf6ce19
Closes-Bug: #1262325
We don't need to have the vi modelines in each source file,
it can be set in a user's vimrc if required.
Also a check is added to hacking to detect if they are re-added.
Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6
Closes-Bug: #1229324
Previous _ was monkey patched into builtins whenever
certain modules were imported. This removes that and
simply imports it when it is needed.
Change-Id: I0af2c6d8a230e94440d655d13cab9107ac20d13c
The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.
Add the 2013.1b4 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.
Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.
Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.
Make tools/hacking.py include oslo in IMPORT_EXCEPTIONS like it already
does for paste. It turns out imp.find_module() doesn't correct handle
namespace packages.
Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).
Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
context_is_admin role is used by nova to check if
the current user is the admin. But it can only check
role rules. The fix allow generic rules in context_is_admin.
DocImpact
Fixes bug 1118142
Change-Id: Ib4823a67fe63d5356fc8c9280a2013b8855f5217
Modules import nova.config for two reasons right now - firstly, to
reference nova.config.CONF and, secondly, if they use one of the
options defined in nova.config.
Often modules import nova.openstack.common.cfg and nova.config
which is a bit pointless since they could just use cfg.CONF if
they just want to nova.config in order to reference CONF.
Let's just use cfg.CONF everywhere and we can explicitly state
where we actually require options defined in nova.config.
Change-Id: Ie4184a74e3e78c99658becb18dce1c2087e450bb
Now that options have all moved from nova.flags to nova.config, we can
safely remove the nova.flags imports and replace them with nova.config
imports.
Change-Id: Ic077a72dd6419bbf1e1babe71acfa43c4e8b55c8
Use the global CONF variable instead of FLAGS. This is purely a cleanup
since FLAGS is already just another reference to CONF.
We leave the nova.flags imports until a later cleanup commit since
removing them may cause unpredictable problems due to config options not
being registered.
Change-Id: Ib110ba8d1837780e90b0d3fe13f8e6b68ed15f65
Merge in update openstack-common policy code.
Updates Nova-specific policy glue code to eliminate deprecated
openstack-common policy interfaces. Also cleans up policy code
to allow for returning fine-grained policy values.
As a side effect, fixes bug 1039132.
Change-Id: I2951a0de3751bd2ec868e7a661070fed624e4af2
When we create an admin context internally, we need to make sure
that all of the policy actions that are available to that context
are still available. This wasn't working in some cases because
we were looking for a hard-coded role called 'admin'.
Fixes bug 1039093
Change-Id: I939f834a63c9e6e6e7c87b115bd469466da66a69
bug 1037786
This change removed hard-coded 'admin' role checking, and use policy check to
decide is_admin at the time of context construction. And also set
default admin role to 'admin', in policy template, to keep backward
compatibility.
Change-Id: I56b45bfdfba1b8e1391bde7fec3d4cf26c66324c
The original policy framework allowed new policy checks to be created
through inheritance. This is somewhat clunky and unnecessary in
Python. This change refactors policy.py to allow new policy checks
to be registered using an @register() decorator. One consequence is
that HttpBrain is deprecated.
Care has been taken to ensure backwards compatibility; deprecation
warnings will be emitted for uses of HttpBrain or the inheritance-
based checks.
(Pull-up from openstack-common, with deprecation of HttpBrain.)
Change-Id: Ia9a6039a82fe4ebfa9b18e5eb93c21fffee90f09
Use cfg's new helper method to find policy config file. The basic
behavior is "look alongside the config file" with a fall back to
the standard default config paths.
Change-Id: I763097107c53fc9b7d4e783ee92bf16989d92cff
blueprint sphinx-doc-cleanup
bug 945160
- Correct parameter declarations, list formatting, cross-references, etc.
- We don't need "let" in generate_autodoc_index.sh since we aren't doing math.
- Change conf.py to not prefix class and function names with full namespace
in generated output to save width on the screen.
Change-Id: I9adc8681951913fd291d03e7142146e9d46841df
Make FLAGS a ConfigOpts instance and fix up all the places where we
expected FlagValues behaviour.
Change-Id: I8f96f42e0d8d30ba6b362d29861e717cf0fa9e89
Move it here so that it can be kept in sync with openstack-common using
the new update.py script for code in openstack-common's incubation area.
See here for more details:
http://wiki.openstack.org/CommonLibrary#Incubation
Note: this commit just moves the existing code in Nova with no other
changes. A subsequent commit will sync it with latest openstack-common
so that it is easier see the new changes.
Change-Id: If88d678b1b9bad3d37117de7f7159d7fea8ab4c8
The next obvious step in porting to cfg is to define all options using
cfg schemas directly rather than using the flags.DEFINE_* helpers.
This is a large change, but it is almost entirely pure refactoring and
does not result in any functional changes.
The only change to note is that the default values for glance_host,
glance_api_servers and default_publisher_id options are now using opt
value interpolation i.e.
-glance_host=_get_my_ip()
+glance_host='$my_ip'
-glance_api_servers=['%s:%d' % (FLAGS.glance_host, FLAGS.glance_port)]
+glance_api_servers=['$glance_host:$glance_port']
-default_publisher_id=FLAGS.host
+default_publisher_id='$host'
Also note that the lower_bound check on the {report,periodic}_interval
options are no more, but this has been true since cfg was first added.
Change-Id: Ia58c8f0aaf61628bb55b1b8485118a2a9852ed17
Sean Mooney