After updating pylint, it started emitting additional "R"
warnings in some cases, fix some of them.
use-a-generator,
unnecessary-lambda-assignment,
consider-using-max-builtin,
consider-using-generator,
consider-using-in,
use-list-literal,
consider-using-from-import
Trivialfix
Change-Id: Ife6565cefcc30b4e8a0df9121c9454cf744225df
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
some remaining ones in miscellaneous directories.
Also cleanup any remaining code that I missed in this
series, or has changed since I started.
Trivialfix
Change-Id: I17b4779020a7bfb369c3e721ab6638cd4a6ab50c
In the policy_enforcement module if policy.enforce() will raise
PolicyNotAuthorized exception, there is additional check if user is
trying to modify own or someone else resource. In case when user is not
allowed to show resource even, error 404 is raised to "hide" any
information about requested resource.
But that was also the case for POST (create resource) requests and 404
error when user is trying e.g. create network is confusing.
So this patch modifies that logic and in case of "create_" actions it
will return 403 if user was not authorized to do such operation.
Closes-Bug: #1965294
Change-Id: I80b0616c335134a564361137b2a00ff86dcbdf1c
When new default policy rules and scope enforcement are enabled, Neutron
needs to handle properly not only PolicyNotAuthorized exception from
oslo_policy module but also InvalidScope exception.
This patch adds handling of that exception to the neutron policy
modules.
In the check() method from the neutron.policy module we are calling
ENFORCER.enforce() method with do_raise=False which means that
PolicyNotAuthorized isn't rasised. Unfortunately it seems that there is
bug in oslo.policy module and InvalidScope is raised even with
do_raise=False.
For now, lets workaround it in Neutron by properly handling InvalidScope
exception in the check() method.
This workaround can be cleaned when bug [1] will be fixed in
oslo.policy.
[1] https://bugs.launchpad.net/oslo.policy/+bug/1965315
Partial-Bug: #1959333
Change-Id: I973f8896248c8222031c53343bb53ce48254da74
The goal of [1] is to, in case of failing when removing the quota
reservation, continue the operation. Any expired reservation will
be removed automatically in any driver.
If the DB transaction fails, it should affect only to the reservation
trying to be deleted. This is why this patch isolates the
"remove_reservation" method and guarantees it is called outside an
active DB session. That guarantees, in case of failure, no other DB
operation will be affected.
This patch also partially reverts [2] but still checks the security
group rule quota when a new security group is created. Instead of
creating and releasing a quota reservation for the security group
rules created, now only the available quota limit is checked before
creating them. That won't prevent another operation to create security
group rules in parallel, exceeding the available quota. However, this
is not even guaranteed with the current quota driver.
[1]https://review.opendev.org/c/openstack/neutron/+/805031
[2]https://review.opendev.org/c/openstack/neutron/+/701565
Closes-Bug: #1943714
Change-Id: Id73368576a948f78a043d7cf0be16661a65626a9
The quota driver ``ConfDriver`` was deprecated in Liberty release.
``NullQuotaDriver`` is created for testing although it could be used
in production if no quota enforcement is needed. However, because
the Quota engine is not plugable (is an extension always loaded), it
could be interesting to make it plugable as any other plugin.
This patch also creates a Quota engine driver API class that should be
used in any Quota engine driver. Currently it is used in the three
in-tree drivers implemented: ``NullQuotaDriver``, ``DbQuotaDriver``
and ``DbQuotaNoLockDriver``.
Change-Id: Ib4af80e18fac52b9f68f26c84a215415e63c2822
Closes-Bug: #1928211
While implementing some Neutron extension, I noticed that whenever a parameter
that is expected by the API is not entered, we receive an internal error
message. However, from the user/operator/dev calling the API, it might not be
clear what the error is. Therefore, we should at least, log these validation
errors.
This patch proposes to enclose the call to Controller.prepare_request_body with
a try/except block, and then we log the exception in warning level. After
logging the exception, we raise it to allow the normal execution flow to
continue.
Change-Id: I855442ee8c0c4b4f0aff19968a5db07fca898146
The hook starts a DB transaction and should be covered with
DB retry decorator.
Closes-Bug: #1777965
Closes-Bug: #1771293
Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
Neutron-LBaaS has now been retired and there will be no Train
release[1]. This patch removes neutron-lbaas references from
neutron.
[1] https://review.opendev.org/658494
Closes-Bug: #1833125
Change-Id: I0fe3fbaf4adf7fb104632fd94cd093e701e12289
In current neutron, when I update resource with incorrect body, neutron
server return 500 NeutronError. It should be fixed and BadRequest (400)
should be returned.
This patch fixed it.
Change-Id: I024b8ef73be69af069e4516dc501df3f5c145419
Partial-bug: #1838587
This change fixes an 'RuntimeError: dictionary changed size during
iteration' error that is raised because of different behaviour between
python2 and python3. An earlier attempt [1] to use the six library did
not fix the issue.
[1] https://review.opendev.org/660427
Resolves-Bug: #1829304
Change-Id: Ic4c5790a5d66e10a606327f4f4e4c92611f161d8
This change fixes an 'RuntimeError: dictionary changed size during
iteration' error that is raised because of different behaviour between
python2 and python3. We use the six library to ensure that the behavior
is compatible across versions.
Change-Id: I0723ae10825e1e2d86789627895e3286d8c97602
Resolves-Bug: #1829304
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
The neutron.common.rpc module has been in neutron-lib for awhile now and
neutron is shimmed to use neutron-lib already.
This patch removes neutron.common.rpc and switches the code over to use
neutron-lib's implementation where needed.
NeutronLibImpact
Change-Id: I733f07a8c4a2af071b3467bd710290eee11a4f4c
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.
NeutronLibImpact
Change-Id: I9704f20eb21da85d2cf024d83338b3d94593671e
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
wsgi call to best_match() generates the following warning:
DeprecationWarning: The behavior of AcceptValidHeader.best_match
is currently being maintained for backward compatibility, but it
will be deprecated in the future, as it does not conform to the RFC.
Change to use acceptable_offers() as was done in nova code.
Simlar changes in the wsgi best language code as well, except
new call is to lookup().
Required bump to get WebOb 1.8.2.
Trivialfix
Change-Id: I45a222c098658eef6a49adcc168b0f5625ea49db
Enforce validation on filter parameters on list requests.
If an API request contains an unknown or unsupported parameter,
the server will return a 400 response instead of silently ignoring
the invalid input.
In resource attributes map, all filter parameters are annotated by
the ``is_filter`` keyword. Attributes with is_filter set to True
are candidates for validation.
Enabling filter validation requires support from core plugin and
all service plugins so each plugin need to indicate if it supports
the validation by setting ``__filter_validation_support`` to True.
If this field is not set, the default is False and validation is
turned off. Right now, the ML2 plugin and all the in-tree service
plugin support filter validation. Out-of-tree plugins will have
filter validation disabled by default.
An API extension is introduced to allow API users to discover this
new API behavior. This feature can be disabled by cloud operators
if they choose to do that. If it is disabled, the extension won't
be presented.
Depends-On: Ic3ab5b3ffdc378d570678b9c967cb42b0c7a8a9b
Depends-On: I4397df1c35463a8b532afdc9c5d28b37224a37b4
Depends-On: I3f2e6e861adaeef81a1a5819a57b28f5c6281d80
Depends-On: I1189bc9a50308df5c7e18c329f3a1262c90b9e12
Depends-On: I057cd917628c77dd20c0ff7747936c3fec7b4844
Depends-On: I0b24a304cc3466a2c05426cdbb6f9d99f1797edd
Change-Id: I21bf8a752813802822fd9966dda6ab3b6c4abfdc
Partial-Bug: #1749820
The externally consumed APIs from neutron.db.api were rehomed into
neutron-lib with https://review.openstack.org/#/c/557040/
This patch consumes the retry_db_errors function from lib by:
- Removing retry_db_errors from neutron.db.api
- Updating the imports for retry_db_errors to use it from lib
- Using the DB API retry fixture from lib in the UTs where applicable
- Removing the UTs for neutron.db.api as they are now covered in lib
NeutronLibImpact
Change-Id: I1feb842d3e0e92c945efb01ece29856335a398fe
This patch is a fix for the issue B101 assert_used found by bandit
scanner:
https://docs.openstack.org/bandit/latest/plugins/b101_assert_used.html
As assert functionality could be turned off by the -O option for python
interpreter, assert statements were replaced to raise AssertionError.
The main reasoning for that is rest code after assert could be not
ready for values that assert filters.
Change-Id: I92a871b1b496c96c0c76cb37e4dda51bfc007e38
Related-Bug: #1759250
Today we shim the RESOURCE_ATTRIBUTE_MAP in neutron; it references the
equivelant in neutron-lib named RESOURCES.
This patch removes neutron's RESOURCE_ATTRIBUTE_MAP and cleans up
neutron.api.v2.attributes in prep to delete it.
To do so:
- CORE_RESOURCES and RESOURCE_FOREIGN_KEYS are moved to the single
module that references them respectively and the are made private (no
consumers use them).
- get_collection_info is removed and instead the 2 uses in neutron
just use the get() method of the RESOURCES map. There are no external
uses of get_collection_info.
- References using RESOURCE_ATTRIBUTE_MAP are switched over to
neutron-lib's RESOURCES.
- The neutron.api.v2.attributes module is removed as it's empty now.
- A few api attribute UTs are removed; there's nothing to test as per
this patch.
NeutronLibImpact
Change-Id: Iaacee584d499c4d33d6d2dd9609c7ac0f2cfc386
This patch switches callbacks over to the payload object style events
[1] for BEFORE_RESPONSE and AFTER_REQUEST based notifications. To do
so an APIEventPayload object is used with the publish() method to
pass along the API related data. In addition a few UTs are updated to
work with the changes.
NeutronLibImpact
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads
Change-Id: Ibd8559e0db9dcc995abf8937a0cb764b21a18531
Idfd956fa213e1ba6c3b1c983f246c5b515951e6d rehomed the FAULT_MAP into
neutron-lib. This patch consumes it from lib and removes its definition
from neutron.
NeutronLibImpact
Change-Id: Ib051555ea8496ed7eec8ed30733c964012e31c32
The neutron-lib commit I360545b6ee4291547e0c5c8e668ad03d3efa4725 moved
the externally consumed globals from neutron.common.constants into lib.
With the exception of PROVISIONAL_IPV6_PD_PREFIX all other constants
in neutron.common.constants should only be used in neutron, and will
hopefully remain that way. External consumers needing access to other
common constants should move them into lib first.
NeutronLibImpact
Change-Id: Ie4bcffccf626a6e1de84af01f3487feb825f8b65
Avoid logging about a missing resource for normal cases
where there is no resource in the request (e.g. HTTP GET's,
custom actions, etc).
Change-Id: I04c6b6737f238ca6250869b92e1d0771b4f95f22
Partial-Bug: #1707307
Change [1] altered the behavior of the legacy API controller
to do the sane thing and return an HTTP 403 instead of a 404
whenever a user got a policy authorization failure when trying
to mutate a resource they have the permission to view.
This carries the same logic over to the pecan API.
This also adjusts the logic for GET requests to return 404s
instead of 403s to match the resource hiding behavior of the
old controller.
1. I7a5b0a9e89c8a71490dd74497794a52489f46cd2
Closes-Bug: #1714388
Change-Id: I9e0d288a42bc63c2927bebe9c581b83e6fbe010b
This adds the validation to ensure that the plugin supports
native sorting when native pagination is used.
This patch doesn't add a unit test for this because it will
be covered in the switch to pecan for the existing unit tests
in I76dc23fb7b96d82b0da50285bd0aac76142e81e5 (which is how this
bug was discovered).
Closes-Bug: #1714381
Change-Id: I6443832357c91fe791853a374cdec11dd1f968ea
This changes the pecan body validation to bring parity with the
old legacy controller code.
* If a body is present on POST/PUT, it must be a JSON dict
* DELETEs to an item must not contain a body
* A POST request to the standard collection controller must have
resources in the body.
Closes-Bug: #1714348
Change-Id: I1568285c28d227bacf038b3667466a20d3947ca9
The legacy API controller included the original object in the delete
notification body payload. This adjusts pecan to do the same.
Closes-Bug: #1714386
Change-Id: I5b2f87e7b1eccceedb821506bb2de43b764fde8d
Remove duplicated and empty fields from users requests
in Pecan to preserve the old legacy API controller behavior.
Closes-Bug: #1714384
Change-Id: I1afc24b146a8fcc6c8ebae708f32dd7c1795292e
Brian Haley