The policy rule ``shared_security_group`` allows to create new policy
rules checking if a security group rule belongs or not to the project
default security group.
By default the behaviour has not changed. If an administrator wants
to prevent a non-privileged user from creating or deleting rules in the
default security group, the ``create_security_group_rule`` and
``delete_security_group_rule`` can be overriden. An example is provided
in the unit tests.
Closes-Bug: #2019960
Change-Id: I6c90b61df0e726ef07f177801069baf30c49de67
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/api.
Trivialfix
Change-Id: I1258b04f64a18036407e1d9de9ddca7472af0d11
When new default policy rules and scope enforcement are enabled, Neutron
needs to handle properly not only PolicyNotAuthorized exception from
oslo_policy module but also InvalidScope exception.
This patch adds handling of that exception to the neutron policy
modules.
In the check() method from the neutron.policy module we are calling
ENFORCER.enforce() method with do_raise=False which means that
PolicyNotAuthorized isn't rasised. Unfortunately it seems that there is
bug in oslo.policy module and InvalidScope is raised even with
do_raise=False.
For now, lets workaround it in Neutron by properly handling InvalidScope
exception in the check() method.
This workaround can be cleaned when bug [1] will be fixed in
oslo.policy.
[1] https://bugs.launchpad.net/oslo.policy/+bug/1965315
Partial-Bug: #1959333
Change-Id: I973f8896248c8222031c53343bb53ce48254da74
The goal of [1] is to, in case of failing when removing the quota
reservation, continue the operation. Any expired reservation will
be removed automatically in any driver.
If the DB transaction fails, it should affect only to the reservation
trying to be deleted. This is why this patch isolates the
"remove_reservation" method and guarantees it is called outside an
active DB session. That guarantees, in case of failure, no other DB
operation will be affected.
This patch also partially reverts [2] but still checks the security
group rule quota when a new security group is created. Instead of
creating and releasing a quota reservation for the security group
rules created, now only the available quota limit is checked before
creating them. That won't prevent another operation to create security
group rules in parallel, exceeding the available quota. However, this
is not even guaranteed with the current quota driver.
[1]https://review.opendev.org/c/openstack/neutron/+/805031
[2]https://review.opendev.org/c/openstack/neutron/+/701565
Closes-Bug: #1943714
Change-Id: Id73368576a948f78a043d7cf0be16661a65626a9
The quota driver ``ConfDriver`` was deprecated in Liberty release.
``NullQuotaDriver`` is created for testing although it could be used
in production if no quota enforcement is needed. However, because
the Quota engine is not plugable (is an extension always loaded), it
could be interesting to make it plugable as any other plugin.
This patch also creates a Quota engine driver API class that should be
used in any Quota engine driver. Currently it is used in the three
in-tree drivers implemented: ``NullQuotaDriver``, ``DbQuotaDriver``
and ``DbQuotaNoLockDriver``.
Change-Id: Ib4af80e18fac52b9f68f26c84a215415e63c2822
Closes-Bug: #1928211
While implementing some Neutron extension, I noticed that whenever a parameter
that is expected by the API is not entered, we receive an internal error
message. However, from the user/operator/dev calling the API, it might not be
clear what the error is. Therefore, we should at least, log these validation
errors.
This patch proposes to enclose the call to Controller.prepare_request_body with
a try/except block, and then we log the exception in warning level. After
logging the exception, we raise it to allow the normal execution flow to
continue.
Change-Id: I855442ee8c0c4b4f0aff19968a5db07fca898146
As a complement to the create_port_bulk optimizations that took place in
the Stein cycle, create the ability to have an ML2 plugin optionally
expose an 'update_port_bulk' operation as well. This will permit ML2
plugins to optimize this code path for bulk operations as well.
Change-Id: Id2c88a82715bccd1c23213828ef904322ab4386f
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
The neutron.common.constants.EXT_PARENT_RESOURCE_MAPPING constant lives
in neutron-lib. This patch consumes it by removing it from neutron
and switching all uses over to libs constant.
NeutronLibImpact
Change-Id: Ib7ea6dea58bf211a2da2b103cb526233df04ba49
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
The neutron.common.rpc module has been in neutron-lib for awhile now and
neutron is shimmed to use neutron-lib already.
This patch removes neutron.common.rpc and switches the code over to use
neutron-lib's implementation where needed.
NeutronLibImpact
Change-Id: I733f07a8c4a2af071b3467bd710290eee11a4f4c
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.
NeutronLibImpact
Change-Id: I9704f20eb21da85d2cf024d83338b3d94593671e
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
Add common parent owner check for the resources which introduced by
service plugin.
Then port forwarding resource will share the same tenant_id with
floatingip. That means only the fip owner can create/update/get/delete
the associated port forwarding resource.
Partially-Implements: blueprint port-forwarding
Partial-Bug: #1491317
Change-Id: I450c674e55ca15e1d9a6a6224138f3305427da68
Enforce validation on filter parameters on list requests.
If an API request contains an unknown or unsupported parameter,
the server will return a 400 response instead of silently ignoring
the invalid input.
In resource attributes map, all filter parameters are annotated by
the ``is_filter`` keyword. Attributes with is_filter set to True
are candidates for validation.
Enabling filter validation requires support from core plugin and
all service plugins so each plugin need to indicate if it supports
the validation by setting ``__filter_validation_support`` to True.
If this field is not set, the default is False and validation is
turned off. Right now, the ML2 plugin and all the in-tree service
plugin support filter validation. Out-of-tree plugins will have
filter validation disabled by default.
An API extension is introduced to allow API users to discover this
new API behavior. This feature can be disabled by cloud operators
if they choose to do that. If it is disabled, the extension won't
be presented.
Depends-On: Ic3ab5b3ffdc378d570678b9c967cb42b0c7a8a9b
Depends-On: I4397df1c35463a8b532afdc9c5d28b37224a37b4
Depends-On: I3f2e6e861adaeef81a1a5819a57b28f5c6281d80
Depends-On: I1189bc9a50308df5c7e18c329f3a1262c90b9e12
Depends-On: I057cd917628c77dd20c0ff7747936c3fec7b4844
Depends-On: I0b24a304cc3466a2c05426cdbb6f9d99f1797edd
Change-Id: I21bf8a752813802822fd9966dda6ab3b6c4abfdc
Partial-Bug: #1749820
The externally consumed APIs from neutron.db.api were rehomed into
neutron-lib with https://review.openstack.org/#/c/557040/
This patch consumes the retry_db_errors function from lib by:
- Removing retry_db_errors from neutron.db.api
- Updating the imports for retry_db_errors to use it from lib
- Using the DB API retry fixture from lib in the UTs where applicable
- Removing the UTs for neutron.db.api as they are now covered in lib
NeutronLibImpact
Change-Id: I1feb842d3e0e92c945efb01ece29856335a398fe
Handle an edge case that API users send a POST/PUT request with
invalid data in request body.
Closes-Bug: #1780327
Change-Id: I3877c18a18ac506dc8f4a9ded2a18b53b9f6cfae
This patch switches callbacks over to the payload object style events
[1] for BEFORE_RESPONSE and AFTER_REQUEST based notifications. To do
so an APIEventPayload object is used with the publish() method to
pass along the API related data. In addition a few UTs are updated to
work with the changes.
NeutronLibImpact
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads
Change-Id: Ibd8559e0db9dcc995abf8937a0cb764b21a18531
Idfd956fa213e1ba6c3b1c983f246c5b515951e6d rehomed the FAULT_MAP into
neutron-lib. This patch consumes it from lib and removes its definition
from neutron.
NeutronLibImpact
Change-Id: Ib051555ea8496ed7eec8ed30733c964012e31c32
Idfd956fa213e1ba6c3b1c983f246c5b515951e6d rehomed the FAULT_MAP into
neutron-lib. Since consumers expect to access the latest faults in this
map which are updated by neutron at runtime, we can't just move
consumers over to lib's version (it won't be updated by neutron).
Therefore this patch shims neutron's FAULT_MAP to use lib's. Once
this shim lands consumers can move to lib's FAULT_MAP (neutron will
ref it via shim) and finally we can then consume in neutron and remove
in a subsequent patch.
The shim herein shouldn't impact consumers so should be free to land
without worry of their impact.
Change-Id: I8e79f29206ec244d870e19f99c18ccaaa50b4761
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
neutron-lib now contains the functionality from neutron's attributes
module. This patch moves the local references over to neturon-lib's
implementation and removes the functionality from the attributes
module.
NeutronLibImpact
Change-Id: I773702ebd4b66d28059cb3ced64b2b477ceff8af
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
according to https://wiki.openstack.org/wiki/Python3, now we should avoid
using six.iteritems and replace it with dict.items.
Change-Id: I58a399baa2275f280acc0e6d649f81838648ce5c
Closes-Bug: #1680761
When a user is not authorized to see a given resource, we need to
convert HTTP 403s into HTTP 404s to avoid giving away information
that the resource exists. However, the previous code was being
overaggressive and doing this conversion even in some cases where
the user is allowed to see the resource and really needs to know
that what they were trying to do is forbidden, not be told that the
resource doesn't exist. This fixes that logic to only do the 403
to 404 conversion when truly appropriate.
Change-Id: I7a5b0a9e89c8a71490dd74497794a52489f46cd2
Closes-Bug: 1682621
By changing pecan to store plugins by collection (plural) instead of
resource (singular) we don't need REVERSED_PLURALS and
get_resource_info().
These are not used outside of neutron core.
Consequently, this exposed a bug with some controllers not honoring an
extension's intention of disallowing pagination and sorting. This is
also fixed in this patch to get successful test runs.
Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Change-Id: I9d529d0a2ad369e2be0a8df3c6f06a6532e8b13d
In some places, indentation was wrong.
In this patchset, they are fixed.
Also, I did some readibility improvements which I've found
during indentation fixing.
Change-Id: I298b9b0c557c07848c69ed23b0faa3a3147d055d
Update the API to accept project_id in requests and return
project_id in responses.
For now, the API treats tenant_id and project_id equivalently.
It accepts either or both in requests.
It returns both in responses, depending on filters.
We include an extension to indicate that support for project_id
is enabled in the API.
Completes: blueprint keystone-v3
APIImpact: Describe how the Networking API supports Keystone V3.
Co-Authored-By: Henry Gessau <HenryG@gessau.net>
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Change-Id: I8775aa8a477191ef21e7c3c6da31d098befefc3c
Retrying mutating operations at the API layer caused a
couple of problems. First, when components would call
the core plugin using the neutron manager, they would
have to handle the retriable failures on their own or
undo any work they had done so far and allow retriable
failures to be propagated up to the API. Second, retrying
at the API makes composite operations (e.g. auto allocate,
add_router_interface, etc) painful because they have to
consider if exceptions are retriable before raising
fatal exceptions on failures of core plugin calls.
This patch begins the process of moving them down to the
core operations with a new decorator called
'retry_if_session_inactive', which ensures that the
retry logic isn't triggered if there is an ongoing transaction
since retrying inside of a transaction is normally ineffective.
Follow-up patches apply them to various parts of the code-base.
Additionally, the args and kwargs of the method are automatically
deep copied in retries to ensure that any mangling the methods
do to their arguments don't impact their retriability.
Finally, since we are leaving the API decorators in place for now,
the retry logic will not be triggered by another decorator if an
exception has already been retried. This prevents an exponential
explosion of retries on nested retry decorators.
The ultimate goal will be to get rid of the API decorators entirely
so retries are up to each individual plugin.
Partial-Bug: #1596075
Partial-Bug: #1612798
Change-Id: I7b8a4a105aabfa1b5f5dd7a638099007b0933e66
The legacy pecan uri routing allowed the definition of member actions.
These are API resources that hang off a particular resource_id. An
example of this is:
PUT /routers/{router_id}/add_router_interface
The legacy seemed to do only PUTs and GETs so that has been implemented
in Pecan. Other methods can easily be added if needed.
Also, with the generic handling of this came the decision to remove the
specific pecan routers controller because it's only function was to
facilitate its member actions to work. It is no longer needed.
Change-Id: If776476545edca0c4c43ce3969bb5d1af79f4382
QoS rule can be edited or created only by admin users. Because
rule object don't have tenant_id attribute there was error
in checking tenant_id of object and neutron server got internal
server error.
Now if updating object doesn't have tenant_id at all (like in
case of QoS rules), 404 error will be returned to the user.
New API tests for checking this case are added for QoS.
Change-Id: Ia82ad84a3a07df4df8eaeaed6c47d31be9493cbd
Closes-Bug: #1515564
Refactors the current pecan code to use predefined methods to get
the plugin methods the controllers will call. This also adds in
parent resource information to build those method names, but parent
support will not be complete until a follow up patch.
Change-Id: I4e0e6ef96364991a9393bc5fba495e37dade68a1
Closes-Bug: #1596754
This fix will allow the delete event to have the required data
in the notification payload.
Change-Id: I57a001ca2fddc2a750026e7da7980bfd8e5aab40
Closes-Bug: 1593127
Once the decoupling started with 60c05a6d57, this cleanup
patch continues the effort by improving the decoupling
between the API layer and Nova.
Change-Id: I1d7d4b80ee77deefce18df22f76cab81750c0397
Implements pagination and sorting in the QueryParametersHook. Doing
this in a hook keeps the main controller code clean and also doesn't
require implementing pagination and sorting in every controller.
Change-Id: I5f439cf2457984d6b62ce5fed9d5d471aafbde81
Closes-Bug: #1591338
Bug 1591766 unveiled an issue where calling the plugin API does not trigger
DHCP notifications. This is required by the auto-allocated-topology service
plugin that calls core_plugin.update_network(), and expect notifications
to be sent out on state changes. To accomplish this, the logic has been
encapsulated in the DHCP module, and leveraged via callback mechanisms.
For this reason, new events have been introduced, AFTER_REQUEST, and
BEFORE_RESPONSE. The latter in particular is the one needed to hook up
dhcp notifications in order to preserve backward compatibility.
More precisely, core plugins that use DHCP as is or implement their own,
(with or without an agent) should already instantiate their own notifier,
and if they do not, this should be rectified.
A search on codesearch.openstack.org reveals that out-of-tree plugins
already specify their own notifiers, and the default initialization is
clearly redundant now.
Related-bug: #1591766
Change-Id: I7440becb6d30af7159ecaeba09d7a28eceb71bea
The legacy controllers used their own references to the resource
attribute info while pecan was looking for it in the
attributes.RESOURCE_ATTRIBUTE_MAP. This aligns pecan controllers
with that of the legacy controllers, in that the pecan controllers
now store the their own resource attribute info.
This also fixes a bug that was unnoticed until the above change was
made. The ItemController's index method was not passing the *args and
**kwargs to the get method.
Change-Id: I1cc85daecfb19f73092e52678fd19ff1a2912195
Previously the pecan controller creation was based on the
attributes.RESOURCE_ATTRIBUTE_MAP's keys after it had been fully
populated by all active extensions. The problem with the problem
with this strategy is that those keys did not necessarily define
the actual URI path, and if any extensions ever had subresources
(.i.e. /qos/policies/{policy_id}/bandwidth_limit_rules), the
attributes.RESOURCE_ATTRIBUTE_MAP was never populated with those.
Getting those subresources would require some very unsightly code
and strategies.
What this does is it grabs the resources that every extension
returns in the get_resources method. The get_resources method
returns a list of ResourceExtensions that do infact define the
URI, and does have a clear linkage to subresources. With this
in mind, the pecan initializer can now translate what get_resources
returns into pecan controllers with the correct paths.
The subresource work still needs to be done and will be done in a
follow-up patch. Also, there is an issue with the
auto-allocate-topology tests, but this too can be solved in
another patch so this review does not get bloated.
Change-Id: I3b42b4b603af97538f08b401d2740d3049276a43
This adjusts the update handler in the API layer to pass a deepcopy
of the original update body to ensure that the original is used on
any generated retry requests.
Without this update handlers that modified the body as they processed
it (e.g. the external gateway update for routers) would be missing
data on a retry request.
This adds a test case for the bug that was caused by this.
Closes-Bug: #1584920
Change-Id: I88fb9d45eb82ef48e0d865decdcc7fad1c5fb361
When delete request is thrown with "request body", it returns 500.
This commit fixes the return code from 500 to 400(BadRequest).
Change-Id: Iadf1bc724f9bd1728d3f9d3c0da4f0675da97d30
Closes-Bug: #1519342
Rodolfo Alonso Hernandez