In the policy_enforcement module if policy.enforce() will raise
PolicyNotAuthorized exception, there is additional check if user is
trying to modify own or someone else resource. In case when user is not
allowed to show resource even, error 404 is raised to "hide" any
information about requested resource.
But that was also the case for POST (create resource) requests and 404
error when user is trying e.g. create network is confusing.
So this patch modifies that logic and in case of "create_" actions it
will return 403 if user was not authorized to do such operation.
Closes-Bug: #1965294
Change-Id: I80b0616c335134a564361137b2a00ff86dcbdf1c
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This completes removal of all references to "import mock",
changing to "from unittest import mock" in fullstack and
functional tests.
Added a hacking check to enforce it in future patches.
Change-Id: Ifcaf1c21bea0ec3c35278e49cecc90a101a82113
Today we shim the RESOURCE_ATTRIBUTE_MAP in neutron; it references the
equivelant in neutron-lib named RESOURCES.
This patch removes neutron's RESOURCE_ATTRIBUTE_MAP and cleans up
neutron.api.v2.attributes in prep to delete it.
To do so:
- CORE_RESOURCES and RESOURCE_FOREIGN_KEYS are moved to the single
module that references them respectively and the are made private (no
consumers use them).
- get_collection_info is removed and instead the 2 uses in neutron
just use the get() method of the RESOURCES map. There are no external
uses of get_collection_info.
- References using RESOURCE_ATTRIBUTE_MAP are switched over to
neutron-lib's RESOURCES.
- The neutron.api.v2.attributes module is removed as it's empty now.
- A few api attribute UTs are removed; there's nothing to test as per
this patch.
NeutronLibImpact
Change-Id: Iaacee584d499c4d33d6d2dd9609c7ac0f2cfc386
This patch switches callbacks over to the payload object style events
[1] for BEFORE_RESPONSE and AFTER_REQUEST based notifications. To do
so an APIEventPayload object is used with the publish() method to
pass along the API related data. In addition a few UTs are updated to
work with the changes.
NeutronLibImpact
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads
Change-Id: Ibd8559e0db9dcc995abf8937a0cb764b21a18531
Change [1] altered the behavior of the legacy API controller
to do the sane thing and return an HTTP 403 instead of a 404
whenever a user got a policy authorization failure when trying
to mutate a resource they have the permission to view.
This carries the same logic over to the pecan API.
This also adjusts the logic for GET requests to return 404s
instead of 403s to match the resource hiding behavior of the
old controller.
1. I7a5b0a9e89c8a71490dd74497794a52489f46cd2
Closes-Bug: #1714388
Change-Id: I9e0d288a42bc63c2927bebe9c581b83e6fbe010b
The legacy API controller included the original object in the delete
notification body payload. This adjusts pecan to do the same.
Closes-Bug: #1714386
Change-Id: I5b2f87e7b1eccceedb821506bb2de43b764fde8d
Allows posting revision number matching in the If-Match header
so updates/deletes will only be satisfied if the current revision
number of the object matches.
DocImpact: The Neutron API now supports conditional updates to resources
that contain the standard 'revision_number' attribute by
setting the revision_number in an HTTP If-Match header.
APIImpact
Partial-Bug: #1493714
Partially-Implements: blueprint push-notifications
Change-Id: I7d97d6044378eb59cb2c7bdc788dc6c174783299
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
policy.init() is called in the before hook, but this isn't
invoked on GET calls so if the first call a thread receives
is a GET call, we can end up with an uninitialized policy
enforcer and get a traceback.
This just calls policy.init() in the after hook as well.
Change-Id: I29ebc9a91b98a27e707d5b35ad1a24a26e8c8f44
Closes-Bug: #1671267
Remove The following _MAX_LEN constants from
neutron/api/v2/attributes.py and use the corresponding DB field size
constants from neutron_lib.db.constants instead.
NAME_MAX_LEN --> NAME_FIELD_SIZE
TENANT_ID_MAX_LEN --> PROJECT_ID_FIELD_SIZE
DESCRIPTION_MAX_LEN --> DESCRIPTION_FIELD_SIZE
LONG_DESCRIPTION_MAX_LEN --> LONG_DESCRIPTION_FIELD_SIZE
DEVICE_ID_MAX_LEN --> DEVICE_ID_FIELD_SIZE
DEVICE_OWNER_MAX_LEN --> DEVICE_NAME_FIELD_SIZE
In alembic migration scripts, the raw numerical value is used.
For more information, see:
http://lists.openstack.org/pipermail/openstack-dev/2016-October/105789.html
NeutronLibImpact
Change-Id: I734890372584fe27e5d6ec38c0cad2de882ff11c
It's written to but never read from!
Related-Bug: #1642426
This patch is required by neutron-lib adoption because it is part of
rehoming neutron/api/v2/attributes.py. All subprojects that reference
PLURALS must be patched to remove the reference.
NeutronLibImpact
Change-Id: Ifdf29b8fc4c824c0ee840c4f51593a5aa8a22127
By changing pecan to store plugins by collection (plural) instead of
resource (singular) we don't need REVERSED_PLURALS and
get_resource_info().
These are not used outside of neutron core.
Consequently, this exposed a bug with some controllers not honoring an
extension's intention of disallowing pagination and sorting. This is
also fixed in this patch to get successful test runs.
Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Change-Id: I9d529d0a2ad369e2be0a8df3c6f06a6532e8b13d
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
[1] and [2] were added to use the registry callback notifier for dhcp and nova
notifications for the legacy wsgi layer. This adds the same functionality
to the pecan wsgi layer. It just so happens to clean the code up nicely
and also fixes a bug that was introduced by [3] that caused an ever
increasing number of subscriptions to the registry callback notifier.
[1] I7440becb6d30af7159ecaeba09d7a28eceb71bea
[2] I1d7d4b80ee77deefce18df22f76cab81750c0397
[3] I607635601caff0322fd0c80c9023f5c4f663ca25
Change-Id: I0a8b64e7742283a9d6c6b42ebc27887836df69ec
Closes-Bug: #1633296
This makes the notifier subscribe to core resource events
and leverage them if they are available. This solves the
issue where internal core plugin calls from service plugins
were not generating DHCP agent notifications.
Closes-Bug: #1621345
Change-Id: I607635601caff0322fd0c80c9023f5c4f663ca25
The legacy pecan uri routing allowed the definition of member actions.
These are API resources that hang off a particular resource_id. An
example of this is:
PUT /routers/{router_id}/add_router_interface
The legacy seemed to do only PUTs and GETs so that has been implemented
in Pecan. Other methods can easily be added if needed.
Also, with the generic handling of this came the decision to remove the
specific pecan routers controller because it's only function was to
facilitate its member actions to work. It is no longer needed.
Change-Id: If776476545edca0c4c43ce3969bb5d1af79f4382
The legacy controllers used their own references to the resource
attribute info while pecan was looking for it in the
attributes.RESOURCE_ATTRIBUTE_MAP. This aligns pecan controllers
with that of the legacy controllers, in that the pecan controllers
now store the their own resource attribute info.
This also fixes a bug that was unnoticed until the above change was
made. The ItemController's index method was not passing the *args and
**kwargs to the get method.
Change-Id: I1cc85daecfb19f73092e52678fd19ff1a2912195
If there was a bad status code, none of the notification
logic should attempt to process. A bad status code means
that the assumptions the notification code makes about
original resources being present may not be true.
This also fixes the exception translator so the correct
status code is set by the hook on exceptions that have
a fault map entry.
Closes-Bug: #1557516
Change-Id: Ibff419e7caa6de422f04eb1e43fd2aed304bdd97
This adds the standard 'object.(create|update|delete).(start|end)'
notifications to the Pecan notification hook and adds unit tests
to exercise them.
This patch also corrects the on_error handler for untranslated
exceptions which was incorrectly raising the exception rather than
returning it. This was resulting in the other hooks not getting
the correct status code on an untranslated exception.
Closes-Bug: #1552979
Change-Id: I400f8d3988db204caed25e7c848a415b45d47172
This patch adds a the Nova notifier calls to the pecan
notification hook and adds some simple functional tests
for them as well.
This patch also alters the operations of the policy enforcement
hook, which now stores in the request context the original
value of the object rather than the value of the object as it
would have appeared to the plugin - that is to say a merge
between the original objects and the parameters passed on to
the request body. Such 'frankenobject' is indeed of no use for
the notifiers.
Partial-Bug: #1552979
Change-Id: I4a37197eb38afe15d2b368f4e355226824478792
This change updates the quota enforcement hook to employ reservations
in place of the simple limit check currently used.
As a part of this change, functional tests for the quota enforcement
hook are also added.
Change-Id: I1e8a59ffbb38c252a824337c556249a3e395484d
Related-blueprint: wsgi-pecan-switch
This patch adds support for DHCP notifications into the Notifier hook,
which so far has been pretty much a placeholder for future work.
To this aim its priority has been changed in order to ensure the
'after' hook is executed after the 'after' hook for the policy engine.
This will ensure that any 'alien' data returned from the plugin is
stripped off before being sent to the notifiers, as well as any
attribute that the user is not authorized to access.
Since delete operations require to send the original object to the
notifier, this patch leverages the "prefetch" feature of the policy
hook to avoid loading again the object from the plugin. This is not
ideal and will be fixed in another patch where prefetching will be
performed in its own hook.
The ACTION_MAP constant has been factored out in an appropriate
module for constants as it is now used by the NotifierHook class as
well. The decision of using a new constant module is rather
arbitrary as the module neutron.common.constants could have been
used as well.
With this patch, the notifier hook only sends events signalling
completion of operations (e.g.: network.create.end) as these are
the only events processed by the DHCP agent. Support for 'start'
events will be added in a subsequent patch.
Related-Blueprint: pecan-wsgi-switch
Change-Id: I69680952f99c404d4535db48db73fc815977f2ee
This patch refactors the pecan controllers and functional tests
by spliting the code in several modules in order to avoid excessive
growth and ensure some level of consistency as regards function
and classes defined in each module.
To this aim:
- The generic resource controllers are moved into their own module
- Other controllers, such as extension and quotas, are moved into
their own module
- Functional tests are split into 3 modules: one for generic tests,
one for request processing tests, and the latter for testing hooks.
Related blueprint wsgi-pecan-switch
Change-Id: Id76cd88208dfd273bccadc15a1c6febbcc4fc030
Slawek Kaplonski