Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This completes removal of all references to "import mock",
changing to "from unittest import mock" in fullstack and
functional tests.
Added a hacking check to enforce it in future patches.
Change-Id: Ifcaf1c21bea0ec3c35278e49cecc90a101a82113
When bulk ports are created with a security group supplied, the
resulting port(s) should only have that security group assigned. But the
resulting ports are getting both the requested security group as well as
the tenant default security group assigned. This fixes that condition.
In order to ensure that bulk port creation results in the proper
assignment of security groups, add some testing.
Change-Id: I65aca7cd14447cc988e4bc4ab62bc7b9279e7522
Fixes-Bug: #1842666
Sorting and filtering will rely on the attributes information. It is
necessary to populate project info before using it to sort/filter.
Closes-Bug: #1765452
Change-Id: Ife90268530b6e86a0b0d213e4742a2ef81cb2395
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.
NeutronLibImpact
Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
This changes the pecan body validation to bring parity with the
old legacy controller code.
* If a body is present on POST/PUT, it must be a JSON dict
* DELETEs to an item must not contain a body
* A POST request to the standard collection controller must have
resources in the body.
Closes-Bug: #1714348
Change-Id: I1568285c28d227bacf038b3667466a20d3947ca9
Remove duplicated and empty fields from users requests
in Pecan to preserve the old legacy API controller behavior.
Closes-Bug: #1714384
Change-Id: I1afc24b146a8fcc6c8ebae708f32dd7c1795292e
This adds the simple bulk emulation logic that was present
in the old legacy controller for plugins that do not support
bulk operations.
Closes-Bug: #1714355
Change-Id: I4ff02b9c5c007847edd18ec4ad6794257dd79576
Add logic to pecan to add 'project_id' as a mandatory policy
field whenever 'tenant_id' is specified as a policy field.
This logic was added to the previous controller code to ensure
that 'project_id' was queried in the database whenever 'tenant_id'
was required by the policy engine and the user had fields set.
Closes-Bug: #1714378
Change-Id: I3652fbc50ce0c9a7cd1cc193e0933cf0373ecb54
Separate user-applied filters from policy-enforcement
filters processing and put it at the end of the pipeline
so users can't put filters on the API request that impact
the fields available to the hooks.
This prevents a filter excluding the ID field from breaking
pagination.
Change-Id: I05f4582fb1e8809740d473e24fa54483e040a6c8
Closes-Bug: #1714131
The well known service type constants are in
neutron_lib.plugins.constants, but for legacy reasons a few still exist
and are referenced from neutron_lib.constants that we'd like to remove.
This patch switches references over to neutron_lib's plugin constants.
Change-Id: I1861448cec303725b30cef8f42029f467f9e03a3
This patch fixes the missing custom tenant_id and project_id
matching on policy_enforcement.py
Change-Id: I278759f6b65cce7caa4f66fa694488b75f0459b6
Closes-Bug: #1674517
The legacy framework return a list of resources as the response to:
GET /v2.0/
Current pecan 2.0 controller seems return the same as root controller
content. This patch will make it return the same response as legacy
framework
Change-Id: Icae8c1b417d4d5a65a98eeafe80a26886aa254a3
Openstack common has a wrapper for generating uuids. We should
use that function when generating uuids for consistency.
Change-Id: I9e8097f7254e9cd63f27898ab49c473aa6177121
Closes-Bug: #1082248
At this point, if we decide to bump a quota for a resource, we would
need to change the test. It suggests that those tests are not doing the
right thing (they catch any change, not just a regression).
The right thing for the tests to validate is not a particular default
value for a resource quota, but the fact that resources indeed get
default values applied, irrespective of particular values that we tend
to choose at this point in time.
Change-Id: I84afd9e2b9fed8f6eae1743a34bb8064739123a6
This sets up the factory methods needed to have paste deploy create the
pecan app and return it. It also changes the legacy factory methods to
conditionally use the pecan factory methods if the web_framework config
option is set to 'pecan'. This way, all deployments of neutron will not
need to change their api-paste.ini files to get pecan toggled on. It
should just happen without notice once pecan becomes the default.
Also, by moving this to be loaded by paste deploy, there is a good chunk of
code that has been removed because it is no longer necessary.
Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Change-Id: I8b1bbea8d90fdc62715cd8b6738ad955df53d7cd
Policy checks were sending the list action to the policy.check method
whereas the legacy code sends the show action when the API lists.
Change-Id: I89a05910a1c89c7694a0c7de1fe68905cfe008cc
THe policy enforcement hook was not inserting the parent resource name
when checking that a policy action was supported.
Change-Id: Iee5dd06c4c2ff8e73337946ab40faa455873aa4d
Pecan json renderer converts a None response from a controller
into a string 'null' in the HTTP response. This directly conflicts
with us setting an HTTP 204 in the response to HTTP DELETE calls.
The legacy API did not return data on HTTP DELETE calls.
The result of saying we were providing no content and then providing
content was strange behavior with Apache2 mod_proxy (the way devstack
deploys Neutron) where connections would be unusable after a DELETE
call. Since mod_proxy re-uses connections, this would cause long
pauses until connection timeouts occurred.
Change-Id: Ie2fd9108f7b8a60c2574dab31b586ee43fd2a789
Previously, pecan was using custom controllers to handle quotas API
calls. This resulted in the pecan custom controllers becoming out of
date because it was only changed in the original controllers. This
patch removes the use of the pecan controllers and uses shim controllers
to redirect to the original quota controllers.
Change-Id: If673216db51439b9b404d9f18ee5dbf7228c8d2a
They were deprecated in Newton. This patch cleans them up.
Note: it does not mean that the features will not work anymore. On the
contrary, now API will consistently sort and paginate for plugins that
honour the relevant sorting/pagination parameters.
Note2: base resource controller still allows to pass
allow_pagination=False and allow_sorting=False parameters to disable the
features if a registered plugin does not support the features yet.
Change-Id: I5fd30b20f645846d9366740372c4815c4b33e2eb
Related-Bug: #1566514
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
If a bulk create happened with a single item, the pecan code would
return back to the user the singular version of the resource in the
body. For example, to bulk create many security group rules, the user
would give in the body of the request the json with a parent key of
"security-group-rules" with a value of a list of json security group
rules. If this list is of length one, then after the creation of the
one security group rule, the API would return back to the client
"security-group-rule" as the parent key. This is not how the legacy
wsgi layer behaved. The behavior is expected when nova creates security
group rules by calling the neutron API through its deprecated security
groups API.
Change-Id: I8757630403e4d486cd3c8dd6f041e9ee326ba3b4
Closes-Bug: #1633671
[1] and [2] were added to use the registry callback notifier for dhcp and nova
notifications for the legacy wsgi layer. This adds the same functionality
to the pecan wsgi layer. It just so happens to clean the code up nicely
and also fixes a bug that was introduced by [3] that caused an ever
increasing number of subscriptions to the registry callback notifier.
[1] I7440becb6d30af7159ecaeba09d7a28eceb71bea
[2] I1d7d4b80ee77deefce18df22f76cab81750c0397
[3] I607635601caff0322fd0c80c9023f5c4f663ca25
Change-Id: I0a8b64e7742283a9d6c6b42ebc27887836df69ec
Closes-Bug: #1633296
The legacy pecan uri routing allowed the definition of member actions.
These are API resources that hang off a particular resource_id. An
example of this is:
PUT /routers/{router_id}/add_router_interface
The legacy seemed to do only PUTs and GETs so that has been implemented
in Pecan. Other methods can easily be added if needed.
Also, with the generic handling of this came the decision to remove the
specific pecan routers controller because it's only function was to
facilitate its member actions to work. It is no longer needed.
Change-Id: If776476545edca0c4c43ce3969bb5d1af79f4382
This is the initial support for flavors and multiple service
providers with the built-in L3 service plugin.
This patch handles a few key components:
* Adds an optional flavor_id to the router data model
* Adds a new driver controller that performs the following tasks:
* Loads up the configured drivers and 4 default drivers representing
the current matrix of ha/dvr options (single node, ha, dvr, and ha+dvr)
* Associates every router with a driver based on ha/dvr attributes
or the flavor_id if specified
Note that the current drivers are very limited because they don't do anything.
All of the complex logic for the in-tree drivers is still tied up in the giant
mixin the service plugin inherits. Breaking that apart will be in follow-up
patches.
Partially-Implements: blueprint multi-l3-backends
Change-Id: Idce75bf0fc1375dcbbff9b9803fd2fe97d158cff
Implements adding a subresource (of extensions) to its parents
A resource and a subresource has a parent-child relationship, for
example:
A qos resource has a subresource of bandwith_limit_rules. This patch
will allow urls like : /qos/policies/{policy_id}/bandwidth_limit_rules to
work.
Change-Id: Ib2288234710fd6eed7fb9f6b880f57c9dd3beade
Implements pagination and sorting in the QueryParametersHook. Doing
this in a hook keeps the main controller code clean and also doesn't
require implementing pagination and sorting in every controller.
Change-Id: I5f439cf2457984d6b62ce5fed9d5d471aafbde81
Closes-Bug: #1591338
The logic for massaging the fields and filters data was being done in
the controllers. This change pulls that logic out of the controllers
and puts it in a generic hook so that it's done for all controllers.
This will later be used for pagination, but thats not the main
motivation of this patch.
Ideally, the filters and fields would be passed into the controller
methods themselves as kwargs, but that is not easily done due to pecan
not easily allowing overriding the args and kwargs passed to the
controller methods. For now, the fields and filters will have to be
pulled out of the request context.
Change-Id: Ibea9087c7036faa064b87acb52e09c9a3c74148a
This also correctly handles the case where no fields are requested which seems
to have started to break.
Closes-Bug: #1590588
Change-Id: Ida1e3ff575c7fe6c3199c5f4393679bbf89c0fe1
Neutron plugin are able to do field selection on responses (for
instance only returning id & name for a resource). However,
Pecan does not leverage this capability and always tells the
plugin to fetch all fields, doing field selection while
processing the response.
This patch ensures that pecan send the field list down to the
plugin.
As a part of this patch TestRequestProcessing has been update
to inherit from TestRootController rather than
TestResourceController. Inheriting from the latter was causing
tests to be executed twice for no reason, beyong using
TestResourceController's 'port' attribute, which was however
unnecessary as this change proves.
Closes-Bug: #1570259
Change-Id: Iac930cd3bb14dfdda78e6a94d2c8bef2b5c4b9a5
Previously the pecan controller creation was based on the
attributes.RESOURCE_ATTRIBUTE_MAP's keys after it had been fully
populated by all active extensions. The problem with the problem
with this strategy is that those keys did not necessarily define
the actual URI path, and if any extensions ever had subresources
(.i.e. /qos/policies/{policy_id}/bandwidth_limit_rules), the
attributes.RESOURCE_ATTRIBUTE_MAP was never populated with those.
Getting those subresources would require some very unsightly code
and strategies.
What this does is it grabs the resources that every extension
returns in the get_resources method. The get_resources method
returns a list of ResourceExtensions that do infact define the
URI, and does have a clear linkage to subresources. With this
in mind, the pecan initializer can now translate what get_resources
returns into pecan controllers with the correct paths.
The subresource work still needs to be done and will be done in a
follow-up patch. Also, there is an issue with the
auto-allocate-topology tests, but this too can be solved in
another patch so this review does not get bloated.
Change-Id: I3b42b4b603af97538f08b401d2740d3049276a43
A simple replace of hyphens to underscores is needed on a
check to see if a collection needed to be shimmed or not.
This also adds a fake extension and service plugin to test
this. This should allow for easier testing of other test
cases.
Change-Id: Id2ddd01a4c437f14c67aed37a182871c1fceff13
This modifies the expose and when decorators in
pecan to also include the olso db API retry logic
so all controllers that leverage these automatically
get the retry logic to handle temporary DB errors.
To accomplish this, the ATTR_NOT_SPECIFIED sentinel
was modified to be friendly to copy.deepcopy since
unlike in the old API, the new logic will be sending
copies to the core plugin after the attributes have
been populated.
This also includes a workaround for a bad assumption
in the pecan decorator utils that assumes all decorators
have a __code__ attribute.
Change-Id: If4e502d25856190495b160a589232de258364c4f
Closes-Bug: #1557509
Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Fix the status codes returned by the shim controller
to match the expectations of the API tests.
Closes-bug: #1567801
Change-Id: Ibfede6b6bc4ba24cf2b9e4ff5540c2249695252f
The current API allows listings of the neturon server
version without authentication. Because pecan was treating
this like a normal controller, it was requiring keystone
authentication.
This adjusts the version listing to behave as a wrapper so
it can be placed outside of the keystone authentication
wrapper to allow anonymous queries.
Closes-Bug: #1556038
Change-Id: I9f5aa3bea0e11c5e179fc286f9fa350b3930364f
Rodolfo Alonso Hernandez