Commit Graph

26 Commits

Author SHA1 Message Date
Chris Dent 787bb33606 Use external placement in functional tests
Adjust the fixtures used by the functional tests so they
use placement database and web fixtures defined by placement
code. To avoid making redundant changes, the solely placement-
related unit and functional tests are removed, but the placement
code itself is not (yet).

openstack-placement is required by the functional tests. It is not
added to test-requirements as we do not want unit tests to depend
on placement in any way, and we enforce this by not having placement
in the test env.

The concept of tox-siblings is used to ensure that the
placement requirement will be satisfied correctly if there is a
depends-on. To make this happen, the functional jobs defined in
.zuul.yaml are updated to require openstack/placement.

tox.ini has to be updated to use a envdir that is the same
name as job. Otherwise the tox siblings role in ansible cannot work.

The handling of the placement fixtures is moved out of nova/test.py
into the functional tests that actually use it because we do not
want unit tests (which get the base test class out of test.py) to
have anything to do with placement. This requires adjusting some
test files to use absolute import.

Similarly, a test of the comparison function for the api samples tests
is moved into functional, because it depends on placement functionality,

TestUpgradeCheckResourceProviders in unit.cmd.test_status is moved into
a new test file: nova/tests/functional/test_nova_status.py. This is done
because it requires the PlacementFixture, which is only available to
functional tests. A MonkeyPatch is required in the test to make sure that
the right context managers are used at the right time in the command
itself (otherwise some tables do no exist). In the test itself, to avoid
speaking directly to the placement database, which would require
manipulating the RequestContext objects, resource providers are now
created over the API.

Co-Authored-By: Balazs Gibizer <balazs.gibizer@ericsson.com>
Change-Id: Idaed39629095f86d24a54334c699a26c218c6593
2018-12-12 18:46:49 +00:00
rajat29 d27d8e44a3 Normalize dashless 'resource provider create' uuid
When creating resource provider with '--uuid' argument, nova
accept uuid without dash('-') too, which some time results in,
resource provider with same uuid i.e one with dash and one without.

This patch attempts to fix it by transforming dashless UUID into
dashed one before inserting it into the database.

Co-Authored-By: Chen <dstbtgagt@foxmail.com>

Change-Id: I2685eb65907adbd22b2d09264b110692e100eaf9
Closes-Bug: #1758057
2018-08-23 11:20:42 +08:00
Zuul 6ca30fd832 Merge "Resource_provider API handler does not return specific error codes" 2018-07-16 14:52:50 +00:00
deepak_mourya a6411afbfa Resource_provider API handler does not return specific error codes
There are a few places in the resource_provider.py placement
API handler module that need to be updated to return custom
error codes.

Change-Id: I7efdf7f82d581cf5dc9a3decc063b69c8cef22ff
Closes-Bug: #1778929
2018-07-06 15:38:44 +05:30
Balazs Gibizer 00b13a2e2f Prevent updating an RP's parent to form a loop
Placement had RP loop detection for RP creation but if an RP is created
without a parent (e.g. root RP) then the parent can be set later with a
PUT /resource_providers/{uuid} request by providing the UUID of the
parent. In this code path the loop detection was missing from the
validation. Moreover there are different loop cases for create than for
set. For create the only possible loop is when the RP being created is
points to itself as a parent. However when the parent is provided later
in a PUT the RP being updated can have descendant RPs. Setting a parent
to a descendant also creates a loop.

This patch adds the missing check and returns HTTP 400 if loop is detected.

Closes-Bug: #1779635
Change-Id: I42c91f5f752f0a4fba8b1d95489fc3f87a1c5b6e
2018-07-03 10:08:04 +02:00
Balazs Gibizer 16812e8979 Regression test for bug 1779635
This patch adds a test that shows that placement allows setting the
parent of an RP to itself.

Change-Id: Ie81de7bdd43bfdb2ba48996f79d41efc2d70c1cd
Related-Bug: #1779635
2018-07-02 12:13:56 +02:00
deepak.mourya 1ea3d5ec74 Handle CannotDeleteParentResourceProvider to 409 Conflict
Error handling to raise the exception 409 while deleting
the parent if it has children.

Closes-Bug: #1770636
Change-Id: I87df68992e4e635f009974f5205ca4919a4f2576
2018-06-27 13:28:46 +00:00
Matt Riedemann 0a461979df Implement granular policy rules for placement
This adds a granular policy checking framework for
placement based on nova.policy but with a lot of
the legacy cruft removed, like the is_admin and
context_is_admin rules.

A new PlacementPolicyFixture is added along with
a new configuration option, [placement]/policy_file,
which is needed because the default policy file
that gets used in config is from [oslo_policy]/policy_file
which is being used as the nova policy file. As
far as I can tell, oslo.policy doesn't allow for
multiple policy files with different names unless
I'm misunderstanding how the policy_dirs option works.

With these changes, we can have something like:

  /etc/nova/policy.json - for nova policy rules
  /etc/nova/placement-policy.yaml - for placement rules

The docs are also updated to include the placement
policy sample along with a tox builder for the sample.

This starts by adding granular rules for CRUD operations
on the /resource_providers and /resource_providers/{uuid}
routes which use the same descriptions from the placement
API reference. Subsequent patches will add new granular
rules for the other routes.

Part of blueprint granular-placement-policy

Change-Id: I17573f5210314341c332fdcb1ce462a989c21940
2018-05-17 11:12:16 -04:00
Chris Dent 4e07d81260 [placement] Support forbidden traits in API
In a new microversion (1.22) expose support for processing
forbidden traits in GET /resource_providers and GET
/allocation_candidates. A forbidden trait is expressed as
part of the required parameter with a "!" prefix:

    required=CUSTOM_FAST,!CUSTOM_SLOW

This change uses db and query processing code adjustments
already present in the code but guarded by a flag. If the
currently requested microversion matches 1.22 or beyond
that flag is True, otherwise False.

Reno, api-ref update and api history update are included.
Because this microversion changes the value of an existing
parameter it was unclear how to best express that in the
api-ref. In this case existing parameter references were
annotated.

Partially implements blueprint placement-forbidden-traits

Change-Id: I43e92bc5f97db7a2b09e64c6cb953c07d0561e63
2018-04-13 19:24:08 +01:00
Eric Fried 388db7e6e2 placement: Return new provider from POST /rps
To facilitate opaqueness of resource provider generation internals, we
need to return the (initial) generation when a provider is created. For
consistency with other APIs, we will do this by returning the entire
resource provider record (which includes the generation) from POST
/resource_providers.

Change-Id: I8624e194fe0173531c5aa2119c903e3c68b8c6cd
blueprint: generation-from-create-provider
2018-03-14 17:08:55 -05:00
Eric Fried 558540a27c rp: GET /resource_providers?required=<traits>
Introduce placement microversion 1.18 with a new ?required=<trait list>
query parameter accepted on the GET /resource_providers API.  Results
are filtered by providers possessing *all* of the specified traits.
Empty/invalid traits result in 400 errors.

Change-Id: I8191c9a390cb02b2a38a3f1c6e29457435994981
blueprint: traits-on-list-resource-providers
2018-02-23 12:08:32 -06:00
Zuul b9cdcc3f80 Merge "[placement] add name to resource provider create error" 2017-12-14 14:26:07 +00:00
Chris Dent 83030804cc [placement] Add cache headers to placement api requests
In relevant requests to the placement API add last-modified
and cache-control headers.

According the HTTP 1.1 RFC headers last-modified headers SHOULD always
be sent and should have a tie to the real last modified time. If we do
send them, we need Cache-Control headers to prevent inadvertent caching
of resources.

This change adds a microversion 1.15 which adds the headers to GET
requests and some PUT or POST requests.

Despite what it says 'no-cache' means "check to see if the version you
have is still valid as far as the server is concerned". Since our server
doesn't currently validate conditional requests and will always return an
entity, it ends up meaning "don't cache" (which is what we want).

The main steps in the patch are:

* To both the get single entity and get collection handlers add
  response.cache_control = 'no-cache'
* For single entity add response.last_modified = obj.updated_at or
  obj.created_at
* For collections, discover the max modified time when traversing the
  list of objects to create the serialized JSON output. In most of
  those loops an optimization is done where we only check for
  last-modified information if we have a high enough microversion such
  that the information will be used. This is not done when listing
  inventories because the expectation is that no single resource
  provider will ever have a huge number of inventory records.
* Both of the prior steps are assisted by a new util method:
  pick_last_modfied.

Where a time cannot be determined the current time is used.

In typical placement framework fashion this has been done in a very
explicit way, as it makes what the handler is doing very visible, even
though it results in a bit of boilerplate.

For those requests that are created from multiple objects or by doing
calculations, such as usages and aggregate associations, the current time
is used.

The handler for PUT /traits is modified a bit more extensively than some
of the others: This is because the method can either create or validate
the existence of the trait. In the case where the trait already exists,
we need to get it from the DB to get its created_at time. We only do
this if the microversion is high enough (at least 1.15) to warrant
needing the info.

Because these changes add new headers (even though they don't do
anything) a new microversion, 1.15, is added.

Partial-Bug: #1632852
Partially-Implements: bp placement-cache-headers

Change-Id: I727d4c77aaa31f0ef31c8af22c2d46cad8ab8b8e
2017-12-12 15:51:58 +00:00
Chris Dent 885ad10edc [placement] add name to resource provider create error
When a resource provider is created an ObjectActionError can
happen. The API handler for POST /resource_provider catches
this and sends out an error response. Prior to this change the
response only identified the resource provider by uuid. Since
it is possible to create a resource provider by only providing
a name, this error response may not be super useful.

This change adds the name of the proposed resource provider to
the message, to make sure things are clear.

However, it's worth noting that at the current time most
situations which would cause an ObjectActionError are not
possible from the API code path: they are caught by the
JSONSchema validation. The one clear path that is not caught,
and thus is the place where a validation of the response has
been done in a gabbit, is when a resource provider tries to
set its parent uuid to its own uuid. In order for the caller
to do this, the caller has to know its own uuid, so the
ambiguity problem is moot.

As we might make additional changes in the future, adding
the name seems the most flexible thing to do, so here it
is.

Change-Id: I7fee5344fc7cb2e4ba3f46e192ba0122edeb0605
Closes-Bug: #1735405
2017-12-08 15:28:15 +00:00
Stephen Finucane 109f21f3c8 placement: adds REST API for nested providers
Adds a new microversion (1.14) to the placement REST API for supporting
nested resource providers.

For POST /resource_providers and PUT /resource_providers/{uuid}, a new
optional 'parent_provider_uuid' field is added to the request payload.

For GET /resource_providers/{uuid} responses, the
'parent_provider_uuid' field and a convenience field called
'root_provider_uuid' are provided.

For GET /resource_providers, a new '?in_tree=<rp_uuid>' parameter is
supported. This parameter accepts a UUID of a resource provider. This
will cause the resulting list of resource providers to be only the
providers within the same "provider tree" as the provider identified by
<rp_uuid>

Clients for the placement REST API can specify either
'OpenStack-API-Version: placement 1.14' or 'placement latest' to handle
the new 'parent_provider_uuid' attribute and to query for resource
providers in a provider tree.

Change-Id: I4db74e4dc682bc03df6ec94cd1c3a5f5dc927a7b
blueprint: nested-resource-providers
APIImpact
2017-12-06 10:48:09 -06:00
Eric Fried a9105b4904 Include /resource_providers/uuid/allocations link
/resource_providers/{rp_uuid}/allocations has been available since
microversion 1.0 [1], but wasn't listed in the "links" section of the
GET /resource_providers response.  This change adds the link in a new
microversion, 1.11

[1] https://review.openstack.org/#/c/366789/

Closes-Bug: #1714275

Change-Id: I6a1d320ce914926791d5f45e89bf4c601a6b10a0
2017-10-23 14:08:05 -05:00
Chris Dent 863f0ce0b5 [placement] Disambiguate resource provider conflict message
When creating a resource provider, if the uuid or name (or both)
of the resource provider is already being used a 409 response is
returned. Prior to this change the error message always mentioned
the name of the resource provider, even if it was not the cause of
the duplication. That's misleading and unfriendly.

The new code will list the field and the value of the field that
was duplicated. From looking at the olso_db code that creates a
DBDuplicateEntry it appears that sometimes, but not always, the
exception could list both fields, so allowance for that is made
in the message output and the related gabbi tests.

Because this changes the error message, but not the error response
code, this does not violate the api interoperability guidelines[1],
so I'm willing to say this doesn't require a microversion, and thus
does not require a spec.

[1] http://specs.openstack.org/openstack/api-wg/guidelines/api_interoperability.html
If we do think this is a violation, then the guideline should be
made more explicit.

Change-Id: Ibafbfd8302977a2b4e9250fbb5ada283b69d3c25
Closes-Bug: #1693349
2017-06-08 09:57:38 +00:00
Chris Dent 72960a33e0 [placement] adjust resource provider links by microversion
The resource provider representation includes links to other related
resources. Those links should vary depending on microversion as some
of the related resources did not show up until a later microversion.

This change makes it so 'aggregates' show up at microverison 1.1 and
beyond and traits in microversion 1.6 and beyond. A new gabbit has
been added, resource-provider-links which exercises this. Only the
single resource provider representation is tested because the same
code is used for single and collection representations.

Change-Id: Ife64922ee91a775fabdcf718e6eb0e0c7ddfde7a
Closes-Bug: #1693353
2017-05-29 16:38:59 +00:00
He Jie Xu 9c975b6bd8 placement: Add Traits API to placement service
This patch adds support for a REST API for CRUD operations on traits.

  GET /traits: Returns all resource classes.
  PUT /traits/{name}: To insert a single custom trait.
  GET /traits/{name}: To check if a trait name exists.
  DELETE /traits/{name}: To delete the specified trait.
  GET /resource_providers/{uuid}/traits: a list of traits associated
  with a specific resource provider
  PUT /resource_providers/{uuid}/traits: Set all the traits for a
  specific resource provider
  DELETE /resource_providers/{uuid}/traits: Remove any existing trait
  associations for a specific resource provider

Partial implement blueprint resource-provider-traits

Change-Id: Ia027895cbb4f1c71fd9470d8f9281d2bebb6d8a2
2017-04-11 10:31:37 +08:00
liyingjun d88a2f00e1 Error message should not include SQL command
There is SQL command included in the http response for conflicting
resource provider. We should remove SQL related errors from the
error message.

Change-Id: I47726919f9a2d2a767564909f454c334e42ffbe0
Closes-bug: 1664117
2017-03-08 17:05:25 +08:00
Pushkar Umaranikar 14ed7ad64d Placement api: set custom json_error_formatter in resource_provider
Change decorator for resource based actions in resource_provider
handler to wsgi_wrapper.PlacementWsgify

This is a newly introduced wrapper class around webob.dec.wsgify
to set json formatter in case of webob exceptions.

Change-Id: I18b062bf4e53a2ea4c2f7e4d8b205f8fe8a7513d
Partial-Bug: #1635182
2017-02-13 16:56:51 +00:00
Sylvain Bauza a5dc7f1a11 Expose a REST API for a specific list of RPs
Now that we merged the object method for getting the list of ResourceProviders
based on a specific amount request, we need to expose that method into a REST
API call so that the scheduler client could be calling it.

Co-Authored-By: Jay Pipes <jaypipes@gmail.com>

Change-Id: Ia8b534d20c064eb3a767f95ca22814925acfaa77
Implements: blueprint resource-providers-get-by-request
2017-01-13 17:57:29 -05:00
bhagyashris 9bbce28c56 Return 400 when name is more than 200 characters
APIs listed below are returning 500 error if you pass name more than
200 characters.
1. create resource_providers
2. update resource_providers

Added maxLength check in schema to ensure name should not be more than
200 characters.

Closes-Bug: #1642485
Change-Id: Ia22ae86702dabe18bc577b17164a81c363da6d41
2016-12-07 12:29:10 +05:30
Pushkar Umaranikar f87af91d90 Placement api: 404 response do not indicate what was not found
Add informative messages to the 404 error response from
nova/objects/resource_provider.py

For the method _update_inventory_in_db() from resource_provider.py,
gabbi test validation is not added since it is not being used and
from the placement API you create an inventory but then call
set_inventory or add_inventory on a resource_provider.

Similar change, i.e. adding informative messages to 404 error can be
done for api tests from inventory.yaml and usage.yaml

It includes adding informative logging in code for 404 exceptions and
adding gabbi tests for those.
This can be done in follow up patch.

Change-Id: If53f84ac5f7521e9926b97bdcce3cf77ec5b4ffd
Closes-Bug: #1634115
2016-11-09 21:11:07 +00:00
Roman Podoliaka b838f2225c placement: add filtering by attrs to resource_providers
Make it possible to GET /resource_providers passing filters as query
params to get only a subset of resource providers.

Partially-Implements: blueprint generic-resource-pools

Change-Id: I2c905d55ac20fd1c5a285b1e460cdd31dff8c7dd
2016-08-16 13:27:07 +00:00
Chris Dent 125cfc97fb Add support for resource_providers urls
GET a list of /resource_providers
POST to /resource_providers for a new one
GET, DELETE, PUT a single /resource_providers/{uuid}

Uses jsonschema to validate input when creating or updating.
A 'uuid' FormatChecker is added in the util module to verify
the format of uuids in request bodies. Note the lengthy comment
adjacent explaining its magic.

The olso_context RequestContext is subclassed to allow it to
be an engine_facade transaction_context_provider. A database
fixture is added to the gabbi tests to create and dispose the
placement database per each gabbi testsuite (i.e. each YAML
file).

A resource_provider_url is added to util.py: given a
resource_provider object, give the url, accounting for any
prefix in SCRIPT_NAME

With the addition of working endpoints, basic_http.yaml
is extended with more complete testing of basic http behaviors.

Tests for resource_providers are in resource-provider.yaml.

Change-Id: I799d839101af78b4d89aca175f647efc2b56c401
Partially-Implements: blueprint generic-resource-pools
2016-08-16 13:24:44 +00:00