Adjust the fixtures used by the functional tests so they
use placement database and web fixtures defined by placement
code. To avoid making redundant changes, the solely placement-
related unit and functional tests are removed, but the placement
code itself is not (yet).
openstack-placement is required by the functional tests. It is not
added to test-requirements as we do not want unit tests to depend
on placement in any way, and we enforce this by not having placement
in the test env.
The concept of tox-siblings is used to ensure that the
placement requirement will be satisfied correctly if there is a
depends-on. To make this happen, the functional jobs defined in
.zuul.yaml are updated to require openstack/placement.
tox.ini has to be updated to use a envdir that is the same
name as job. Otherwise the tox siblings role in ansible cannot work.
The handling of the placement fixtures is moved out of nova/test.py
into the functional tests that actually use it because we do not
want unit tests (which get the base test class out of test.py) to
have anything to do with placement. This requires adjusting some
test files to use absolute import.
Similarly, a test of the comparison function for the api samples tests
is moved into functional, because it depends on placement functionality,
TestUpgradeCheckResourceProviders in unit.cmd.test_status is moved into
a new test file: nova/tests/functional/test_nova_status.py. This is done
because it requires the PlacementFixture, which is only available to
functional tests. A MonkeyPatch is required in the test to make sure that
the right context managers are used at the right time in the command
itself (otherwise some tables do no exist). In the test itself, to avoid
speaking directly to the placement database, which would require
manipulating the RequestContext objects, resource providers are now
created over the API.
Co-Authored-By: Balazs Gibizer <balazs.gibizer@ericsson.com>
Change-Id: Idaed39629095f86d24a54334c699a26c218c6593
`GET /resource_provider/{uuid}/allocations` API didn't
return all the allocations made by multiple users.
This was because the placement wrongly used project table
for user table. This patch fixes it with the test case.
Change-Id: I7c808dec5de1204ced0d1f1b31d8398de8c51679
Closes-Bug: #1785382
`GET /resource_provider/{uuid}/allocations` API currently doesn't
return all the allocations made by multiple users.
This patch adds a test to describe this bug. The fix for this
is coming in a follow up.
Change-Id: I2b01e27922f11bef2defcb01fe415692de1578ea
Partial-Bug: #1785382
There's a special case of changing inventories that we didn't have
gabbit coverage for: For an existing resource provider, we allow its
inventory to be changed to have a lower total than existing allocations
(See commit 09627f2a0b). There was no gabbi coverage for that case.
This adds it.
Change-Id: I34e104a766d4578f83a455b4b900ea50c9a6ec0f
Change I9a833aa35d474caa35e640bbad6c436a3b16ac5e added a custom error
code framework to the placement API and, as an example of a specific
error code, created INVENTORY_INUSE ('placement.inventory.inuse') for
PUT /resource_providers/{uuid}/inventories. We have essentially the
same error condition for DELETE /resource_providers/{uuid}/inventories,
so this change set adds the same error code there.
Change-Id: I52715205e607cef5deee1351099725885b282c08
The API-sig has a guideline[1] for including error codes in error
responses to help distinguish errors with the same status code
from one another. This change provides a simplest-thing-that-could-
possibly-work solution to make that go.
This solution comes about after a few different constraints and attempts:
* We would prefer to go on using the existing webob.exc exceptions, not
make subclasses.
* We already have a special wrapper around our wsgi apps to deal with
setting the json_error_formatter.
* Though webob allows custom Request and Response objects, it uses the
default Response object as the parent of the HTTP exceptions.
* The Response object accepts kwargs, but only if they can be associated
with known attributes on the class. Since we can't subclass...
* The json_error_formatter method is not passed the raw exception, but
it does get the current WSGI environ
* The webob.exc classes take a 'comment' kwarg that is not used, but
is also not passed to the json_error_formatter.
Therefore, when we raise an exception, we can set 'comment' to a code
and then assign that comment to a well known field in the environ and if
that environ is set in json_error_formatter, we can set 'code' in the
output.
This is done in a new microversion, 1.23. Every response gets a default
code 'placement.undefined_code' from 1.23 on. Future development will
add specific codes where required. This change adds a stub code for
inventory in use when doing a PUT to .../inventories but the name
may need improvement.
[1] http://specs.openstack.org/openstack/api-wg/guidelines/errors.html
Implements blueprint placement-api-error-handling
Change-Id: I9a833aa35d474caa35e640bbad6c436a3b16ac5e
In relevant requests to the placement API add last-modified
and cache-control headers.
According the HTTP 1.1 RFC headers last-modified headers SHOULD always
be sent and should have a tie to the real last modified time. If we do
send them, we need Cache-Control headers to prevent inadvertent caching
of resources.
This change adds a microversion 1.15 which adds the headers to GET
requests and some PUT or POST requests.
Despite what it says 'no-cache' means "check to see if the version you
have is still valid as far as the server is concerned". Since our server
doesn't currently validate conditional requests and will always return an
entity, it ends up meaning "don't cache" (which is what we want).
The main steps in the patch are:
* To both the get single entity and get collection handlers add
response.cache_control = 'no-cache'
* For single entity add response.last_modified = obj.updated_at or
obj.created_at
* For collections, discover the max modified time when traversing the
list of objects to create the serialized JSON output. In most of
those loops an optimization is done where we only check for
last-modified information if we have a high enough microversion such
that the information will be used. This is not done when listing
inventories because the expectation is that no single resource
provider will ever have a huge number of inventory records.
* Both of the prior steps are assisted by a new util method:
pick_last_modfied.
Where a time cannot be determined the current time is used.
In typical placement framework fashion this has been done in a very
explicit way, as it makes what the handler is doing very visible, even
though it results in a bit of boilerplate.
For those requests that are created from multiple objects or by doing
calculations, such as usages and aggregate associations, the current time
is used.
The handler for PUT /traits is modified a bit more extensively than some
of the others: This is because the method can either create or validate
the existence of the trait. In the case where the trait already exists,
we need to get it from the DB to get its created_at time. We only do
this if the microversion is high enough (at least 1.15) to warrant
needing the info.
Because these changes add new headers (even though they don't do
anything) a new microversion, 1.15, is added.
Partial-Bug: #1632852
Partially-Implements: bp placement-cache-headers
Change-Id: I727d4c77aaa31f0ef31c8af22c2d46cad8ab8b8e
In a new microversion, 1.12, include project_id and user_id in the
output of GET /allocations/{consumer_uuid} and add JSON schema
to enable PUT to /allocations/{consumer_uuid} using the same dict-based
format for request body that is used in the GET response. In later
commits a similar format will be used in POST /allocations. This
symmetry is general good form and also will make client code a little
easier.
Since GET /allocation_candiates includes objects which are capable
of being PUT to /allocations/{consumer_uuid}, its response body has
been updated as well, to change the 'allocation_requests' object
to use the dict-based format.
Internally to handlers/allocation.py the same method (_set_allocations)
is used for every microversion. Any previous data structure is
transformed into the dict-ish form. This means that pre-existing tests
(like allocation-bad-class.yaml) continue to exercise the problems it
was made for, but needs to be pinned to an older microversion, rather than
being latest.
Info about these changes is added to placement-api-ref,
rest_api_version_history and a reno.
Change-Id: I49f5680c15413bce27f2abba68b699f3ea95dcdc
Implements: bp symmetric-allocations
Closes-Bug: #1708204
Change I30b51a3235e69f4a2993052f7f9ef853858f2d65 regressed
a check added in I10b22606f704abcb970939fb2cd77f026d4d6322
which is meant to avoid dumping errors in the logs when
the scheduler report client fails to delete compute node
resource provider inventory since some resource class inventory
is still in use. It's all very fragile since the check
in the report client is dependent on a regex for the
error message coming back from the Placement API, but until
we have error codes in the API responses, this is what we
have to deal with.
This change puts the original error message back in the
Placement API and adds notes for awareness.
Change-Id: I9a9af0e3e04c7ea7314b7077a681d6a111de6e47
Closes-Bug: #1708031
This adds GET /usages as part of a new microversion 1.9 of the
placement API. Usages can be queried by project or project/user:
GET /usages?project_id=<project id>
GET /usages?project_id=<project id>&user_id=<user id>
and will be returned as a sum of usages, for example:
200 OK
Content-Type: application/json
{
"usages": {
"VCPU": 2,
"MEMORY_MB": 1024,
"DISK_GB": 50,
...
}
}
A new method UsageList.get_all_by_project_user() has been added
for usage queries.
Part of blueprint placement-project-user
Change-Id: I8b948a4dfe6a50bea053b5dcae8f039229e2e364
Deleting an inventory in use should return HTTPConflict 409 with
a specific error message, not just dump the error from the DB API
into the REST API output.
Change-Id: I30b51a3235e69f4a2993052f7f9ef853858f2d65
This method is no longer used, nor should it be. Instead, the
AllocationList.create_all() method should be used to create allocations
in the DB. Tests have been updated to use create_all() instead, and as
create_all() enforces inventory/allocation relationships, several tests
had to be modified so that they properly accounted for this.
Change-Id: I0bccfee155c871babf507cdd1117b5023d5a1361
This patch adds a new method for deleting all inventories for a
resource provider: DELETE /resource-providers/{uuid}/inventories
Return codes:
204 NoContent on success
404 NotFound if the resource provider does not exist
405 MethodNotAllowed if a microversion is specified that is before
this change (1.5)
409 Conflict if inventory in use or if some other request concurrently
updates this resource provider
Change-Id: I1ecb12c888f873e8330367c8411d5a2ef0458495
Implements: bp delete-inventories-placement-api
Now that we merged the object method for getting the list of ResourceProviders
based on a specific amount request, we need to expose that method into a REST
API call so that the scheduler client could be calling it.
Co-Authored-By: Jay Pipes <jaypipes@gmail.com>
Change-Id: Ia8b534d20c064eb3a767f95ca22814925acfaa77
Implements: blueprint resource-providers-get-by-request
Cover a 409 that should happen when trying to delete a resource
provider for which there are allocations.
Change-Id: Id07c8dbc334aff94fd079de867f8db6256bc6973
GET /resources_providers/{uuid}/usages returns a dictionary of resource
usage by resource class.
Change-Id: I3ad0d16594301635966bd9e9352bc3f1f29dab14
Partially-Implements: blueprint generic-resource-pools
Chris Dent