Commit Graph

10 Commits

Author SHA1 Message Date
Chris Dent 787bb33606 Use external placement in functional tests
Adjust the fixtures used by the functional tests so they
use placement database and web fixtures defined by placement
code. To avoid making redundant changes, the solely placement-
related unit and functional tests are removed, but the placement
code itself is not (yet).

openstack-placement is required by the functional tests. It is not
added to test-requirements as we do not want unit tests to depend
on placement in any way, and we enforce this by not having placement
in the test env.

The concept of tox-siblings is used to ensure that the
placement requirement will be satisfied correctly if there is a
depends-on. To make this happen, the functional jobs defined in
.zuul.yaml are updated to require openstack/placement.

tox.ini has to be updated to use a envdir that is the same
name as job. Otherwise the tox siblings role in ansible cannot work.

The handling of the placement fixtures is moved out of nova/test.py
into the functional tests that actually use it because we do not
want unit tests (which get the base test class out of test.py) to
have anything to do with placement. This requires adjusting some
test files to use absolute import.

Similarly, a test of the comparison function for the api samples tests
is moved into functional, because it depends on placement functionality,

TestUpgradeCheckResourceProviders in unit.cmd.test_status is moved into
a new test file: nova/tests/functional/test_nova_status.py. This is done
because it requires the PlacementFixture, which is only available to
functional tests. A MonkeyPatch is required in the test to make sure that
the right context managers are used at the right time in the command
itself (otherwise some tables do no exist). In the test itself, to avoid
speaking directly to the placement database, which would require
manipulating the RequestContext objects, resource providers are now
created over the API.

Co-Authored-By: Balazs Gibizer <balazs.gibizer@ericsson.com>
Change-Id: Idaed39629095f86d24a54334c699a26c218c6593
2018-12-12 18:46:49 +00:00
Chris Dent dd13c2784b Add placement.concurrent_udpate to generation pre-checks
We have two kinds of generation conflicts that happen (for both
resource providers and consumers). In
I5a164a0ee298846231113ad7b62b82a3e8f1f409 error codes are added for
conflicts that happen when incrementing the generation in the database.s

In this change the codes are added for those situations where in code we
are checking an incoming generation against the stored generation.

Since there are incoming genearations in this case, we can add tests
so this does, via gabbi.

Change-Id: I5ed005068c1c19194ed67556376604ae90727412
2018-07-11 15:56:12 +01:00
Takashi NATSUME c23f135b11 [placement] Add functional tests for traits API
Add functional tests for traits API in the following cases.

* Invalid 'resource_provider_generation' in
  PUT /resource_providers/{uuid}/traits
* Invalid 'traits' in
  PUT /resource_providers/{uuid}/traits
* Additional properties in
  PUT /resource_providers/{uuid}/traits
* Earlier microverion (1.5)

Fix a response string to check in the following test.

* Missing 'resource_provider_generation' in
  PUT /resource_providers/{uuid}/traits

Change-Id: I4db0c8a5c55f7fcdebd5fcb04273d922727a4521
2018-02-19 00:41:56 +00:00
Chris Dent 83030804cc [placement] Add cache headers to placement api requests
In relevant requests to the placement API add last-modified
and cache-control headers.

According the HTTP 1.1 RFC headers last-modified headers SHOULD always
be sent and should have a tie to the real last modified time. If we do
send them, we need Cache-Control headers to prevent inadvertent caching
of resources.

This change adds a microversion 1.15 which adds the headers to GET
requests and some PUT or POST requests.

Despite what it says 'no-cache' means "check to see if the version you
have is still valid as far as the server is concerned". Since our server
doesn't currently validate conditional requests and will always return an
entity, it ends up meaning "don't cache" (which is what we want).

The main steps in the patch are:

* To both the get single entity and get collection handlers add
  response.cache_control = 'no-cache'
* For single entity add response.last_modified = obj.updated_at or
  obj.created_at
* For collections, discover the max modified time when traversing the
  list of objects to create the serialized JSON output. In most of
  those loops an optimization is done where we only check for
  last-modified information if we have a high enough microversion such
  that the information will be used. This is not done when listing
  inventories because the expectation is that no single resource
  provider will ever have a huge number of inventory records.
* Both of the prior steps are assisted by a new util method:
  pick_last_modfied.

Where a time cannot be determined the current time is used.

In typical placement framework fashion this has been done in a very
explicit way, as it makes what the handler is doing very visible, even
though it results in a bit of boilerplate.

For those requests that are created from multiple objects or by doing
calculations, such as usages and aggregate associations, the current time
is used.

The handler for PUT /traits is modified a bit more extensively than some
of the others: This is because the method can either create or validate
the existence of the trait. In the case where the trait already exists,
we need to get it from the DB to get its created_at time. We only do
this if the microversion is high enough (at least 1.15) to warrant
needing the info.

Because these changes add new headers (even though they don't do
anything) a new microversion, 1.15, is added.

Partial-Bug: #1632852
Partially-Implements: bp placement-cache-headers

Change-Id: I727d4c77aaa31f0ef31c8af22c2d46cad8ab8b8e
2017-12-12 15:51:58 +00:00
Takashi NATSUME e51a14bc93 [placement] Fix an error message in API validation
Fix the error message in validation of the following API
when the name exceeds max character length(255).

* PUT /traits/{name}

Change-Id: I2bbbadd7842bf4652d3c32ae3e5c74e2e159990b
Closes-Bug: #1735343
2017-12-05 11:08:48 +09:00
Chris Dent f269023c49 [placement] set accept to application/json if accept not set
If an incoming request to placement does not set the accept header
force it to 'application/json'. This not only ensures that error
responses are in JSON but also that errors early in the middleware
stack do not cause a key error (see the #1724065 bug).

This fix is done by checking headers early in the middleware stack
using the requestlog middleware as a convenient place to do the
check. This overloads requestlog's purpose, but avoids adding yet
more middleware (doing so has some small impact per request).

Fixing bug 1724065 fixes a 500. Fixing bug 1674694 changes (for some
requests) the content type of the bodies of 400-499 responses. This
creates a bit of quandry for microversion handling. If a microversion
is considered required here then it's not clear the global fix is
worth doing and the 500 fix should be limited the microversion
middleware. The intent all along has been that responses should
strive to align with the API-WG errors guideline [1], which assumes
application/json.

[1] http://specs.openstack.org/openstack/api-wg/guidelines/errors.html

Change-Id: Ice27c7080fc2df097cb387f7438c0aaf32b4c63d
Closes-Bug: #1724065
Closes-Bug: #1674694
2017-11-07 15:25:31 +11:00
Chris Dent f974e3c356 [placement] avoid case issues microversions in gabbits
If a gabbi file sets a default microversion by setting a header
'OpenStack-API-Version' with a value like 'placement latest' and then
later overrides that in an individual test with a header of
'openstack-api-version' the difference in case can lead to failure.

In the best case the failure is consistent.

In the worst case it can sometimes work, because the header shows up
twice in the request, and the last header wins, order in the headers
and the resulting list dependent on the vagrancies of python ordering.

The solution is to always use the same case, so this change updates
all use to be lowercase, to establish a precedent that future people
will be able to use as an example.

Note that gabbi is case sensitive here in part because of the
implementation but also because it provides the control and possibility
to test exactly this problem.

Change-Id: I1e89e231cf0d46d211d360cda091b33520f85027
Closes-Bug: #1728934
2017-10-31 12:54:17 +00:00
Chris Dent 4b73b3bca9 [placement] cover deleting standard trait
Add a gabbi test that confirms that attepting to delete a standard
trait is a Bad Request (400).

Change-Id: I6360f6304e2c7ce5cfa889a7a1bc90363701034a
2017-07-15 18:05:45 +01:00
Chris Dent e013d1cabe Sync os-traits to Traits database table
When a new version of the os-traits library is released, the Traits
table in the api database needs to be updated to reflect those new
traits. This change does that, once, the first time either
Trait.get_by_name or TraitList.get_all is called in any process that is
using those objects.

This is an alternative to I729e84ea0ff8f333a156b24b15fe4d368209d015.
The major difference here is that there is no trait cache so the
surface area of the change is much smaller.

The list traits test in gabbit/traits.yaml has been changed to not
assert the length of traits returned: this can now change with each
release of the os-traits library.

Depends-on: I7a3f4bb8501fc3edad43e1aae5cb6b9ef1c0b00d
Co-Authored-By: Jay Pipes <jaypipes@gmail.com>
Change-Id: Ia92a4fd20b8991c6f4b34e3546cfb22aa5ed78aa
2017-06-06 10:31:05 +00:00
He Jie Xu 9c975b6bd8 placement: Add Traits API to placement service
This patch adds support for a REST API for CRUD operations on traits.

  GET /traits: Returns all resource classes.
  PUT /traits/{name}: To insert a single custom trait.
  GET /traits/{name}: To check if a trait name exists.
  DELETE /traits/{name}: To delete the specified trait.
  GET /resource_providers/{uuid}/traits: a list of traits associated
  with a specific resource provider
  PUT /resource_providers/{uuid}/traits: Set all the traits for a
  specific resource provider
  DELETE /resource_providers/{uuid}/traits: Remove any existing trait
  associations for a specific resource provider

Partial implement blueprint resource-provider-traits

Change-Id: Ia027895cbb4f1c71fd9470d8f9281d2bebb6d8a2
2017-04-11 10:31:37 +08:00