Adjust the fixtures used by the functional tests so they
use placement database and web fixtures defined by placement
code. To avoid making redundant changes, the solely placement-
related unit and functional tests are removed, but the placement
code itself is not (yet).
openstack-placement is required by the functional tests. It is not
added to test-requirements as we do not want unit tests to depend
on placement in any way, and we enforce this by not having placement
in the test env.
The concept of tox-siblings is used to ensure that the
placement requirement will be satisfied correctly if there is a
depends-on. To make this happen, the functional jobs defined in
.zuul.yaml are updated to require openstack/placement.
tox.ini has to be updated to use a envdir that is the same
name as job. Otherwise the tox siblings role in ansible cannot work.
The handling of the placement fixtures is moved out of nova/test.py
into the functional tests that actually use it because we do not
want unit tests (which get the base test class out of test.py) to
have anything to do with placement. This requires adjusting some
test files to use absolute import.
Similarly, a test of the comparison function for the api samples tests
is moved into functional, because it depends on placement functionality,
TestUpgradeCheckResourceProviders in unit.cmd.test_status is moved into
a new test file: nova/tests/functional/test_nova_status.py. This is done
because it requires the PlacementFixture, which is only available to
functional tests. A MonkeyPatch is required in the test to make sure that
the right context managers are used at the right time in the command
itself (otherwise some tables do no exist). In the test itself, to avoid
speaking directly to the placement database, which would require
manipulating the RequestContext objects, resource providers are now
created over the API.
Co-Authored-By: Balazs Gibizer <balazs.gibizer@ericsson.com>
Change-Id: Idaed39629095f86d24a54334c699a26c218c6593
These tests are very straightforward and do not require any of the setup
and fixtures that nova.test.NoDBTestCase supplies. Also, in the future
of placement extraction we don't want the dependecy.
Note that when placement is extracted, we'll need to create a local
uuidsentinel. That's easiest to do at the time of extraction.
Partially implements blueprint placement-extract
Change-Id: I2f859b25694cd4304a94effce07f0ffd648e9d4e
That change extracts much of the microversion handling code from
placement and uses code from the microversion-parse library instead.
The code in microversion-parse was taken from placement and lightly
adjusted to make it more generally useful.
Depends-On: https://review.openstack.org/555332
Change-Id: I5a70ebf2843d2b50dccb44e66d50e1e5105005f4
In relevant requests to the placement API add last-modified
and cache-control headers.
According the HTTP 1.1 RFC headers last-modified headers SHOULD always
be sent and should have a tie to the real last modified time. If we do
send them, we need Cache-Control headers to prevent inadvertent caching
of resources.
This change adds a microversion 1.15 which adds the headers to GET
requests and some PUT or POST requests.
Despite what it says 'no-cache' means "check to see if the version you
have is still valid as far as the server is concerned". Since our server
doesn't currently validate conditional requests and will always return an
entity, it ends up meaning "don't cache" (which is what we want).
The main steps in the patch are:
* To both the get single entity and get collection handlers add
response.cache_control = 'no-cache'
* For single entity add response.last_modified = obj.updated_at or
obj.created_at
* For collections, discover the max modified time when traversing the
list of objects to create the serialized JSON output. In most of
those loops an optimization is done where we only check for
last-modified information if we have a high enough microversion such
that the information will be used. This is not done when listing
inventories because the expectation is that no single resource
provider will ever have a huge number of inventory records.
* Both of the prior steps are assisted by a new util method:
pick_last_modfied.
Where a time cannot be determined the current time is used.
In typical placement framework fashion this has been done in a very
explicit way, as it makes what the handler is doing very visible, even
though it results in a bit of boilerplate.
For those requests that are created from multiple objects or by doing
calculations, such as usages and aggregate associations, the current time
is used.
The handler for PUT /traits is modified a bit more extensively than some
of the others: This is because the method can either create or validate
the existence of the trait. In the case where the trait already exists,
we need to get it from the DB to get its created_at time. We only do
this if the microversion is high enough (at least 1.15) to warrant
needing the info.
Because these changes add new headers (even though they don't do
anything) a new microversion, 1.15, is added.
Partial-Bug: #1632852
Partially-Implements: bp placement-cache-headers
Change-Id: I727d4c77aaa31f0ef31c8af22c2d46cad8ab8b8e
Treat HTTPMethodNotAllowed as a WSGI application rather than exception
so that it is not treated as an uncaught exception and logged as an ERROR
in the PlacementHandler before being caught in the FaultWrapper middleware.
Bad method detection is happening outside the context of WebOb wsgify
handling (which automatically catches Webob exceptions and transforms
them into appropriate responses) so we need to do the transformation
ourself.
This will help to avoid spurious noise in the logs. See the bug report
for more detail.
Change-Id: I6de7c2bffb08f370fcfbd86070c94263ee202f93
Closes-Bug: #1708167
The CONTENT_LENGTH environ can be present, but empty, which returns
None, and causes a ValueError when attempting to use .int().
This patch removes the setting of CONTENT_LENGTH to an integer, but
instead ensures that if CONTENT_LENGTH is not empty it is an integer, to
prevent a situation where a bogus "CONTENT_LENGTH" header is specified.
Additionally, as the CONTENT_TYPE environ can similarly be present but
empty, we should .get() it in a similar fashion to ensure it isn't
present but None when CONTENT_LENGTH is specified.
Change-Id: I66b6f9afbea8bf037997a59ba0b976f83c9825fb
Closes-Bug: #1681843
mod-wsgi checks that response header values are what's described
as "native strings". This means whatever `str` is in either
python 2 or 3, but never `unicode`. When they are not mod-wsgi
will 500. For the most part this is taken care of by webob, but in
the case of the 405 handling, the webob response is not being
fully massaged.
mod-wsgi is doing this because it supposed to. Python WSGI server
gateways have different expectations of headers depending on whether
the Python is 2 or 3. See
https://www.python.org/dev/peps/pep-3333/#a-note-on-string-types
In addition to the unit test, the gabbi tests are now using a
version of wsgi-intercept that will raise a TypeError when the
application response headers are not using the correct form. This
check needs to be done in wsgi-intercept rather than the gabbi tests
because both wsgi-intercept and the http client makes requests
transform the headers for their own purposes.
This fix ensures that instead of a 500 the correct 405 response
happens.
Closes-Bug: #1626496
Depends-On: I3b8aabda929fe39b60e645abb6fabb9769554829
Change-Id: Ifa436e11e79adc2e159b4c5e7d3623d9a792b5f7
When mounted under a prefix (such as /placement) the service was
only returning the home document at /placement/ not /placement. In
the context of having a prefix, we'd generally like the latter to
work and for '/' to work when there is no prefix. This allows both.
Note that this doesn't make /placement/resource_providers/ work, and
we don't want that.
Change-Id: I0ac92bf9982227d5f4915175182e5230aeb039b4
Closes-Bug: #1626490
Prior to this change, a 404 caused by a not possible route being
requested (such as /placement/hello) would result in a proper 404
response but also a log message about an uncaught exception. This
made it appear that a 404 response is unexpected and something that
needs exceptional attention. It doesn't. This change makes sure
that 404s propagate up the stack in a clean way and are logged only
by the existing INFO log used for all requests.
Change-Id: I21f9411bd5154eddc582d5f38891be747de874b3
Closes-Bug: #1626489
The placement API will initially host resource provider
information and then will grow to provide a full placement
service. Eventually it will be fully extracted from Nova.
To facilitate that extraction, this service is being developed
with few ties to existing nova.wsgi structures. Instead it
uses relatively plain WSGI apps that are:
* uses the Routes library with declarative mapping defined in
ROUTE_DECLARATIONS
* basic wsgi apps, with webob and the request class, for each handler
define as functions
* does not use a paste-ini file to compose middleware, instead code
which minimally inspects the oslo config to know how to adjust
middleware (in the initial case choosing an auth_strategy)
Many of these "features" will be demonstrated in commits that
follow this one that enable specific behaviors for resource
providers and their inventories.
In order to ensure that this change can be merged in an atomic
fashion it includes microversion support that was previously in its
own commit.
The microversion_parse library is used in a WSGI middleware
to parse incoming microversion headers and place the
determined value into the WSGI environment at the
'placement.microversion' key. Response headers are adjusted to
add the required outgoing headers as described in:
http://specs.openstack.org/openstack/api-wg/guidelines/microversion_specification.html
If a client requests an invalid microversion, they will receive
a 400 response. If the microversion is of a valid form but not
available, they will received a 406 response. The format of that
response is dependent on the accept header of the request. If it
is JSON, the min and max available microversions are presented.
A request to '/' will return version discovery information.
Thus far nothing else is done with the microversion information.
It is there for when we need it in the future. For now everything
is version 1.0.
The groundwork for using gabbi to test the API is in place in
nova/tests/functional/api/openstack/placement. The gabbi tests
are included in the functional target. To run just the placement
tests it is possible to run 'tox -efunctional placement'.
Change-Id: Icb9c8f7a1fa8a9eac66c2d72f4b7e4efd4e1944f
Partially-Implements: blueprint generic-resource-pools
Chris Dent