Add keyword option to get_version_data() to allow passing
of the version header so that we can get the microversions.
Specifically, this is so that we can re-use this function
in barbican, which recently implemented microversions, but
doesn't return them by default, for backward compatibility
with old clients.
Change-Id: I909750381a559f9dc61650c9f98c88d4481012b7
Check if the last url segment matches the project id.
Previously the check only confirmed whether the last url segment
endswith the project id which could cause problems with spurious
matches of some legacy integer project ids.
Closes-Bug: 1968793
Change-Id: I7c6c22e41bde2a73508635b7e964c58a02c12146
When a non-keystone plugin is used together with an unversioned endpoint,
we give up on discovery before figuring out both major version and
the correct endpoint. This is because get_endpoint_data is called with
discover_versions=False, so discovery assumes we have all information
already. It may be an issue in discovery itself, but I'm afraid to
touch that code. Instead, if get_endpoint_data returns no API version
with discover_versions=False, try with discover_versions=True, which
matches what the identity plugins do.
Also increase the unit test coverage.
Change-Id: Ie623931b150748d7759cf276e0023a2f06a8d4db
We expect endpoint_override, but these plugins won't necessary
have it, they have endpoint instead.
Co-Authored-By: Dmitry Tantsur <dtantsur@protonmail.com>
Change-Id: Iead4b95c1f5b8d84cec705da32f41049e2eea641
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I07d61e1a8f18d65acdf86cdd61f7d9e28157f1d7
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Some services, like Nova, default to requiring auth for their
versioned discovery documents. This means strict discovery
does not work on them, because discovery as it is now defaults
to not sending auth. Just changing the default would be a behavior
change resulting in sending unneeded data with *every* request.
Instead, respond to Unauthorized exceptions by retrying the request
with auth token. This way discovery will work for services that
are otherwise blocking unauthenticated access, and will get more
efficient over time as those services improve.
Depends-On: https://review.opendev.org/#/c/685999
Change-Id: I8a33e8a05bed0f18e4e42431f6d16b8a6a5270ef
On a cloud that has inaccessible version discovery documents AND uses
integer project ids, the discovery fallback logic can fail because the
project id parses as a (very large) version.
Check to see that the url segment in the fallback code begins with a v,
so that we're only attempting to parse versions from actual candidate
segments.
Closes-Bug: #1806109
Change-Id: Id90b3b9e4852494a4678b0a9bb67362babdc971c
For (unclear) historical reasons the root single version endpoint also
contains "id" and "links" fields. This makes the current workaround
for old-style endpoints take priority over the correct algorithm.
This change reorders the code, so that if "version" is present, it
always take priority over the workaround.
The unit tests are updated to be closer to real output from ironic.
Change-Id: I743b954c6c5b2f986c213acb6ec6af7e08c9f5f8
It's possible in get_api_major_version that the endpoint in question is
not found at all. In that case, we are documented to return None, but
what we do instead is throw an exception trying to get data off of the
None object.
Change-Id: I06ad497854f4e95a1a2a4a93241b244fc476b139
The ironic payload looks like:
{'id': 'v1',
'links': [{"href": "https://bare-metal.example.com/v1/",
"rel": "self"}]}
This does not have version info in it, nor min/max ranges for
microversion discovery. We can't really get any useful information from
this document, but we can at least not fail when trying to deal with it.
This should then be upwards-compatible with ironic adding version discovery
information to the document that is returned.
Change-Id: I47e0f9b295c24ef168f4a033faf573b953025d4c
If the url netloc in the catalog and service's response
are not the same, we should choose the catalog's and
add the version info to it if needed.
Change-Id: If78d368bd505156a5416bb9cbfaf988204925c79
Closes-bug: #1733052
We expose version_to_string and version_match but not version_between.
openstacksdk would really like to use version_between too for matching
microversion suitability. Turn it in to a public function.
Change-Id: I710f9e1441f4caeb9bd9830f9d4a3398a71249ec
A recent change pushed API status definitions into a class:
I93ee971125bc0c7a497e1fb839df38ebd38340e1
We can reuse this in the tests instead of duplicating the same
string across assertions.
Change-Id: I07a602a05f896d7cc70120bd89424e3c553baf9f
The last piece of service type alias support is to handle volumev2,
volumev3, workflowv2, workflowv3 and friends.
Although it's an annoying scenario, luckily legacy code that uses them
has a clear meaning. volumev2, version='3' is just legit not a thing.
Needed-By: https://review.openstack.org/564494
Change-Id: Iec09bcb16d8e9b09e09bf12d03c2a55e679ad70c
For better readability, make normalize_status a class that has constants
for each of the status values.
While in there, add a test for unknown status values.
Co-Authored-By: Eric Fried <efried@us.ibm.com>
Change-Id: I93ee971125bc0c7a497e1fb839df38ebd38340e1
We collect the status when doing discovery, but it's not exposed
anywhere to the use when they look at the EndpointData for an endpoint.
Add a function to normalize the statuses and then add normalized status
to the EndpointData object.
Change-Id: Icf855d7892335b093c1083cd0106946d8911010d
Add the flake8-import-order library to our test requirements so that we
can avoid these PEP8 violations and maintain consistent import ordering.
Also fix our violations.
This library is in requirements but is blacklisted from being
automatically updated since it is never shipped. For now, don't bother
to pin it.
Change-Id: I4e788292b98b7f2f835cc80081763b2d249fe43e
It is useful to be able to dump the contents of an EndpointData for
debugging purposes. This change adds a __str__ method that joins up all
the public attributes/properties.
Change-Id: Ib8985f0fa48a613ab8fca7faffbdf60c19c7cd22
The new 'none' auth plugin and the old 'admin_token' plugin
are subclasses of BaseAuthPluign, not BaseIdentityPlugin.
That means if someone does:
s = session.Session(noauth.NoAuth())
a = adapter.Adapter(s, endpoint_override='https://example.com')
to get an Adapter on an endpoint using the none plugin, then does
either:
a.get_api_major_version()
or:
a.get_endpoint_data()
it will fail because the none plugin doesn't have those methods.
There is, however, nothing about those methods that necessarily needs
authentication. That is, they can work just fine in contexts without
a keystone token or without authentication of any sort.
Ironic/Bifrost is specifically a usecase here, as standalone Ironic
wants to use the 'none' plugin, but consuming the API still needs to
get microversion info from the given endpoint.
Add methods to BaseAuthPlugin that take less arguments since the ones
about finding services in catalogs make zero sense in none/admin_token
context.
Change-Id: Id9bd19cca68206fc64d23b0eaa95aa3e5b01b676
Lance noted [1] that the helper methods' arguments are the reverse of
the (expected, actual) to which consumers of the unittest framework's
assertions are accustomed. This change set reverses them for
readability.
[1] https://review.openstack.org/#/c/483604/11/keystoneauth1/tests/unit/test_discovery.py@372
Change-Id: I3542f34211559420c09df87a0c71c2c3ce642d67
Partly in response to [1], and partly for the sake of making its API
consistent and understandable, discover._version_between now allows
un-normalized input, and responds sensibly to unspecified upper & lower
bounds.
Previously the method treated None as follows:
min_version max_version behavior
None None Always False
not None None ValueError
not None not None Compare min to candidate
None not None Compare max to candidate
That's whack, yo. Took me like ten minutes just to come up with that
chart.
This change allows us to explain clearly what happens when upper or
lower bounds are None: it translates naturally to "there is no bound".
Given that, the first line of the chart (None/None) now always returns
True, which is a behavior change the callers need to be updated to
expect. This is an acceptable sacrifice for making this method sane.
[1] https://review.openstack.org/#/c/483604/9/keystoneauth1/discover.py@306
Change-Id: I09b6cf692c8bfa290b73c8d7498bc12a5e91d690
There's some ongoing concern about exactly how discover.version_between
should work. Since it's new, make it private initially in the release
to give us time to work it out.
Change-Id: I349423f1b3da74b139f5fec838597c9a824ba571
We're discouraging the use of the ambiguous and difficult-to-understand
'version' parameter in new discovery methods, instead encouraging the
use of min_version and max_version.
In order to make it possible to get the same functionality, though, we
need a way to say the same thing as version="M.m", which actually means,
"min version is M.m, and max version is the latest within major version
M".
Introducing 'latest' syntax, which can be used in various ways,
including:
min_version='2.3', max_version='2.latest'
...which is equivalent to the old school version='2.3'
Change-Id: Ife842333e25c33e54bbae4c1adb101014cb8e8db
The API-WG just approved the spec for version discovery documents to
optionally provide "next_min_version" and "not_before" information.
http://specs.openstack.org/openstack/api-wg/guidelines/microversion_specification.html#version-discovery
The intended use of these is to communicate that at a point in the
future the service plans to raise the minimum microversion. It can't say
when that will happen, as a service does not know when deployers will
decide to upgrade their services. But it can communicate the earliest
date it's possible to happen, which would be the first date the service
itself would raise the minimum.
This can be used to emit warnings to users who are using a microversion
less than the next_min_version and to tell them how long they have to
think about it.
Currently keystoneauth will not consume these for that purpose. This
patch is merely about collecting the information from the discovery
document if it is there so a consumer can take action on it if they
wish.
Change-Id: Ibc404ef55eeae721a0d1d16e4e3e51ad77b5a75c
Make Discover.version_data a little more lenient in the JSON data it
accepts with respect to "max_version" and "version". Previously, if the
"max_version" field was present but empty, the "version" field would be
ignored, even if populated with a proper version number. This goes
against the intent of the API. This change causes version_data to
ignore a blank "max_version" if "version" is populated.
This change also introduces a unit test harness for the various
permutations of version_data translating input version data to
min/max_microversion, including cases for the above.
Change-Id: I1cb4ebdd57a6eadc1a16efc0fc1ceeda640f26c2
Closes-Bug: #1703438
Remove some unused variables.
Remove a couple of unused parameters from private method
EndpointData._run_discovery.
Fix some formatting issues to reclaim a line or two, or because my IDE
flags them.
Change-Id: Idcc1f1e4dc6068125ea30ce0ab482a99a9e0b1cc
Fix an edge case where discover.normalize_version_number could return a
one-member tuple rather than the expected >=2-member tuple.
Fix up the docstring for the same, including the above behavior.
Change-Id: Ibe54da05705846e47063f8fc639b31df773bed9d
Closes-Bug: #1703414
EndpointData._run_discovery thought it was caching already-tried URLs,
but it was really caching each letter of each URL, so the cache wasn't
working:
tried = set()
...
if vers_url in tried:
continue
tried.update(vers_url) # Adds each char of `vers_url` to `tried`
Changed that last line to:
tried.add(vers_url)
Added a unit test for same.
Change-Id: I894636d846de4a3b63000f9e2f79f378134c1de8
Closes-Bug: #1703447
Just wanting "latest" isn't the full picture. A client could support,
say, v1 and v2 of an API but not v3 and would like to find an
appropriate matching endpoint.
Add two new arguments, min_version and max_version, rather than
repurpose the version argument.
This changes the behavior of versioned_data_for and versioned_url_for in
the case where version=None. Before that would return None, now it
returns the information about the endpoint that was in the catalog.
The booleans in this are a bit hard to read, as are the fun times with
latest and things being or not being defined. It's time to make the
versions into objects, but we'll do that as a followup.
Change-Id: I8ba948a712002775098b0a86c70f05e0c68250f5
If a user does not specify a version, that means they want whatever is
in the catalog. However, they may still want discovery to be run for
things like microversion information. The new parameter
"discover_versions", if set to True, will run discovery even with no
version parameter. The inverse of this is "skip_discovery" which will
tell keystoneauth to not run discovery even if a version has been given.
Note: This adds some parameters to some methods that get removed by
change I54053336edf1b3c2bd35a77dbd78f56388b8e806 so we should not
actually land this one until we're ready to land the stack up to and
including that one.
Note: This adds two new methods that will have behavior changes in
patch I8ba948a712002775098b0a86c70f05e0c68250f5.
Change-Id: I897c39743089c5994b51336a4ad44eebed33ec35
From the API-WG spec, there are two common patterns for service URLs
that can be interpreted. Trailing project_id and a string that starts
with v. If the project_id is in the URL, it needs to be removed before
discovery can happen, but it needs to be put back on to the url found
via discovery. If the endpointin the catalog has a version, and it
matches the version we're asking for, then we don't need to go hunting
for the unversioned doc.
Also, in the EndpointData we're collecting, we want to grab microversion
info, since we're already there in the discovery doc.
There is one behavior change that can be seen in the tests. If the
attempt at an unversioned discovery endpoint fails, we fall back to the
url from the catalog ... but we attempt to get a discovery document from
it because we need the metadata for microversions. The catalog URL should be
returned as the endpoint even if the second discovery call attempt
succeeds, so the user-facing interface is the same - there will just be,
in some cases, an additional URL fetch behind the scenes.
Change-Id: I2a036d65e4f7dba6f50daf6a0ce4589ee59ae95f
There are a number of fixtures still present in test_discovery for
testing that version discovery works with the different discovery pages
found in glance, cinder and others.
The method names for a lot of this stuff was slightly changed from
keystoneclient to keystoneauth so it looks like these tests were dropped
in the port rather than fixed.
Re-add these tests to ensure other discovery formats are supported.
Change-Id: I311e94437545544917b027de69a89ede9d96f555
Conver from the keystoneauth namespace to keystoneauth1. This is to
ensure that is is possible to install all versions of keystoneauth
side-by-side.
Change-Id: Ibbaf11525980c8edb5968d8b8ee19c55094e77d8
Grzegorz Grasza