This is no longer necessary since we only support Python 3.x.
A note is removed from requirements.txt since it's no longer relevant:
pip 20.3+ has a real dependency resolver.
Change-Id: Ie3006813a79fef1f128d388b906e4f1752347fa4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Co-Authored-By: Grzegorz Grasza <xek@redhat.com>
The trailng slash on a pathless url is not meaningful, but we were
treating the url given to the discovery cache as if it were. In some
circumstances, such as an endpoint_override that didn't match the
found discovery document perfectly, a double-request could be made.
Normalize root urls in the caching code so that https://example.com and
https://example.com/ would be the same.
Change-Id: I70a5911cf0f213a7816fe8d58c6cca4702ff71bb
The get_all_version_data method is useful for getting a full listing of
what's going on with version discovery on a cloud. Sometimes though
people just want to see the versions for a specific service. Add a
filter to allow skipping making the version discovery call in the first
place, instead of needing to do that as a post-filtering step.
Change-Id: Ia3ca4be2976d1a5e7914fa8f2adbf7297e8cb1e1
If the url netloc in the catalog and service's response
are not the same, we should choose the catalog's and
add the version info to it if needed.
Change-Id: If78d368bd505156a5416bb9cbfaf988204925c79
Closes-bug: #1733052
Setting up an Adapter with an endpoint_override, calling
get_endpoint_data() on that adapter, and accessing the `url` @property
of the resulting EndpointData now regurgitates the endpoint_override
initially set on the Adapter.
Change-Id: I1033c97a7b0e19a75fa0d3c5682cf18b75918514
Closes-Bug: #1707993
The last piece of service type alias support is to handle volumev2,
volumev3, workflowv2, workflowv3 and friends.
Although it's an annoying scenario, luckily legacy code that uses them
has a clear meaning. volumev2, version='3' is just legit not a thing.
Needed-By: https://review.openstack.org/564494
Change-Id: Iec09bcb16d8e9b09e09bf12d03c2a55e679ad70c
We've got great discovery support, but if someone is wanting to find out
what is available and doesn't otherwise know what they're looking for,
they're out of luck.
Add a method to EndpointData which will return all of the version data
for a given service, and then add a method to the base auth plugin that
will use that method to collect all of the version discovery documents
for every service in the cloud.
This commit adds os-service-types so that the resulting datastructure
can return only official service type keys. A followup patch will also
use os-service-types to allow catalog lookups by service-type alias.
There is a change to the test_identity_common.V2.get_auth_data method
to remove the public and internal urls for keystone from the catalog.
The V3 catalog only has keystone on admin, so this makes them have
equivilent data.
Change-Id: I07243edb939865a5df8b283e7c626874ffd830db
Modify the error message for failed discovery in an attempt to better
help the user fixing the issue.
Include a reference to the actual exception that occured in the
error message. Add SSLError to the list of caught exceptions so that
we can log this case, too.
Add some unit tests to verify the handling of possible exceptions during
version discovery.
Change-Id: I9c26ab35d5515a937e016421e26e844212cb0bb3
Closes-Bug: 1749144
There's an unfortunately common deployment config issue where the
per-service version discovery document doesn't return scheme and netloc
properly. (Especially common with glance, as there was an actually
upstream bug in the version discovery documents until very recently)
Since up until now nobody has actually been doing version discovery
(all of the python-client libs skip it and cheat by appending strings
to URLs locally) it's a pervassive issue with existing clouds.
The workaround is to grab the scheme and netloc from the catalog url,
since the service_url is a resource on the endpoint described by the
catalog_url, and since the catalog_url has to be correct or else nothing
works. Do this in the url normalization so that it persists in the
cache, and so that things like endpoint_override are not affected.
The need for the workaround and the description of it are documented in
the API-SIG spec on consuming version discovery.
Needed-By: I78019717cdee79cab43b0d11e737327aa281fd03
Change-Id: I29102e08998b662db8136bee32217532a316f263
Similar to get_endpoint, which knows it doesn't need full endpoint_data,
if a user just wants to know what major version the discovery process
wound up with, there are cases in which we do not need to fetch
discovery documents. Provide an API call that a user can use when this
is the information they need to avoid them having to play games with
discover_versions settings.
Change-Id: I204a45d1d139a90176bcc2ef8d46decd09b2cd5b
The _get_discovery_url_choices generator works by taking a starting URL,
splitting it on '/', and working through the parts trying to get a
matching discovery document from it. It makes assumptions about what the
URL might look like: it might have a project ID on the end of it, and a
version before that. If the starting URL has a trailing '/', splitting
the URL results in an empty string at the end of the list of parts,
which is then treated as a version. The real version is left on the URL
while the generator assumes it has already trimmed the URL down to an
unversioned endpoint. If that version does not match the version we're
seeking, the resulting discovery document will be mismatched and the
generator will fail to yield the right endpoint.
This patch normalizes the starting URL by removing the trailing '/', if
there is one. This way every part of the split URL will be meaningful.
Closes-bug: #1709658
Change-Id: I28c48f78d6f07804d6ea228f163dd37b0fcfcd58
Some of the available checks are
disabled by default, like:
[H106] Dont put vim configuration in source files
[H203] Use assertIs(Not)None to check for None
Change-Id: Ic6ffe931f6b5fb7a851d5daecad10e30e3f92b1a
With the addition of input ranges for versions it's important for the
user to be able to know what version was discovered. This is already in
the EndpointData model as "api_version" - the value was just never
populated.
In cases where an Endpoint has been discovered and the user has an
EndpointData that now correctly stores the api_version of the discovered
endpoint, and where that user now wants to get microversion metadata,
getting rid of api_version as an input to get_versoined_data made
it awkward to fetch metadata about the already discovered endpoint.
Add a method so that the user can say "please run discovery on this
endpoint to fill in the data, don't look for different endpoints, and
don't make me do the api-version -> min/max transformation myself".
Change-Id: I3031d1eb0967eaa8b911e8e8a06ba722f9e69063
We test appending a trailing slash vs. returning catalog endpoint
unmodified by happenstance in some other tests. Add two specific for
this behavior so that we don't accidentally lose the coverage if we
refactor the other tests.
Change-Id: I9bdfdad9807ffdf447cd814de81d20a2bac442a2
The previous behavior in keystoneauth for constructing an unversioned
endpoint if there's a versioned endpoint in the catalog would result in
a trailing slash. Ensure that we end up with trailing slashses on
endpoints we construct.
On the other hand, do not add trailing slashes to urls we're actually
using. If someone wants a / the url they register with the catalog, they
can put one there.
Change-Id: I2798216b5b93d49fd5d3d865193cc0be6a1f6f0d
Closes-Bug: #1705770
Among the zillion permutations being tested, there was no test case for
discovery URLs lacking a path component (e.g. 'http://foo:123' as
opposed to 'http://foo:123/v2'). A new discovery code path was indexing
the second-to-last part of the path component, resulting in IndexError
when the discovery data contained a pathless URL.
This change set fixes the IndexError and adds unit test coverage for
pathless URLs in discovery data.
Change-Id: I990a24ea32d0f7123566053046c80e48bce0536b
Partial-Bug: #1705770
We're discouraging the use of the ambiguous and difficult-to-understand
'version' parameter in new discovery methods, instead encouraging the
use of min_version and max_version.
In order to make it possible to get the same functionality, though, we
need a way to say the same thing as version="M.m", which actually means,
"min version is M.m, and max version is the latest within major version
M".
Introducing 'latest' syntax, which can be used in various ways,
including:
min_version='2.3', max_version='2.latest'
...which is equivalent to the old school version='2.3'
Change-Id: Ife842333e25c33e54bbae4c1adb101014cb8e8db
Remove four unneeded things in unit tests, updated a docstring,
and changed TypeError to ValueError.
Change-Id: Iad43ab2afa3767739ef3137b2691875520d052b3
There are a two interrelated pieces in this patch which are around
fixing up places where discovery was being re-run inappropriately.
They fall out from adding tests for the functionality and couldn't
be sanely shifted back further in the stack without a big dance.
Switch the default for "discover_versions" on all of the calls that
return an EndpointData to "True". It's a new feature and is a thing that
doesn't make a ton of sense to call if you don't want discovery run.
However, get_endpoint uses it, so needs to be able to pass in
discover_version=False, so the option is still useful. Make sure that
get_endpoint and other places where ksa calls get_endpoint_data on
behalf of the user work as before without unneeded discovery.
Add tests to show that we use actually use the discovery cache properly when
we've previously done discovery that can satisfy the new request. This
works from the microversion optimization patch, but we had to clean up
a couple of things to show it fully in a test.
Change-Id: I54053336edf1b3c2bd35a77dbd78f56388b8e806
The large set of copy-pasta test urls make it hard to follow what is
being mocked and what should be returned. Put in a class to manage it
with clearly named things.
This exposes a bug, but fixing the bug is hairy. Comment out that bit
with a note to fix it. The fix comes in the next patch, but is hopefully
clearer to read with the test refactor decoupled.
Change-Id: I1435d2dc16db7b1a7f5dfe76562de56ba8450531
Sometimes, especially in places like service-to-service defaults, it's
very helpful to express a list of values. For instance, when thinking
about nova connecting to ironic, nova would like to have the default
value of "interface" be ['internal', 'public'] - which is to say, use
internal if it's there, but otherwise use public. This use case is covered
in the API-WG specs on discoverability.
Change-Id: I9102155c2d4ef1ef8bbb1d0fa26a5b5838108a4c
In the case where we're just asking for an endpoint, and the endpoint in
the catalog matches the requested version, there is no need to fetch the
version discovery document.
Change-Id: I2c14337a3fcb1369652d43ca68e6a572fef6d425
If a user has provided an endpoint_override, they may still be
interested in version discovery data for the endpoint. Doing that is
always an opt-in behavior, so we set the strictness flag to prevent any
URL manipulations. We'll either return data or None.
Change-Id: I673beafd1e55fd096bb221b2ca6794bb124653b8
All of this "get endpoint data" stuff is great, but it's no good if the
user can't ask "hey - what are we working with here". It's safe enough
to not cache this data on the session or adapter objects, because the
source data is all cached anyway. That way calling the method with
different filters will always return the correct data.
Change-Id: I1feb505b2d838bd4b6505336c23b68041a938008
Just wanting "latest" isn't the full picture. A client could support,
say, v1 and v2 of an API but not v3 and would like to find an
appropriate matching endpoint.
Add two new arguments, min_version and max_version, rather than
repurpose the version argument.
This changes the behavior of versioned_data_for and versioned_url_for in
the case where version=None. Before that would return None, now it
returns the information about the endpoint that was in the catalog.
The booleans in this are a bit hard to read, as are the fun times with
latest and things being or not being defined. It's time to make the
versions into objects, but we'll do that as a followup.
Change-Id: I8ba948a712002775098b0a86c70f05e0c68250f5
Setting version to a version returns the version, which is not
surprising. Setting the version to None gets the version from the
catalog. But sometimes you want the latest available, for which the
string 'latest' is the thing you want. This adds support for 'latest'.
Co-Authored-By: Jamie Lennox <jamielennox@gmail.com>
Change-Id: Idbc536835f16ec5369cbf5caf2d034066faf5ef2
If a user does not specify a version, that means they want whatever is
in the catalog. However, they may still want discovery to be run for
things like microversion information. The new parameter
"discover_versions", if set to True, will run discovery even with no
version parameter. The inverse of this is "skip_discovery" which will
tell keystoneauth to not run discovery even if a version has been given.
Note: This adds some parameters to some methods that get removed by
change I54053336edf1b3c2bd35a77dbd78f56388b8e806 so we should not
actually land this one until we're ready to land the stack up to and
including that one.
Note: This adds two new methods that will have behavior changes in
patch I8ba948a712002775098b0a86c70f05e0c68250f5.
Change-Id: I897c39743089c5994b51336a4ad44eebed33ec35
From the API-WG spec, there are two common patterns for service URLs
that can be interpreted. Trailing project_id and a string that starts
with v. If the project_id is in the URL, it needs to be removed before
discovery can happen, but it needs to be put back on to the url found
via discovery. If the endpointin the catalog has a version, and it
matches the version we're asking for, then we don't need to go hunting
for the unversioned doc.
Also, in the EndpointData we're collecting, we want to grab microversion
info, since we're already there in the discovery doc.
There is one behavior change that can be seen in the tests. If the
attempt at an unversioned discovery endpoint fails, we fall back to the
url from the catalog ... but we attempt to get a discovery document from
it because we need the metadata for microversions. The catalog URL should be
returned as the endpoint even if the second discovery call attempt
succeeds, so the user-facing interface is the same - there will just be,
in some cases, an additional URL fetch behind the scenes.
Change-Id: I2a036d65e4f7dba6f50daf6a0ce4589ee59ae95f
As part of implementing the API-WG spec on version discovery, there are
more version manipulations and logic that need to happen between
fetching from the catalog and doing discovery.
Move the logic for doing that into the discovery file and attach it to
the EndpointData object.
This changes 2 interfaces, but neither of those interfaces have been in
a release.
The method "discover_versions" is renamed to "get_versioned_data" - since
the work it does is actually to get a versioned EndpointData object.
It also now returns a new EndpointData object instead of mutating the
existing one. Especially with the name change, mutation seemed really
off.
Change-Id: Ifeeac7af1ebd9d2d59a30d4503c8bcc0137e9370
Allow the user to pass in a cache dict that will be used
in addition to the session and auth level caches. Make Session
always have a discovery_cache attribute and allow the user to
provide the cache at Session creation time. Finally, rename
the private variable to _discovery_cache from _endpoint_cache
since it's caching discovery objects, not endpoints.
Co-Authored-By: Samuel de Medeiros Queiroz <samueldmq@gmail.com>
Change-Id: I0a0f489fd3bbecc4596e99acafcde1bff4e181f7
Whilst historically we always wanted keystoneauth to do the most
permissive thing and allow a versioned or unversioned entry in a service
catalog there are now cases where we would prefer to fail when the
catalog is misconfigured. This will allow a client to opt out of
versioned catalog endpoints to insist that the deployment is correctly
configured.
Closes-Bug: #1668484
Change-Id: Ided0e0c7409994f703175fe61bd4043b840bcf1e
1.As mentioned in [1], we should avoid using
six.iteritems to achieve iterators. We can
use dict.items instead, as it will return
iterators in PY3 as well. And dict.items/keys
will more readable. 2.In py2, the performance
about list should be negligible, see the link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
Change-Id: I9f8f2c35f0d45d866076507a3a167aaafb8382e5
After patch Ia08538ccf00c9063dc0d284c5ece9a969c15500a the urljoin would
ensure that a URL was always returned from the get_endpoint method even
when the version was not available. This breaks plugin discovery and a
number of other areas.
Change-Id: I04014b6e770c2e9708c5f9c81c3160d51603ad0c
Closes-Bug: #1616720
When using a reverse proxy or TLS terminator it can be really hard to
correctly specify a URL in version discovery that is correct for all
situations.
Make it so that a relative URL in a discovery document is joined to the
queried URL to always return a fully qualified URL to the identity
plugins.
Change-Id: Ia08538ccf00c9063dc0d284c5ece9a969c15500a
Closes-Bug: #1614304
The exact parameters we pass to request change occasionally for reasons
unrelated to either of these tests. Requests-mock supports testing for
cert and verify values directly and we can test the auth plugin via what
it sets on the request.
Change-Id: I771db5a061c355b9324e46f601c2dd98cce1ace0
Particularly for allowing the CLI to store and reuse previous
authentication allow an application to extract and reinstall the
auth state from a plugin.
We provide a method that returns a dictionary of all of the identifiable
information that is used to create a plugin. This dictionary is hashed
to uniquely identify the plugin.
We then have a get_auth_state and set_auth_state function, the return of
which is intended to be opaque to the calling application. If the plugin
created returns an ID of an existing authentication you can call
set_auth_state to load that state. If the state is out of date it will
be refreshed as per normal otherwise it will be used instead of
authenticating again.
There is not support for caching federated tokens in this patch. They
will follow the exact same pattern and are not much harder they just
need a way for subclasses to signal they are cachable and so can be done
as a follow up.
Implements: bp cachable-auth
Change-Id: I4eebe7ff8060a37f19af5decfa3a8313cfb7c207
I noticed a lot of tests were still in directories from
the past and things were not as easy as you might hope to find.
Change-Id: Iae3d75cee9f48a09d2c9aa2a5b18301b4832bef0
Zuul