The check_revocation method is called at least 3 times when validating
a token.
Each time, it's doing a heavy SQL statement depending on the size of the
revocation table.
We can save time by adding cache to this method.
Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
Change-Id: I70b4664905bb4360d792ba8bd701674f60538223
The OAuth2.0 Access Token API is modified, support to get an OAuth2.0
certificate-bound access token from the keystone identity server with
OAuth 2.0 credentials and Mutual-TLS certificates.
Co-Authored-By: Hiromu Asahina <hiromu.asahina.az@hco.ntt.co.jp>
Change-Id: I885527bec61429b1437a046097a16491848b5a0a
Implements: blueprint support-oauth2-mtls
If a token is issued with an application credential we need to check
the expiration of the application credential to ensure that the token
does not outlive the application credential. This ensures that if the
token expiration is greaten than that of the application credential it
is reset to the expiration of the application credential and a warning
is logged. Please see CVE-2022-2447 for more information.
Closes-Bug: 1992183
Change-Id: If6f9f72cf25769d022a970fac36cead17b2030f2
Previously, the fernet token provider would log warnings when a fernet
token exceeded 255 characters, which is common for LDAP-backed
deployments. The warning is always issued, even when operators configure
keystone's max_token_size to a higher value, causing confusion because
it appears the configuration value is silently ignored.
This commit fixes that issue by using the max_token_size configuration
parameter consistently in the fernet token provider.
Closes-Bug: 1926483
Change-Id: I4bb54aac9b950d59082a4468203a3249790839d7
msgpack v1.0 changed its data format [1] and during a rolling upgrade, attempts
to unpack cached tokens with old data format with the new default raw=False
result in the following error:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x87 in
position 3: invalid start byte
This passes raw=True to support backward-compat with the old format
until we are guaranteed to have msgpack >= 1.0 in the N-1 release of
a rolling upgrade.
Closes-Bug: #1891244
[1]
https://github.com/msgpack/msgpack-python/blob/v1.0.0/README.md#major-breaking-changes-in-msgpack-10
Change-Id: I6c61df6c097fef698c659c79402c4381ec7f3586
GET /v3/auth/tokens?allow_expired=1 works fine with fernet tokens
returning the expired token data, whereas it returns exception
TokenNotFound for JWT. This patch fixes the same.
Change-Id: I03f6c58dce7d140d62055a97063aeb480498e5e6
Closes-Bug: #1886017
This repo does not support Python 2 anymore, so we don't need
six for compatibility between Python2 and 3, convert six usage to Python
3 code.
Change-Id: Icba56808f38277b27af2ae5aac4b8507dee71b3b
`federation_group_ids` could be zero length list, so deciding whether
a token is federated by checking if it is none.
Change-Id: I0f4b9e24d949aa4838ee721a165999b29c684d32
Closes-Bug: #1856962
In 09088690 we mistakenly added E501 to the flake8 ignore list. Since
then, many new violations have been introduced. This patch re-enables
the check and corrects all violations, except in some cases like unit
test names where the subunit output would suffer if we attempted to
shorten the function name.
This may appear to be a pointless no-op that messes with
git-blameability, and it is, but the reason to do this is that if PEP8
violations are introduced in master and then backported to a stable
branch, most stable branches will fail the pep8 job since the flake8
ignore list is correct for those branches. Rather than loosening the
check in older branches or requiring those backports to fix the linter
errors independently of what's been merged in master, we should fix it
now so that we don't introduce more errors in the future and patches can
more easily be backported.
Change-Id: I9f71926105eb448bb0200201d1838b67d4963cd6
This change adds application credential access rules to the token model
and ensures that only clients (that is, keystonemiddleware) that support
access rule enforcement are allowed to validate tokens containing
access rules.
Depends-on: https://review.openstack.org/633369
bp whitelist-extension-for-app-creds
Change-Id: I301651369cf03e06550bc29eb534506674e56a1f
Ensure that any binary types unpacked from message payloads
are correctly converted from binary to text type.
Under Python 3 msgpack returns the serialized input as a
byte string. Similar to other msgpack'd values in the payload,
we need to explicitly decode it to a string value.
This is specifically more of an issue under Python 3; however
the decode operation is safe back to Python 2 so there is no
need to limit the decode codepath to just Python 3.
Change-Id: Ib1073acf5677a60714d0a386de3bcd14ce6cd134
Closes-Bug: 1832265
Like in 44c1b3d, the unscoped federated token formatter needs to account
for the new user string format too. If it doesn't, the python3 tests may
fail.
Change-Id: I9529d6bee3e5bb1f618f40f225f69e2ad7e3f64a
Now that the user ID for shadowed, federated users is no longer a random
UUID but a sha256 hash, the token formatter shouldn't be trying to
convert it to a byte string, and yet on python3 msgpack does anyway, so
we need to convert it back to a string.
Related-bug: #1641639
Change-Id: Icb2a591642df96d5bbd02428d2b0d0e8090009c0
Keystone already supports a global caching toggle and an option to
configure token caching explicitly. Having a third option to enable
pre-cached tokens is redundant, creates unnecessary complexity that
bleeds through to operators, and causes weird behaviors if token
caching is disabled and pre-caching is not.
This commit deprecates the cache_on_issue configuration option in
favor of just using ``keystone.conf [token] caching`` option instead.
This commit also attempts to clarify the help text so that it
describes the relationship between the various caching options, even
if it is short-lived.
The help text for ``keystone.conf [token] cache_on_issue`` claimed
that it only cached tokens if global caching was enabled through
oslo.cache and if ``keystone.conf [token] caching = True``. However,
the actual implementation doesn't check if ``keystone.conf [token]
caching = True`` at all. Even if token caching is disabled, tokens
will be cached when they are issued.
Change-Id: I1e1117deabadaba26ea8e833a06180529e1e0a4b
This commit introduces a class that implements the JWS token provider
functionality.
bp json-web-tokens
Change-Id: Ie16110894348a83e3a80cba4649e6cccdc3c84b1
We attempt to be clever about string types in the token formatters.
We do this because in some cases, not all items in a token payload
are serialized to byte strings. To add flexibility for this, we use
tuples with a boolean value that denotes if the accompanying value is
a byte string or not. This helps us safely re-inflate the value from
a byte string back to it's .hex representations, typically with UUID
strings.
With python3, we actually hit an interesting case where what we pass
into the token payload doesn't actually maintain that state due to the
usage of msgpack. The msgpack library returns byte strings even though
the initial value may not have been a byte string. This breaks the
logic we have for the associated boolean value because the string type
changes and the boolean does not.
This commit adds a couple of if/statements to detect if we running on
python3 and if the boolean mismatches the actual value type. Then, it
attempts to do the right thing by decoding the string.
We should think about how we want to do this, or if there is a better
way.
Change-Id: Iaecd45ef985cbf5ff4a6a724df96c1304a927247
Closes-Bug: 1813085
Without this patch, if a token fails to decrypt, the token string is
logged at the WARNING level, which means the majority of deployments
will log it. Since keystone tokens are bearer tokens, logging it to disk
is a security risk. Of course the reason for the log is that the token
is invalid, and so it can't be used as-is to gain access, but the token
might be *almost* valid: for example, it might be a token that the user
had intended to use on a different keystone instance, or it might be a
truncated token such that the last few characters could be guessed.
Since the encrypted token is nearly useless to an operator for
debugging, stop logging the token string and just emit a generic
warning.
Change-Id: Id05b506327d22e42b2da3b1a38d8237cbf7786a8
Now that support for sql token storage and uuid tokens has been
removed, it doesn't make sense to still expose an API for listing
revoked tokens. Maintaining this behavior would require keystone to
persist non-persistent tokens, which defeats the purpose.
This change makes the API return either a 410 Gone or a 403 Forbidden
depending on configuration for backwards compatibility. Logic to
list revoked tokens was also removed from the token provider API
since it's no longer called by any controllers.
Change-Id: Ic7bcba148f0a062b144e6dfbe9693f2125008458
This module was a hodge-podge of common utility methods and a basic
implementation of the token provider API interface. In theory, if
something should be done for all providers, we should try and pull
it into a higher layer, like the token provider Manager. This makes
things easier to share without having to worry about reimplementing
something if we override a specific method of the interface. This is
the pattern we're working towards with the TokenModel object.
It was also home to the V3TokenDataHelper, which was ultimately
responsible for making sure the token API contracts were honored. Now
that we've moved token behavior into the TokenModel and the
representation of a token into the controllers, we don't need this
anymore. We should be able to make this much more clear and clean up
the interfaces for people providing their own token providers.
Partial-Bug: 1778945
Change-Id: I6f069c8c94e625ae553e9b41f0c54fd25bad9408
With the removal of token binding, we can clean up some of the logic
in other parts of the token provider API that accounted for it.
Change-Id: I6db60077d829ef78baca80eb79a96d0032ad62be
Since we're no longer supporting persistent tokens in tree and we
removed the uuid token provider, it's the perfect time to clean up a
good amount of confusing technical debt.
The token provider API is historically known for being confusing.
This is mainly because the reference that is intended to be returned
to the user is modified all up and down the API. Different parts of
the API use the reference to invoke call hooks in other method making
the code hard to debug. In order to fully understand how tokens are
built, you need to understand where and how tokens are modified by
different layers of the API according to a specific contract of the
authentication API. Another big problem is that it couples the actual
reference of how a token looks too closely to the business logic for
tokens. Which means you have to write a ton of code if you ever want a
token to look differently, like you would if you wanted to support a
new API version.
A token should be an object that the managers and controllers can
query and reason about. From there they should be able to build token
responses accordingly. This will make the actual token provider API
much simpler because it needs to know less about API contracts that
are the responsibility of the controllers. This should lead to simpler
interfaces when new token providers are added, or maintained out of
tree. This also makes it less likely for APIs to behave differently
based on what token provider is configured by being explicitly
building the token reference in one place.
This commit ports the token business logic out of the
keystone.token.providers.common module and into a dedicated token
object, or model. This will result in a cleaner interface between the
token providers and the token provider API. A subsequent patch will
remove the unused code across the token provider API.
Partial-Bug: 1778945
Change-Id: If9ded94e65bacb0d06f5225bb36f659dc7bb8355
Then `enable` config option of trust feature is depreacted in
Queens. Remove it in Rocky now.
Change-Id: I186b49471cb774e161ff4c35c9879a0a4fa9538f
bp: removed-as-of-rocky.
When checking the current effective trust roles, the current
implementation creates a list from a set, then iterates over the
list to find a single id's existance in the list.
This change keeps the list as a set and utilizes the set
collection's speedier existance check to see if a trust_role_id is
present in the list of current_effective_trustor_roles. This removes
the casting back into a list, as well as a redundant list iteration.
Change-Id: I63ffbcc294e77b96f7a302543d335d3cb630090c
This patch removes the rest of the v2.0 specific code which was being
maintained exclusively due to a copy-paste issue with the deprecation
warning on the EC2 controller(s). With sign off from TC members
we have removed all of the bits except those exclusively tied to
the paste.deploy removal. All paste.deploy specific changes will
be removed in a future patch.
With the conversion to Flask, none of the v2.0 legacy controllers
left were wired up to routes that could be accessed.
Change-Id: I959dac0d0dd2e667982383e1e3d52ab28c4c1e2e
This reverts commit 03ba867327.
Because of the introduction of auth receipts we will be
using fernet for more than just tokens. Lets make this a
generic util for fernet key handling.
Change-Id: I3a870a63239491f84db3350178bd2313eeccdbf3
Basic conversion of Keystone's core application to flask framework.
This doesn't add much in the way of flask-specific-isms but should
get keystone running directly under flask. This implementation does
not use paste-deploy.
Change-Id: Ib4c1ed3f645dd55fbfb76395263ecdaf605caae7
The _verify_token_ref() method is a dead section of code that was
last called in:
I62cc1d5efbe0869fd0e501aaa9405ff0b0da0e5e
This commit removes the method since it is no longer called anywhere
and hasn't since the Icehouse release.
Change-Id: I1b6ed09e9c9ad9214cd6e2c0f5849bb8550fbf3f
Without this patch, the token formatter does not have enough data to
construct a token created with an application credential. This means
that if the token cache is disabled or expired, when keystone goes to
create the token it will not find any application credential information
and will not recreate the application_credential_restricted parameter in
the token data. This patch creates a new Payload class for application
credentials so that the application credential ID is properly persisted
in the msgpack'd payload. It also adds more data to the token data
object so that the application credential ID and name as well as its
restricted status is available when the token is queried.
Co-authored-by: Lance Bragstad <lbragstad@gmail.com>
Change-Id: I322a40404d8287748fe8c3a8d6dc1256d935d84a
Closes-bug: #1750415
The token provider listened for specific callbacks from the
federation and oauth APIs. These were mainly for token invalidation
and token persistence removal. Now that the sql token driver and uuid
token provider have been removed, we can remove the need for
persistence notifications.
Instead, both of these cases can use a generic token cache
invalidation callback. This makes the token provider Manager simpler
because it only has to listen for a single internal notification to
invalidate the token cache, instead of multiple. It also makes the
code sending the notification easier to understand because it's
clearer about what it's doing.
Change-Id: I3281a013ce2764872dda8c183c4e9851974e1d14
The INVALIDATE_USER_TOKEN_PERSISTENCE and
INVALIDATE_USER_PROJECT_TOKEN_PERSISTENCE callbacks were meant to
clean up invalid tokens from the token storage layer. Now that the
sql token driver has been removed, we don't need them any more. This
commit removes those notifications and refactors the places where
notifications are still needed, making them more specific and not
eluding to token persistence.
This commit also removes a significant amount of logic from the
assignment API that used to notify the token API when assignments
were deleted. This made sense when tokens were written to disk
because there was an opportunity to invalidate them when users were
removed from projects. This is no longer needed since we do
validation online and we don't persist tokens anymore.
Change-Id: I100b7416e8ba61eb4ea2c2eb4962e952a53ea388
There are several cases where we need to invalidate the token cache.
Notifications are used to do this and the token provider API uses
callbacks to subscribe to those notifications. This commit makes them
all use a single callback function instead of each defining their own
just to do the same thing. A subsequent patch will refactor the term
"persistence" from the notification names used across keystone.
Change-Id: I5e78f8aafec74e3edb952ab92947503c9be38b91
Since the sql token storage mechanism was removed in Rocky, we no
longer need hooks in the token Manager to determine if a token needs
to be retrieved from or written to a backend somewhere. Instead, token
providers will need to handle storage requirements if they need them.
This will result in a cleaner token provider interface.
Change-Id: Icc095987d41e9c08de2f34dc657b08b98bd944e4
These two were just chillin' as token provider class variables. They
are not used anywhere and we use special notification strings defined
in notification.py for good reason. Keeping things defined in one
place makes maintenance easier. Remove the unused variabels from the
token provider.
Change-Id: I98f29114daa01df94f12813143da9097c294b43a
Both of these drivers were staged for removal in Rocky. Now that
Rocky is open for development we can remove them. This commit removes
just the bare-bones aspects of each. Subsequent patches will do the
following:
- Remove test class that were only meant for sql or uuid scenarios
- Refactor the notification framework to not hint at token storage
- Refactor the token provider API interfaces to be simpler and
cleaner
- Remove the needs_persistence property from the token provider API
and document the ability to push that logic into individual
providers that require it
- Return 403 Forbidden for all requests to fetch a revocation list
- Remove the signing directory configuration options
These changes will result in simpler interfaces which will be
important for people implementing their own token providers and
storage layers.
bp removed-as-of-rocky
Change-Id: I76d5c29f6b1572ee3ec7f2b1af63ff31572de2ce
The only API we're supporting on the v2.0 path until the T release is
the ec2tokens API. This commit removes all routers from the public
and admin v2.0 applications. This includes the extensions API. This
commit also removes unused v2.0 controller logic.
Change-Id: I523c1215899ac9ee605df6bf717643c0ba87c761
Closes-Bug: 1746798
Previously, it was possible to validate a federated keystone token
after the identity provider associated by that token was deleted,
which is a security concern.
This commit does two things. First it makes it so that the token
cache is invalidated when identity providers are deleted. Second,
it validates the identity provider in the token data and ensures it
actually exists in the system before considering the token valid.
Change-Id: I57491c5a7d657b25cc436452acd7fcc4cd285839
Closes-Bug: 1291157
Add an auth plugin for application credentials and update the common
auth utilities to understand an auth method of 'application_credential'
and validate and scope accordingly.
By default, application credentials should not be allowed to be used for
creating other application credentials or trusts. If a user creates an
application credential with flag `allow_application_credential_creation`
then that application should be allowed to be used for creating and
deleting other application credentials and trusts. Ensure a flag is set
in the token if this property is set to allow this behavior.
bp application-credentials
Change-Id: I15a03e79128a11314d06751b94343f22d533243a
This commit exposes the necessary bits to expose system-scoped
token authenticate and validation via the API
bp system-scope
Change-Id: I572a8e48953f493d521fd2aa00007df46e562e2e
This change incorporates system-scope changes into the token provider
API. A subsequent patch will wire up the necessary controller bits
that expose the functionality via the API.
bp system-scope
Change-Id: I15ed4b96d3e0d78f85ca61a3d85274c770c4c7cf
This commit adds a new payload type that is meant to allow
system-scoped token. A subsequent patch will actually wire this up
to the token provider API and another patch to expose it via the
authentication API.
bp system-scope
Change-Id: I26357b6d62ce88ad116e0231145b2367dda62fa2
Arnaud Morin