The OAuth2.0 Access Token API is modified, support to get an OAuth2.0
certificate-bound access token from the keystone identity server with
OAuth 2.0 credentials and Mutual-TLS certificates.
Co-Authored-By: Hiromu Asahina <hiromu.asahina.az@hco.ntt.co.jp>
Change-Id: I885527bec61429b1437a046097a16491848b5a0a
Implements: blueprint support-oauth2-mtls
Previously, the fernet token provider would log warnings when a fernet
token exceeded 255 characters, which is common for LDAP-backed
deployments. The warning is always issued, even when operators configure
keystone's max_token_size to a higher value, causing confusion because
it appears the configuration value is silently ignored.
This commit fixes that issue by using the max_token_size configuration
parameter consistently in the fernet token provider.
Closes-Bug: 1926483
Change-Id: I4bb54aac9b950d59082a4468203a3249790839d7
The mock library is a third party lib that attempted to bridge the gap
between Python 2 and Python 3 mocking. Now that we have moved to py3
only, there is no need to use a third party lib and we can use the
standard built-in mocking support.
Change-Id: I8bbcedb7ad3f0bc2e06dfa13878a97411ee1dc6d
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This repo does not support Python 2 anymore, so we don't need
six for compatibility between Python2 and 3, convert six usage to Python
3 code.
Change-Id: Icba56808f38277b27af2ae5aac4b8507dee71b3b
`federation_group_ids` could be zero length list, so deciding whether
a token is federated by checking if it is none.
Change-Id: I0f4b9e24d949aa4838ee721a165999b29c684d32
Closes-Bug: #1856962
In 09088690 we mistakenly added E501 to the flake8 ignore list. Since
then, many new violations have been introduced. This patch re-enables
the check and corrects all violations, except in some cases like unit
test names where the subunit output would suffer if we attempted to
shorten the function name.
This may appear to be a pointless no-op that messes with
git-blameability, and it is, but the reason to do this is that if PEP8
violations are introduced in master and then backported to a stable
branch, most stable branches will fail the pep8 job since the flake8
ignore list is correct for those branches. Rather than loosening the
check in older branches or requiring those backports to fix the linter
errors independently of what's been merged in master, we should fix it
now so that we don't introduce more errors in the future and patches can
more easily be backported.
Change-Id: I9f71926105eb448bb0200201d1838b67d4963cd6
Ensure that any binary types unpacked from message payloads
are correctly converted from binary to text type.
Under Python 3 msgpack returns the serialized input as a
byte string. Similar to other msgpack'd values in the payload,
we need to explicitly decode it to a string value.
This is specifically more of an issue under Python 3; however
the decode operation is safe back to Python 2 so there is no
need to limit the decode codepath to just Python 3.
Change-Id: Ib1073acf5677a60714d0a386de3bcd14ce6cd134
Closes-Bug: 1832265
Like in 44c1b3d, the unscoped federated token formatter needs to account
for the new user string format too. If it doesn't, the python3 tests may
fail.
Change-Id: I9529d6bee3e5bb1f618f40f225f69e2ad7e3f64a
Now that the user ID for shadowed, federated users is no longer a random
UUID but a sha256 hash, the token formatter shouldn't be trying to
convert it to a byte string, and yet on python3 msgpack does anyway, so
we need to convert it back to a string.
Related-bug: #1641639
Change-Id: Icb2a591642df96d5bbd02428d2b0d0e8090009c0
The unit test uses sqlite for test which closes db foreign keys
function by default. This patch enabled the sqlite foreign keys
function for unit test by default.
The "project" table is a self referencing FK table(id <-> domain_id
column). So when the FK is enabled, there must exists a root record
before insert data to this table. It's <<keystone.domain.root>>.
Usually, the <<keystone.domain.root>> recored is inserted into the
table once operators run "keystone-manage db_sync" command when
deploy Keystone. But the unit test code doesn't run this command,
it initialise the db schema by reading sqlalchemy object model, so
the <<keystone.domain.root>> record is missed. Then we can't create
any project record, it'll raise FK error.
So in this patch, before creating any projects in the test, we must
ensure the <<keystone.domain.root>> record exists first.
Change-Id: I565d12395ca39a58ba90faf8641a9e02d986aeb9
Closes-Bug: #1744195
Without this patch, if a token fails to decrypt, the token string is
logged at the WARNING level, which means the majority of deployments
will log it. Since keystone tokens are bearer tokens, logging it to disk
is a security risk. Of course the reason for the log is that the token
is invalid, and so it can't be used as-is to gain access, but the token
might be *almost* valid: for example, it might be a token that the user
had intended to use on a different keystone instance, or it might be a
truncated token such that the last few characters could be guessed.
Since the encrypted token is nearly useless to an operator for
debugging, stop logging the token string and just emit a generic
warning.
Change-Id: Id05b506327d22e42b2da3b1a38d8237cbf7786a8
This module was a hodge-podge of common utility methods and a basic
implementation of the token provider API interface. In theory, if
something should be done for all providers, we should try and pull
it into a higher layer, like the token provider Manager. This makes
things easier to share without having to worry about reimplementing
something if we override a specific method of the interface. This is
the pattern we're working towards with the TokenModel object.
It was also home to the V3TokenDataHelper, which was ultimately
responsible for making sure the token API contracts were honored. Now
that we've moved token behavior into the TokenModel and the
representation of a token into the controllers, we don't need this
anymore. We should be able to make this much more clear and clean up
the interfaces for people providing their own token providers.
Partial-Bug: 1778945
Change-Id: I6f069c8c94e625ae553e9b41f0c54fd25bad9408
With the removal of token binding, we can clean up some of the logic
in other parts of the token provider API that accounted for it.
Change-Id: I6db60077d829ef78baca80eb79a96d0032ad62be
Since we're no longer supporting persistent tokens in tree and we
removed the uuid token provider, it's the perfect time to clean up a
good amount of confusing technical debt.
The token provider API is historically known for being confusing.
This is mainly because the reference that is intended to be returned
to the user is modified all up and down the API. Different parts of
the API use the reference to invoke call hooks in other method making
the code hard to debug. In order to fully understand how tokens are
built, you need to understand where and how tokens are modified by
different layers of the API according to a specific contract of the
authentication API. Another big problem is that it couples the actual
reference of how a token looks too closely to the business logic for
tokens. Which means you have to write a ton of code if you ever want a
token to look differently, like you would if you wanted to support a
new API version.
A token should be an object that the managers and controllers can
query and reason about. From there they should be able to build token
responses accordingly. This will make the actual token provider API
much simpler because it needs to know less about API contracts that
are the responsibility of the controllers. This should lead to simpler
interfaces when new token providers are added, or maintained out of
tree. This also makes it less likely for APIs to behave differently
based on what token provider is configured by being explicitly
building the token reference in one place.
This commit ports the token business logic out of the
keystone.token.providers.common module and into a dedicated token
object, or model. This will result in a cleaner interface between the
token providers and the token provider API. A subsequent patch will
remove the unused code across the token provider API.
Partial-Bug: 1778945
Change-Id: If9ded94e65bacb0d06f5225bb36f659dc7bb8355
This patch removes the rest of the v2.0 specific code which was being
maintained exclusively due to a copy-paste issue with the deprecation
warning on the EC2 controller(s). With sign off from TC members
we have removed all of the bits except those exclusively tied to
the paste.deploy removal. All paste.deploy specific changes will
be removed in a future patch.
With the conversion to Flask, none of the v2.0 legacy controllers
left were wired up to routes that could be accessed.
Change-Id: I959dac0d0dd2e667982383e1e3d52ab28c4c1e2e
This reverts commit 03ba867327.
Because of the introduction of auth receipts we will be
using fernet for more than just tokens. Lets make this a
generic util for fernet key handling.
Change-Id: I3a870a63239491f84db3350178bd2313eeccdbf3
In some rare cases, an empty key file can get created within the fernet
key repository. When keystone tries to load the keys from disk, it will
fail with an invalid fernet key ValueError.
This change adds a check for empty files with a valid numerical name
within the key repository when rotating keys and loading keys. If an
empty file exists, it will be ignored when loading keys, reported in the
logs, and overwritten with a valid key upon rotation.
Change-Id: Ic19dd02d38e8f6a05c8951ec3dd13659aab98259
Closes-Bug: 1728907
The fernet token unit tests had a bunch of config_fixture overrides to
populate the fernet max_active_keys configuration option with its
default value. Since it is just setting these to the default, it's
not necessary to provide overrides when it's already established in
keystone.tests.unit.core.TestCase.
Change-Id: Id09459104d9b5ec412ec60211ef71b7c7c62b14f
This assertion was a bit misleading in that it seemed to validate
one key at a time. But the implementation of the assertion actually
assumes it is given a list of string values. This commit fixes the
name of the assertion to be more clear and it also makes the
assertion iterate all values given, which ensures each key in the
list is valid, not just the first one.
Change-Id: Ifc9542e4e7f57f54d473183c97f616f92cb0961b
Without this patch, the token formatter does not have enough data to
construct a token created with an application credential. This means
that if the token cache is disabled or expired, when keystone goes to
create the token it will not find any application credential information
and will not recreate the application_credential_restricted parameter in
the token data. This patch creates a new Payload class for application
credentials so that the application credential ID is properly persisted
in the msgpack'd payload. It also adds more data to the token data
object so that the application credential ID and name as well as its
restricted status is available when the token is queried.
Co-authored-by: Lance Bragstad <lbragstad@gmail.com>
Change-Id: I322a40404d8287748fe8c3a8d6dc1256d935d84a
Closes-bug: #1750415
Since the sql token storage mechanism was removed in Rocky, we no
longer need hooks in the token Manager to determine if a token needs
to be retrieved from or written to a backend somewhere. Instead, token
providers will need to handle storage requirements if they need them.
This will result in a cleaner token provider interface.
Change-Id: Icc095987d41e9c08de2f34dc657b08b98bd944e4
Just like the APIs in keystone, we should be using the
`keystone.common.provider_api.ProviderAPIs` to load managers instead
of self.
This makes those changes for a few of the test modules. Finding
occurrences can be done with `grep -R 'self.*_api' keystone/tests/`.
Change-Id: Ic2094dca56158d8e4cd843eadff837f3a17ea38f
Previously, it was possible to validate a federated keystone token
after the identity provider associated by that token was deleted,
which is a security concern.
This commit does two things. First it makes it so that the token
cache is invalidated when identity providers are deleted. Second,
it validates the identity provider in the token data and ensures it
actually exists in the system before considering the token valid.
Change-Id: I57491c5a7d657b25cc436452acd7fcc4cd285839
Closes-Bug: 1291157
This commit adds a new payload type that is meant to allow
system-scoped token. A subsequent patch will actually wire this up
to the token provider API and another patch to expose it via the
authentication API.
bp system-scope
Change-Id: I26357b6d62ce88ad116e0231145b2367dda62fa2
This change moves the token_formatter under fernet to be located
under the token directory. This is to allow for the functionality
of token_formatter to be easily used with other token providers.
Change-Id: I3a1ee30f69c33c9dd98c7e39e552b7908cc836e1
This change renames the fernet_utils from keystone.common to
token_utils in order to provide more generic tools for the
implementation of alternative token providers.
Change-Id: I85091709ed4ed7bdd2c2d053e342e4b1b615ef08
This expanded test illustrates that the key rotation mechanism can
recover from repeated rotation attempts while the disk was full.
Change-Id: I042a15929c5e61aeaf357d88f224cdc026b5a5ed
As of Mitaka we no longer need to validate both padded and unpadded
fernet tokens.
Implements: bp removed-as-of-pike
Change-Id: Iaff357efdda7b08b0555b7c56d68222f5610ad80
Moved AuthContext and AuthInfo to keystone.auth.core as they are shared
code bits and not exclusively controller specific.
Change-Id: I649690d9e39057249e674500d85a053e0c28b30e
FernetUtils is giving incorrect error messages to administrators
indicating that [fernet_tokens] option is always where the
fernet repository information is stored even if it is referenced
from the [credential] option group.
Change-Id: I7b8344bb306eeb0a9e1cf5093dfd42d3e6c2dd1b
Force the keystone.auth.controllers.Auth.authenticate method to
require the use of an AuthContext object instead of something
duck-typed (dictionary). This is done to ensure the security and
integrity of IDENTITY_KEYS are covered and values are not changed
by a plugin due to the security built into AuthContext being
circumvented since it was not used. This is not pythonic, this
is being done for hardening purposes.
Change-Id: I013846af59587d17b15ca4cf546e6372231f576e
Closes-Bug: #1656076
As a follow up from comments at https://review.openstack.org/#/c/412083,
this patch adds a new assertion to improve the validation in the test.
Change-Id: Ib8dd06a2098730a58dc2a3015dae67cd58fb5eb3
Co-Authored-By: Lance Bragstad <lbragstad@gmail.com>
_create_new_key() is broke down into 2 parts:
1. _create_tmp_new_key()
2. _become_valid_new_key()
This can avoid empty Fernet keys when the write to the
staged key fails. The _become_valid_new_key() is called
only after a successful call to _create_tmp_new_key().
Change-Id: Iaf33e2b291f13b9eb9464ef345a8664a634121ff
Closes-Bug: #1642457
Signed-off-by: John Lin <johnlinp@gmail.com>
Since we don't have a reason to hold on to a version specific
token method name, we can rename it to be just issue_token.
Now the token provider interface has two intergral methods,
validate_token and issue_token. From a third-party perspective, this
makes it easier to maintain a proprietary token provider since it
eliminates a lot of version specific things from the provider
implementation. From a keystone perspective, we are isolating the
numerous token validation and issuance paths to a since route. This
will make understanding the code easier for other developers and
eventually easier to optimize.
Change-Id: I71a04b42e931338b8bd59e479636b6199c7c2d76
The token provider module (core) has a list of methods that are called
by backend modules. This patch moves those methods to the backend and
removes dependencies where backend code references code in the core.
Change-Id: I59dab2efc5b743508ee9ecebcdc7c07b9f66791d
Closes-Bug: #1563101
The new user attribute, password_expires_at, is not being returned
during auth; this patch adds it.
bp password-expires-validation
Change-Id: I1f17a849d9da4067d6be7d612c5a561bcb247ebb
Regardless of persistence requirements or format, let's perform
token validation one way.
This simplifies the validation path of the token provider API.
Change-Id: Idb5de4459fd8bf83973ed74fccc275a64873c88c
Because sharing is caring... and it helps us test credential encryption by
allowing us to reuse the KeyRepository fixture for credentials and fernet.
bp credential-encryption
Change-Id: I50a4e663385a0070ee1fd2c83c2fe5913f5a0ad0
This makes FernetUtils configurable for different things to user fernet
utilities. This should help us use fernet for credential encryption.
bp credential-encryption
Change-Id: Ib4a7309844a2e03a5ca920b5512cf7efe1c97867
By converting our module-level fernet utilities to a class, we can
extend it or modify it in ways to make the utilities work for outside
the fernet token provider. This change is in preparation to use fernet
to encrypt credentials at rest.
bp credential-encryption
Change-Id: Ia4e9fd2b8597993f006d9fea82b782085f2cdbc2
If we are going to be using fernet utilities to implement encryption of
credentials at rest, we should put the utilities in a more available
place.
bp credential-encryption
Change-Id: I06aff77836527885b3df1cbd25834906cda73910
Currently, in both unscoped and scoped federated tokens, the
username value in the token is equal to the userid and not to
the value of the username in the external identity provider.
This makes WebSSO login to show the userid of the logged-in
user in the Horizon dashboard, whereas before it was showing
the actual user name.
This patch fixes the value of the username in the federated
tokens, which will fix the WebSSO issue as well, since Horizon
looks at the username value and displays that as the logged-in user.
Closes-Bug: #1597101
Closes-Bug: #1482701
Change-Id: I33a0274641c4e6bc4e127f5206ba9bc7dbd8e5a8
keystone.common.config is 1200+ lines of super dense, merge-conflict
prone, difficult to navigate, and finicky to maintain code. Let's follow
nova's lead and break it down into more manageable modules.
This patch creates a new Python package, keystone.conf, and moves all of
our configuration options into it, mirroring nova's nova.conf package.
There are a couple special modules in keystone.conf introduced here as
well:
- keystone.conf.__init__: This causes all of Keystone options to be
registered on import, so consumers of keystone.conf don't have
races with config initialization code while trying to use
oslo_config.cfg.CONF directly (keystone.conf replaces all uses for
oslo_config.cfg.CONF in keystone).
- keystone.conf.base: Keystone's [DEFAULT] group options. I'd prefer
this to be called 'default.py', but I'm just copying nova's lead here.
- keystone.conf.opts: The entry point for oslo.config itself.
- keystone.conf.constants: There are a few constants (deprecation
messages, default paths, etc) that are used by multiple configuration
modules, so they need to live in a common place.
Change-Id: Ia3daffe3fef111b42de203762e966cd14d8927e2
sunyonggen