Currently all token validation evaluate all events
related to trust deletion even if current token is neihter
trust scoped or if trust_ids do not match token trust.
This involves token validation time variation in environment
where trust deletion is high.
This change reduce token validation time and make it stable
during token life cycle by filtering also on trust_id when
querying revocation_event table.
Change-Id: If592599a12035769491eaed3df1541b5afe43e3f
The pep257 dependency does not support Python 3.10 and has been
deprecated in favour of flake8-docstrings. While we're here, we bump the
other linter dependencies and remove a note regarding the order of
dependencies, which is no longer true with the new dependency resolver
introduced in pip 20.3. We also remove an import exception for six.moves
since we no longer use six.
Change-Id: I4aae75f513568126230becf27b2e07d6682d35a1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This repo does not support Python 2 anymore, so we don't need
six for compatibility between Python2 and 3, convert six usage to Python
3 code.
Change-Id: Icba56808f38277b27af2ae5aac4b8507dee71b3b
This patch removes a method that wasn't being
used anymore anywhere except for a single unit
test. Since keystone now validates Fernet
tokens the same way - by rebuilding the token
context at validation time, we no longer need
to persist certain types of revocation events.
Change-Id: Ic9dcc6a5fd40e504ec04dcefab995d03927362bc
partial-bug: #1671887
Basic conversion of Keystone's core application to flask framework.
This doesn't add much in the way of flask-specific-isms but should
get keystone running directly under flask. This implementation does
not use paste-deploy.
Change-Id: Ib4c1ed3f645dd55fbfb76395263ecdaf605caae7
The INVALIDATE_USER_TOKEN_PERSISTENCE and
INVALIDATE_USER_PROJECT_TOKEN_PERSISTENCE callbacks were meant to
clean up invalid tokens from the token storage layer. Now that the
sql token driver has been removed, we don't need them any more. This
commit removes those notifications and refactors the places where
notifications are still needed, making them more specific and not
eluding to token persistence.
This commit also removes a significant amount of logic from the
assignment API that used to notify the token API when assignments
were deleted. This made sense when tokens were written to disk
because there was an opportunity to invalidate them when users were
removed from projects. This is no longer needed since we do
validation online and we don't persist tokens anymore.
Change-Id: I100b7416e8ba61eb4ea2c2eb4962e952a53ea388
The only API we're supporting on the v2.0 path until the T release is
the ec2tokens API. This commit removes all routers from the public
and admin v2.0 applications. This includes the extensions API. This
commit also removes unused v2.0 controller logic.
Change-Id: I523c1215899ac9ee605df6bf717643c0ba87c761
Closes-Bug: 1746798
This change converts the usage of self.<provider_api> to
keystone.common.providers_api.ProviderAPIs.<provider_api> in manager
and controller logic. This is the correct way to reference
providers from other managers and controllers now that dependency
injection has been eliminated.
Change-Id: I262536f96c5a1944308252ee2834082b66b1a8cd
Refactors all of keystone's dependency injection to maintain a
single centralized repository of instantiated objects. This
means that we are no longer having to resolve order. All
objects that need to reference the various manager APIs simply
do so via the __getattr__ built into the Manager common object
or the ProviderAPIMixin object.
This is also the first step towards correcting our tests to
where they cannot run "load_backends" multiple times.
This forces any/all managers to properly run super()
as the way to register the api is via __init__.
This eliminates all use of the @dependency.requires and
@dependency.provides decorators, simplifying the objects
all around.
Any instantiations of a Manager after keystone is running
will now generate an error, ensuring everything for keystone
is running before handling requests. An exception is for
CLI and CLI tests, as the CLI may directly instantiate
managers and will not lock the registry.
Change-Id: I4ba17855efd797c0db9f4824936b49e4bff54b6a
Keystone's performance degrades as the `revocation_event` table grows
in size. This patch reduces the total number of events written to the
table by not persisting events when a domain or project is disabled.
The main reason for persisting a revocation event when a project or
domain is disabled is to make sure tokens associated to those targets
are considered invalid. Instead of relying on revocation events, we
can check if the project or domain is enabled when we validate the
token. We take the same approach when we validate a user's role
assignments instead of relying on an ever-growing database table.
Co-Authored-By: Lance Bragstad <lbragstad@gmail.com>
Closes-Bug: 1524030
Change-Id: I76330567e0df2d9f2af88ef9b6b98b8c379e7406
This patch removes a method that wasn't being used anymore anywhere
except for a single unit test. In addition, we no longer need to store
the revocation event when the token provider API will rebuild the
token context at validation time, and this revocation method isn't being
exposed to an external API.
Among some of the test coverage for this can be found here:
https://github.com/openstack/keystone/blob/master/keystone/tests/unit/test_v3_auth.py#L2030-L2060
Change-Id: Ie92b238b0968a23bca3f0f57879369ea74298b8d
partial-bug: 1671887
With [1], we no longer need revoke_by_grant callback. It isn't being
used anywhere and token providers rebuild tokens at validation time.
[1]: https://review.openstack.org/#/c/447562/
Change-Id: I4e7e2f29d3db0eb8486173d4fb9134d61aab6dab
partial-bug: 1671887
This is no longer needed since the token provider API will
rebuild the token context at validation time. The revocation event
is not needed and we no longer need to store it.
Change-Id: I91315f620534974ab1102d693fbdff45e4ae8887
partial-bug: 1671887
This is no longer needed since the token provider API will
rebuild the token context at validation time. The revocation event
is not needed and we no longer need to store it.
Change-Id: I4dc766981a29b0afd0a44718c1c5d81155163982
partial-bug: 1671887
The openstack.org pages now support https and our references to
the site should by default be one signed by the organization.
Change-Id: I30a462e03d1fd7852511e22cac34c6bc0e8917f4
Do not call `to_dict` outside of a session context as if to_dict
interacts with lazy-loaded relationships it can cause errors. For
the most part these are edge-cases and unlikely to happen.
A couple FIXMEs were added to restructure the calls to allow for
`to_dict` to be moved into a method that will act within a session
context.
Change-Id: I769c2cdea1b08a780093d27cdc70bce9f004017b
This patch improves check_token validation performance by only pulling
revocation events based on the token issued_at value, taking advantage
of the table index. In this way, only a subset of relevant events will
be returned for validation.
Benchmarks can be seen at [1], but included here as well:
Time per Request for Old Method
-------------------------------
10 revokes at 7.908
100 revokes at 18.224
1,000 revokes at 110.155
10,000 revokes at 1998.220
Time per Request New Method
---------------------------
10 revokes at 17.636ms,
100 revokes at 17.279ms,
1,000 revokes at 17.370,
10,000 revokes w/all revokes issued before token at 17.263 (best case)
10,000 revokes w/all revokes after token creation 44.934ms (worst case)
[1] https://gist.github.com/csrichard1/4b7b8527ee5a6565a84956cff33cf29b
Change-Id: I9c2f067d870d542ec5909eaf8b24ded07b75f433
Partial-Bug: 1524030
This patch adds indexes to the revocation_event table as part of an
effort to improve performance during token validation.
Partial-Bug: 1524030
Change-Id: I73ff077bb6dc3ca8821f8cc14639bf986517d158
dogpile.cache's region invalidation is not designed to work across
processes. This patch enables distributed invalidation of keys in a
region.
Instead of using a static cache key, we use the original cache key
and append a dynamic value to it. This value is looked up in
memcached using the region name as a key. So anytime the value of
the region key changes the cache keys in that region are
effectively invalidated.
Closes-Bug: #1590779
Change-Id: Ib80d41d43ef815b37282d72ad68e7aa8e1ff354e
Since the API docs have been moved to api-ref, and the docs maintained
in keystone-specs repo have been moved to attic. The href attribute
that points to old `specs.openstack.org` will not be a valid link, this
patch update them to point to the new site.
Change-Id: Idc4995509d262c74ce6d49c6c18b6452fae186d6
The context['query_string'] is just a dictionary copy of the original
params object that comes from a request. Just use the existing params
instead.
Change-Id: I0ecd7a09e36b39a105c150b3affcbbcd26a544c2
keystone.common.config is 1200+ lines of super dense, merge-conflict
prone, difficult to navigate, and finicky to maintain code. Let's follow
nova's lead and break it down into more manageable modules.
This patch creates a new Python package, keystone.conf, and moves all of
our configuration options into it, mirroring nova's nova.conf package.
There are a couple special modules in keystone.conf introduced here as
well:
- keystone.conf.__init__: This causes all of Keystone options to be
registered on import, so consumers of keystone.conf don't have
races with config initialization code while trying to use
oslo_config.cfg.CONF directly (keystone.conf replaces all uses for
oslo_config.cfg.CONF in keystone).
- keystone.conf.base: Keystone's [DEFAULT] group options. I'd prefer
this to be called 'default.py', but I'm just copying nova's lead here.
- keystone.conf.opts: The entry point for oslo.config itself.
- keystone.conf.constants: There are a few constants (deprecation
messages, default paths, etc) that are used by multiple configuration
modules, so they need to live in a common place.
Change-Id: Ia3daffe3fef111b42de203762e966cd14d8927e2
Instead of the unformed context dictionary pass a full request object
with access to the context_dict so that existing functions still work.
After this we can replace smaller usages of the context dict with
functions and properties on the request directly.
Change-Id: Ibe822ed7c76a24a7d31d98ce62f873a01e5fb213
The RevokeTree was built out of an attempt to optimize the search for
a match between a candidate token and the list of revocation events.
The performance proved to be poor, mostly due to the cost of creating
and checking hash values.
The RevokeTree code is also so complex that most of the team could not
understand it or troubleshoot it. There are some subtle bugs due to
race conditions with revocation events, and it is impossible to track
them down due to the code complexity.
This change replaces the tree based search with a linear search through
the list of revocation events. A failure-to-match will pass through
the entire list. A revoked token should match on O(n/2) comparisons.
With the past year of Fernet tokens in deployment, the feedback is that
the number of revocation events is small, and they only are kept for
the lifetime of the tokens (usually 1-8 hours) so the linear search is
not expected to slow down token validations in live deployments.
Future work will also reduce the number of revocation events.
Change-Id: Ib6a686494e897840b09d134ecf1ca50ce712f281
revoke_by_expiration is only useful if a token does not have an
audit_id or audit_chain_id. tokens always have an audit ID, so
this function seems redundant. it was also deprecated in the
J release, with no timeline for its removal
Change-Id: Ieb92a70ab782fa8ceb59dc807ea8647587be9e2b
bp: removed-as-of-newton
This patch moves the revoke abstract base class out of core and into
backends/base.py
This removes dependencies where backend code references code in the
core. The reasoning being that the core should know about the backend
interface, but the backends should not know anything about the core
(separation of concerns). And part of the risk here is a potential for
circular dependencies.
Partial-Bug: #1563101
Change-Id: I0550e0fedfb28b16e9803d225df4d2337e6c65e6
Currently tox ignores D401.
D400: First line should end with a period.
This change removes it and make keystone docstring compliantwith it.
Change-Id: I9a9520e69701718ff471eebbcc52199dacdd9c68
Currently tox ignores D401 (401: First line should be in imperative mood).
This change removes it and make keystoneauth docstring compliantwith it.
Change-Id: I136cf810f47c4c19f29216907a63f226930b5082
Partial-Bug: 1570049
Modified the revocation_event id column to be an auto-incremented
integer. The id column was a varchar(64), however it is only used
internally and an integer column will help with large deletes.
Change-Id: Ibbac98ec50583579ca8fcb8f928d4882f571b07c
Closes-Bug: 1290625
Use a request local cache, where possible, to offload requests
to the backend.
Change-Id: I8b744f75f21e9dd669a735a7717fd0d044d6d8ed
Depends-On: I6d1d28f5b974e79d44d1e86ea53c666e3f5771df
EngineFacade is deprecated. This partially switches keystone to
use oslo.db.sqlalchemy.enginefacade. 'get_session' and 'get_engine'
methods are still used in sql migrations and related tests.
Change-Id: I221232d50821fe2adb9881f237f06714003ce79d
Partial-Bug: #1490571
`dict.get()` returns `None` by default, if a key wasn't found.
Removing `None` as second argument to avoid redundancy.
Change-Id: I810854f50257b9dc805248ff1017521cc82d5d73
Extensions links are out-of-date. They are either
invalid or pointing to an old repository [1].
This patch fixes such links.
[1] https://github.com/openstack/identity-api
Change-Id: I6cd461b3ba24a042fd188d6398212ded582eb365
Keystone API routers are exposed at their package
level (in __init__.py files). This causes them to
be unnecessarily executed each time something
whithin that package is used.
For example, simply importing
keystone.federation.constants would make the
federation routers code to be executed.
This patch remove routers exposure from package
level and import them directly in services.py,
which is the single place that needs them.
Change-Id: If68184c871ac77659ad2e64aa5f0aafac7a4bf70
There are several issues in the docstring, the format is not
correct, not using the full path of class etc.
This patch corrects all of them, so the docstring will be rendered
correctly.
Change-Id: I04d5818f38b5e75b6f6197ef5e13dcd64ed91bf4
Remove revoke as an extension and move it to a core resource.
For now we leave the database migrations in the extension directory
until we have a general policy for merging these into core.
DocImpact: update keystone-paste and remove revoke from pipeline
Change-Id: I2d6f425a508b7acb4b4d079e4387f25bf7555683
Implements: bp move-extensions
Alexandre Arents