We propose to extend Keystone identity provider (IdP) attribute mapping
schema to make Keystone honor the `domain` configuration that we have
on it.
Currently, that configuration is only used to define a default domain
for groups (and then each group there, could override it). It is
interesting to expand this configuration (as long as it is in the root
of the attribute mapping) to be also applied for users and projects.
Moreover, to facilitate the development and extension concerning
attribute mappings for IdPs, we changed the way the attribute mapping
schema is handled. We introduce a new configuration
`federation_attribute_mapping_schema_version`, which defaults to "1.0".
This attribute mapping schema version will then be used to control the
validation of attribute mapping, and also the rule processors used to
process the attributes that come from the IdP. So far, with this PR,
we introduce the attribute mapping schema "2.0", which enables
operators to also define a domain for the projects they want to assign
users. If no domain is defined either in the project or in the global
domain definition for the attribute mapping, we take the IdP domain
as the default.
Change-Id: Ia9583a254336fad7b302430a38b538c84338d13d
Implements: https://bugs.launchpad.net/keystone/+bug/1887515
Closes-Bug: #1887515
The check_revocation method is called at least 3 times when validating
a token.
Each time, it's doing a heavy SQL statement depending on the size of the
revocation table.
We can save time by adding cache to this method.
Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
Change-Id: I70b4664905bb4360d792ba8bd701674f60538223
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Fix problems found.
Update local hacking checks for new flake8.
Change-Id: Ic440219814ee0c2b98217e9a821f38f5baf482ec
This patch adds functionality to allow an operator to pass in a
federated attribute when updating a user. When a user is updated
the federated objects in the federated attribute will be updated
and associated along with the user.
Co-Authored-By: Kristi Nikolla <knikolla@bu.edu>
Partial-Bug: 1816076
Change-Id: I8ee43b437b551858c198320204b768cdba311506
This patch adds functionality to allow an operator to pass in a
federated attribute when creating a user. When a user is created
the federated objects in the federated attribute will be created
and associated along with the user.
Co-Authored-By: Kristi Nikolla <knikolla@bu.edu>
Partial-Bug: 1816076
Change-Id: I6db03af81099a7509635881f05adf5a7257466a7
This patch adds functionality to get_user that allows it to pull all
associated federated objects and tack it on to be displayed to the
user.
Partial-Bug: 1816076
Change-Id: I8d69ef68153d6650652e1081e5e7b9e5e31a3ed1
When a federated user authenticates, they are added to their
mapped groups during shadowing.
Closes-Bug: 1809116
Change-Id: I19dc400b2a7aa46709b242cdeef82beaca975ff3
Modify the base driver and SQL driver to support expiring group
memberships.
Additions to the SQL Driver to support listing expiring groups
for user.
Change-Id: I7d52cd2003f511483619a429de57201df4990209
Partial-Bug: 1809116
Depends-On: I4294a879071dde07e5eb1da4df133de8032e1059
As LDAP is now read-only, trying to remove it was throwing an error.
We now only try to delete it when the driver is sql-based.
Change-Id: I15b92b35b31d0e5d735a629e7c154ddd7bdda03d
Closes-bug: #1848238
If we do not pass kwargs to exception, the parameter will be deemed
as message and will be displayed directly. This is to pass kwargs to
InvalidOperatorError and URLValidationError, to get better format of
error message.
Change-Id: I5220d7af077a6c2eb1fe49cbbd7421169fa5b015
Without this change, when an admin tries to delete an LDAP-backed
domain, it fails due to the foreign key relationship in the users table.
Previously, we were assuming that LDAP users existed solely in the LDAP
directory, but this is not true with shadow users. This patch fixes the
logic to delete the shadow users upon domain deletion.
Change-Id: I12a08001e3aa08e4db9438cae425ad1a0a8070f7
Closes-bug: #1801873
The request input format validation now is handled by jsonschema
check. The clean file is useless now.
Change-Id: I9806722cca8bcd1d4c73618cf1b36107929d37b0
Convert /v3/users to use flask native dispatching.
The following test changes were required:
* Application Credentials did not have the plural form
in the JSON Home document. The JSON Home document was
corrected both in code and in tests.
* Application Credentials "patch" test needed to be
refactored to look for METHOD_NOT_ALLOWED instead
of NOT FOUND for invalid/unimplemented methods.
The "assertValidErrorResponse" method was
insufficient and the test now uses the flask
test_client mechanism instead.
Change-Id: Iedaf405d11450b11e2d1fcdfae45ccb8eeb6f255
Partial-Bug: #1776504
Convert the /auth paths to flask native dispatching.
A minor change to additional_urls was implemented to ensure all
urls are added at once instead of individually (causing an over-
write issue within flask as a single resource may only have a
single set of URL mappings).
Alternate URLs now support adding alternate JSON Home rel links.
This is to support the case of OS-FEDERATION auth routes moving
to /auth. The old JSON Home entries must exist but reference
the new paths.
This port includes the following test changes (needed due to the
way flask handles requests and the way requests are passed through
the auth system):
* Implemented keystone.common.render_token (module)
containing render_token_response_from_model and use it instead
of keystone.common.controller.render_token_response_from_model.
Minor differences occur in render_token_response_from_model in
the keystone.common.render_token module, this is simply
for referencing data from flask instead of the request object.
* Test cases have been modified to no longer rely on the auth
controller(s) directly
* Test cases now use "make_request" as a context manager
since authenticate/authenticate_for_token directly
reference the flask contexts and must have an explicit
context pushed.
* Test cases no longer pass request objects into methods
such as authenticate/authenticate_for_token or similar
methods on the auth plugins
* Test cases for federation reference the token model now
where possible instead of the rendered token response.
Rendered token responses are generated where needed.
* Auth Plugin Configuration is done in test core as well.
This is because Auth controller does not exist.
NOTE: This is a massive change, but must of these changes
were now easily uncoupled because of how far reaching auth
is.
Change-Id: I636928102875760726cc3493775a2be48e774fd7
Partial-Bug: #1776504
Keystone uses translated strings both in
logging an exceptions. This is incorrect.
All strings that are passed to logging
should remain un-translated. This patch
addresses above issue.
Change-Id: Idf4f0bc1bd63eb8dc6dc61d8a49a9e2a93320474
Closes-Bug: #1777671
This change refactors _handle_shadow_and_local_users to remove
an extra looping over the hint filters, as well as moving the
deepcopy down to only be performed if needed.
Also modifies a test to indicate that this change does not
break behavior as pointed out here[0].
[0] https://review.openstack.org/#/c/553880/
Change-Id: I30e434facf5b8720dd19194dd7e99ff28ea6661e
This change refactors both functions listed for single refs and
a list of refs for _set_domain_id_and_mapping to do separate checks
for conditions such as _is_domain_aware and _is_mapping_needed. This
is due to how it is currently handled, it uses the same logic between
both functions, which provides more overhead for a list than a single
ref.
Change-Id: I325627f511fe7d52a191e64eada030e84d756d7e
with many users and groups in a domain fetching all mappings (for both
users and groups) may become inefficient.
In an environment with approx 125k users and 150 groups in the mapping
table and SAML2+LDAP auth/backend, this patch reduced the time
for first (uncached) 'openstack token issue' command from 12 to 3 seconds.
Similar improvements were seen with time to login to Horizon as well.
Change-Id: Iccbef534ff7e723f8b1461bb1169e2da66cc1dea
Closes-Bug: #1775207
When deleting a user, the cache for the related shadow user should
be invalidated as well. Otherwise the federation authentication
will not work well and will raise 404 UserNotFound error.
This patch fixes the bug and adds a new function for shadow backend
to get the shadow user information.
Change-Id: I3882f0dc6e8f8f618bb89ebd699736bc4b352261
Closes-bug: #1760205
When the federated rule contains 'email' in user and we should set
email for the federated user. Also, if the federated user changes the
email info, it should be chenged too.
Change-Id: Ib17172c34bd65d5236cbfc192b3a3f2b221411ef
Closes-Bug: #1746599
The INVALIDATE_USER_TOKEN_PERSISTENCE and
INVALIDATE_USER_PROJECT_TOKEN_PERSISTENCE callbacks were meant to
clean up invalid tokens from the token storage layer. Now that the
sql token driver has been removed, we don't need them any more. This
commit removes those notifications and refactors the places where
notifications are still needed, making them more specific and not
eluding to token persistence.
This commit also removes a significant amount of logic from the
assignment API that used to notify the token API when assignments
were deleted. This made sense when tokens were written to disk
because there was an opportunity to invalidate them when users were
removed from projects. This is no longer needed since we do
validation online and we don't persist tokens anymore.
Change-Id: I100b7416e8ba61eb4ea2c2eb4962e952a53ea388
Since the users table has a foreign key to the projects table[1], users
must be deleted before the domain can be deleted. However, the
notification emitted from the domain deletion comes too late, and
keystone runs into a foreign key reference error before it can delete
the users. This patch addresses the problem by adding a new internal
notification to alert the identity manager that users should be deleted.
This uses a new notification rather than the existing notification
because the existing one is used to alert listeners that the domain
deletion has been fully completed, whereas this one must happen in the
middle of the domain delete process.
The callback must also only try to delete SQL users. The LDAP driver
doesn't support deleting users, and we can't assume other drivers
support it either. Moreover, the foreign key reference is only a problem
for SQL users anyway.
Because our backend unit tests run with SQLite and foreign keys do not
work properly, we can't properly expose this bug in our unit tests, but
there is an accompanying tempest test[2][3] to validate this fix.
[1] https://github.com/openstack/keystone/blob/2bd88d3/keystone/common/sql/expand_repo/versions/014_expand_add_domain_id_to_user_table.py#L140-L141
[2] https://review.openstack.org/#/c/509610
[3] https://review.openstack.org/#/c/509947
Change-Id: If5bdb6f5eef80b50b000aed5188ce7da4dfd1083
Closes-bug: #1718747
When attempting to filter users by name, it works
for local users, but doesn't work for federated users.
This patch fixed this error.
Change-Id: I1bee51c2be81dbddd9d849731ab53728c86b2765
Closes-bug: #1738895
This change converts the usage of self.<provider_api> to
keystone.common.providers_api.ProviderAPIs.<provider_api> in manager
and controller logic. This is the correct way to reference
providers from other managers and controllers now that dependency
injection has been eliminated.
Change-Id: I4c9428f916fd28ee6df701aee26c5e2c516c4913
Refactors all of keystone's dependency injection to maintain a
single centralized repository of instantiated objects. This
means that we are no longer having to resolve order. All
objects that need to reference the various manager APIs simply
do so via the __getattr__ built into the Manager common object
or the ProviderAPIMixin object.
This is also the first step towards correcting our tests to
where they cannot run "load_backends" multiple times.
This forces any/all managers to properly run super()
as the way to register the api is via __init__.
This eliminates all use of the @dependency.requires and
@dependency.provides decorators, simplifying the objects
all around.
Any instantiations of a Manager after keystone is running
will now generate an error, ensuring everything for keystone
is running before handling requests. An exception is for
CLI and CLI tests, as the CLI may directly instantiate
managers and will not lock the registry.
Change-Id: I4ba17855efd797c0db9f4824936b49e4bff54b6a
Previously, the default behavior for the callback that unset
default project ids was to only call the method for the default
domain's identity driver. This meant that when a project was deleted,
only the default identity backend would have references to that
project removed. This means it would be possible for other identity
backends to still have references to a project that doesn't exist
because the callback wasn't invoked for that specific backend.
This commit ensures each backend clears project id from a user's
default_project_id attribute when a project is deleted.
Change-Id: Ibb5396f20101a3956fa91d6ff68155d4c00ab0f9
Closes-Bug: 1705072
The identity backend registers a callback that listens for when a
project is deleted. When it receives a notification, it uses the
project ID send in the notification and removes all references to it
from the identity backend, where users might have it referenced in
their `default_project_id` attribute. The original fix for this did
not account for LDAP backends being read-only. This caused an issue
where DELETE /v3/projects/{project_id} actually caused an HTTP 403
Forbidden exception because the LDAP backend wasn't writeable,
despite that project actually being deleted.
This change makes the identity API manager handle the exception
and tests it specifically for LDAP, or read-only, backends.
Change-Id: I16f4fcb289dad2fe752f3188476329c95cf777c9
Closes-Bug: 1705081
The usage of `enforce_type` from oslo.config will be removed in the
4.0 release of oslo.config. The default behavior has been
incorporated into set_override/set_default as of:
Ifa552de0a994e40388cbc9f7dbaa55700ca276b0
We no longer need to specify `enforce_type=True` and we should
remove it since it will be removed from oslo.config.
This commit also fixes violations with enforce_type=True.
Change-Id: I8222e84583aaa4de4c7c36ec8cec5e35c2e7e253
Related-Bug: 1517839
The revocation API was listed as a dependency of the identity API,
but it was never used. If it was no longer being used, we shouldn't
make the identity API load it.
Change-Id: I8137b1e9f7058572c1cf8de2ead4d5b42212f098
partial-bug: 1671887
Currently when a project is deleted, the project ID details
still exists in user information. After this fix, when a project
is deleted the default project ID in user
information will be cleared.
Closes-Bug: #1523369
Signed-off-by: Kalaswan Datta <kalaswan.datta@nectechnologies.in>
Change-Id: I3db0cf27d3cfdf6cf7c5bb34ec1b27ef80c139f4
When a file-config-based domain-specific backend was loaded the
local config files from /etc/keystone/keystone.conf.d/ were also read. The
local config dir should not be used in this case.
Change-Id: Ib576c8f12a7cc4272e07bb057bf028d69649b65d
Related-Bug: #1489118
When an SQL-config-based domain-specific backend was loaded the
local config dir (/etc/keystone/keystone.conf.d) is also read. The
local config files should not be used in this case.
This is a followup fix for Idd095b2df375329f579c164d00dfd50b41b0e96d
Related-Bug: #1489118
Change-Id: I14008656a538ca7641aefffe08b9d1c23b7b87d2
This patch updates the domain for federated users to be the domain of
the Identity Provider (IdP).
Closes-Bug: #1642687
Partially-Implements: bp support-federated-attr
Depends-On: If8c8ad39c4c55a2d800bf4432411db59799e84e6
Change-Id: Iccfad6f39dc339ca054bedf3c6882c3701dcf0ec
Code moving users, groups, and projects between domains has been
removed and errors consolidated at the manager level (instead of
the controller level).
Change-Id: I73e77cc6dd5081d25dcd85ff2d2a61555829066d
bp: removed-as-of-ocata
New query added for determining when passwords will expire.
The following are the new queries:
/v3/groups/{group_id}/users?password_expires_at={operator}:{timestamp}
- Lists users belonging to a group whose password will expire based
on the operator given.
/v3/users?password_expires_at={operator}:{timestamp}
- Lists users whose password will expire based on the operator given.
{timestamp} is a datetime in the format "YYYY-MM-DDTHH:mm:ssZ".
{operator} is one of lt, lte, gt gte, eq, and neq to filter in.
If no operator is given, it is treated as eq.
Examples:
- GET /v3/users?password_expires_at=lt:2016-11-06T15:32:17Z
- GET /v3/groups/079c578fd99b428ab61fcd4c9bd88ecd/users?password_expires_at=gt:2016-12-08T22:02:00Z
Partially-Implements: bp pci-dss-query-password-expired-users
Change-Id: If0b9cc3c8af92b2ea5d41a0e8afeb78e12b7689c
This patch adds filters to list_user that enable the user to query for
unique_id, idp_id, protocol_id, or a mix of these to get back the
corresponding users of the federated attributes.
Partially-Implements: bp support-federated-attr
Change-Id: Iea5681791e521e9b8d96137fe30c388c10a02b30
Currently, if a users password expires, they must contact an
administrator in order to have their password reset for them.
This change allows a user to perform the change_password call
without a token, which will allow a user with an expired password
to change it if they are using PCI-DSS related features. This
removes the issue of needing an administrator to reset any
user's password that has expired.
Also updated the api-ref with the related changes.
Change-Id: I4d3421c56642cfdbb25cb33b3aaaacbac4c64dd1
Closes-Bug: #1641645
This adds a reason to the CADF event notifications that are emitted
for the following events related to PCI-DSS:
- Change user passwords/passphrases at least once every X days
- Limit repeated access attempts by locking out the user ID after
not more than X attempts
- Do not allow an individual to submit a new password/phrase that
is the same as any of the last X passwords/phrases he or she has used
- Passwords/phrases must meet the specificed regex
- User attempting to change password early
Implements: bp pci-dss-notifications
Co-Authored-By: Tin Lam <tinlam@gmail.com>
Change-Id: Ia678d25bdfa151c95483f5fcb77853184fbecfd1
Zuul