A primary key is automatically unique, therefore if one or columns is
included in a primary key constraint there is no need to add a separate
unique constraint for these columns. Remove it.
Note that this only affects MySQL. Both SQLite and PostgreSQL appear to
ignore the duplicate unique constraint. As a result, it was necessary to
run auto-generation against MySQL instead of the default SQLite. The
actual command used was similar to what we normally do, however.
$ python keystone/common/sql/migrations/manage.py revision \
--autogenerate --message 'Remove duplicate constraints'
As always, the resulting schema migrations then needed some manual
tweaks to remove "please adjust!" comments and unnecessary imports but
they are correct.
Change-Id: I64252086f994901a5ebe05afec37a6afd3a192ee
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following LegacyAPIWarning warning:
The Query.get() method is considered legacy as of the 1.x series of
SQLAlchemy and becomes a legacy construct in 2.0. The method is now
available as Session.get()
Change-Id: I30d0bccaddff6a1d91fcd5660f490f904e7c8965
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Fix problems found.
Update local hacking checks for new flake8.
Change-Id: Ic440219814ee0c2b98217e9a821f38f5baf482ec
This repo does not support Python 2 anymore, so we don't need
six for compatibility between Python2 and 3, convert six usage to Python
3 code.
Change-Id: Icba56808f38277b27af2ae5aac4b8507dee71b3b
This reverts commit 3d46c8a5d9.
In the last commit, the foreign key constraints between the project
table and other tables were dropped, which allows us to restore the
configurability of the resource driver.
Change-Id: Iba4951e2d3965be5acec705385967d312456f1c7
Add in support for immutable roles and projects (including domains).
If the immutable option is set for a role or a project that
resource may not:
* Be Deleted
* Be Updated, except to change the value of "immutable" from
`True` to `False` or `None` (None explicitly unsets the
resource option).
* For projects (and domains), project tags cannot be created,
updated, or deleted.
The immutable check is performed at the manager layer allowing
for exceptional code-cases to work directly with the driver.
Change-Id: I2027b1235a260b7ae5d66cbd6c369773d9e99876
Partial-bug: #1823258
Add in support for resource options for roles and projects (including
domains). No options are currently implemented for roles or projects.
Scaffolding has been implemented so that adding options should be
straight forward. This will allow for implementing options such
as an immutable flag.
As a mechanism to isolate SQL Models from the Driver implementation
especially when adding in complexity of the resource options, the
models for the Resource backend and the Role Backend (SQL) have been
move to their own module.
Partial-Bug: #1807751
Depends-On: https://review.opendev.org/678379
Required-By: https://review.opendev.org/678380
Change-Id: I456a7c19506d28d5846534f884b8abe0d3079c96
This allows domain_ids to match across distinct Keystone
deployments The domain_id is used to create unique
identifiers with the mapping backend. When this
option is used, mapped user identifiers can be
consistant across different Keystone servers.
closes-bug: 1794527
Change-Id: I100bca162e71a9d394ed5787b976b13b1e57987f
Now `domain` is considered as the first level of project depth.
Update the check function to make sure the depth check is correct.
Change-Id: I38411001271518b9c2f58d53b2f654c361e952e8
bp: domain-level-limit
The [assignment] driver now defaults to sql.
Logic to determine the default assignment
driver if one wasn’t supplied through
configuration has been removed. Same for
[resource] driver.
This patch removes the func default_assignment
_driver used nowhere except test case
from assignment as well as resource.
Change-Id: Idb6c96dcabe9d93ecce6b8857f8f5bebba9dee27
The request input format validation now is handled by jsonschema
check. The clean file is useless now.
Change-Id: I9806722cca8bcd1d4c73618cf1b36107929d37b0
Convert the projects API to Flask native dispatching.
Change-Id: I3406284acfb7950b701f6a98a3a173a427415f97
Co-Authored-By: Morgan Fainberg <morgan.fainberg@gmail.com>
Partial-Bug: #1776504
The unit test uses sqlite for test which closes db foreign keys
function by default. This patch enabled the sqlite foreign keys
function for unit test by default.
The "project" table is a self referencing FK table(id <-> domain_id
column). So when the FK is enabled, there must exists a root record
before insert data to this table. It's <<keystone.domain.root>>.
Usually, the <<keystone.domain.root>> recored is inserted into the
table once operators run "keystone-manage db_sync" command when
deploy Keystone. But the unit test code doesn't run this command,
it initialise the db schema by reading sqlalchemy object model, so
the <<keystone.domain.root>> record is missed. Then we can't create
any project record, it'll raise FK error.
So in this patch, before creating any projects in the test, we must
ensure the <<keystone.domain.root>> record exists first.
Change-Id: I565d12395ca39a58ba90faf8641a9e02d986aeb9
Closes-Bug: #1744195
This patched removed some redundant "get_project" calls when
deleting projects/domains.
Change-Id: Ife4dd18962077bac30fa1cecf7621cc86a62929c
Closes-bug: #1764282
When deleting a project, the limits belong to the project should
be deleted as well.
Change-Id: I5f7eed9bf4f69cb7d79e44acac27c6c5881be22e
Closes-Bug: #1779903
Add a check function for project hierarchical tree check
when starting Keystone. If the tree depth exceed the
enforcement model's depth, fail to start keystone process
Change-Id: I4ce6a48505b8b9688bbdd18ee46ce035ee0938ed
bp: strict-two-level-model
when a hierarchical limit enforcement model is deployed
by keystone, the project hierarchical depth should
not break the limit model as well.
This patch updated the project depth check function
to fix the problem.
bp: strict-two-level-model
Change-Id: I695fedcf83bdca2946a2c5d876052c02b0f63810
Add include_limits filter for get project to support
fetching project hierarchy limits.
This filter should be used together with "subtree_as_list"
or "parents_as_list" filter
bp: strict-two-level-model
Change-Id: Ib602887c92b89be0ffec1394a3076f5dd5671511
Without it, listing projects results were missing project on which the
user had an inherited role.
Change-Id: If8edb3d1d1d3a0dab691ab6c81dd4b42e3b10ab3
Closes-Bug: #1780159
Basic conversion of Keystone's core application to flask framework.
This doesn't add much in the way of flask-specific-isms but should
get keystone running directly under flask. This implementation does
not use paste-deploy.
Change-Id: Ib4c1ed3f645dd55fbfb76395263ecdaf605caae7
This patch changed the sql query for domain list
to ensure the return count is correct if list_limit
is set for resource.
Closes-bug: #1760521
Change-Id: I690a9815fe2c35e6191ab3ac789e34736e0d637c
Currently when using the "tags" filter to search projects by tag names,
the filter only matches projects that have an exact match. Projects that
contain the exact tags given, but with additional tags are excluded.
This behavior is not compatible with the use cases defined in both the
keystone[0] and api-wg[1] specs, notibly with the "tags" and "tags-any"
interaction.
This change makes it so that "tags" filtering will be performed by
matching a subset containing given tags against projects, rather than
exact matching. This allows the "tags" and "tags-any" filters to work as
described in both [0] and [1].
[0] https://specs.openstack.org/openstack/keystone-specs/specs/keystone/queens/project-tags.html
[1] https://specs.openstack.org/openstack/api-wg/guidelines/tags.html#filtering-and-searching-by-tags
Co-Authored By: Nicolas Helgeson <nh202b@att.com>
Closes-Bug: #1756190
Change-Id: I632efdf0af2969be0a59dc5928a6c036eeca6051
This change makes tags a property of Project instead of a hybrid_property
since we will always have a Project contain some list of tags.
Change-Id: I1033321132cb3ec71bf94b8293cef91dfc6b8272
Co-Authored-By: Morgan Fainberg <morgan.fainberg@gmail.com>
Closes-Bug: #1752301
The INVALIDATE_USER_TOKEN_PERSISTENCE and
INVALIDATE_USER_PROJECT_TOKEN_PERSISTENCE callbacks were meant to
clean up invalid tokens from the token storage layer. Now that the
sql token driver has been removed, we don't need them any more. This
commit removes those notifications and refactors the places where
notifications are still needed, making them more specific and not
eluding to token persistence.
This commit also removes a significant amount of logic from the
assignment API that used to notify the token API when assignments
were deleted. This made sense when tokens were written to disk
because there was an opportunity to invalidate them when users were
removed from projects. This is no longer needed since we do
validation online and we don't persist tokens anymore.
Change-Id: I100b7416e8ba61eb4ea2c2eb4962e952a53ea388
Since the users table has a foreign key to the projects table[1], users
must be deleted before the domain can be deleted. However, the
notification emitted from the domain deletion comes too late, and
keystone runs into a foreign key reference error before it can delete
the users. This patch addresses the problem by adding a new internal
notification to alert the identity manager that users should be deleted.
This uses a new notification rather than the existing notification
because the existing one is used to alert listeners that the domain
deletion has been fully completed, whereas this one must happen in the
middle of the domain delete process.
The callback must also only try to delete SQL users. The LDAP driver
doesn't support deleting users, and we can't assume other drivers
support it either. Moreover, the foreign key reference is only a problem
for SQL users anyway.
Because our backend unit tests run with SQLite and foreign keys do not
work properly, we can't properly expose this bug in our unit tests, but
there is an accompanying tempest test[2][3] to validate this fix.
[1] https://github.com/openstack/keystone/blob/2bd88d3/keystone/common/sql/expand_repo/versions/014_expand_add_domain_id_to_user_table.py#L140-L141
[2] https://review.openstack.org/#/c/509610
[3] https://review.openstack.org/#/c/509947
Change-Id: If5bdb6f5eef80b50b000aed5188ce7da4dfd1083
Closes-bug: #1718747
This change converts the usage of self.<provider_api> to
keystone.common.providers_api.ProviderAPIs.<provider_api> in manager
and controller logic. This is the correct way to reference
providers from other managers and controllers now that dependency
injection has been eliminated.
Change-Id: Id0305ce4e782dc8e8977dc7f802110148da2c901
Refactors all of keystone's dependency injection to maintain a
single centralized repository of instantiated objects. This
means that we are no longer having to resolve order. All
objects that need to reference the various manager APIs simply
do so via the __getattr__ built into the Manager common object
or the ProviderAPIMixin object.
This is also the first step towards correcting our tests to
where they cannot run "load_backends" multiple times.
This forces any/all managers to properly run super()
as the way to register the api is via __init__.
This eliminates all use of the @dependency.requires and
@dependency.provides decorators, simplifying the objects
all around.
Any instantiations of a Manager after keystone is running
will now generate an error, ensuring everything for keystone
is running before handling requests. An exception is for
CLI and CLI tests, as the CLI may directly instantiate
managers and will not lock the registry.
Change-Id: I4ba17855efd797c0db9f4824936b49e4bff54b6a
With the removal of the V2.0 API in Queens, this functionality
was only used with the v2 resource controller and can now be
removed. Also removed the associated tests.
Change-Id: Id0503bc0610cff5fb635cd021088658bc06aef26
This change adds the new API routes for project tags as well as
the controller methods for listening for /v3/project/[id]/tags.
Change-Id: Ic2e5874a427341f2edb6e9122386fb98af2c24ef
Partially-Implements: bp project-tags
Co-Authored-By: Jaewoo Park <jp655p@att.com>
Co-Authored-By: Nicolas Helgeson <nh202b@att.com>
Depends-On: I00f094a5584be40ab477cbf680a5f6d1afb4d21b
Depends-On: Ieb68bd2c9c216b25ad74d320a1c9a297d2b251e7
This change adds the tags attribute of project into the resource
manager. This change builds off of the backend logic.
Change-Id: Ie988b7a2065f0ecdb8ec953a1d01f5e3cbd67a1b
Partially-Implements: bp project-tags
Co-Authored-By: Jaewoo Park <jp655p@att.com>
Co-Authored-By: Nicolas Helgeson <nh202b@att.com>
Depends-On: I00f094a5584be40ab477cbf680a5f6d1afb4d21b
Depends-On: I5f8e4a53089b9fcc38084bb958d09f63ccc59d2a
This change adds in tags as a hybrid property for projects.
Also adds in relevant tests for backend logic.
Change-Id: Ie153130b794e3541f197f8fa1e73bf30a64f12dc
Partially-Implements: bp project-tags
Co-Authored-By: Jaewoo Park <jp655p@att.com>
Co-Authored-By: Nicolas Helgeson <nh202b@att.com>
Depends-On: I00f094a5584be40ab477cbf680a5f6d1afb4d21b
Depends-On: Ib3db74cb08e7fdec01d55d8988e0906948d80a32
This change adds json schema for project tags validation, with
the limits for number of characters per tag as well as the
limit for the number of tags a project can have per update.
Co-Authored-By: Jaewoo Park <jp655p@att.com>
Co-Authored-By: Nicolas <nh202b@att.com>
Change-Id: I5f8e4a53089b9fcc38084bb958d09f63ccc59d2a
Partially-Implements: bp project-tags
When is_domain is false, the project must contain a valid
domain_id and parent_id. We check the domain_id specified
is consistent with the domain of its parent.
The log messages is not very clear, this patch optimize it.
Change-Id: Ie83d295f036f94d36c9dfa88ae03e074cdfd9810
This change has been implemented to avoid the need to backport
signficantly impactful Foreign Key dropping backports.
Resource is highly relational data and it makes sense to allow the
use of FKs from other subsystems to project/domains.
Change-Id: Ic3831d1c7ae41fe4d406d60a013770cc1258584f
Listing projects and domains for a user based on their role
assignments was noted as being really slow, especially when users
have a lot of assignments. This commit implements caching to mitigate
the issue while we continue to investigate ways to speed up the
assignment API.
Change-Id: I72e398c65f01aa4f9a37f817d184a13ed01089ce
Closes-Bug: 1700852
The domain configuration API had several GET calls that didn't
support HEAD. The documentation for the added HEAD APIs is going
to be proposed at a later date as a blanket statement saying all
APIs the support GET also support HEAD. This reduces duplication
of documentation for a relatively straight-forward concept.
This commit also fixes some inconsistencies in the domain
configuration documentation with colons and newlines.
Change-Id: I7a6ab2f05600c8d562d5be83d651ff735c9a68a4
Partial-Bug: 1696574