Commit Graph

259 Commits

Author SHA1 Message Date
Stephen Finucane 535bc8e22e sql: Remove duplicate constraints
A primary key is automatically unique, therefore if one or columns is
included in a primary key constraint there is no need to add a separate
unique constraint for these columns. Remove it.

Note that this only affects MySQL. Both SQLite and PostgreSQL appear to
ignore the duplicate unique constraint. As a result, it was necessary to
run auto-generation against MySQL instead of the default SQLite. The
actual command used was similar to what we normally do, however.

  $ python keystone/common/sql/migrations/manage.py revision \
      --autogenerate --message 'Remove duplicate constraints'

As always, the resulting schema migrations then needed some manual
tweaks to remove "please adjust!" comments and unnecessary imports but
they are correct.

Change-Id: I64252086f994901a5ebe05afec37a6afd3a192ee
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-04-06 11:23:03 +01:00
Stephen Finucane 5d2ab6c63b db: Replace use of Query.get()
Resolve the following LegacyAPIWarning warning:

  The Query.get() method is considered legacy as of the 1.x series of
  SQLAlchemy and becomes a legacy construct in 2.0. The method is now
  available as Session.get()

Change-Id: I30d0bccaddff6a1d91fcd5660f490f904e7c8965
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-02-28 17:26:39 +00:00
Andreas Jaeger f36111954b Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found.

Update local hacking checks for new flake8.

Change-Id: Ic440219814ee0c2b98217e9a821f38f5baf482ec
2020-04-15 07:17:58 +02:00
Vishakha Agarwal 4530041931 Remove six usage
This repo does not support Python 2 anymore, so we don't need
six for compatibility between Python2 and 3, convert six usage to Python
3 code.

Change-Id: Icba56808f38277b27af2ae5aac4b8507dee71b3b
2020-01-30 06:06:51 +00:00
Colleen Murphy 9607ed3266 Revert "Resource backend is SQL only now"
This reverts commit 3d46c8a5d9.

In the last commit, the foreign key constraints between the project
table and other tables were dropped, which allows us to restore the
configurability of the resource driver.

Change-Id: Iba4951e2d3965be5acec705385967d312456f1c7
2019-10-11 14:13:04 -07:00
morgan fainberg a38d4a650f Add immutable option for roles and projects
Add in support for immutable roles and projects (including domains).
If the immutable option is set for a role or a project that
resource may not:

* Be Deleted

* Be Updated, except to change the value of "immutable" from
  `True` to `False` or `None` (None explicitly unsets the
  resource option).

* For projects (and domains), project tags cannot be created,
  updated, or deleted.

The immutable check is performed at the manager layer allowing
for exceptional code-cases to work directly with the driver.

Change-Id: I2027b1235a260b7ae5d66cbd6c369773d9e99876
Partial-bug: #1823258
2019-09-10 00:03:16 -07:00
morgan fainberg b31ff3f991 Implement resource options for roles and projects
Add in support for resource options for roles and projects (including
domains). No options are currently implemented for roles or projects.
Scaffolding has been implemented so that adding options should be
straight forward. This will allow for implementing options such
as an immutable flag.

As a mechanism to isolate SQL Models from the Driver implementation
especially when adding in complexity of the resource options, the
models for the Resource backend and the Role Backend (SQL) have been
move to their own module.

Partial-Bug: #1807751
Depends-On:  https://review.opendev.org/678379
Required-By: https://review.opendev.org/678380
Change-Id: I456a7c19506d28d5846534f884b8abe0d3079c96
2019-09-09 22:07:30 +00:00
Adam Young 4cd99e7197 Allow an explicit_domain_id parameter when creating a domain
This allows domain_ids to match across distinct Keystone
deployments The domain_id is used to create unique
identifiers with the mapping backend.  When this
option is used, mapped user identifiers can be
consistant across different Keystone servers.

closes-bug: 1794527

Change-Id: I100bca162e71a9d394ed5787b976b13b1e57987f
2019-04-09 16:29:52 +00:00
wangxiyuan 8667f42beb Update project depth check
Now `domain` is considered as the first level of project depth.

Update the check function to make sure the depth check is correct.

Change-Id: I38411001271518b9c2f58d53b2f654c361e952e8
bp: domain-level-limit
2019-02-19 11:09:13 +08:00
Vishakha Agarwal fe5ee0a36a Removing default_assigment_driver
The [assignment] driver now defaults to sql.
Logic to determine the default assignment
driver if one wasn’t supplied through
configuration has been removed. Same for
[resource] driver.

This patch removes the func default_assignment
_driver used nowhere except test case
from assignment as well as resource.

Change-Id: Idb6c96dcabe9d93ecce6b8857f8f5bebba9dee27
2018-11-13 07:45:28 +00:00
wangxiyuan e5def7c3ad Remove useless "clean" file
The request input format validation now is handled by jsonschema
check. The clean file is useless now.

Change-Id: I9806722cca8bcd1d4c73618cf1b36107929d37b0
2018-10-25 14:09:23 +08:00
Harry Rybacki 46380baeb7 Convert projects API to Flask
Convert the projects API to Flask native dispatching.

Change-Id: I3406284acfb7950b701f6a98a3a173a427415f97
Co-Authored-By: Morgan Fainberg <morgan.fainberg@gmail.com>
Partial-Bug: #1776504
2018-10-11 15:27:46 -07:00
wangxiyuan 012dac29b8 Enable foreign keys for unit test
The unit test uses sqlite for test which closes db foreign keys
function by default. This patch enabled the sqlite foreign keys
function for unit test by default.

The "project" table is a self referencing FK table(id <-> domain_id
column). So when the FK is enabled, there must exists a root record
before insert data to this table. It's <<keystone.domain.root>>.

Usually, the <<keystone.domain.root>> recored is inserted into the
table once operators run "keystone-manage db_sync" command when
deploy Keystone. But the unit test code doesn't run this command,
it initialise the db schema by reading sqlalchemy object model, so
the <<keystone.domain.root>> record is missed. Then we can't create
any project record, it'll raise FK error.

So in this patch, before creating any projects in the test, we must
ensure the <<keystone.domain.root>> record exists first.

Change-Id: I565d12395ca39a58ba90faf8641a9e02d986aeb9
Closes-Bug: #1744195
2018-10-09 09:50:21 +08:00
Gage Hugo 296f20f0a7 Convert domains api to flask
Convert domains api to flask native dispatching.

Change-Id: I974ab112986e6c242a139f7d7dc14fd54a48ee42
Partial-Bug: #1776504
2018-09-12 15:26:37 +00:00
wangxiyuan 44da48f013 Remove redundant get_project call
This patched removed some redundant "get_project" calls when
deleting projects/domains.

Change-Id: Ife4dd18962077bac30fa1cecf7621cc86a62929c
Closes-bug: #1764282
2018-08-07 14:26:30 +08:00
Zuul 7a45fd8671 Merge "Delete project limits when deleting project" 2018-07-25 02:34:21 +00:00
wangxiyuan 022217e003 Delete project limits when deleting project
When deleting a project, the limits belong to the project should
be deleted as well.

Change-Id: I5f7eed9bf4f69cb7d79e44acac27c6c5881be22e
Closes-Bug: #1779903
2018-07-19 03:49:45 +00:00
wangxiyuan 8e44aa3b97 Add project hierarchical tree check when Keystone start
Add a check function for project hierarchical tree check
when starting Keystone. If the tree depth exceed the
enforcement model's depth, fail to start keystone process

Change-Id: I4ce6a48505b8b9688bbdd18ee46ce035ee0938ed
bp: strict-two-level-model
2018-07-19 11:46:00 +08:00
wangxiyuan a7d4a9bd4c Update project depth check
when a hierarchical limit enforcement model is deployed
by keystone, the project hierarchical depth should
not break the limit model as well.

This patch updated the project depth check function
to fix the problem.

bp: strict-two-level-model
Change-Id: I695fedcf83bdca2946a2c5d876052c02b0f63810
2018-07-19 11:20:17 +08:00
wangxiyuan 8038c70abf Add include_limits filter
Add include_limits filter for get project to support
fetching project hierarchy limits.

This filter should be used together with "subtree_as_list"
or "parents_as_list" filter

bp: strict-two-level-model
Change-Id: Ib602887c92b89be0ffec1394a3076f5dd5671511
2018-07-19 11:20:12 +08:00
Sami MAKKI 83e72d7443 Invalidate 'computed assignments' cache when creating a project.
Without it, listing projects results were missing project on which the
user had an inherited role.

Change-Id: If8edb3d1d1d3a0dab691ab6c81dd4b42e3b10ab3
Closes-Bug: #1780159
2018-07-11 08:47:41 +02:00
Morgan Fainberg 4ec6bc5a44 Convert Keystone to use Flask
Basic conversion of Keystone's core application to flask framework.

This doesn't add much in the way of flask-specific-isms but should
get keystone running directly under flask. This implementation does
not use paste-deploy.

Change-Id: Ib4c1ed3f645dd55fbfb76395263ecdaf605caae7
2018-06-04 20:14:41 -07:00
wangxiyuan 245a0c718b Fix list_limit doesn't work correctly for domain
This patch changed the sql query for domain list
to ensure the return count is correct if list_limit
is set for resource.

Closes-bug: #1760521

Change-Id: I690a9815fe2c35e6191ab3ac789e34736e0d637c
2018-04-03 14:26:59 +08:00
Gage Hugo 4b572e564d Make tags filter match subset rather than exact
Currently when using the "tags" filter to search projects by tag names,
the filter only matches projects that have an exact match. Projects that
contain the exact tags given, but with additional tags are excluded.
This behavior is not compatible with the use cases defined in both the
keystone[0] and api-wg[1] specs, notibly with the "tags" and "tags-any"
interaction.

This change makes it so that "tags" filtering will be performed by
matching a subset containing given tags against projects, rather than
exact matching. This allows the "tags" and "tags-any" filters to work as
described in both [0] and [1].

[0] https://specs.openstack.org/openstack/keystone-specs/specs/keystone/queens/project-tags.html
[1] https://specs.openstack.org/openstack/api-wg/guidelines/tags.html#filtering-and-searching-by-tags

Co-Authored By: Nicolas Helgeson <nh202b@att.com>

Closes-Bug: #1756190
Change-Id: I632efdf0af2969be0a59dc5928a6c036eeca6051
2018-03-27 13:38:47 -05:00
Gage Hugo 94ef945885 Remove @expression from tags
This change makes tags a property of Project instead of a hybrid_property
since we will always have a Project contain some list of tags.

Change-Id: I1033321132cb3ec71bf94b8293cef91dfc6b8272
Co-Authored-By: Morgan Fainberg <morgan.fainberg@gmail.com>
Closes-Bug: #1752301
2018-02-28 11:23:16 +00:00
Lance Bragstad c7658abfd6 Simplify token persistence callbacks
The INVALIDATE_USER_TOKEN_PERSISTENCE and
INVALIDATE_USER_PROJECT_TOKEN_PERSISTENCE callbacks were meant to
clean up invalid tokens from the token storage layer. Now that the
sql token driver has been removed, we don't need them any more. This
commit removes those notifications and refactors the places where
notifications are still needed, making them more specific and not
eluding to token persistence.

This commit also removes a significant amount of logic from the
assignment API that used to notify the token API when assignments
were deleted. This made sense when tokens were written to disk
because there was an opportunity to invalidate them when users were
removed from projects. This is no longer needed since we do
validation online and we don't persist tokens anymore.

Change-Id: I100b7416e8ba61eb4ea2c2eb4962e952a53ea388
2018-02-16 21:40:07 +00:00
Colleen Murphy 62ee18b359 Delete SQL users before deleting domain
Since the users table has a foreign key to the projects table[1], users
must be deleted before the domain can be deleted. However, the
notification emitted from the domain deletion comes too late, and
keystone runs into a foreign key reference error before it can delete
the users. This patch addresses the problem by adding a new internal
notification to alert the identity manager that users should be deleted.
This uses a new notification rather than the existing notification
because the existing one is used to alert listeners that the domain
deletion has been fully completed, whereas this one must happen in the
middle of the domain delete process.

The callback must also only try to delete SQL users. The LDAP driver
doesn't support deleting users, and we can't assume other drivers
support it either. Moreover, the foreign key reference is only a problem
for SQL users anyway.

Because our backend unit tests run with SQLite and foreign keys do not
work properly, we can't properly expose this bug in our unit tests, but
there is an accompanying tempest test[2][3] to validate this fix.

[1] https://github.com/openstack/keystone/blob/2bd88d3/keystone/common/sql/expand_repo/versions/014_expand_add_domain_id_to_user_table.py#L140-L141
[2] https://review.openstack.org/#/c/509610
[3] https://review.openstack.org/#/c/509947

Change-Id: If5bdb6f5eef80b50b000aed5188ce7da4dfd1083
Closes-bug: #1718747
2018-02-08 21:19:02 +01:00
Lance Bragstad bf548cb0fd Use keystone.common.provider_api for resource APIs
This change converts the usage of self.<provider_api> to
keystone.common.providers_api.ProviderAPIs.<provider_api> in manager
and controller logic. This is the correct way to reference
providers from other managers and controllers now that dependency
injection has been eliminated.

Change-Id: Id0305ce4e782dc8e8977dc7f802110148da2c901
2018-01-02 15:32:49 +00:00
Gage Hugo db12357a6a Refactor project tags encoding
This change moves the tags conversion to unicode into the Project class
tags setter.

Change-Id: Idc7b2de24252f3199fcdf19ca7bb822e7322d3f9
2017-12-19 22:03:49 -06:00
Morgan Fainberg 81f9fe6fed Remove Dependency Injection
Refactors all of keystone's dependency injection to maintain a
single centralized repository of instantiated objects. This
means that we are no longer having to resolve order. All
objects that need to reference the various manager APIs simply
do so via the __getattr__ built into the Manager common object
or the ProviderAPIMixin object.

This is also the first step towards correcting our tests to
where they cannot run "load_backends" multiple times.

This forces any/all managers to properly run super()
as the way to register the api is via __init__.

This eliminates all use of the @dependency.requires and
@dependency.provides decorators, simplifying the objects
all around.

Any instantiations of a Manager after keystone is running
will now generate an error, ensuring everything for keystone
is running before handling requests. An exception is for
CLI and CLI tests, as the CLI may directly instantiate
managers and will not lock the registry.

Change-Id: I4ba17855efd797c0db9f4824936b49e4bff54b6a
2017-12-13 10:59:39 -08:00
Zuul c4b7bcf452 Merge "Remove ensure_default_domain_exists" 2017-11-04 11:06:30 +00:00
Zuul b9dee73323 Merge "Remove v2 schema and validation tests" 2017-11-04 00:12:37 +00:00
Gage Hugo e2295edab8 Remove ensure_default_domain_exists
With the removal of the V2.0 API in Queens, this functionality
was only used with the v2 resource controller and can now be
removed. Also removed the associated tests.

Change-Id: Id0503bc0610cff5fb635cd021088658bc06aef26
2017-11-01 00:07:51 -05:00
Gage Hugo 665cca0161 Remove v2 schema and validation tests
With the removal of the v2.0 APIs in Queens, the schema
for specific v2.0 actions can be removed.

Change-Id: Id63dbdab0b13995e6d42d19f6c224f23a2ce8d19
2017-10-19 17:20:47 -05:00
Gage Hugo de788453d9 Implement project tags API controller and router
This change adds the new API routes for project tags as well as
the controller methods for listening for /v3/project/[id]/tags.

Change-Id: Ic2e5874a427341f2edb6e9122386fb98af2c24ef
Partially-Implements: bp project-tags
Co-Authored-By: Jaewoo Park <jp655p@att.com>
Co-Authored-By: Nicolas Helgeson <nh202b@att.com>
Depends-On: I00f094a5584be40ab477cbf680a5f6d1afb4d21b
Depends-On: Ieb68bd2c9c216b25ad74d320a1c9a297d2b251e7
2017-10-19 12:18:01 -05:00
Gage Hugo ee900029db Implement project tags logic into manager
This change adds the tags attribute of project into the resource
manager. This change builds off of the backend logic.

Change-Id: Ie988b7a2065f0ecdb8ec953a1d01f5e3cbd67a1b
Partially-Implements: bp project-tags
Co-Authored-By: Jaewoo Park <jp655p@att.com>
Co-Authored-By: Nicolas Helgeson <nh202b@att.com>
Depends-On: I00f094a5584be40ab477cbf680a5f6d1afb4d21b
Depends-On: I5f8e4a53089b9fcc38084bb958d09f63ccc59d2a
2017-10-19 12:17:48 -05:00
Gage Hugo 6d320f75fd Implement backend logic for project tags
This change adds in tags as a hybrid property for projects.
Also adds in relevant tests for backend logic.

Change-Id: Ie153130b794e3541f197f8fa1e73bf30a64f12dc
Partially-Implements: bp project-tags
Co-Authored-By: Jaewoo Park <jp655p@att.com>
Co-Authored-By: Nicolas Helgeson <nh202b@att.com>
Depends-On: I00f094a5584be40ab477cbf680a5f6d1afb4d21b
Depends-On: Ib3db74cb08e7fdec01d55d8988e0906948d80a32
2017-10-19 12:17:41 -05:00
Gage Hugo 5329071174 Add JSON schema validation for project tags
This change adds json schema for project tags validation, with
the limits for number of characters per tag as well as the
limit for the number of tags a project can have per update.

Co-Authored-By: Jaewoo Park <jp655p@att.com>
Co-Authored-By: Nicolas <nh202b@att.com>

Change-Id: I5f8e4a53089b9fcc38084bb958d09f63ccc59d2a
Partially-Implements: bp project-tags
2017-10-17 10:15:08 -05:00
Jenkins 30eb190995 Merge "Remove v2.0 resource APIs" 2017-09-12 23:45:48 +00:00
Jenkins fe87d28422 Merge "Confusing log messages in project hierarchy checking" 2017-09-05 01:58:54 +00:00
zlyqqq b5c3dec60f Confusing log messages in project hierarchy checking
When is_domain is false, the project must contain a valid
domain_id and parent_id. We check the domain_id specified
is consistent with the domain of its parent.
The log messages is not very clear, this patch optimize it.

Change-Id: Ie83d295f036f94d36c9dfa88ae03e074cdfd9810
2017-09-04 08:36:13 +00:00
Lance Bragstad f2d2bcbf2b Remove v2.0 resource APIs
bp removed-as-of-queens

Change-Id: I2ac92c1f541b6348006d315f7ada6f6022944db4
2017-08-31 22:21:01 +00:00
Morgan Fainberg 3d46c8a5d9 Resource backend is SQL only now
This change has been implemented to avoid the need to backport
signficantly impactful Foreign Key dropping backports.

Resource is highly relational data and it makes sense to allow the
use of FKs from other subsystems to project/domains.

Change-Id: Ic3831d1c7ae41fe4d406d60a013770cc1258584f
2017-08-14 19:01:02 +00:00
Lance Bragstad 63124f703a Cache list projects and domains for user
Listing projects and domains for a user based on their role
assignments was noted as being really slow, especially when users
have a lot of assignments. This commit implements caching to mitigate
the issue while we continue to investigate ways to speed up the
assignment API.

Change-Id: I72e398c65f01aa4f9a37f817d184a13ed01089ce
Closes-Bug: 1700852
2017-08-09 14:45:58 +00:00
ForestLee d02a66d6bf Add a hacking rule for string interpolation at logging
String interpolation should be delayed to be handled by
the logging code, rather than being done at the point
of the logging call.
See the oslo i18n guideline
* https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html#adding-variables-to-log-messages
and
* https://github.com/openstack-dev/hacking/blob/master/hacking/checks/other.py#L39
Closes-Bug: #1596829

Change-Id: Iba231be2476dcbeeb0edd76d6a921e549d183758
2017-07-18 14:11:28 +08:00
Lance Bragstad b56dc82822 Add HEAD API to domain config
The domain configuration API had several GET calls that didn't
support HEAD. The documentation for the added HEAD APIs is going
to be proposed at a later date as a blanket statement saying all
APIs the support GET also support HEAD. This reduces duplication
of documentation for a relatively straight-forward concept.

This commit also fixes some inconsistencies in the domain
configuration documentation with colons and newlines.

Change-Id: I7a6ab2f05600c8d562d5be83d651ff735c9a68a4
Partial-Bug: 1696574
2017-06-12 20:50:49 +00:00
Feng Shengqin 19559b69c0 Remove unused CONF
Change-Id: Ie748beb4236b13508f66b396efbeca4965d3da14
2017-04-22 15:22:05 +08:00
Jenkins 39c6b0ff53 Merge "Make use of Dict-base including extras explicit" 2017-03-31 20:11:14 +00:00
Jenkins a2e8d7a9ba Merge "Remove revocation API dependency from resource API" 2017-03-30 06:26:52 +00:00
Jenkins 6d77eefaac Merge "Add group_members_are_ids to whitelisted options" 2017-03-30 05:50:30 +00:00