This adds the ability to create users and projects directly from
keystone-manage. We also add the ability to specify specific UUIDs
for both users and projects via the creation functions.
Change-Id: Icd193eff25556d21ec26bb29908b8ad6548fdc91
We propose to extend Keystone identity provider (IdP) attribute mapping
schema to make Keystone honor the `domain` configuration that we have
on it.
Currently, that configuration is only used to define a default domain
for groups (and then each group there, could override it). It is
interesting to expand this configuration (as long as it is in the root
of the attribute mapping) to be also applied for users and projects.
Moreover, to facilitate the development and extension concerning
attribute mappings for IdPs, we changed the way the attribute mapping
schema is handled. We introduce a new configuration
`federation_attribute_mapping_schema_version`, which defaults to "1.0".
This attribute mapping schema version will then be used to control the
validation of attribute mapping, and also the rule processors used to
process the attributes that come from the IdP. So far, with this PR,
we introduce the attribute mapping schema "2.0", which enables
operators to also define a domain for the projects they want to assign
users. If no domain is defined either in the project or in the global
domain definition for the attribute mapping, we take the IdP domain
as the default.
Change-Id: Ia9583a254336fad7b302430a38b538c84338d13d
Implements: https://bugs.launchpad.net/keystone/+bug/1887515
Closes-Bug: #1887515
The current shebang requires python 2 instead of python 3. Because
the shebang is not really requires this removes it instead of fixing
it.
Change-Id: I4c82281a67de53c6c8d788000b695a22360c3dd6
These practices cause conflicts periodically. Not right now:
the gate is okay with the current values, which this patch
deletes. However, like sun raising in the east it is sure
to happen again. This patch lets workarounds work that the
infra team puts in place. The downside is, we need to fix
the code once in a while as new checks get added.
Change-Id: Ia7a96fb4b6de4251862a8a96c995cefa94dbc271
Comma is missing from the 'rules' list at check_trust_policies_are_not_empty().
Closes-Bug: #2037052
Change-Id: I47f3a7649b7e6022bea477caff8b081f352b0af3
sqlalchemy-migrate does not (and will not) support sqlalchemy 2.0. We
need to drop these migrations to ensure we can upgrade our sqlalchemy
version.
Change-Id: I31ba9e4f129a7cc28744e814b5fd28eb284ae3de
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The pep257 dependency does not support Python 3.10 and has been
deprecated in favour of flake8-docstrings. While we're here, we bump the
other linter dependencies and remove a note regarding the order of
dependencies, which is no longer true with the new dependency resolver
introduced in pip 20.3. We also remove an import exception for six.moves
since we no longer use six.
Change-Id: I4aae75f513568126230becf27b2e07d6682d35a1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Switch to alembic for real by integrating it into the 'db sync' command
flow. From a user-facing perspective, things should remain pretty much
the same as before, with the key difference being that version
information (i.e. what's shown by 'keystone-manage db_sync --check' or
'keystone-manage db_version') will now take the form of a hash rather
than an integer. There are a few differences for contributors however.
The changes are described in the included release note and
documentation.
Note that there are a couple of important design decisions here that are
worth examining:
- We drop the idea of the 'data_migration' branch entirely and the
'keystone-manage db_sync --migrate' command is now a no-op. Neutron
doesn't do data migrations like we do and yet they manage just fine.
Dropping this gets us closer to neutron's behavior, which is a good
thing for users.
- We haven't re-added the ability to specify a version when doing
'db_sync'. Neutron has this, but the logic needed to get this working
is complex and of questionable value. We've managed without the
ability to sync to a version since Newton and can continue to do so
until someone asks for it (and does the work).
- sqlalchemy-migrate is not removed entirely. Instead, upon doing a
'db_sync' we will apply all sqlalchemy-migrate migrations up to the
final '079_expand_update_local_id_limit' migration and dummy apply the
initial alembic migration, after which we will switch over to alembic.
In a future release we can remove the sqlalchemy-migrate migrations
and rely entirely on alembic. Until then, keeping this allows fast
forward upgrades to continue as a thing.
- Related to the above, we always apply *all* sqlalchemy-migrate
migrations when calling 'db_sync', even if this command is called with
e.g. '--expand' (meaning only apply the expand branch). This is
because there is at most one "real" migration to apply, the Xena-era
'079_expand_update_local_id_limit' migration, which is an expand-only
migration. There is no risk to applying the empty "data_migration" and
"contract" parts of this migration, and applying everything in one go
results in *much* simpler logic.
Future changes will update documentation and add developer tooling for
(auto-)generating new migrations, a la 'neutron-db-manage revision'.
Change-Id: Ia376cb87f5159a4e79e2cfbab8442b6bcead708f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Nothing functional here. We simply switch from using "repo_name"
terminology and move/rename some helper functions. Branches aren't
really a SQLAlchemy-Migrate thing but it's close enough to do.
Change-Id: I005d20ef21b6c8122be90e8afb38abd902fdfc6e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This is deprecated and will be removed in a future release of oslo.db.
Even without that stick to prod us, we're going to need to use some of
the sqlalchemy-migrate APIs and it's simpler to talk to this for
everything rather than using oslo.db for some stuff and
sqlalchemy-migrate for the remainder.
Change-Id: Ib25c75a99794a04b6549e6b5184a2029955befc1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Remove option from 'db sync' and 'db version' commands of
'keystone-manage'. We also remove handling code for providing '--expand'
and '--migrate' in the same invocation, which isn't possible since
they're in a mutually exclusive group. Finally, we clean some formatting
things up in preparation for the alembic integration.
Change-Id: I198822d6f55353f1adeea9024db67abb24be54c7
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:
1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.
2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.
Also replace policy.json to policy.yaml ref from doc and tests.
[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: Ic65d2fd6ce7215b4a47a6fb41b9cbf991f27773b
This makes it easier for operators to troubleshoot connection issues to
Memcached.
Related-Bug: 1332058
Change-Id: I6e67363822480314b93608bb1eae3514f1480f6d
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Fix problems found.
Update local hacking checks for new flake8.
Change-Id: Ic440219814ee0c2b98217e9a821f38f5baf482ec
This repo does not support Python 2 anymore, so we don't need
six for compatibility between Python2 and 3, convert six usage to Python
3 code.
Change-Id: Icba56808f38277b27af2ae5aac4b8507dee71b3b
The bootstrap logic doesn't take into consideration multiple roles
with the same name. If bootstrap is unable to determine which role to
use and accidentally uses a domain-specific role with the same name
as a default role, bootstrap will fail in unexpected ways.
Closes-Bug: 1856881
Change-Id: Iddc364d8c934b6e54d1e8c75b8b159faadbf865d
In 09088690 we mistakenly added E501 to the flake8 ignore list. Since
then, many new violations have been introduced. This patch re-enables
the check and corrects all violations, except in some cases like unit
test names where the subunit output would suffer if we attempted to
shorten the function name.
This may appear to be a pointless no-op that messes with
git-blameability, and it is, but the reason to do this is that if PEP8
violations are introduced in master and then backported to a stable
branch, most stable branches will fail the pep8 job since the flake8
ignore list is correct for those branches. Rather than loosening the
check in older branches or requiring those backports to fix the linter
errors independently of what's been merged in master, we should fix it
now so that we don't introduce more errors in the future and patches can
more easily be backported.
Change-Id: I9f71926105eb448bb0200201d1838b67d4963cd6
The ``keystone-manage mapping_engine`` help text wasn't formatting
an example assertion properly. The example uses newlines but the
default formatter drops them when formatting and printing help text.
Instead, we can set a raw formatter for the ``mapping_engine``
parser so that it includes newlines. This means all help text needs to
be properly formatted in code even though we only need it for a single
option. This commit introduces a raw formatter and adds new lines to
all help text so that it renders the same way as when we were using
the default formatter.
Change-Id: I7f8051e008ffea1ebaa4b6176f3d1a9f8208eaee
Without this change, policy enforcement for the GET
/OS-TRUST/trusts/{trust_id}/roles/{role_id} API is hardcoded in the
flask dispatcher code. This is a problem because this enforcement can't
be controlled by the operator, as is the norm. Moreover, it makes the
transition to system-scope and default-roles-aware policies more
difficult because there's no sensible migration from "" to a logical
role-based check string.
This converts the hardcoded enforcement to enforcement via default
policies for GET /OS-TRUST/trusts/{trust_id}/roles/{role_id}. The API
specifically blocks the is_admin user from using it, and since policies
aren't loaded for the is_admin user we need to continue explicitly
blocking it.
This change does not use the formal oslo.policy deprecation system
because "" OR'd with the new default is entirely useless as a policy.
Change-Id: Ib5a6a87313aa7b2a73211f512b8a8c675a21b52f
Partial-bug: #1818850
Partial-bug: #1818846
Without this change, policy enforcement for the GET
/OS-TRUST/trusts/{trust_id}/roles API is hardcoded in the flask
dispatcher code. This is a problem because this enforcement can't be
controlled by the operator, as is the norm. Moreover, it makes the
transition to system-scope and default-roles-aware policies more
difficult because there's no sensible migration from "" to a logical
role-based check string.
This converts the hardcoded enforcement to enforcement via default
policies for GET /OS-TRUST/trusts/{trust_id}/roles. The API specifically
blocks the is_admin user from using it, and since policies aren't loaded
for the is_admin user we need to continue explicitly blocking it.
This change does not use the formal oslo.policy deprecation system
because "" OR'd with the new default is entirely useless as a policy.
Change-Id: Ib339852c9d619b8cbf7a00d45da461377991ba6f
Partial-bug: #1818850
Partial-bug: #1818846
Without this change, policy enforcement for the GET
/OS-TRUST/trusts/{trust_id} API is hardcoded in the flask dispatcher
code. This is a problem because this enforcement can't be controlled by
the operator, as is the norm. Moreover, it makes the transition to
system-scope and default-roles-aware policies more difficult because
there's no sensible migration from "" to a logical role-based check
string.
This converts the hardcoded enforcement to enforcement via default
policies for GET /OS-TRUST/trusts/{trust_id}. The API specifically
blocks the is_admin user from using it, and since policies aren't loaded
for the is_admin user we need to continue explicitly blocking it.
This change does not use the formal oslo.policy deprecation system
because "" OR'd with the new default is entirely useless as a policy.
Change-Id: I3c0718330d5a18c0c79e8f12509200fd97a55913
Partial-bug: #1818850
Partial-bug: #1818846
Without this change, policy enforcement for the DELETE
/OS-TRUST/trusts/{trust_id} API is hardcoded in the flask dispatcher
code. This is a problem because this enforcement can't be controlled by
the operator, as is the norm. Moreover, it makes the transition to
system-scope and default-roles-aware policies more difficult because
there's no sensible migration from "" to a logical role-based check
string.
This converts the hardcoded enforcement to enforcement via default
policies for DELETE /OS-TRUST/trusts/{trust_id}. Currently only the
trustor or the is_admin user can access this API (since the is_admin
user bypasses the policy loading). This behavior will be changed in a
future patch that will allow the system admin to access this API.
This change does not use the formal oslo.policy deprecation system
because "" OR'd with the new default is entirely useless as a policy.
Change-Id: I1aaba72b69b389ffbfcf7d5b8cc70453ffa59e73
Partial-bug: #1818850
Partial-bug: #1818846
Without this change, policy enforcement for the GET /OS-TRUST/trusts API
is hardcoded in the flask dispatcher code. This is a problem because
this enforcement can't be controlled by the operator, as is the norm.
Moreover, it makes the transition to system-scope and
default-roles-aware policies more difficult because there's no sensible
migration from "" to a logical role-based check string.
This patch starts the conversion from hardcoded enforcement to
enforcement via default policies for GET /OS-TRUST/trusts. To do this,
we add two new policy rules, "identity:list_trusts_for_trustor" and
"identity:list_trusts_for_trustee". We need to do this so that we can
keep backwards compatibility with the bizarre behavior that an admin can
list all trusts (GET /OS-TRUST/trusts) but not list trusts for a trustor
or trustee (GET /OS-TRUST/trusts?trustor_user_id={} and
GET/OS-TRUST/trusts?trustee_user_id={}). The tricky part is that it's
plausible that operators may have incorporated the hardcoded empty
default for "identity:list_trusts" into their on-disk policy
configuration, either by never removing the old default policy file that
used to come packaged with keystone, or by generating a sample file and
applying that to disk (we don't recommend that but we don't expressly
forbid or discourage it either). To overcome
this, the trust API code checks whether the "identity:list_trusts" rule
is "" and re-applies the enforcement with a warning. We don't need to do
this for the two new policies because they are initially enforced
in-code and an operator would have to take explicit action on upgrade to
override them.
This change does not use the formal oslo.policy deprecation system
because "" OR'd with the new default is entirely useless as a policy.
Partial-bug: #1818850
Partial-bug: #1818846
Change-Id: I6c1a4ecd756519f7f807c9d28960482e7f0d235b
When keystone-manage bootstrap is ran without providing a value
to set as the admin password, keystone-manage will throw an
unhandled ValueError while displaying the proper warning
message.
This change removes the ValueError and simply has the CLI
exit out when this condition is met.
Closes-Bug: #1804700
Change-Id: I4e7d5eeb2e48ff354b44196bd11d62d51a73357b
The current bootstrap command does not update the endpoints, but
does update many other settings.
It's very convenient to have a way to update endpoints from the
bootstrap command. One such use case is when bootstrap is run from
a configuration management tool such as ansible, and the endpoint
URLs need to be updated.
Change-Id: I2b1cb53701c3e5fb2d91c14d9c792beadd1dbbc2
This is to add missing ws separator between words which
is introduced in I65130eb93a7e13d14d3ca9c325bf0bb0477ea45a.
Change-Id: I1052d19b3b6921548ec8352d6e7104838260f8bc
Since in a usual deployment, the default user, project and role are all
named "admin", logging
Granted admin on admin to user admin
is a bit ambigous. Add in some more words in order to make the message
more easily interpreted.
Change-Id: I65130eb93a7e13d14d3ca9c325bf0bb0477ea45a
Thw JSON Web Token provider implementation is going to need keys in
order to issue and validate tokens, very similar to how the fernet
provider operates, but using asymmetric signing instead of symmetric
encryption.
This commit addes a new subcommand to the keystone-manage binary that
creates a ECDSA key pair for creating and validating JWS tokens.
bp json-web-tokens
Change-Id: I9cf5c168bae2a90aba3d696e3f6ce3028998121a
The commands "receipt_setup" and "receipt_rotate" are used for
control receipt keys. They should be exposed to operators.
Update the cli doc in alpha order as well.
Change-Id: Id8f062ce1d713cd029a11fe8fdc81c256a771471
When the token and receipt directories don't match perfectly
(e.g. trailing slash vs not), we would get double key rotation.
Use os.path to make sure we are indeed checking for the same path.
Change-Id: I34c5e4891773bcd6a799f3dbfa7174718030c4e4
Dave Wilde