This adds the ability to create users and projects directly from
keystone-manage. We also add the ability to specify specific UUIDs
for both users and projects via the creation functions.
Change-Id: Icd193eff25556d21ec26bb29908b8ad6548fdc91
We propose to extend Keystone identity provider (IdP) attribute mapping
schema to make Keystone honor the `domain` configuration that we have
on it.
Currently, that configuration is only used to define a default domain
for groups (and then each group there, could override it). It is
interesting to expand this configuration (as long as it is in the root
of the attribute mapping) to be also applied for users and projects.
Moreover, to facilitate the development and extension concerning
attribute mappings for IdPs, we changed the way the attribute mapping
schema is handled. We introduce a new configuration
`federation_attribute_mapping_schema_version`, which defaults to "1.0".
This attribute mapping schema version will then be used to control the
validation of attribute mapping, and also the rule processors used to
process the attributes that come from the IdP. So far, with this PR,
we introduce the attribute mapping schema "2.0", which enables
operators to also define a domain for the projects they want to assign
users. If no domain is defined either in the project or in the global
domain definition for the attribute mapping, we take the IdP domain
as the default.
Change-Id: Ia9583a254336fad7b302430a38b538c84338d13d
Implements: https://bugs.launchpad.net/keystone/+bug/1887515
Closes-Bug: #1887515
These practices cause conflicts periodically. Not right now:
the gate is okay with the current values, which this patch
deletes. However, like sun raising in the east it is sure
to happen again. This patch lets workarounds work that the
infra team puts in place. The downside is, we need to fix
the code once in a while as new checks get added.
Change-Id: Ia7a96fb4b6de4251862a8a96c995cefa94dbc271
sqlalchemy-migrate does not (and will not) support sqlalchemy 2.0. We
need to drop these migrations to ensure we can upgrade our sqlalchemy
version.
Change-Id: I31ba9e4f129a7cc28744e814b5fd28eb284ae3de
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Switch to alembic for real by integrating it into the 'db sync' command
flow. From a user-facing perspective, things should remain pretty much
the same as before, with the key difference being that version
information (i.e. what's shown by 'keystone-manage db_sync --check' or
'keystone-manage db_version') will now take the form of a hash rather
than an integer. There are a few differences for contributors however.
The changes are described in the included release note and
documentation.
Note that there are a couple of important design decisions here that are
worth examining:
- We drop the idea of the 'data_migration' branch entirely and the
'keystone-manage db_sync --migrate' command is now a no-op. Neutron
doesn't do data migrations like we do and yet they manage just fine.
Dropping this gets us closer to neutron's behavior, which is a good
thing for users.
- We haven't re-added the ability to specify a version when doing
'db_sync'. Neutron has this, but the logic needed to get this working
is complex and of questionable value. We've managed without the
ability to sync to a version since Newton and can continue to do so
until someone asks for it (and does the work).
- sqlalchemy-migrate is not removed entirely. Instead, upon doing a
'db_sync' we will apply all sqlalchemy-migrate migrations up to the
final '079_expand_update_local_id_limit' migration and dummy apply the
initial alembic migration, after which we will switch over to alembic.
In a future release we can remove the sqlalchemy-migrate migrations
and rely entirely on alembic. Until then, keeping this allows fast
forward upgrades to continue as a thing.
- Related to the above, we always apply *all* sqlalchemy-migrate
migrations when calling 'db_sync', even if this command is called with
e.g. '--expand' (meaning only apply the expand branch). This is
because there is at most one "real" migration to apply, the Xena-era
'079_expand_update_local_id_limit' migration, which is an expand-only
migration. There is no risk to applying the empty "data_migration" and
"contract" parts of this migration, and applying everything in one go
results in *much* simpler logic.
Future changes will update documentation and add developer tooling for
(auto-)generating new migrations, a la 'neutron-db-manage revision'.
Change-Id: Ia376cb87f5159a4e79e2cfbab8442b6bcead708f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Nothing functional here. We simply switch from using "repo_name"
terminology and move/rename some helper functions. Branches aren't
really a SQLAlchemy-Migrate thing but it's close enough to do.
Change-Id: I005d20ef21b6c8122be90e8afb38abd902fdfc6e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This is deprecated and will be removed in a future release of oslo.db.
Even without that stick to prod us, we're going to need to use some of
the sqlalchemy-migrate APIs and it's simpler to talk to this for
everything rather than using oslo.db for some stuff and
sqlalchemy-migrate for the remainder.
Change-Id: Ib25c75a99794a04b6549e6b5184a2029955befc1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Remove option from 'db sync' and 'db version' commands of
'keystone-manage'. We also remove handling code for providing '--expand'
and '--migrate' in the same invocation, which isn't possible since
they're in a mutually exclusive group. Finally, we clean some formatting
things up in preparation for the alembic integration.
Change-Id: I198822d6f55353f1adeea9024db67abb24be54c7
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The ``keystone-manage mapping_engine`` help text wasn't formatting
an example assertion properly. The example uses newlines but the
default formatter drops them when formatting and printing help text.
Instead, we can set a raw formatter for the ``mapping_engine``
parser so that it includes newlines. This means all help text needs to
be properly formatted in code even though we only need it for a single
option. This commit introduces a raw formatter and adds new lines to
all help text so that it renders the same way as when we were using
the default formatter.
Change-Id: I7f8051e008ffea1ebaa4b6176f3d1a9f8208eaee
When keystone-manage bootstrap is ran without providing a value
to set as the admin password, keystone-manage will throw an
unhandled ValueError while displaying the proper warning
message.
This change removes the ValueError and simply has the CLI
exit out when this condition is met.
Closes-Bug: #1804700
Change-Id: I4e7d5eeb2e48ff354b44196bd11d62d51a73357b
Thw JSON Web Token provider implementation is going to need keys in
order to issue and validate tokens, very similar to how the fernet
provider operates, but using asymmetric signing instead of symmetric
encryption.
This commit addes a new subcommand to the keystone-manage binary that
creates a ECDSA key pair for creating and validating JWS tokens.
bp json-web-tokens
Change-Id: I9cf5c168bae2a90aba3d696e3f6ce3028998121a
The commands "receipt_setup" and "receipt_rotate" are used for
control receipt keys. They should be exposed to operators.
Update the cli doc in alpha order as well.
Change-Id: Id8f062ce1d713cd029a11fe8fdc81c256a771471
When the token and receipt directories don't match perfectly
(e.g. trailing slash vs not), we would get double key rotation.
Use os.path to make sure we are indeed checking for the same path.
Change-Id: I34c5e4891773bcd6a799f3dbfa7174718030c4e4
Adds a new model and provider for receipts which are
very similar to tokens (fernet based), and share the
same fernet mechanisms.
Adds changes to the auth layer to handle the creation,
validation, and consumptions of receipts as part of
the auth process.
Change-Id: Iccb6e6fc7aee57c58a53f90c1d671402b8efcdbb
bp: mfa-auth-receipt
Before keystone needed to be run on two separate ports to accommodate
the Identity v2 API which ran a separate admin-only service commonly
on port 35357. With the removal of the v2 API, keystone runs on port
5000.
Change-Id: I7744f664a9477430fb350889dc46cdf63dee17f9
This patch adds functionality to purge
expired and soft-deleted trusts older
than the given date.
Change-Id: I0bd47e57f8650182e38b4f70e04cb53338fce474
Related-Bug: #1473292
This patch removes the deprecated token_flush
in stein release of openstack.
Change-Id: Ifc6d162923072d657fd5c163b94ffcd88dc80692
Implements: bp removed-as-of-stein
This patch adds the functionality for purging both
expired trusts as well as non-expired soft-deleted
trusts, since those soft-deleted trusts are
as likely to bloat the database as expired trusts.
Related to patch-
https://review.openstack.org/#/c/589378/
Change-Id: I3c74f2345a944ce03a8189c4e66c3c37350cd34f
Related-Bug: #1473292
Mapping Engine Tester is untested. Looking
at a coverage report for the Keystone CLI
shows missing test cases for MappingEngine
Tester. This patch adds the test cases and
also fixes the compatibility issues of
MappingEngine with python3.
Change-Id: Id0844ba1f4e2979d91414d3bc821ab25650c6935
Closes-Bug: #1782197
Keystone was logging a warning saying "no configuration file was
found" when one was clearly passed to keystone-manage. This was
due to the fact that keystone-manage was really just checking for
default configuration file values. This means that users passing
a specific configuration file location to keystone-manage would
see a warning that wasn't necessarily true since keystone-manage
passes sys.argv to oslo.config, which resolves the argument
properly.
This change enhances the check to make sure keystone-manage isn't
dealing with a configuration file passed in from an end-user before
emitting the warning.
This change also cleans up some of the interactions between
keystone.cmd.manage and keystone.cmd.cli to make the variables
more clear about their purpose.
Change-Id: I6d58d9c499c447ef1cfd6edd4aeb1176130fb6ad
Closes-Bug: 1782704
metadata.to_string() was generating a bytes string, which python3
prepends with b'' when using the print function. Seems like
removing the to_string() call and just calling print on the object
makes it work correctly under both python3 and python2.
Change-Id: I546ba0b15f6196be7bbb5940e9f00434d5e6b1ff
Closes-Bug: #1780377
"keystone-manage mapping_purge" has "--type" option in order to select
mappings which have to be deleted. However, the option doesn't work
and all mappings are purged when specifying this option.
The command passes a value of the option to purge_mappings API with
a key name "type", but the API expects "entity_type".
Change-Id: Ib222d70d4cf6bb61847cef049ec1ded5c2ff2c8b
Closes-Bug: #1757022
Expand bootstrap process to include creation of roles outlined in
basic default roles spec.
The bootstrap process now creates two new roles, 'reader' and 'member,
in addition to the well established 'admin' role. During this process,
a role implication[1] chain is created: 'admin' implies 'member' and
'member' implies 'reader'.
[1] - https://developer.openstack.org/api-ref/identity/v3/#create-role-inference-rule
Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
bp basic-default-roles
Depends-On: https://review.openstack.org/574149
Change-Id: Ie18a269e3d1075d955fe494acaf634a393c6bd7b
This reverts commit 03ba867327.
Because of the introduction of auth receipts we will be
using fernet for more than just tokens. Lets make this a
generic util for fernet key handling.
Change-Id: I3a870a63239491f84db3350178bd2313eeccdbf3
We have a cli command that's useful for bootstrapping new
deployments, resetting administrator passwords, creating the default
domain, et cetera. But this functionality isn't just useful for
operators. We could actually benefit from using the same process
in our tests.
Many test cases actually do a bunch of their own setup. Sometimes
they duplicate a lot of what bootstrap already provides. Other times
test modules just blindly inherit from ``core.TestCase`` to get
things works. This isn't ideal because it isn't DRY. It also becomes
really easy to tack on extra cycles to each test when things are
added to the generic ``load_fixtures`` method that is typically run
for each test.
Instead, we can try and reuse what we give to operators for our
testing. This will:
- Reduce duplicate setup scenarios across test modules
- Reduce extra resource creation and cleanup when it's not needed
for a particular set of test
- Enforces better readability if each test case assumes the same
basic structure of data present after bootstrap, test classes
that need addition resources should include those in their
setUp or setUpClass methods
- We engrain the testing of a tool we provide to operators into
our testing environment
This commit helps us move in that direction by encapsulating the
logic for bootstrapping deployments into a reusable object that is
accessible by both the tests and the cli tooling.
Change-Id: I84c75076405e6af9356f79778f2d2e67855f4fe0
The `keystone-manage token_flush` command was useful when we
supported token format that were actually written to disk. Now that
keystone has moved towards formats that don't require physical
storage in the name of scalability, we no longer need the
token_flush utility.
This commit logs a warning when using `keystone-manage token_flush`
so that people are aware they don't need to use it anymore if they
are using supported upstream non-persistent token providers.
Change-Id: I14d1e8a3ae367c8f13b6819a1a216f3f8f6b001a
Closes-Bug: 1759289
This change adds new token_setup and token_rotate commands that uses the
same functionality as fernet_setup and fernet_rotate. These commands are
to move users to a more generic use case.
Change-Id: I118f9a89d28d8ec1ccaf88a90053fa723ef98cff
Now that we have system scope in place, we should make sure at least
one user has a role assignment on the system. We can do this at the
same time we grant the user a role on a project during bootstrap.
This is backwards compatible because even if a deployment doesn't use
system-scope, the assignment will just sit there. The deployment will
have to opt into enforcing scope by updating configuration options
for oslo.policy to enforce scoping.
This shouldn't prevent deployments from fixing bug 968696 and using
system scope.
Closes-Bug: 1749268
Change-Id: I6b7196a28867d9a699716c8fef2609d608a5b2a2
``member_role_id`` and ``member_role_name`` config options
are only used for V2. Instead of removing, just deprecate
them because that maybe some consumers still use them
for V3.
This patch also removed the usage in
``keystone-manage bootstrap`` as well.
Closes-bug: #1728690
bp: deprecated-as-of-queens
bp: removed-as-of-queens
Change-Id: Ib85479442ec68f9a67615c23e5c39bd217c9b109
This change renames the fernet_utils from keystone.common to
token_utils in order to provide more generic tools for the
implementation of alternative token providers.
Change-Id: I85091709ed4ed7bdd2c2d053e342e4b1b615ef08
Dave Wilde