diff --git a/doc/source/index.rst b/doc/source/index.rst index 3b9228b..607b657 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -6,10 +6,7 @@ Barbican Project Specifications Train approved specs: -.. - disabled to not break builds, once first spec is added, - this needs enabling. - .. toctree:: +.. toctree:: :glob: :maxdepth: 1 diff --git a/specs/train/secret-consumers.rst b/specs/train/secret-consumers.rst new file mode 100644 index 0000000..7d07648 --- /dev/null +++ b/specs/train/secret-consumers.rst @@ -0,0 +1,362 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +================ +Secret Consumers +================ + +https://storyboard.openstack.org/#!/story/2005770 + +This spec proposes an addition to the Barbican Secrets API to allow +other OpenStack projects to add references to individual Secrets when +those secrets are being used by them. + +This spec also proposes a change to both the Python and CLI clients in +python-barbicanclient in how they handle the deletion of secrets. +Clients would be changed such that deleting a secret will result in an +error when they are still being consumed by another project unless a `force` +parameter is provided. + +This spec is part of a larger effort to provide Encrypted Images +to OpenStack clouds. + +Problem Description +=================== + +Other OpenStack projects would like to make use of an end user's secrets +e.g. A Secret that contains an encryption key for Image Encryption. +But there is currently no way for those projects to let the user know +that they are using the Secret. This lack of awareness may lead to errors +if the user deletes a Secret that is still in use by other projects. + +On the other hand, users should be allowed to delete secrets whenever they +want, so a Secret being used by other projects should not prevent deletion. + +Proposed Change +=============== + +Add a new API to Secrets to register Secret Consumers (similar, but not +identical to the Containers Consumer API [1]). + +With this new API, other OpenStack projects would register as a consumer +of a secret by sending a request to Barbican. Barbican stores the service +type of the requesting service, as well as both the resource type and +resource ID of the resource that is using the Secret. + +See REST API Impact below for details of the API changes. + +Clients to barbican would change the semantics for deleting secrets by +returning an error when trying to delete a secret if that secret has one +or more consumers. Clients will also accept an additional boolean parameter +to delete a secret regardless of how many consumers it has. + +See Python and Command Line Client Impact below for details of the client +changes. + +Alternatives +------------ + +One alternative would be to implement Secret Consumers just like Container +Consumers, which uses a URL instead of the consuming entity type and ID. + +Another alternative approach that was considered was to have each project +clone the secret when they need to use it. This alternative has some +downsides, however. For one, an end user may not be able to delete +those copies. + +Data model impact +----------------- + +A new model and associated data table will need to be added. For example, +a new class SecretConsumerMetadatum with a secret_consumer_metadata table. + +The new class will have references to both the secret_id as well as the +project_id which owns the secret. + +REST API impact +--------------- + +POST /v1/secrets/{secret_id}/consumers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add a new resource as a consumer to a secret. + +Body Parameters ++++++++++++++++ + ++---------------------+--------+--------------------------------------------------------+ +| Name | Type | Description | ++---------------------+--------+--------------------------------------------------------+ +| service | string | Consumer's OpenStack service type as shown in | +| | | https://service-types.openstack.org/service-types.json | ++---------------------+--------+--------------------------------------------------------+ +| resource_type | string | Name of the resource type using the secret | +| (or resource_path?) | | e.g. "images" or "lbaas/loadbalancers" | ++---------------------+--------+--------------------------------------------------------+ +| resource_id | string | Unique identifier for the resource using this secret. | ++---------------------+--------+--------------------------------------------------------+ + +Barbican will consider the resource_id to be a unique consumer. This assumes +that resource_id is a UUID, and that duplicate IDs for different projects +is not likely to ever happen in a single cloud. + +resource_type should be meaningful to the individual projects, and should +be used to identify the resource in the consuming service. For example, +Glance could use "images" as the value of the resource type to indicate that +the resrouce_id refers to an image. + +Request ++++++++ + + POST /v1/secrets/{secret_id}/consumers + Headers: + X-Auth-Token: {token} + X-Content-Type: application/json + + { + "service": "image", + "resource_type": "images", + "resource_id": "{image_id}" + } + +Responses ++++++++++ + ++------+--------------------------------------------------------------------+ +| Code | Description | ++======+====================================================================+ +| 200 | OK | ++------+--------------------------------------------------------------------+ +| 401 | Unauthorized - X-Auth-Token is invalid | ++------+--------------------------------------------------------------------+ +| 403 | Forbidden - X-Auth-Token is valid, but the associated project does | +| | not have the appropriate role/scope | ++------+--------------------------------------------------------------------+ + +GET /v1/secrets/{secret_id}/consumers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +List consumers for a particular Secret. + +Parameters +++++++++++ + ++---------+---------+---------+-------------------------------------------------+ +| Name | Type | Default | Description | ++=========+=========+=========+=================================================+ +| offset | integer | 0 | Offset to start consumer response | ++---------+---------+---------+-------------------------------------------------+ +| limit | integer | 10 | Number of consumer entries returned in response | ++---------+---------+---------+-------------------------------------------------+ +| service | string | None | Filter by service type | ++---------+---------+---------+-------------------------------------------------+ + +Request ++++++++ + + GET /v1/secrets/{secret_id}/consumers + Headers: + X-Auth-Token: {token} + +OK Response ++++++++++++ + + 200 OK + + { + "total": 1, + "consumers": [ + { + "service": "image", + "resource_type": "images", + "resource_id" : "{image_id}" + } + ] + } + +Other Responses ++++++++++++++++ + ++------+--------------------------------------------------------------------+ +| Code | Description | ++======+====================================================================+ +| 401 | Unauthorized - X-Auth-Token is invalid | ++------+--------------------------------------------------------------------+ +| 403 | Forbidden - X-Auth-Token is valid, but the associated project does | +| | not have the appropriate role/scope | ++------+--------------------------------------------------------------------+ + +DELETE /v1/secrets/{secret_id}/consumers/{resource_id} +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Delete a consumer. ie. The resource is being deleted and it longer needs +to access this secret. + +Request ++++++++ + + DELETE v1/secrets/{secret_id}/consumers/{resource_id} + +Responses ++++++++++ + ++------+--------------------------------------------------------------------+ +| Code | Description | ++======+====================================================================+ +| 200 | OK | ++------+--------------------------------------------------------------------+ +| 401 | Unauthorized - X-Auth-Token is invalid | ++------+--------------------------------------------------------------------+ +| 403 | Forbidden - X-Auth-Token is valid, but the associated project does | +| | not have the appropriate role/scope | ++------+--------------------------------------------------------------------+ +| 404 | Not Found - Consumer record for given resource_id was not found. | ++------+--------------------------------------------------------------------+ + +Security impact +--------------- + +Because the consumers are stored in the database, there is the possibility +that a bad actor could add many consumers to try to fill the database disk +space. Secret Consumers should be limited to the same quota as Container +Consumers to mitigate this risk. For example: + + [quota] + quota_consumers=10000 + +Would limit both Container Consumers and Secret Consumers to a maximum +of 10,000 consumers each for both a single Container or a single Secret. + +Notifications & Audit Impact +---------------------------- + +The new API endpoints should be audited as usual. + +Python and Command Line Client Impact +------------------------------------- + +The Secret class in python-barbicanclient should be updated to add new +methods such as: + + class Secret(...): + ... + + def add_consumer(self, service_type, resource_type, resource_id): + ... + + def remove_consumer(self, service_type, resource_type, resource_id): + ... + +Both methods should raise appropriate exceptions when the API returns an error. +Additionally, the Secret.delete() method should be updated to take a new *force* +parameter and throw an exception when delete() is called with force=False, +and the secret still has consumers: + + class Secret(...): + ... + + def delete(self, force=False): + ... + +The CLI client should be changed to add new consumer options, such as: + + openstack secret consumer add --service-type=image --resource-type=image \ + --resource-id=XXXX-XXXX-XXXX-XXXX + + openstack secret consumer remove --service-type=image --resource-type=image \ + --resource-id=XXXX-XXXX-XXXX-XXXX + +The secret delete command should be changed to take a *--force* parameter: + + openstack secret delete --force {secret_uuid} + +This command should return an error when a secret has one or more consumers +and the --force flag is not used: + + openstack secret delete {secret_uuid_with_consumers} + ERROR: Secret has one or more consumers. Use --force to delete anyway. + +These changes will require a new Major version for python-barbicanclient +because the default --force=False option could cause some scripts to break in +certain scenarios where secrets are currently being deleted that do have +consumers associated with them. + +Other end user impact +--------------------- + +Currently there is no other impact to the end user other than the CLI changes +listed above. In the future, when a barbican-ui for Horizon is developed, +it should use the consumers to present confirmation dialogs to the user +when deleting Secrets which have consumers. + +It should be noted that Deleting Secrets in the Barbican REST API +has not changed, and a client using the API directly will be able to delete +a secret regardless of the presence of consumers. + +Performance Impact +------------------ + +Deleting secrets using the CLI or the Python client will be affected as we +will likely need to perform additional requests to the API to get the list of +consumers for a secret before sending a DELETE request. + +Other deployer impact +--------------------- + +When python-barbican changes are merged, some automation scripts that use +secret deletion may break if the secrets being deleted have consumers. + +Any automation scripts should be updated to use the --force flag if needed. + +Developer impact +---------------- + +Developers of other projects that want to make use of this feature will +need to use python-barbicanclient to integrate with the Key Manager service. + +Implementation +============== + +Assignee(s) + +Primary assignee: + Douglas Mendizábal (Freenode: redrobot) + +Other contributors: + Moisés Guimarães (Freenode: moguimar) + +Work Items +---------- + +* Implement Model changes and database migration +* Implement API changes +* Implement python-barbicanclient changes (both python client and CLI) + +Dependencies +============ + +None. + +Testing +======= + +Tempest test cases should be added to test adding/removing Secret Consumers +using a service-user that is not barbican. + +Documentation Impact +==================== + +All API changes should be documented in the API reference, as well as the +API Guide. + +References +========== + +[1] Container Consumers API: +https://docs.openstack.org/barbican/stein/api/reference/consumers.html + +Barbican Train PTG Etherpad: +https://etherpad.openstack.org/p/barbican-train-ptg diff --git a/specs/train/template.rst b/specs/train/template.rst deleted file mode 100644 index 8721844..0000000 --- a/specs/train/template.rst +++ /dev/null @@ -1,342 +0,0 @@ -.. - This work is licensed under a Creative Commons Attribution 3.0 Unported - License. - - http://creativecommons.org/licenses/by/3.0/legalcode - -========================================== -Example Spec - The title of your blueprint -========================================== - -Include the URL of your launchpad blueprint: - -https://blueprints.launchpad.net/barbican/+spec/example - -Include the URL of your client blueprint: - -https://blueprints.launchpad.net/python-barbicanclient/example - -Introduction paragraph -- why are we doing anything? A single paragraph of -prose that operators can understand. - -Some notes about using this template: - -* Your spec should be in ReSTructured text, like this template. - -* Please wrap text at 79 columns. - -* The filename in the git repository should match the launchpad URL, for - example a URL of: https://blueprints.launchpad.net/barbican/+spec/awesome-thing - should be named awesome-thing.rst - -* Please do not delete any of the sections in this template. If you have - nothing to say for a whole section, just write: None - -* For help with syntax, see http://sphinx-doc.org/rest.html - -* To test out your formatting, build the docs using tox, or see: - http://rst.ninjs.org - -* If you would like to provide a diagram with your spec, ascii diagrams are - required. http://asciiflow.com/ is a very nice tool to assist with making - ascii diagrams. The reason for this is that the tool used to review specs is - based purely on plain text. Plain text will allow review to proceed without - having to look at additional files which can not be viewed in gerrit. It - will also allow inline feedback on the diagram itself. - -* If your specification proposes any changes to the Barbican REST API such - as changing parameters which can be returned or accepted, or even - the semantics of what happens when a client calls into the API, then - you should add the APIImpact flag to the commit message. Specifications with - the APIImpact flag can be found with the following query:: - - https://review.openstack.org/#/q/status:open+project:openstack/barbican-specs+message:apiimpact,n,z - - -Problem Description -=================== - -A detailed description of the problem: - -* For a new feature this might be use cases. Ensure you are clear about the - actors in each use case: End User vs Deployer - -* For a major reworking of something existing it would describe the - problems in that feature that are being addressed. - - -Proposed Change -=============== - -Here is where you cover the change you propose to make in detail. How do you -propose to solve this problem? - -If this is one part of a larger effort make it clear where this piece ends. In -other words, what's the scope of this effort? - -Alternatives ------------- - -What other ways could we do this thing? Why aren't we using those? This doesn't -have to be a full literature review, but it should demonstrate that thought has -been put into why the proposed solution is an appropriate one. - -Data model impact ------------------ - -Changes which require modifications to the data model often have a wider impact -on the system. The community often has strong opinions on how the data model -should be evolved, from both a functional and performance perspective. It is -therefore important to capture and gain agreement as early as possible on any -proposed changes to the data model. - -Questions which need to be addressed by this section include: - -* What new data objects and/or database schema changes is this going to - require? - -* What database migrations will accompany this change (if any)? - -* How will the initial set of new data objects be generated? For example, if you - need to take into account existing keys, or modify other existing data - describe how that will work. - -REST API impact ---------------- - -Each API method which is either added or changed should have the following - -* Specification for the method - - * A description of what the method does suitable for use in - user documentation - - * Method type (POST/PUT/GET/DELETE) - - * Normal http response code(s) - - * Expected error http response code(s) - - * A description for each possible error code should be included - describing semantic errors which can cause it such as - inconsistent parameters supplied to the method, or when an - instance is not in an appropriate state for the request to - succeed. Errors caused by syntactic problems covered by the JSON - schema defintion do not need to be included. - - * URL for the resource - - * Parameters which can be passed via the url - - * JSON schema definition for the body data if allowed - - * JSON schema definition for the response data if any - -* Example use case including typical API samples for both data supplied - by the caller and the response - -* Discuss any policy changes, and discuss what things a deployer needs to - think about when defining their policy. - -Example JSON schema definitions can be found in the Nova tree -http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/schemas/v3 - -Note that the schema should be defined as restrictively as -possible. Parameters which are required should be marked as such and -only under exceptional circumstances should additional parameters -which are not defined in the schema be permitted (eg -additionaProperties should be False). - -Reuse of existing predefined parameter types such as regexps for -passwords and user defined names is highly encouraged. - -Security impact ---------------- - -Describe any potential security impact on the system. Some of the items to -consider include: - -* Does this change touch sensitive data such as tokens, keys, or user data? - -* Does this change alter the API in a way that may impact security, such as - a new way to access sensitive information or a new way to login? - -* Does this change involve cryptography or hashing? - -* Does this change require the use of sudo or any elevated privileges? - -* Does this change involve using or parsing user-provided data? This could - be directly at the API level or indirectly such as changes to a cache layer. - -* Can this change enable a resource exhaustion attack, such as allowing a - single API interaction to consume significant server resources? Some examples - of this include launching subprocesses for each connection, or entity - expansion attacks in XML. - -* Does this change the need for auditing in any way? - -For more detailed guidance, please see the OpenStack Security Guidelines as -a reference (https://wiki.openstack.org/wiki/Security/Guidelines). These -guidelines are a work in progress and are designed to help you identify -security best practices. For further information, feel free to reach out -to the OpenStack Security Group at openstack-security@lists.openstack.org. - -Notifications & Audit Impact ----------------------------- - -Please specify any changes to notifications or auditing. Be that an extra notification, -changes to an existing notification, or removing a notification. - -Python and Command Line Client Impact -------------------------------------- - -Please specify any changes to the python and command line clients (CLI). Consider -the OpenStack unified clients as well as the soon to be deprecated Barbican clients. - -Other end user impact ---------------------- - -Aside from the API, are there other ways a user will interact with this -feature? - -* Does this change have an impact on python-novaclient? What does the user - interface there look like? - -Performance Impact ------------------- - -Describe any potential performance impact on the system, for example -how often will new code be called, and is there a major change to the calling -pattern of existing code. - -Examples of things to consider here include: - -* A periodic task might look like a small addition but if it calls conductor or - another service the load is multiplied by the number of nodes in the system. - -* Scheduler filters get called once per host for every instance being created, - so any latency they introduce is linear with the size of the system. - -* A small change in a utility function or a commonly used decorator can have a - large impacts on performance. - -* Calls which result in a database queries (whether direct or via conductor) - can have a profound impact on performance when called in critical sections of - the code. - -* Will the change include any locking, and if so what considerations are there - on holding the lock? - -Other deployer impact ---------------------- - -Discuss things that will affect how you deploy and configure OpenStack -that have not already been mentioned, such as: - -* What config options are being added? Should they be more generic than - proposed (for example a flag that other hypervisor drivers might want to - implement as well)? Are the default values ones which will work well in - real deployments? - -* Is this a change that takes immediate effect after its merged, or is it - something that has to be explicitly enabled? - -* If this change is a new binary, how would it be deployed? - -* Please state anything that those doing continuous deployment, or those - upgrading from the previous release, need to be aware of. Also describe - any plans to deprecate configuration values or features. For example, if we - change the directory name that instances are stored in, how do we handle - instance directories created before the change landed? Do we move them? Do - we have a special case in the code? Do we assume that the operator will - recreate all the instances in their cloud? - -Developer impact ----------------- - -Discuss things that will affect other developers working on OpenStack, -such as: - -* If the blueprint proposes a change to the driver API, discussion of how - other hypervisors would implement the feature is required. - - -Implementation -============== - -Assignee(s) ------------ - -Who is leading the writing of the code? Or is this a blueprint where you're -throwing it out there to see who picks it up? - -If more than one person is working on the implementation, please designate the -primary author and contact. - -Primary assignee: - - -Other contributors: - - -Work Items ----------- - -Work items or tasks -- break the feature up into the things that need to be -done to implement it. Those parts might end up being done by different people, -but we're mostly trying to understand the timeline for implementation. - - -Dependencies -============ - -* Include specific references to specs and/or blueprints in nova, or in other - projects, that this one either depends on or is related to. - -* If this requires functionality of another project that is not currently used - by Nova (such as the glance v2 API when we previously only required v1), - document that fact. - -* Does this feature require any new library dependencies or code otherwise not - included in OpenStack? Or does it depend on a specific version of library? - - -Testing -======= - -Please discuss how the change will be tested. We especially want to know what -tempest tests will be added. It is assumed that unit test coverage will be -added so that doesn't need to be mentioned explicitly, but discussion of why -you think unit tests are sufficient and we don't need to add more tempest -tests would need to be included. - -Is this untestable in gate given current limitations (specific hardware / -software configurations available)? If so, are there mitigation plans (3rd -party testing, gate enhancements, etc). - - -Documentation Impact -==================== - -What is the impact on the docs team of this change? Some changes might require -donating resources to the docs team to have the documentation updated. Don't -repeat details discussed above, but please reference them here. - - -References -========== - -Please add any useful references here. You are not required to have any -reference. Moreover, this specification should still make sense when your -references are unavailable. Examples of what you could include are: - -* Links to mailing list or IRC discussions - -* Links to notes from a summit session - -* Links to relevant research, if appropriate - -* Related specifications as appropriate (e.g. if it's an EC2 thing, link the - EC2 docs) - -* Anything else you feel it is worthwhile to refer to