diff --git a/doc/source/index.rst b/doc/source/index.rst index 4140a5c..9e468bf 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -22,6 +22,15 @@ Mitaka specs/mitaka/approved/* +Newton +------ + +.. toctree:: + :glob: + :maxdepth: 1 + + specs/newton/approved/* + Implemented specifications ========================== @@ -38,6 +47,15 @@ Mitaka specs/mitaka/implemented/* +Newton +------ + +.. toctree:: + :glob: + :maxdepth: 1 + + specs/newton/implemented/* + Indices and tables ================== diff --git a/specs/mitaka/implemented/mistral-rerun-update-env.rst b/specs/mitaka/implemented/mistral-rerun-update-env.rst new file mode 100644 index 0000000..737cc53 --- /dev/null +++ b/specs/mitaka/implemented/mistral-rerun-update-env.rst @@ -0,0 +1,128 @@ +=============================== +Allow env update on task re-run +=============================== + +https://blueprints.launchpad.net/mistral/+spec/mistral-rerun-update-env + +Problem description +=================== +On rerunning (and resuming) a workflow execution, allow changes to the +environment variables that were provided at the start of workflow execution. + +Use Cases +--------- +Given the use case where a workflow execution failed because of environment +related issue(s) (i.e. endpoint unavailable, etc.), it is possible that as +part of resolving the environment related issue(s), the endpoint is replaced +(i.e. different host/ip) or that the token passed as credential has expired. +Endpoints and credentials can be passed on workflow invocation under the env +param and then accessed by workflow tasks using the env() function. In these +circumstances, the user will need to be able to update the env variables prior +to re-running the workflow task(s). This also applies to workflow that are +manually paused (i.e. for maintenance) and now resumed but the token passed as +credential in the env has expired. + +Proposed change +=============== +To change environment variables, this will be a two step process. First is to +overlay the new set of env variables to the workflow execution context so any +new task executions will pick up the changes. Second to overlay the new set +to the in context of the existing tasks to be rerun. Any existing tasks that +have completed successfully will not be modified. + +Alternatives +------------ +None + +Data model impact +----------------- +- New env property for the Task API resource model to pass the new set of + environment variables. + +REST API impact +--------------- +Update to the env is only permitted on task re-run or workflow resume. + +For workflow resume, the PUT method of the execution controller will be +affected. The user will pass the new set of environment variables via +params in the Execution resource model. Then the put operation for the +executions controller will pass the updated env to resume_workflow (i.e. +rpc.engineclient().resume_workflow(wf_ex_id, env=env)). The +resume_workflow method will merge the new set of env to the workflow +execution appropriately. + +The following is the data for the PUT request to the execution controller. + +.. code-block:: json + + { + 'state': 'RUNNING', + 'params': '{"env": {"k1": "v1"}}' + } + +For task re-run, the PUT method of the task controller will be affected. +The user will pass the new set of environment variables via the env +property in the Task resource model. Then the put operation for the +tasks controller will pass the updated env to rerun_workflow (i.e. +rpc.engineclient().rerun_workflow(wf_ex_id, task_ex_id, env=env)). The +rerun_workflow method will merge the new set of env to the workflow +execution and the task execution appropriately. + +The following is the data for the PUT request to the task controller. + +.. code-block:: json + + { + 'state': 'RUNNING', + 'reset': True, + 'env': '{"k1": "v1"}' + } + +End user impact +--------------- +- Add --env option to `mistral execution-update` to pick up a json string or + path to a json file containing the list of variables to update. +- Add --env option to `mistral task-rerun` to pick up a json string or + path to a json file containing the list of variables to update. + +Performance Impact +------------------ +None + +Deployer impact +--------------- +None + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + m4dcoder + +Work Items +---------- +- Add DB API method to update env in execution. +- Update resume_workflow in default engine. +- Update rerun_workflow in default engine. +- Update PUT in execution controller. +- Update Task API resource model. +- Update PUT in task controller. +- Update execution-update command in mistral client. +- Update task-rerun command in mistral client. + +Dependencies +============ +None + +Testing +======= +- Test that environment is updated and workflow can rerun successfully. +- Test update of workflow execution and task execution in different states. + Test exception cases where certain states are not allowed (i.e. SUCCESS). + +References +========== +None diff --git a/specs/mitaka/implemented/mistral-workflow-resource-sharing.rst b/specs/mitaka/implemented/mistral-workflow-resource-sharing.rst new file mode 100644 index 0000000..e597372 --- /dev/null +++ b/specs/mitaka/implemented/mistral-workflow-resource-sharing.rst @@ -0,0 +1,194 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +=================================== +Support workflow sharing in Mistral +=================================== + +https://blueprints.launchpad.net/mistral/+spec/mistral-workflow-resource-sharing + + +Problem description +=================== + +Currently, we support creating public scope resource in Mistral, a public +resource(e.g. workflow) could be visible and used by any other tenant of the +system. The motivation of this feature is to avoid too many 'noisy' workflows +when you do ``mistral workflow-list`` in CLI, because you will passively see +all the resources with 'public' property, which you don't care about and will +never use, the situation will get worse especially in scenario of public +cloud. + +Use Cases +--------- + +* As a tenant, I want to share my workflows to a specific tenant, rather than + all other tenants in the system. + +* As a tenant, I want to have the capability to accept or reject workflows + shared to me. + +* As a tenant, I can see the workflows list containing all of the following: + all public workflows, my own workflows and the workflows that I am a member + of. + + +Proposed change +=============== + +To solve those problems, I propose adding new resource sharing API for this +feature, please see detailed information in the following sections. + +Users should always use UUID for using workflow sharing feature, because using +name will cause a lot of confusion and lead to plenty of bugs, e.g. different +tenants could have same name workflows, they can share these workflows to +another tenant, who will be confused when he operates with the workflow name. +Workflow UUID in API has been supported[1] in Mistral. + +Alternatives +------------ + +None + +Data model impact +----------------- + +A new table is needed for this feature, the following table constructs would +suffice :: + + __tablename__ = 'resource_members_v2' + __table_args__ = ( + sa.UniqueConstraint( + 'resource_identifier', + 'resource_type', + 'member_id' + ), + ) + + id = mb.id_column() + resource_identifier = sa.Column(sa.String(80), nullable=False) + resource_type = sa.Column( + sa.String(50), + nullable=False, + default='workflow' + ) + project_id = sa.Column(sa.String(80), default=security.get_project_id) + member_id = sa.Column(sa.String(80), nullable=False) + status = sa.Column(sa.String(20), nullable=False, default="pending") + +Database migration is also needed. + +REST API impact +--------------- + +1. Shares the workflow to a new member. + +POST http://127.0.0.1:8989/v2/workflows//members + +request body:: + + {'member_id': XXX} + +response:: + + { + 'resource_identifier': , + 'resource_type': 'workflow', + 'project_id': , + 'member_id': , + 'status': 'pending' + } + +2. Sets the status for a workflow member. + +PUT http://127.0.0.1:8989/v2/workflows//members/ + +request body:: + + {'status': } + +Only user with whom this workflow is shared could make this call, to +accept or reject the sharing, or remain what the status was. Other users +making this call may get HTTP 404 status code. + +3. Shows workflow member details. + +GET http://127.0.0.1:8989/v2/workflows//members/ + +Response body is a single workflow member entity, user must be the owner +or a member of the workflow. + +4. Return all members with whom the workflow has been shared. + +GET http://127.0.0.1:8989/v2/workflows//members + +If a user with whom this workflow is shared makes this call, the member +list contains only information for that user. +If a user with whom this workflow has not been shared makes this call, the +call returns the HTTP 404 status code. + +5. Deletes a member from the member list of a workflow. + +DELETE http://127.0.0.1:8989/v2/workflows//members/ + +Users making this call must be the owner of the workflow. Please note, +check should be done before the member relationship is deleted, since +after deletion, other users can not use that workflow any more, which may +cause error to existing executions or cron-triggers. + +End user impact +--------------- + +Besides the new API, users can use new commands in CLI for resource sharing +feature. For instance:: + + mistral resource-member-create --type workflow --id \ + --member + +Performance Impact +------------------ + +None + +Deployer impact +--------------- + +None + + +Implementation +============== + +Assignee(s) +----------- + +kong + +Work Items +---------- + +* Add new db schema for resource_member. +* Add db operations for resource members. +* Add new API for workflow sharing. +* Including workflows shared to user, when user query workflows. + + +Dependencies +============ + +None + + +Testing +======= + +Tests should cover all the scenarios memtioned in use cases section. + + +References +========== + +[1]: https://blueprints.launchpad.net/python-mistralclient/+spec/support-id-in-workflow-operation diff --git a/specs/mitaka/implemented/use-workflow-id-in-rest-api.rst b/specs/mitaka/implemented/use-workflow-id-in-rest-api.rst new file mode 100644 index 0000000..53cd863 --- /dev/null +++ b/specs/mitaka/implemented/use-workflow-id-in-rest-api.rst @@ -0,0 +1,142 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +========================================= +Workflow UUID support in Mistral REST API +========================================= + +https://blueprints.launchpad.net/mistral/+spec/use-workflow-id-in-rest-api + + +Problem description +=================== + +Currently, we identify a workflow by its name, of course, the workflow name is +unique within a tenant's scope. However, when we use 'public' workflows or we +want to get benifits from workflow sharing feature[1], we may see more than +one workflows with the same name (you can see a related bug which has been +fixed here[2]). Even worse, users will never see other 'same-name' workflows +when performing ``mistral workflow-get `` command, since it always +returns the first one. + +Look at almost all other projects, they always use UUID as globally unique +resource identifier, especially in the REST API, the resource name is a string +that can be duplicated throughout the whole system, and may be changed +frequently as the time goes by. + +So, I propose we use UUID as the workflow global unique identifier in the REST +API. What's more, we can consider using UUID for other resources after that. + +Use Cases +--------- + +* As a user, I want to see the definition of a public scope workflow, whose + name is the same with one of my private workflows or another public + workflow. + +* As a user or application developer, I want to send REST API to Mistral with + resource UUID contained, rather than resource name. + + +Proposed change +=============== + +We need to support workflow UUID as a parameter of pecan WorkflowsController +related methods(GET, PUT, DELETE), we still support workflow name for backward +compatibility, the magic will be happened in the db api layer. At the +meanwhile, workflow UUID needs to be exposed to end users, using which users +could do operations they want. + +Things will be a little complicated for PUT method. Before, when we want to +update a workflow definition, Mistral only accepts URL like +http://localhost:8989/v2/workflows and the new workflow definition content as +request body, which means workflow name is an identifier when updating +workflow definition, and can't be changed. In order to support UUID, a new +parameter with an appropriate default value will be added to PUT method, with +the UUID, workflow name can be changed. In addition, when updating a workflow +with UUID provided, only one workflow definition could be contained in request +body. + +Since UUID is so important right now, users should see that both via REST API +or Mistral client, a good news is, UUID has already be supported from both +server side and client side[3]. + +Changes should also be made for db interface, operations should be supported +based on UUID. + +Alternatives +------------ + +Without the UUID support in REST API, those problems mentioned in the first +section can't be solved totally. + +Data model impact +----------------- + +None + +REST API impact +--------------- + +All the REST API requests for workflow resource will support UUID, the request +and response body are the same with before, the only exception to this is PUT +HTTP method. + +When updating a workflow with UUID provided, only one workflow definition +could be contained in request body. + +End user impact +--------------- + +It's strongly recommended that users use UUID in URL of HTTP request or in the +command line. + +Performance Impact +------------------ + +None + +Deployer impact +--------------- + +None + + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + kong + +Work Items +---------- + +* Add UUID support for GET, PUT, DELETE mothod of workflow REST API. + + +Dependencies +============ + +None + + +Testing +======= + +* Test the UUID support in REST API and/or Mistral client side. + + +References +========== + +[1]: https://blueprints.launchpad.net/mistral/+spec/mistral-resource-sharing + +[2]: https://bugs.launchpad.net/python-mistralclient/+bug/1518276 + +[3]: https://review.openstack.org/248031 diff --git a/specs/newton/approved/example.rst b/specs/newton/approved/example.rst new file mode 100644 index 0000000..cb39cdb --- /dev/null +++ b/specs/newton/approved/example.rst @@ -0,0 +1,239 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +========================================== +Example Spec - The title of your blueprint +========================================== + +Include the URL of your launchpad blueprint: + +https://blueprints.launchpad.net/mistral/+spec/example + +Introduction paragraph -- why are we doing anything? A single paragraph of +prose that operators can understand. The title and this first paragraph +should be used as the subject line and body of the commit message respectively. + +Some notes about the mistral-spec and blueprint process: + +* Not all blueprints need a spec. If a new feature is straightforward enough + that it doesn't need any design discussion, then no spec is required. It + should be decided on IRC meeting with the whole core team. + +* The aim of this document is first to define the problem we need to solve, + and second agree the overall approach to solve that problem. + +* This is not intended to be extensive documentation for a new feature. + For example, there is no need to specify the exact configuration changes, + + nor the exact details of any DB model changes. But you should still define + that such changes are required, and be clear on how that will affect + upgrades. + +* You should aim to get your spec approved before writing your code. + While you are free to write prototypes and code before getting your spec + approved, its possible that the outcome of the spec review process leads + you towards a fundamentally different solution than you first envisaged. + +* But, API changes are held to a much higher level of scrutiny. + As soon as an API change merges, we must assume it could be in production + somewhere, and as such, we then need to support that API change forever. + To avoid getting that wrong, we do want lots of details about API changes + upfront. + +Some notes about using this template: + +* Your spec should be in ReSTructured text, like this template. + +* Please wrap text at 79 columns. + +* The filename in the git repository should match the launchpad URL, for + example a URL of: https://blueprints.launchpad.net/mistral/+spec/awesome-thing + should be named awesome-thing.rst. + +* Please do not delete any of the sections in this template. If you have + nothing to say for a whole section, just write: None. + +* For help with syntax, see http://sphinx-doc.org/rest.html + + +Problem description +=================== + +A detailed description of the problem. What problem is this blueprint +addressing? + +Use Cases +--------- + +What use cases does this address? What impact on actors does this change have? +Ensure you are clear about the actors in each use case: Developer, End User, +Deployer etc. + + +Proposed change +=============== + +Here is where you cover the change you propose to make in detail. How do you +propose to solve this problem? + +If this is one part of a larger effort make it clear where this piece ends. In +other words, what's the scope of this effort? + +Alternatives +------------ + +What other ways could we do this thing? Why aren't we using those? This doesn't +have to be a full literature review, but it should demonstrate that thought has +been put into why the proposed solution is an appropriate one. + +Data model impact +----------------- + +Changes which require modifications to the data model often have a wider impact +on the system. The community often has strong opinions on how the data model +should be evolved, from both a functional and performance perspective. It is +therefore important to capture and gain agreement as early as possible on any +proposed changes to the data model. + +Questions which need to be addressed by this section include: + +* What new database schema changes is this going to require? + +* What database migrations will accompany this change. + +* How will the initial set of new data objects be generated, for example if you + need to take into account existing workflow/execution, or modify other + existing data, please describe how that will work. + +REST API impact +--------------- + +Each API method which is either added or changed should have the following: + +* Specification for the method. + + * A description of the added or changed method. + + * Method type (POST/PUT/GET/DELETE). + + * Normal http response code(s). + + * Expected error http response code(s). + + * URL for the resource. + + * Parameters which can be passed via the url. + +* Example use case including typical API samples for both data supplied + by the caller and the response. + +End user impact +--------------- + +Aside from the API, are there other ways a user will interact with this +feature? + +* Does this change have an impact on python-mistralclient? What does the user + interface there look like? + +Performance Impact +------------------ + +Describe any potential performance impact on the system, for example +how often will new code be called, and is there a major change to the calling +pattern of existing code. + +Examples of things to consider here include: + +* A small change in a utility function or a commonly used decorator can have a + large impacts on performance. + +* Calls which result in a database queries can have a profound impact on + performance when called in critical sections of the code. + +* Will the change include any locking, and if so what considerations are there + on holding the lock? + +Deployer impact +--------------- + +Discuss things that will affect how you deploy and configure OpenStack +that have not already been mentioned, such as: + +* What config options are being added? Are the default values ones which will + work well in real deployments? + +* Is this a change that takes immediate effect after its merged, or is it + something that has to be explicitly enabled? + +* If this change is a new binary, how would it be deployed? + +* Please state anything that those doing continuous deployment, or those + upgrading from the previous release, need to be aware of. Also describe + any plans to deprecate configuration values or features. + + +Implementation +============== + +Assignee(s) +----------- + +Who is leading the writing of the code? Or is this a blueprint where you're +throwing it out there to see who picks it up? + +If more than one person is working on the implementation, please designate the +primary author and contact. + +Primary assignee: + + +Other contributors: + + +Work Items +---------- + +Work items or tasks -- break the feature up into the things that need to be +done to implement it. Those parts might end up being done by different people, +but we're mostly trying to understand the timeline for implementation. + + +Dependencies +============ + +* Include specific references to specs and/or blueprints in mistral, or in + other projects, that this one either depends on or is related to. + +* Does this feature require any new library dependencies or code otherwise not + included in Mistral? Or does it depend on a specific version of library? + + +Testing +======= + +Please discuss the important scenarios needed to test here, as well as +specific edge cases we should be ensuring work correctly. + +Please discuss how the change will be tested, e.g. how Mistral is deployed? +Does this change need some specific config options? Does this change need +some 3rd party services pre-installed? + + +References +========== + +Please add any useful references here. You are not required to have any +reference. Moreover, this specification should still make sense when your +references are unavailable. Examples of what you could include are: + +* Links to mailing list or IRC discussions + +* Links to notes from a summit session + +* Links to relevant research, if appropriate + +* Anything else you feel it is worthwhile to refer to diff --git a/specs/newton/implemented/example.rst b/specs/newton/implemented/example.rst new file mode 100644 index 0000000..cb39cdb --- /dev/null +++ b/specs/newton/implemented/example.rst @@ -0,0 +1,239 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +========================================== +Example Spec - The title of your blueprint +========================================== + +Include the URL of your launchpad blueprint: + +https://blueprints.launchpad.net/mistral/+spec/example + +Introduction paragraph -- why are we doing anything? A single paragraph of +prose that operators can understand. The title and this first paragraph +should be used as the subject line and body of the commit message respectively. + +Some notes about the mistral-spec and blueprint process: + +* Not all blueprints need a spec. If a new feature is straightforward enough + that it doesn't need any design discussion, then no spec is required. It + should be decided on IRC meeting with the whole core team. + +* The aim of this document is first to define the problem we need to solve, + and second agree the overall approach to solve that problem. + +* This is not intended to be extensive documentation for a new feature. + For example, there is no need to specify the exact configuration changes, + + nor the exact details of any DB model changes. But you should still define + that such changes are required, and be clear on how that will affect + upgrades. + +* You should aim to get your spec approved before writing your code. + While you are free to write prototypes and code before getting your spec + approved, its possible that the outcome of the spec review process leads + you towards a fundamentally different solution than you first envisaged. + +* But, API changes are held to a much higher level of scrutiny. + As soon as an API change merges, we must assume it could be in production + somewhere, and as such, we then need to support that API change forever. + To avoid getting that wrong, we do want lots of details about API changes + upfront. + +Some notes about using this template: + +* Your spec should be in ReSTructured text, like this template. + +* Please wrap text at 79 columns. + +* The filename in the git repository should match the launchpad URL, for + example a URL of: https://blueprints.launchpad.net/mistral/+spec/awesome-thing + should be named awesome-thing.rst. + +* Please do not delete any of the sections in this template. If you have + nothing to say for a whole section, just write: None. + +* For help with syntax, see http://sphinx-doc.org/rest.html + + +Problem description +=================== + +A detailed description of the problem. What problem is this blueprint +addressing? + +Use Cases +--------- + +What use cases does this address? What impact on actors does this change have? +Ensure you are clear about the actors in each use case: Developer, End User, +Deployer etc. + + +Proposed change +=============== + +Here is where you cover the change you propose to make in detail. How do you +propose to solve this problem? + +If this is one part of a larger effort make it clear where this piece ends. In +other words, what's the scope of this effort? + +Alternatives +------------ + +What other ways could we do this thing? Why aren't we using those? This doesn't +have to be a full literature review, but it should demonstrate that thought has +been put into why the proposed solution is an appropriate one. + +Data model impact +----------------- + +Changes which require modifications to the data model often have a wider impact +on the system. The community often has strong opinions on how the data model +should be evolved, from both a functional and performance perspective. It is +therefore important to capture and gain agreement as early as possible on any +proposed changes to the data model. + +Questions which need to be addressed by this section include: + +* What new database schema changes is this going to require? + +* What database migrations will accompany this change. + +* How will the initial set of new data objects be generated, for example if you + need to take into account existing workflow/execution, or modify other + existing data, please describe how that will work. + +REST API impact +--------------- + +Each API method which is either added or changed should have the following: + +* Specification for the method. + + * A description of the added or changed method. + + * Method type (POST/PUT/GET/DELETE). + + * Normal http response code(s). + + * Expected error http response code(s). + + * URL for the resource. + + * Parameters which can be passed via the url. + +* Example use case including typical API samples for both data supplied + by the caller and the response. + +End user impact +--------------- + +Aside from the API, are there other ways a user will interact with this +feature? + +* Does this change have an impact on python-mistralclient? What does the user + interface there look like? + +Performance Impact +------------------ + +Describe any potential performance impact on the system, for example +how often will new code be called, and is there a major change to the calling +pattern of existing code. + +Examples of things to consider here include: + +* A small change in a utility function or a commonly used decorator can have a + large impacts on performance. + +* Calls which result in a database queries can have a profound impact on + performance when called in critical sections of the code. + +* Will the change include any locking, and if so what considerations are there + on holding the lock? + +Deployer impact +--------------- + +Discuss things that will affect how you deploy and configure OpenStack +that have not already been mentioned, such as: + +* What config options are being added? Are the default values ones which will + work well in real deployments? + +* Is this a change that takes immediate effect after its merged, or is it + something that has to be explicitly enabled? + +* If this change is a new binary, how would it be deployed? + +* Please state anything that those doing continuous deployment, or those + upgrading from the previous release, need to be aware of. Also describe + any plans to deprecate configuration values or features. + + +Implementation +============== + +Assignee(s) +----------- + +Who is leading the writing of the code? Or is this a blueprint where you're +throwing it out there to see who picks it up? + +If more than one person is working on the implementation, please designate the +primary author and contact. + +Primary assignee: + + +Other contributors: + + +Work Items +---------- + +Work items or tasks -- break the feature up into the things that need to be +done to implement it. Those parts might end up being done by different people, +but we're mostly trying to understand the timeline for implementation. + + +Dependencies +============ + +* Include specific references to specs and/or blueprints in mistral, or in + other projects, that this one either depends on or is related to. + +* Does this feature require any new library dependencies or code otherwise not + included in Mistral? Or does it depend on a specific version of library? + + +Testing +======= + +Please discuss the important scenarios needed to test here, as well as +specific edge cases we should be ensuring work correctly. + +Please discuss how the change will be tested, e.g. how Mistral is deployed? +Does this change need some specific config options? Does this change need +some 3rd party services pre-installed? + + +References +========== + +Please add any useful references here. You are not required to have any +reference. Moreover, this specification should still make sense when your +references are unavailable. Examples of what you could include are: + +* Links to mailing list or IRC discussions + +* Links to notes from a summit session + +* Links to relevant research, if appropriate + +* Anything else you feel it is worthwhile to refer to