From cf0cc484bc0709f56779ada07d39e6c564c616f1 Mon Sep 17 00:00:00 2001 From: Renat Akhmerov Date: Tue, 1 Mar 2016 18:52:33 +0600 Subject: [PATCH] Add asynchronous actions doc Change-Id: Idef3fed51099875d2200b1a29af152be23a0215f Implements: blueprint mistral-async-actions-doc --- doc/source/developer/asynchronous_actions.rst | 141 ++++++++++++++++++ doc/source/developer/index.rst | 1 + doc/source/terminology/actions.rst | 3 + 3 files changed, 145 insertions(+) create mode 100644 doc/source/developer/asynchronous_actions.rst diff --git a/doc/source/developer/asynchronous_actions.rst b/doc/source/developer/asynchronous_actions.rst new file mode 100644 index 000000000..3940b1274 --- /dev/null +++ b/doc/source/developer/asynchronous_actions.rst @@ -0,0 +1,141 @@ +===================================== +How to work with asynchronous actions +===================================== + +******* +Concept +******* + +.. image:: /img/Mistral_actions.png + +During a workflow execution Mistral eventually runs actions. Action is a particular +function (or a piece of work) that a workflow task is associated to. + +Actions can be synchronous and asynchronous. + +Synchronous actions are actions that get completed without a 3rd party, i.e. by +Mistral itself. When Mistral engine schedules to run a synchronous action it sends +its definition and parameters to Mistral executor, then executor runs it and upon +its completion sends a result of the action back to Mistral engine. + +In case of asynchronous actions executor doesn't send a result back to Mistral. +In fact, the concept of asynchronous action assumes that a result won't be known +at a time when executor is running it. It rather assumes that action will just +delegate actual work to a 3rd party which can be either a human or a computer +system (e.g. a web service). So an asynchronous action's run() method is supposed +to just send a signal to something that is capable of doing required job. + +Once the 3rd party has done the job it takes responsibility to send result of +the action back to Mistral via Mistral API. Effectively, the 3rd party just needs +to update the state of corresponding action execution object. To make it possible +it must know corresponding action execution id. + +It's worth noting that from Mistral engine perspective the schema is essentially +the same in case of synchronous and asynchronous actions. If action is synchronous, +then executor immediately sends a result back with RPC mechanism (most often, +a message queue as a transport) to Mistral engine after action completion. But +engine itself is not waiting anything proactively, its architecture is fully on +asynchronous messages. So in case of asynchronous action the only change is that +executor is not responsible for sending action result, something else takes over. + +Let's see what we need to keep in mind when working with asynchronous actions. + +****** +How to +****** + +Currently, Mistral comes with one asynchronous action out of the box, "mistral_http". +There's also "async_noop" action that is also asynchronous but it's mostly useful +for testing purposes because it does nothing. "mistral_http" is an asynchronous +version of action "http" sending HTTP requests. Asynchrony is controlled by +action's method is_sync() which should return *True* for synchronous actions and +*False* for asynchronous. + +Let's see how "mistral_http" action works and how to use it step by step. + +We can imagine that we have a simple web service playing a role of 3rd party system +mentioned before accessible at http://my.webservice.com. And if we send an HTTP +request to that url then our web service will do something useful. To keep it +simple, let's say our web service just calculates a sum of two numbers provided +as request parameters "a" and "b". + +1. Workflow example +=================== + + .. code-block:: yaml + + --- + version: '2.0' + + my_workflow: + tasks: + one_plus_two: + action: mistral_http url=http://my.webservice.com + input: + params: + a: 1 + b: 2 + +So our workflow has just one task "one_plus_two" that sends a request to our web +service and passes parameters "a" and "b" in a query string. Note that we specify +"url" right after action name but "params" in a special section "input". This is +because there's no one-line syntax for dictionaries currently in Mistral. But both +"url" and "params" are basically just parameters of action "mistral_http". + +It is important to know that when "mistral_http" action sends a request it includes +special HTTP headers that help identify action execution object. These headers are: + +- **Mistral-Workflow-Name** +- **Mistral-Workflow-Execution-Id** +- **Mistral-Task-Id** +- **Mistral-Action-Execution-Id** +- **Mistral-Callback-URL** + +The most important one is "Mistral-Action-Execution-Id" which contains an id of +action execution that we need to calculate result for. Using that id a 3rd party +can deliver a result back to Mistral once it's calculated. If a 3rd party is a +computer system it can just call Mistral API via HTTP using header +"Mistral-Callback-URL" which contains a base URL. However, a human can also do +it, the simplest way is just to use Mistral CLI. + +Of course, this is a practically meaningless example. It doesn't make sense to use +asynchronous actions for simple arithmetic operations. Real examples when asynchronous +actions are needed may include: + +- **Analysis of big data volumes**. E.g. we need to run an external reporting tool. +- **Human interaction**. E.g. an administrator needs to approve allocation of resources. + +In general, this can be anything that takes significant time, such as hours, days +or weeks. Sometimes duration of a job may be even unpredictable (it's reasonable +though to try to limit such jobs with timeout policy in practice). +The key point here is that Mistral shouldn't try to wait for completion of such +job holding some resources needed for that in memory. + +An important aspect of using asynchronous actions is that even when we interact +with 3rd party computer systems a human can still trigger action completion by +just calling Mistral API. + + +2. Pushing action result to Mistral +=================================== + +Using CLI: + + .. code-block:: console + + $ mistral action-execution-update --state SUCCESS --output 3 + +This command will update "state" and "output" of action execution object with +corresponding id. That way Mistral will know what the result of this action +is and decide how to proceed with workflow execution. + +Using raw HTTP: + + .. code-block:: HTTP + + POST /v2/action-executions/ + + { + "state": "SUCCESS", + "output": 3 + } diff --git a/doc/source/developer/index.rst b/doc/source/developer/index.rst index 951fe744d..3e3b6de3b 100644 --- a/doc/source/developer/index.rst +++ b/doc/source/developer/index.rst @@ -5,5 +5,6 @@ Developer's Reference :maxdepth: 3 creating_custom_action + asynchronous_actions devstack troubleshooting diff --git a/doc/source/terminology/actions.rst b/doc/source/terminology/actions.rst index b8959f23f..d594da54d 100644 --- a/doc/source/terminology/actions.rst +++ b/doc/source/terminology/actions.rst @@ -15,6 +15,8 @@ and result. Third party service should do a request to Mistral API and provide i .. image:: /img/Mistral_actions.png +:doc:`How to work with asynchronous actions ` + System Actions -------------- @@ -23,6 +25,7 @@ actions for specific Mistral installation via a special plugin mechanism. :doc:`How to write an Action Plugin ` + Ad-hoc Actions --------------