Changes made to comply with OpenStack writing style

The writing styles at few places were not in compliance with
the OpenStack writing style as mentioned in
http://docs.openstack.org/contributor-guide/writing-style
/general-writing-guidelines.html. This patch changes the heading
styles and adds a few other minor changes in the documents.

Change-Id: I0d90ba32ddcef0427b1dc4358206210f166e798a
Partial-Implements: blueprint mistral-doc
This commit is contained in:
Sharat Sharma 2016-10-04 19:17:19 +05:30
parent e6134eeec0
commit c07cf48a60
12 changed files with 254 additions and 170 deletions

View File

@ -3,8 +3,14 @@ Architecture
.. image:: img/mistral_architecture.png
* Engine - handles control and data flow of workflow executions. Computes which tasks are ready and places them in a queue. Passes the data from task to task, deals with condition transitions, etc.
* Task Executors - execute task Actions. Pick up tasks from the queue, run actions, and send results back to the engine.
* Engine - handles control and data flow of workflow executions. Computes which
tasks are ready and places them in a queue. Passes the data from task to
task, deals with condition transitions, etc.
* Task Executors - execute task Actions. Pick up tasks from the queue, run
actions, and send results back to the engine.
* API server - exposes REST API to operate and monitor workflow executions.
* Scheduler - stores and executes delayed calls. It is the important Mistral component since it interacts with engine and executors. It also triggers workflows on events (e.g., periodic cron event)
* Persistence - stores workflow definitions, current execution states, and past execution results.
* Scheduler - stores and executes delayed calls. It is the important Mistral
component since it interacts with engine and executors. It also triggers
workflows on events (e.g., periodic cron event)
* Persistence - stores workflow definitions, current execution states, and
past execution results.

View File

@ -3,8 +3,8 @@ Mistral Devstack Installation
1. Download DevStack::
git clone https://github.com/openstack-dev/devstack.git
cd devstack
$ git clone https://github.com/openstack-dev/devstack.git
$ cd devstack
2. Add this repo as an external repository, edit ``localrc`` file::

View File

@ -10,9 +10,9 @@ Tutorial
.. code-block:: bash
mkdir my_project
cd my_project
vim setup.py
$ mkdir my_project
$ cd my_project
$ vim setup.py
.. code-block:: python
@ -68,9 +68,9 @@ the paths or file names used.
.. code-block:: bash
mkdir -p my_package/sub_package
touch my_package/__init__.py
touch my_package/sub_package/__init__.py
$ mkdir -p my_package/sub_package
$ touch my_package/__init__.py
$ touch my_package/sub_package/__init__.py
3. Write a function in ``yaql.py``.
@ -79,8 +79,8 @@ YAQL context available inside the function.
.. code-block:: bash
cd my_package/sub_package
vim yaql.py
$ cd my_package/sub_package
$ vim yaql.py
.. code-block:: python
@ -120,15 +120,15 @@ For more information about YAQL, read the `official YAQL documentation <http://y
.. code-block:: bash
curl https://bootstrap.pypa.io/get-pip.py | python
pip install --upgrade setuptools
cd -
$ curl https://bootstrap.pypa.io/get-pip.py | python
$ pip install --upgrade setuptools
$ cd -
5. Install the package (note that there is a dot ``.`` at the end of the line).
.. code-block:: bash
pip install .
$ pip install .
6. The YAQL function can be called in Mistral using its name ``random_uuid``.
@ -165,8 +165,8 @@ parent folder of ``my_package`` is ``/path/to/my_project``.
.. code-block:: bash
export PYTHONPATH=$PYTHONPATH:/path/to/my_project
vim $(find / -name "mistral.*egg-info*")/entry_points.txt
$ export PYTHONPATH=$PYTHONPATH:/path/to/my_project
$ vim $(find / -name "mistral.*egg-info*")/entry_points.txt
.. code-block:: ini

View File

@ -1,7 +1,7 @@
Troubleshooting And Debugging
=============================
Mistral-Dashboard Debug Instructions
Mistral-Dashboard debug instructions
------------------------------------
**Pycharm**
@ -66,6 +66,6 @@ After a successful login bring the port back - it will continue your session.
- If you would like to manually restart the apache server,
open a terminal and run::
sudo service apache2 restart
$ sudo service apache2 restart
*(if not under Ubuntu, replace "sudo" with an identical command)*

View File

@ -1,28 +1,30 @@
V2 API
======
This API describes the ways of interacting with Mistral service via HTTP protocol using Representational State Transfer concept (ReST).
This API describes the ways of interacting with Mistral service via HTTP protocol
using Representational State Transfer concept (ReST).
Basics
-------
Media Types
Media types
^^^^^^^^^^^
Currently this API relies on JSON to represent states of REST resources.
Error States
Error states
^^^^^^^^^^^^
The common HTTP Response Status Codes (https://github.com/for-GET/know-your-http-well/blob/master/status-codes.md) are used.
Application Root [/]
Application root [/]
^^^^^^^^^^^^^^^^^^^^
Application Root provides links to all possible API methods for Mistral. URLs for other resources described below are relative to Application Root.
Application Root provides links to all possible API methods for Mistral. URLs
for other resources described below are relative to Application Root.
API v2 Root [/v2/]
API v2 root [/v2/]
^^^^^^^^^^^^^^^^^^
All API v2 urls are relative to API v2 root.
@ -32,7 +34,11 @@ Workbooks
.. autotype:: mistral.api.controllers.v2.resources.Workbook
:members:
`name` is immutable. tags is a list of values associated with a workbook that a user can use to group workbooks by some criteria (deployment workbooks, Big Data processing workbooks etc.). Note that name and tags get inferred from workbook definition when Mistral service receives a POST request. So they can't be changed in another way.
`name` is immutable. tags is a list of values associated with a workbook that
a user can use to group workbooks by some criteria (deployment workbooks,
Big Data processing workbooks etc.). Note that name and tags get inferred from
workbook definition when Mistral service receives a POST request. So they
can't be changed in another way.
.. autotype:: mistral.api.controllers.v2.resources.Workbooks
:members:
@ -47,7 +53,10 @@ Workflows
.. autotype:: mistral.api.controllers.v2.resources.Workflow
:members:
`name` is immutable. tags is a list of values associated with a workflow that a user can use to group workflows by some criteria. Note that name and tags get inferred from workflow definition when Mistral service receives a POST request. So they can't be changed in another way.
`name` is immutable. tags is a list of values associated with a workflow that
a user can use to group workflows by some criteria. Note that name and tags get
inferred from workflow definition when Mistral service receives a POST request.
So they can't be changed in another way.
.. autotype:: mistral.api.controllers.v2.resources.Workflows
:members:
@ -84,7 +93,9 @@ Executions
Tasks
-----
When a workflow starts Mistral creates an execution. It in turn consists of a set of tasks. So Task is an instance of a task described in a Workflow that belongs to a particular execution.
When a workflow starts Mistral creates an execution. It in turn consists of a
set of tasks. So Task is an instance of a task described in a Workflow that
belongs to a particular execution.
.. autotype:: mistral.api.controllers.v2.resources.Task
@ -103,7 +114,9 @@ When a workflow starts Mistral creates an execution. It in turn consists of a se
Action Executions
-----------------
When a Task starts Mistral creates a set of Action Executions. So Action Execution is an instance of an action call described in a Workflow Task that belongs to a particular execution.
When a Task starts Mistral creates a set of Action Executions. So Action Execution
is an instance of an action call described in a Workflow Task that belongs to a
particular execution.
.. autotype:: mistral.api.controllers.v2.resources.ActionExecution
@ -121,7 +134,10 @@ When a Task starts Mistral creates a set of Action Executions. So Action Executi
Cron Triggers
-------------
Cron trigger is an object that allows to run Mistral workflows according to a time pattern (Unix crontab patterns format). Once a trigger is created it will run a specified workflow according to its properties: pattern, first_execution_time and remaining_executions.
Cron trigger is an object that allows to run Mistral workflows according to a time
pattern (Unix crontab patterns format). Once a trigger is created it will run a
specified workflow according to its properties: pattern, first_execution_time and
remaining_executions.
.. autotype:: mistral.api.controllers.v2.resources.CronTrigger
@ -137,16 +153,23 @@ Cron trigger is an object that allows to run Mistral workflows according to a ti
Environments
------------
Environment contains a set of variables which can be used in specific workflow. Using Environment possible to create and map action default values - just provide '__actions' key in 'variables'. All these variables can be accessed in workflow DSL by <% $.__env %> expression.
Environment contains a set of variables which can be used in specific workflow.
Using Environment possible to create and map action default values - just provide
'__actions' key in 'variables'. All these variables can be accessed in workflow
DSL by <% $.__env %> expression.
Example of using in DSL::
Example of using in DSL
.. code-block:: yaml
workflow:
tasks:
task1:
action: std.echo output=<% $.__env.my_echo_output %>
Example of creating action defaults::
Example of creating action defaults
.. code-block:: yaml
...ENV...
@ -174,9 +197,17 @@ Note: using CLI, Environment can be created via JSON or YAML file.
Services
--------
Through service management API, system administrator or operator can retrieve Mistral services information of the system, including service group and service identifier. The internal implementation of this feature make use of tooz library, which needs coordinator backend(the most commonly used at present is Zookeeper) installed, please refer to tooz official documentation for more detailed instruction.
Through service management API, system administrator or operator can retrieve
Mistral services information of the system, including service group and service
identifier. The internal implementation of this feature make use of tooz library,
which needs coordinator backend(the most commonly used at present is Zookeeper)
installed, please refer to tooz official documentation for more detailed
instruction.
There are three service groups according to Mistral architecture currently, namely api_group, engine_group and executor_group. The service identifier contains name of the host the service is running on and the process identifier of the service on that host.
There are three service groups according to Mistral architecture currently, namely
api_group, engine_group and executor_group. The service identifier contains name
of the host the service is running on and the process identifier of the service on
that host.
.. autotype:: mistral.api.controllers.v2.resources.Service
:members:
@ -190,7 +221,8 @@ There are three service groups according to Mistral architecture currently, name
Validation
----------
Validation endpoints allow to check correctness of workbook, workflow and ad-hoc action DSL without having to upload them into Mistral.
Validation endpoints allow to check correctness of workbook, workflow and ad-hoc
action DSL without having to upload them into Mistral.
**POST /v2/workbooks/validation**
Validate workbook content (DSL grammar and semantics).
@ -201,4 +233,5 @@ Validation endpoints allow to check correctness of workbook, workflow and ad-hoc
**POST /v2/actions/validation**
Validate ad-hoc action content (DSL grammar and semantics).
These endpoints expect workbook, workflow or ad-hoc action text (DSL) correspondingly in a request body.
These endpoints expect workbook, workflow or ad-hoc action text (DSL) correspondingly
in a request body.

View File

@ -103,7 +103,7 @@ This example workflow simply sends a command to OpenStack Compute
service Nova to start creating a virtual machine and wait till it's
created using special "retry" policy.
Workflow Types
Workflow types
^^^^^^^^^^^^^^
Mistral DSL v2 introduces different workflow types and the structure of
@ -117,10 +117,11 @@ Mistral provides two workflow types:
See corresponding sections for details.
Common Workflow Attributes
Common workflow attributes
^^^^^^^^^^^^^^^^^^^^^^^^^^
- **type** - Workflow type. Either 'direct' or 'reverse'. *Optional*. 'direct' by default.
- **type** - Workflow type. Either 'direct' or 'reverse'. *Optional*. 'direct'
by default.
- **description** - Arbitrary text containing workflow description. *Optional*.
- **input** - List defining required input parameter names and
optionally their default values in a form "my_param: 123". *Optional*.
@ -175,7 +176,7 @@ ahead it's worth saying that Mistral provides a lot of actions out of
the box (including actions for most of the core OpenStack services) and
it's also easy to plug new actions into Mistral.
Common Task Attributes
Common task attributes
''''''''''''''''''''''
All Mistral tasks regardless of workflow type have the following common
@ -183,7 +184,9 @@ attributes:
- **description** - Arbitrary text containing task description.
*Optional*.
- **action** - Name of the action associated with the task. *Mutually exclusive with* **workflow**. If neither action nor workflow are provided then the action 'std.noop' will be used.
- **action** - Name of the action associated with the task.
*Mutually exclusive with* **workflow**. If neither action nor workflow are
provided then the action 'std.noop' will be used.
- **workflow** - Name of the workflow associated with the task.
*Mutually exclusive with* **action**.
- **input** - Actual input parameter values of the task. *Optional*.
@ -266,8 +269,8 @@ automatically by engine if hasn't completed.
**concurrency**
Defines a max number of actions running simultaneously in a task. *Applicable* only for tasks that
have *with-items*.
Defines a max number of actions running simultaneously in a task. *Applicable*
only for tasks that have *with-items*.
**retry**
@ -283,7 +286,8 @@ Defines a pattern how task should be repeated in case of an error.
considered error.
- **continue-on** - Defines a YAQL expression that will continue iteration
loop if it evaluates to 'true'. If it fires then the task is
considered successful. If it evaluates to 'false' then policy will break the iteration.
considered successful. If it evaluates to 'false' then policy will break the
iteration.
Retry policy can also be configured on a single line as:
@ -295,7 +299,7 @@ Retry policy can also be configured on a single line as:
All parameter values for any policy can be defined as YAQL expressions.
Simplified Input Syntax
Simplified input syntax
'''''''''''''''''''''''
When describing a workflow task it's possible to specify its input
@ -343,7 +347,7 @@ keyword *input*. In this case all the parameters will be effectively
merged. If the same parameter is specified in both ways then the one
under *input* keyword takes precedence.
Direct Workflow
Direct workflow
^^^^^^^^^^^^^^^
Direct workflow consists of tasks combined in a graph where every next
@ -393,7 +397,7 @@ YAML example
   send_success_email:
     action: send_email to_addrs=['admin@mysite.org'] body='Vm is successfully created and its id <% $.vm_id %>'
Direct Workflow Task Attributes
Direct workflow task attributes
'''''''''''''''''''''''''''''''
- **on-success** - List of tasks which will run after the task has
@ -515,7 +519,7 @@ In this case instead of 1 it is possible to specify special
string value "one" which is introduced for symmetry with "all". However,
it's up to the user whether to use "1" or "one".
Reverse Workflow
Reverse workflow
^^^^^^^^^^^^^^^^
In reverse workflow all relationships in workflow task graph are
@ -571,13 +575,13 @@ YAML example
     action: send_email to='admin@mysite.org' body='Vm is created and id <% $.vm_id %> and ip address <% $.vm_ip %>'
     requires: [create_vm, associate_ip]
Reverse Workflow Task Attributes
Reverse workflow task attributes
''''''''''''''''''''''''''''''''
- **requires** - List of tasks which should be executed before this
task. *Optional*.
Processing Collections
Processing collections
^^^^^^^^^^^^^^^^^^^^^^
YAML example
@ -657,7 +661,7 @@ similar to a regular function in general purpose programming language
like Python. It has a name and parameters. Mistral distinguishes 'system
actions' and 'Ad-hoc actions'.
System Actions
System actions
^^^^^^^^^^^^^^
System actions are provided by Mistral out of the box and can be used by
@ -815,20 +819,20 @@ its wrapper - PyV8. For installing it, do the next steps:
1. Install required libraries - boost, g++, libtool, autoconf, subversion, libv8-legacy-dev: On Ubuntu::
sudo apt-get install libboost-all-dev g++ libtool autoconf libv8-legacy-dev subversion make
$ sudo apt-get install libboost-all-dev g++ libtool autoconf libv8-legacy-dev subversion make
2. Checkout last version of PyV8::
svn checkout http://pyv8.googlecode.com/svn/trunk/ pyv8
cd pyv8
$ svn checkout http://pyv8.googlecode.com/svn/trunk/ pyv8
$ cd pyv8
3. Build PyV8 - it will checkout last V8 trunk, build it, and then build PyV8::
sudo python setup.py build
$ sudo python setup.py build
4. Install PyV8::
sudo python setup.py install
$ sudo python setup.py install
Example:
@ -887,7 +891,7 @@ Another example for getting the current date and time:
        publish:
          current_date: <% task(get_date_task).result %>
Ad-hoc Actions
Ad-hoc actions
^^^^^^^^^^^^^^
Ad-hoc action is a special type of action that can be created by user.
@ -1042,7 +1046,7 @@ Attributes
- **actions** - Dictionary containing ad-hoc action definitions.
*Optional*.
Predefined Values/Functions in execution data context
Predefined values/Functions in execution data context
-----------------------------------------------------
Using YAQL it is possible to use some predefined values in Mistral DSL.
@ -1062,8 +1066,10 @@ OpenStack context is available by **$.openstack**. It contains
Task result
^^^^^^^^^^^
Task result is available by **task(<task_name>).result**. It contains task result and directly depends on action output
structure. Note that the *task(<task_name>)* function itself returns more than only task result. It returns the following fields of task executions:
Task result is available by **task(<task_name>).result**. It contains task result
and directly depends on action output structure. Note that the *task(<task_name>)*
function itself returns more than only task result. It returns the following
fields of task executions:
* **id** - task execution UUID.
* **name** - task execution name.

View File

@ -4,15 +4,17 @@ Mistral Configuration Guide
Mistral configuration is needed for getting it work correctly
either with real OpenStack environment or without OpenStack environment.
**NOTE:** The most of the following operations should performed in mistral directory.
**NOTE:** The most of the following operations should performed in mistral
directory.
1. Generate *mistral.conf* (if it does not already exist)::
oslo-config-generator --config-file tools/config/config-generator.mistral.conf --output-file etc/mistral.conf
$ oslo-config-generator --config-file tools/config/config-generator.mistral.conf --output-file etc/mistral.conf
2. Edit file **etc/mistral.conf**.
3. **If you are not using OpenStack, skip this item.** Provide valid keystone auth properties::
3. **If you are not using OpenStack, skip this item.** Provide valid keystone
auth properties::
[keystone_authtoken]
auth_uri = http://<Keystone-host>:5000/v3
@ -22,7 +24,8 @@ either with real OpenStack environment or without OpenStack environment.
admin_password = <password>
admin_tenant_name = <tenant>
4. If you want to configure SSL for Mistral API server, provide following options in config file::
4. If you want to configure SSL for Mistral API server, provide following options
in config file::
[api]
enable_ssl_api = True
@ -32,40 +35,46 @@ either with real OpenStack environment or without OpenStack environment.
cert_file = <path-to-certificate file>
key_file = <path-to-key file>
5. **If you don't use OpenStack or you want to disable authentication for the Mistral service**, provide ``auth_enable = False`` in the config file::
5. **If you don't use OpenStack or you want to disable authentication for the
Mistral service**, provide ``auth_enable = False`` in the config file::
[pecan]
auth_enable = False
6. **If you are not using OpenStack, skip this item**. Register Mistral service and Mistral endpoints on Keystone::
6. **If you are not using OpenStack, skip this item**. Register Mistral service
and Mistral endpoints on Keystone::
$ MISTRAL_URL="http://[host]:[port]/v2"
$ openstack service create workflow --name mistral --description 'OpenStack Workflow service'
$ openstack endpoint create workflow --publicurl $MISTRAL_URL --adminurl $MISTRAL_URL --internalurl $MISTRAL_URL
7. Configure transport properties in the corresponding config section: for RabbitMQ it is **oslo_messaging_rabbit**::
7. Configure transport properties in the corresponding config section: for
RabbitMQ it is **oslo_messaging_rabbit**::
[oslo_messaging_rabbit]
rabbit_userid = <user_id>
rabbit_password = <password>
rabbit_host = <host>
**NOTE**: Make sure that backend transport configuration is correct. Example for RabbitMQ::
**NOTE**: Make sure that backend transport configuration is correct. Example for
RabbitMQ::
[DEFAULT]
rpc_backend = rabbit
8. Configure database. **SQLite can't be used in production**. Use *MySQL* or *PostgreSQL* instead. Here are the steps how to connect *MySQL* DB to Mistral:
8. Configure database. **SQLite can't be used in production**. Use *MySQL* or
*PostgreSQL* instead. Here are the steps how to connect *MySQL* DB to Mistral:
Make sure you have installed **mysql-server** package on your Database machine (it can be your Mistral machine as well).
Make sure you have installed **mysql-server** package on your Database machine
(it can be your Mistral machine as well).
Install MySQL driver for python::
pip install mysql-python
$ pip install mysql-python
Create the database and grant privileges::
mysql -u root -p
$ mysql -u root -p
CREATE DATABASE mistral;
USE mistral
@ -80,14 +89,21 @@ either with real OpenStack environment or without OpenStack environment.
connection = postgresql://<user>:<password>@<database-host>:5432/mistral
9. **If you are not using OpenStack, skip this item.** Update mistral/actions/openstack/mapping.json file which contains all allowed OpenStack actions, according to the specific client versions of OpenStack projects in your deployment. Please find more detailed information in tools/get_action_list.py script.
9. **If you are not using OpenStack, skip this item.**
Update mistral/actions/openstack/mapping.json file which contains all allowed
OpenStack actions, according to the specific client versions of OpenStack
projects in your deployment. Please find more detailed information in
tools/get_action_list.py script.
10. Configure Task affinity feature if needed. It is needed for distinguishing either single task executor or one task executor from group of task executors::
10. Configure Task affinity feature if needed. It is needed for distinguishing
either single task executor or one task executor from group of task executors::
[executor]
host = my_favorite_executor
Then, this executor can be referred in DSL by::
Then, this executor can be referred in DSL by
.. code-block:: yaml
...Workflow YAML...
my_task:
@ -104,5 +120,5 @@ either with real OpenStack environment or without OpenStack environment.
12. After that try to run mistral engine and see it is running without any error::
mistral-server --config-file <path-to-config> --server engine
$ mistral-server --config-file <path-to-config> --server engine

View File

@ -31,7 +31,9 @@ The following should get you started:
$ cp -b <mistral-dashboard>/_50_mistral.py.example <horizon>/openstack_dashboard/local/enabled/_50_mistral.py
3. Since Mistral only supports Identity v3, you must ensure that the dashboard points the proper OPENSTACK_KEYSTONE_URL in <horizon>/openstack_dashboard/local/local_settings.py file::
3. Since Mistral only supports Identity v3, you must ensure that the dashboard
points the proper OPENSTACK_KEYSTONE_URL in
<horizon>/openstack_dashboard/local/local_settings.py file::
OPENSTACK_API_VERSIONS = {
"identity": 3,
@ -39,7 +41,9 @@ The following should get you started:
OPENSTACK_KEYSTONE_URL = "http://%s:5000/v3" % OPENSTACK_HOST
4. Also, make sure you have changed OPENSTACK_HOST to point to your Keystone server and check all endpoints are accessible. You may want to change OPENSTACK_ENDPOINT_TYPE to "publicURL" if some of them are not.
4. Also, make sure you have changed OPENSTACK_HOST to point to your Keystone
server and check all endpoints are accessible. You may want to change
OPENSTACK_ENDPOINT_TYPE to "publicURL" if some of them are not.
5. When you're ready, you would need to either restart your apache::
@ -50,6 +54,6 @@ The following should get you started:
$ cd ../horizon/
$ tox -evenv -- python manage.py runserver
Debug Instructions
Debug instructions
------------------
Please refer to :doc:`Mistral Troubleshooting <../developer/troubleshooting>`
Please refer to :doc:`Mistral Troubleshooting <../developer/troubleshooting>`

View File

@ -1,75 +1,75 @@
Mistral Upgrade Guide
=====================
Database Upgrade
Database upgrade
----------------
The migrations in ``alembic_migrations/versions`` contain the changes needed to migrate
between Mistral database revisions. A migration occurs by executing a script that
details the changes needed to upgrade the database. The migration scripts
are ordered so that multiple scripts can run sequentially. The scripts are executed by
Mistral's migration wrapper which uses the Alembic library to manage the migration. Mistral
supports migration from Kilo or later.
The migrations in ``alembic_migrations/versions`` contain the changes needed to
migrate between Mistral database revisions. A migration occurs by executing a
script that details the changes needed to upgrade the database. The migration
scripts are ordered so that multiple scripts can run sequentially. The scripts are
executed by Mistral's migration wrapper which uses the Alembic library to manage
the migration. Mistral supports migration from Kilo or later.
You can upgrade to the latest database version via:
::
mistral-db-manage --config-file /path/to/mistral.conf upgrade head
$ mistral-db-manage --config-file /path/to/mistral.conf upgrade head
You can populate the database with standard actions and workflows:
::
mistral-db-manage --config-file /path/to/mistral.conf populate
$ mistral-db-manage --config-file /path/to/mistral.conf populate
To check the current database version:
::
mistral-db-manage --config-file /path/to/mistral.conf current
$ mistral-db-manage --config-file /path/to/mistral.conf current
To create a script to run the migration offline:
::
mistral-db-manage --config-file /path/to/mistral.conf upgrade head --sql
$ mistral-db-manage --config-file /path/to/mistral.conf upgrade head --sql
To run the offline migration between specific migration versions:
::
mistral-db-manage --config-file /path/to/mistral.conf upgrade <start version>:<end version> --sql
$ mistral-db-manage --config-file /path/to/mistral.conf upgrade <start version>:<end version> --sql
Upgrade the database incrementally:
::
mistral-db-manage --config-file /path/to/mistral.conf upgrade --delta <# of revs>
$ mistral-db-manage --config-file /path/to/mistral.conf upgrade --delta <# of revs>
Or, upgrade the database to one newer revision:
::
mistral-db-manage --config-file /path/to/mistral.conf upgrade +1
$ mistral-db-manage --config-file /path/to/mistral.conf upgrade +1
Create new revision:
::
mistral-db-manage --config-file /path/to/mistral.conf revision -m "description of revision" --autogenerate
$ mistral-db-manage --config-file /path/to/mistral.conf revision -m "description of revision" --autogenerate
Create a blank file:
::
mistral-db-manage --config-file /path/to/mistral.conf revision -m "description of revision"
$ mistral-db-manage --config-file /path/to/mistral.conf revision -m "description of revision"
This command does not perform any migrations, it only sets the revision.
Revision may be any existing revision. Use this command carefully.
::
mistral-db-manage --config-file /path/to/mistral.conf stamp <revision>
$ mistral-db-manage --config-file /path/to/mistral.conf stamp <revision>
To verify that the timeline does branch, you can run this command:
::
mistral-db-manage --config-file /path/to/mistral.conf check_migration
$ mistral-db-manage --config-file /path/to/mistral.conf check_migration
If the migration path has branch, you can find the branch point via:
::
mistral-db-manage --config-file /path/to/mistral.conf history
$ mistral-db-manage --config-file /path/to/mistral.conf history

View File

@ -1,4 +1,4 @@
How to use YAQL in Mistral
==========================
TBD
TBD

View File

@ -2,20 +2,22 @@ Mistral Main Features
=====================
Task result / Data Flow
Task result / Data flow
-----------------------
Mistral supports transferring data from one task to another. In other words, if *taskA* produces a value then
*taskB* which follows *taskA* can use it. In order to use this data Mistral relies on a query language called
`YAQL <https://github.com/openstack/yaql>`_. YAQL is a powerful yet simple tool
that allows the user to filter information,
Mistral supports transferring data from one task to another. In other words,
if *taskA* produces a value then *taskB* which follows *taskA* can use it.
In order to use this data Mistral relies on a query language called `YAQL <https://github.com/openstack/yaql>`_.
YAQL is a powerful yet simple tool that allows the user to filter information,
transform data and call functions. Find more information about it in the
`YAQL official documentation <http://yaql.readthedocs.org>`_ . This mechanism for transferring data plays a
central role in the workflow concept and is referred to as Data Flow.
`YAQL official documentation <http://yaql.readthedocs.org>`_ . This mechanism
for transferring data plays a central role in the workflow concept and is
referred to as Data Flow.
Below is a simple example of how Mistral Data Flow looks like from the DSL (workflow language) perspective:
Below is a simple example of how Mistral Data Flow looks like from the DSL
(workflow language) perspective:
::
.. code-block:: yaml
version: '2.0'
@ -35,10 +37,11 @@ Below is a simple example of how Mistral Data Flow looks like from the DSL (work
task2:
action: do_something data=<% task(task1).result %>
The task called "task1" produces a result that contains a list of the files in a user's home folder on a host
(both username and host are provided as workflow input) and the task "task2" uses this data using the YAQL
expression "task(task1).result".
"task()" here is a function registered in YAQL by Mistral to get information about a task by its name.
The task called "task1" produces a result that contains a list of the files in
a user's home folder on a host(both username and host are provided as workflow
input) and the task "task2" uses this data using the YAQLexpression
"task(task1).result". "task()" here is a function registered in YAQL by
Mistral to get information about a task by its name.
Task affinity
-------------
@ -49,8 +52,8 @@ tasks on specific Mistral executors. In fact, there are 2 cases:
1. You need to execute the task on a single executor.
2. You need to execute the task on any executor within a named group.
To enable the task affinity feature, edit the "host" property in the "executor" section of the configuration
file::
To enable the task affinity feature, edit the "host" property in the
"executor" section of the configuration file::
[executor]
host = my_favorite_executor
@ -67,13 +70,14 @@ this executor in Workflow DSL::
Task policies
-------------
Any Mistral task regardless of its workflow type can optionally have configured policies.
Policies control the flow of the task - for example, a policy can delay task execution before the task starts
or after the task completes.
Any Mistral task regardless of its workflow type can optionally have
configured policies. Policies control the flow of the task - for example,
a policy can delay task execution before the task starts or after the task
completes.
YAML example
^^^^^^^^^^^^
::
.. code-block:: yaml
my_task:
action: my_action
@ -90,19 +94,24 @@ There are different types of policies in Mistral.
1. **pause-before**
Specifies whether Mistral Engine should put the workflow on pause or not before starting a task.
Specifies whether Mistral Engine should put the workflow on pause or not
before starting a task.
2. **wait-before**
Specifies a delay in seconds that Mistral Engine should wait before starting a task.
Specifies a delay in seconds that Mistral Engine should wait before starting
a task.
3. **wait-after**
Specifies a delay in seconds that Mistral Engine should wait after a task has completed before starting the tasks specified in *'on-success'*, *'on-error'* or *'on-complete'*.
Specifies a delay in seconds that Mistral Engine should wait after a task
has completed before starting the tasks specified in *'on-success'*,
*'on-error'* or *'on-complete'*.
4. **timeout**
Specifies a period of time in seconds after which a task will be failed automatically by the engine if it hasn't completed.
Specifies a period of time in seconds after which a task will be failed
automatically by the engine if it hasn't completed.
5. **retry**
@ -110,29 +119,34 @@ There are different types of policies in Mistral.
* *count* - Specifies a maximum number of times that a task can be repeated.
* *delay* - Specifies a delay in seconds between subsequent task iterations.
* *break-on* - Specifies a YAQL expression that will break the iteration loop if it evaluates to *'true'*. If
it fires then the task is considered to have experienced an error.
* *continue-on* - Specifies a YAQL expression that will continue the iteration loop if it evaluates to
*'true'*. If it fires then the task is considered successful.
* *break-on* - Specifies a YAQL expression that will break the iteration loop
if it evaluates to *'true'*. If it fires then the task is considered to
have experienced an error.
* *continue-on* - Specifies a YAQL expression that will continue the iteration
loop if it evaluates to *'true'*. If it fires then the task is considered
successful.
A retry policy can also be configured on a single line, as follows::
A retry policy can also be configured on a single line, as follows
.. code-block:: yaml
task1:
action: my_action
retry: count=10 delay=5 break-on=<% $.foo = 'bar' %>
All parameter values for any policy can be defined as YAQL expressions.
All parameter values for any policy can be defined as YAQL expressions.
Join
----
Join flow control allows to synchronize multiple parallel workflow branches and aggregate their data.
Join flow control allows to synchronize multiple parallel workflow branches
and aggregate their data.
**Full Join (join: all)**.
**Full join (join: all)**.
YAML example
^^^^^^^^^^^^
::
.. code-block:: yaml
register_vm_in_load_balancer:
...
@ -153,17 +167,17 @@ YAML example
join: all
action: send_email
When a task has property *"join"* assigned with value *"all"* the task will run only
if all upstream tasks (ones that lead to this task) are completed and corresponding
conditions have triggered. Task A is considered an upstream task of Task B if Task A
has Task B mentioned in any of its *"on-success"*, *"on-error"* and *"on-complete"* clauses
regardless of YAQL guard expressions.
When a task has property *"join"* assigned with value *"all"* the task will
run only if all upstream tasks (ones that lead to this task) are completed
and corresponding conditions have triggered. Task A is considered an upstream
task of Task B if Task A has Task B mentioned in any of its *"on-success"*,
*"on-error"* and *"on-complete"* clauses regardless of YAQL guard expressions.
**Partial Join (join: 2)**
**Partial join (join: 2)**
YAML example
^^^^^^^^^^^^
::
.. code-block:: yaml
register_vm_in_load_balancer:
...
@ -184,10 +198,11 @@ YAML example
join: 2
action: send_email
When a task has a numeric value assigned to the property *"join"*, then the task
will run once at least this number of upstream tasks are completed and the
corresponding conditions have triggered. In the example above, the task
"wait_for_two_registrations" will run if two any of the "register_vm_xxx" tasks are complete.
When a task has a numeric value assigned to the property *"join"*, then the
task will run once at least this number of upstream tasks are completed and
the corresponding conditions have triggered. In the example above, the task
"wait_for_two_registrations" will run if two any of the "register_vm_xxx"
tasks are complete.
**Discriminator (join: one)**
@ -196,12 +211,12 @@ In this case instead of 1 it is possible to specify the special string value *"o
which is introduced for symmetry with *"all"*. However, it's up to the user whether to use *"1"* or *"one"*.
Processing Collections (with-items)
Processing collections (with-items)
-----------------------------------
YAML example
^^^^^^^^^^^^
::
.. code-block:: yaml
---
version: '2.0'
@ -231,14 +246,17 @@ YAML example
delay: 5
count: <% $.vm_names.len() * 10 %>
The workflow *"create_vms"* in this example creates as many virtual servers as we
provide in the *"vm_names"* input parameter. E.g., if we specify *vm_names=["vm1", "vm2"]*
then it'll create servers with these names based on the same image and flavor.
This is possible because we are using the *"with-items"* keyword that associates an action
or a workflow with a task run multiple times. The value of the *"with-items"*
task property contains an expression in the form: **<variable_name> in <% YAQL_expression %>**.
The workflow *"create_vms"* in this example creates as many virtual servers
as we provide in the *"vm_names"* input parameter. E.g., if we specify
*vm_names=["vm1", "vm2"]* then it'll create servers with these names based on
the same image and flavor. This is possible because we are using the *"with-items"*
keyword that associates an action or a workflow with a task run multiple times.
The value of the *"with-items"* task property contains an expression in the
form: **<variable_name> in <% YAQL_expression %>**.
The most common form is::
The most common form is
.. code-block:: yaml
with-items:
- var1 in <% YAQL_expression_1 %>
@ -248,16 +266,17 @@ The most common form is::
where collections expressed as YAQL_expression_1, YAQL_expression_2,
YAQL_expression_N must have equal sizes. When a task gets started Mistral
will iterate over all collections in parallel, i.e. the number of iterations will
be equal to the length of any of the collections.
will iterate over all collections in parallel, i.e. the number of iterations
will be equal to the length of any of the collections.
Note that in the *"with-items"* case, the task result (accessible in workflow
context as <% $.task_name %>) will be a list containing results of corresponding
action/workflow calls. If at least one action/workflow call has failed then
the whole task will get into *ERROR* state. It's also possible to apply retry
policy for tasks with a *"with-items"* property. In this case the retry policy will
relaunch all action/workflow calls according to the *"with-items"*
configuration. Other policies can also be used in the same way as with regular non-*"with-items"* tasks.
context as <% $.task_name %>) will be a list containing results of
corresponding action/workflow calls. If at least one action/workflow call has
failed then the whole task will get into *ERROR* state. It's also possible to
apply retry policy for tasks with a *"with-items"* property. In this case the
retry policy will relaunch all action/workflow calls according to the
*"with-items"* configuration. Other policies can also be used in the same way
as with regular non-*"with-items"* tasks.
Execution expiration policy
---------------------------

View File

@ -7,17 +7,17 @@ Prerequisites
Before you start following this guide, make sure you have completed these
three prerequisites.
Install and Run Mistral
Install and run Mistral
~~~~~~~~~~~~~~~~~~~~~~~
Go through the installation manual: :doc:`Mistral Installation Guide </guides/installation_guide>`
Install Mistral Client
Install Mistral client
~~~~~~~~~~~~~~~~~~~~~~
To install mistralclient, please refer to :doc:`Mistral Client / CLI Guide </guides/mistralclient_guide>`
Export Keystone Credentials
Export Keystone credentials
~~~~~~~~~~~~~~~~~~~~~~~~~~~
To use the OpenStack command line tools you should specify environment
@ -32,7 +32,7 @@ with a user ``admin`` in the ``admin`` tenant whose password is ``password``:
$ export OS_USERNAME=admin
$ export OS_PASSWORD=password
Write a Workflow
Write a workflow
----------------
For example, we have the following workflow.
@ -64,7 +64,7 @@ then stores the word "Done" as a result of the second task (`task2`).
To learn more about the Mistral Workflows and what you can do, read the
:doc:`Mistral DSL specification </dsl/dsl_v2>`
Upload the Workflow
Upload the workflow
-------------------
Use the *Mistral CLI* to create the workflow::
@ -80,7 +80,7 @@ The output should look similar to this::
+-------------+--------+---------+---------------------+------------+
Run the Workflow and Check the Result
Run the workflow and check the result
-------------------------------------
Use the *Mistral CLI* to start the new workflow, passing in a list of names