6356bce814
When using the command `mistral run-action`, by default it will run the action
synchronously unless the action can only be used asynchronously
(action.is_sync returns False). When it runs synchronously the result of the
action is not saved. When it is ran asynchronously the result is saved, as you
need to retrieve it from Mistral afterwards.
There is a argument on the command `--save-result` that can be used, it causes
the action to be ran asynchronously and the result is saved. There is no way
to have the action run synchronously and have the result be saved.
This patch adds a new API parameter `run_sync` which will be exposed by the
CLI as `--run-sync`. This new argument is intended to be used with `--save-
result` but can be be used independently to ensure an action isn't ran
synchronously by mistake. With the new argument the behaviour of the command
is now as follows:
* `mistral run-action` This behaves as it did before, it runs synchronously if
it can, or it schedules for later and saves the action.
* `mistral run-action --save-result` Again, this is the same as before, it
schedules the action to run later and the action is saved.
* `mistral run-action --run-sync` This is similar to having no argument
passed, however, if you try to run an action that can't be used
synchronously it will be rejected and an error is returned.
* `mistral run-action --run-sync --save-result` The combination of the two
arguments runs the actions synchronously and saves the result. If the action
can't be ran synchronously then an error is returned.
(This commit message uses the CLI to demonstrate the API usage, the new
argument is added in in a mistralclient patch. It can of course be used
directly via the API also.)
Change-Id: I4417750fd5ff47016357655370410e9e7348cc25
(cherry picked from commit
|
||
---|---|---|
devstack | ||
doc | ||
etc | ||
functionaltests | ||
mistral | ||
mistral_tempest_tests | ||
rally-jobs | ||
releasenotes | ||
tools | ||
.coveragerc | ||
.gitignore | ||
.gitreview | ||
.pylintrc | ||
.testr.conf | ||
CONTRIBUTING.rst | ||
LICENSE | ||
README.rst | ||
docker_image_build.sh | ||
requirements.txt | ||
run_functional_tests.sh | ||
run_tests.sh | ||
setup.cfg | ||
setup.py | ||
test-requirements.txt | ||
tox.ini |
README.rst
Mistral
Workflow Service for OpenStack cloud.
Installation
Prerequisites
It is necessary to install some specific system libs for installing Mistral. They can be installed on most popular operating systems using their package manager (for Ubuntu - apt, for Fedora, CentOS - yum, for Mac OS - brew or macports).
The list of needed packages is shown below:
- python-dev
- python-setuptools
- python-pip
- libffi-dev
- libxslt1-dev (or libxslt-dev)
- libxml2-dev
- libyaml-dev
- libssl-dev
In case of ubuntu, just run:
apt-get install python-dev python-setuptools libffi-dev \
libxslt1-dev libxml2-dev libyaml-dev libssl-dev
Mistral can be used without authentication at all or it can work with OpenStack.
In case of OpenStack, it works only with Keystone v3, make sure Keystone v3 is installed.
Install Mistral
First of all, clone the repo and go to the repo directory:
$ git clone https://git.openstack.org/openstack/mistral.git
$ cd mistral
Devstack installation
Information about how to install Mistral with devstack can be found here.
Virtualenv installation:
$ tox
This will install necessary virtual environments and run all the project tests. Installing virtual environments may take significant time (~10-15 mins).
Local installation:
$ pip install -e .
or:
$ pip install -r requirements.txt
$ python setup.py install
Configuring Mistral
Mistral configuration is needed for getting it work correctly with and without an OpenStack environment.
Install and configure a database which can be MySQL or PostgreSQL (SQLite can't be used in production.). Here are the steps to connect Mistral to a MySQL database.
Make sure you have installed
mysql-server
package on your Mistral machine.Install MySQL driver for python:
$ pip install mysql-python
or, if you work in virtualenv, run:
$ tox -evenv -- pip install mysql-python
NOTE: If you're using Python 3 then you need to install
mysqlclient
instead ofmysql-python
.Create the database and grant privileges:
$ mysql -u root -p
CREATE DATABASE mistral; USE mistral GRANT ALL ON mistral.* TO 'root'@'localhost';
Generate
mistral.conf
file:$ oslo-config-generator \ --config-file tools/config/config-generator.mistral.conf \ --output-file etc/mistral.conf
Edit file
etc/mistral.conf
according to your setup. Pay attention to the following sections and options:[oslo_messaging_rabbit] rabbit_host = <RABBIT_HOST> rabbit_userid = <RABBIT_USERID> rabbit_password = <RABBIT_PASSWORD> [database] # Use the following line if *PostgreSQL* is used # connection = postgresql://<DB_USER>:<DB_PASSWORD>@localhost:5432/mistral connection = mysql://<DB_USER>:<DB_PASSWORD>@localhost:3306/mistral
If you are not using OpenStack, add the following entry to the
/etc/mistral.conf
file and skip the following steps:[pecan] auth_enable = False
Provide valid keystone auth properties:
[keystone_authtoken] auth_uri = http://<Keystone-host>:5000/v3 identity_uri = http://<Keystone-host:35357/ auth_version = v3 admin_user = <user> admin_password = <password> admin_tenant_name = <tenant>
Register Mistral service and Mistral endpoints on Keystone:
$ MISTRAL_URL="http://[host]:[port]/v2" $ openstack service create --name mistral workflowv2 $ openstack endpoint create \ --publicurl $MISTRAL_URL \ --adminurl $MISTRAL_URL \ --internalurl $MISTRAL_URL \ mistral
Update the
mistral/actions/openstack/mapping.json
file which contains all available OpenStack actions, according to the specific client versions of OpenStack projects in your deployment. Please find more detailed information in thetools/get_action_list.py
script.
Before the First Run
After local installation you will find the commands
mistral-server
and mistral-db-manage
available
in your environment. The mistral-db-manage
command can be
used for migrating database schema versions. If Mistral is not installed
in system then this script can be found at
mistral/db/sqlalchemy/migration/cli.py
, it can be executed
using Python command line.
To update the database schema to the latest revision, type:
$ mistral-db-manage --config-file <path_to_config> upgrade head
For more detailed information about mistral-db-manage
script please check file
mistral/db/sqlalchemy/migration/alembic_migrations/README.md
.
** NOTE: For users want a dry run with SQLite backend(not used in
production), mistral-db-manage
is not recommended for
database initialization due to SQLite limitations. Please
use sync_db
script described below instead for database
initialization.
Before starting Mistral server, run sync_db
script. It
prepares the DB, creates in it with all standard actions and standard
workflows which Mistral provides for all mistral users.
If you are using virtualenv:
$ tools/sync_db.sh --config-file <path_to_config>
Or run sync_db
directly:
$ python tools/sync_db.py --config-file <path_to_config>
Running Mistral API server
To run Mistral API server:
$ tox -evenv -- python mistral/cmd/launch.py \
--server api --config-file <path_to_config>
Running Mistral Engines
To run Mistral Engine:
$ tox -evenv -- python mistral/cmd/launch.py \
--server engine --config-file <path_to_config>
Running Mistral Task Executors
To run Mistral Task Executor instance:
$ tox -evenv -- python mistral/cmd/launch.py \
--server executor --config-file <path_to_config>
Note that at least one Engine instance and one Executor instance should be running in order for workflow tasks to be processed by Mistral.
If you want to run some tasks on specific executor, the task affinity feature can be used to send these tasks directly to a specific executor. You can edit the following property in your mistral configuration file for this purpose:
[executor]
host = my_favorite_executor
After changing this option, you will need to start (restart) the
executor. Use the target
property of a task to specify the
executor:
... Workflow YAML ...
task1:
...
target: my_favorite_executor
... Workflow YAML ...
Running Multiple Mistral Servers Under the Same Process
To run more than one server (API, Engine, or Task Executor) on the same process:
$ tox -evenv -- python mistral/cmd/launch.py \
--server api,engine --config-file <path_to_config>
The value for the --server
option can be a
comma-delimited list. The valid options are all
(which is
the default if not specified) or any combination of api
,
engine
, and executor
.
It's important to note that the fake
transport for the
rpc_backend
defined in the configuration file should only
be used if all
Mistral servers are launched on the same
process. Otherwise, messages do not get delivered because the
fake
transport is using an in-process queue.
Mistral Client
The Mistral command line tool is provided by the
python-mistralclient
package which is available here.
Debugging
To debug using a local engine and executor without dependencies such
as RabbitMQ, make sure your etc/mistral.conf
has the
following settings:
[DEFAULT]
rpc_backend = fake
[pecan]
auth_enable = False
and run the following command in pdb, PyDev or PyCharm:
mistral/cmd/launch.py --server all --config-file etc/mistral.conf --use-debugger
Running examples
To run the examples find them in mistral-extra repository (https://github.com/openstack/mistral-extra) and follow the instructions on each example.
Tests
You can run some of the functional tests in non-openstack mode locally. To do this:
set
auth_enable = False
in themistral.conf
and restart Mistralexecute:
$ ./run_functional_tests.sh
To run tests for only one version need to specify it:
$ bash run_functional_tests.sh v1
More information about automated tests for Mistral can be found on Mistral Wiki.