Commit Graph

71 Commits

Author SHA1 Message Date
Renat Akhmerov 06a0f33476 Refactor Mistral with Action Providers
* This patch refactors Mistral with the action provider concept
  that is responsible for delivering actions to the system. So
  it takes all the burden of managing action definitions w/o
  having to spread that across multiple subsystems like Engine
  and API and w/o having to assume that action definitions are
  always stored in DB.
* Added LegacyActionProvider  that represents the old way of
  delivering action definitions to the system. It pretty much just
  analyses what entries are configured in the entry point
  "mistral.actions" in setup.cfg and build a collection of
  corresponding Python action classes in memory accessible by names.
* The module mistral/services/actions.py is now renamed to
  adhoc_actions.py because it's effectively responsible only for
  ad-hoc actions (those defined in YAML).
* Added the new entry point in setup.cfg "mistral.action.providers"
  to register action provider classes
* Added the module mistral/services/actions.py that will be a facade
  for action providers. Engine and other subsystems will need to
  work with it.
* Other small code changes.

Depends-On: I13033253d5098655a001135c8702d1b1d13e76d4
Depends-On: Ic9108c9293731b3576081c75f2786e1156ba0ccd
Change-Id: I8e826657acb12bbd705668180f7a3305e1e597e2
2020-09-24 11:10:33 +00:00
Corey Bryant 84041d67b4 Monkey patch original current_thread _active
Monkey patch the original current_thread to use the up-to-date _active
global variable. This solution is based on that documented at:
https://github.com/eventlet/eventlet/issues/592

Change-Id: I98e80bcbc7637bbb6063935043e530718be87f7b
Closes-Bug: #1863021
2020-05-12 06:23:23 +00:00
Renat Akhmerov 753f1bc03f Fix YAQL engine initialization
* When Mistral is launched via launch.py script then YAQL engine class
  initialization logic doesn't use the right properties from the config
  file. This is because this initialization is caused by the chain of
  imports taking its start in launch.py, and this happens before
  the config file is parsed in the main() function of launch.py.
  Using lazy initialization of YAQL engine class (YAQL_ENGINE in
  yaql_expression.py module) solves this issue because now Mistral
  doesn't initialize it immediately before parsing the config and
  waits for the first usage of it.
* Minor style changes per Mistral coding guidelines.

Change-Id: If3367493803b57ef8bc281b1f64f2a223ac86f85
Closes-Bug: #1864785
2020-02-26 11:53:47 +07:00
Eyal f23d3c07b2 Move registration of CLI options to launch.py
This the only place where it is used, should fix the gate
failure of the docs.

Change-Id: Ib4a13acbc65612e3dd23a8a13baad5856777de99
2020-02-18 05:04:20 +00:00
Renat Akhmerov f61929a3c8 Implement engine graceful shutdown
* The functionality of graceful engine shutdown is now possible
  due to correct calculation of the "graceful" flag in the engine
  server's stop() method. Unfortunately, the Oslo Service framework
  doesn't pass it correctly, it simply ignores it in the call chain.
  So the only way to understand if the shutdown is graceful is to
  peek at the configuration property "graceful_shutdown_timeout"
  provided by Oslo Service. If it's greater than zero then we can
  treat it as graceful.
* Oslo Service handles only four OS signals: SIGTERM, SIGINT,
  SIGHUP and SIGALRM. Only sending SIGTERM to the process leads
  to a graceful shutdown. For example, SIGINT (which is equal to
  ctrl + C in a unix shell) interrupts the process immediately.
  So the only way to do a graceful shutdown of an engine instance
  using a unix shell is to run the "kill <PID>" command. This
  needs to be taken into account when using it.
* The patch also changes the order in which the engine server
  stops its inner services so that the underlying RPC server
  (currently Oslo Messaging based or Kombu based) stops first.
  This is needed to make sure that, first of all, no new RPC
  calls can arrive, and thereby, let all active DB transactions
  finish normally w/o starting new ones. Stopping the RPC server
  may be a heavy operation if there are already lots of RPC
  messages waiting for processing that are polled from the queue.
  So to the great extent the entire functionality of graceful
  shutdown will depend on whether an underlying RPC server
  implements the corresponding functionality in the proper way,
  i.e. after calling stop(graceful=True) it will stop receiving
  new calls and wait till all buffered RPC messages are processed
  normally.
* The maximum time given to graceful shutdown is controlled via
  the "graceful_shutdown_timeout" configuration option, which is
  60 seconds, by default.
* Minor refactoring

Implements blueprint: mistral-graceful-scale-in

Change-Id: I6d1234dfa21b1e3420ec9ca2c5235dee973748ee
2019-12-06 09:29:26 +00:00
Eyal 9b576aec7c Use raw strings in all pattern matching strings
This removes some of the deprecation warnings on
unsupported escape sequence for python3.6 and up

Change-Id: I2a6b99aeb3db3b45e0a4c76f1ea4af3b798de73d
2019-08-06 17:57:13 +03:00
Bob Haddleton 488b40834f Update version.version_string to actually be a string
pbr provides VersionInfo.version_string() as a method to determine
the version.  mistral.version.version_string can hide this and
provide a uniform string interface to config and launch.

This fixes a problem with mistral-server --version generating
an exception because the version passed to argparse was a method
instead of a string.

Change-Id: Ie468685e4360bfaec5d82b02f8cf1a27a93bcd94
Closes-Bug: 1796921
2018-10-09 11:30:10 -05:00
Anusree d2f03bc7b0 Enable mutable config in mistral
New releases of oslo.config support a 'mutable' parameter to Opts.
oslo.service provides an option here Icec3e664f3fe72614e373b2938e8dee53cf8bc5e
allows services to tell oslo.service they want mutate_config_files to be
called by passing a parameter.

This commit is to use the same. This allows mistral to benefit from
I1e7a69de169cc85f4c09954b2f46ce2da7106d90, where the 'debug' option
(owned by oslo.log) is made mutable. we should be able to turn debug
logging on and off by changing the config.

tc goal:
https://governance.openstack.org/tc/goals/rocky/enable-mutable-configuration.html

Change-Id: I8ea7085b9343b07f5fe419d30a68c00caec1b745
2018-07-25 03:40:34 +05:30
wangxiyuan dcb4355e59 Use register_session_conf_options API
Using this API is needed to correctly initialize the configuration.

[keystone_authtoken] group is used for keystonemiddleware to validating
token.

The new [keystone] group is used for keystoneauth to initialization
the keystone session.

Co-Authored-By: wangxiyuan<wangxiyuan@huawei.com>

Change-Id: Ie3ab512b0ab42c0f97b3099e0787f4edee08ddce
Partial-Bug: #1775140
2018-07-20 09:22:59 +08:00
Renat Akhmerov afb0f584c5 Use "passive_deletes=True" in ORM relationships
* Using "passive_deletes=True" when configuring relationships
  with SQLAlchemy significantly reduces memory footprint when
  deleting a graph of objects (e.g. root workflow execution
  with all dependent tasks, actions and sub workflows) from DB.
  This happens because in this case SQLAlchemy doesn't load
  those dependent objects that are not yet in the session and
  rather relies on cascading mechanisms of databases which are
  activated by the proper configuration of foreign key columns.
  For example, "ON DELETE CASCADE" clause in the foreign key
  definition in case of for MySQL.
  Particularly, this change reduces memory footprint of API
  processes when deleting large graphs of executions and
  execution expiration policy mechanism where the same happens
  but on the engine side.
* Added a test to make sure that cascaded deletion works well
  with this new property.

Related-Bug: #1757966
Change-Id: I36988817fb8e7695094ef98958b8aa151208fadb
2018-05-15 16:04:02 +07:00
Mike Fedosin 6fb10f5498 Fix server info string
Now there is a discrepancy between how to display the string with
information about the server with the started with parameter "all"
and all other cases.

'all': [api,engine,executor,event-engine,notifier]
other: [['api', 'engine', 'executor']]

TrivialFix

This commit fixes this situation by unifying the string format.

Change-Id: If223fc87db8a48ad64868f5e847d54279f251e40
2018-03-13 23:32:12 +01:00
Winson Chan 3f48e24dc4 Implement notification of execution events
Introduce execution events and notification server and plugins for
publishing these events for consumers. Event notification is defined per
workflow execution and can be configured to notify on all the events or
only for specific events.

Change-Id: I9820bdc4792a374dad9ad5310f84cd7aaddab8ca
Implements: blueprint mistral-execution-event-subscription
2018-02-24 07:25:55 +00:00
Renat Akhmerov 05252da835 Added session.flush() before update_on_match()
* It turns out that update_on_match() from oslo_db breaks fields
  of a persistent object (attached to the session) in some cases.
  For example, when we use PostgreSQL as a DB it doesn't properly
  merge fields of the given specimen onto the fields of the object
  attached to the SQLAlchemy session. For some reason, it cleans up
  the history of ORM events registered before this operation so
  that if some update took place in this session and they are not
  flushed yet to DB they simply get lost. Hence adding flush()
  on the session right before this operation helps.
* NOTE: this is rather a workaround than a fix of the root cause.
  Root cause still needs to be found.

Change-Id: I546badd8b1c96b68567287fc9d38b07738272503
Closes-Bug: #1736821
2018-01-22 16:16:48 +07:00
Renat Akhmerov eeed0730d7 Revert "Enable eventlet monkey patching for MySQLdb driver"
This reverts commit 89f98b7659.

Change-Id: I88471b5ff1d7159aed14dc9923c670b931c36ff8
2017-10-16 07:02:34 +00:00
Mike Fedosin 89f98b7659 Enable eventlet monkey patching for MySQLdb driver
Using MySQLdb driver for MySQL is not compatible with
greenthreads provided by eventlet library. If we issue
an SQL query through it from within a green thread then
eventlet dispatches doesn't switch to another green
thread although it is an IO operation.

We need to apply monkey patching to MySQLdb explicitly
as it was fixed some time ago but doesn't apply by default.

Change-Id: Id32c65b628d8ffec5452cc89d7cf5a0a24f0312d
Closes-bug: #1721218
2017-10-06 03:40:48 +00:00
Winson Chan 8a4fe7f146 Fix services launcher to handle shutdown properly
When Mistral services are configured with systemd, the services time out
on shutdown. The root cause is that each of the server thread is
assigned to a different instance of the oslo.service ServiceLauncher.
The ServiceLauncher can manage multiple server threads. When the server
threads are added to the same instance of the ServiceLauncher, SIGTERM
is handled properly for each server threads.

Change-Id: I8b739844f6073890324189aee028c0d7da3cc76e
Closes-Bug: #1714351
2017-09-01 18:54:12 +00:00
Adriano Petrich 1f68141b3d Change the logo to lowercase
Move from
|\\    //| //   // |||||| |||\\       /\      ||
||\\  //||    //     ||   ||  ||     //\\     ||
|| \\// || || ||     ||   || //     //  \\    ||
||  \/  || ||  \\    ||   || \\    //-||-\\   ||
||      || ||   ||   ||   ||  ||  //      \\  ||
||      || || _//    ||   ||  || //        \\ |||||

to:
|\\    //|           ||                       ||
||\\  //||      __   ||      __      __       ||
|| \\// || ||  //  ||||||  ||  \\  //  \\     ||
||  \/  ||     \\    ||    ||     ||    \\    ||
||      || ||   \\   ||    ||     ||    /\\   ||
||      || || __//   ||_// ||      \\__// \\_ ||

Change-Id: I2c96f05ca32f3fbb9a1b0a5c3cd2f3b063242702
2017-07-21 10:54:07 +01:00
Winson Chan eb09a90749 Make rpc_backend not engine specific
The rpc_backend with kombu and oslo are being used by the executor
and event engine as well. This patch move the rpc_backend up one
level so it's not engine specific. Also Event engine has its own module
and the EventEngine class is defined in the engine module. This patch
moves the EventEngine to it's own base file in the event_engine module.

Implements: blueprint mistral-actions-run-by-engine

Change-Id: Ie814a26e05f5ca6bfba10f20a7d5921836aa7602
2017-04-27 20:02:40 +00:00
Winson Chan 62d8c5edaf Add option to run actions locally on the engine
Make executor pluggable and allow option to run the executor
locally on the engine or remotely over RPC.

Change-Id: I7cfb13068aa1d1f88136eaa092e629c34b78adf2
Implements: blueprint mistral-actions-run-by-engine
2017-04-27 20:02:27 +00:00
Dougal Matthews 8a52e9e8fc Display all the possible server values.
Change-Id: I53d8330247ccd8f1c677a2e45dc35098fee0a162
2016-12-15 09:24:53 +00:00
Renat Akhmerov a5d284aa4f Fix launch process of Mistral components
* Fixed the bug #1622534
* Introduced MistralService class extending oslo.service.Service
  that all Mistral components running standalone should extend
* Refactored engine, executor and event engine with MistralService
* Moved most of the startup logic from launch.cmd to individual
  corresponding components
* Cluster membership is now under control of MistralService
* Fixed test_join.test_full_join_with_conditions() which previously
  had a bug and we were just 'lucky' that it passed due to a
  different work of scheduler
* Fixed a number of test cases
* Other minor changes

TODO:
* We now use many launchers (from oslo.service) to launch
  services whereas we could use just one which is recommended by
  oslo. However, we can't do that because of the api service
  that uses many workers. We'll need to look at how to refactor it
  moving forward.
* Write tests for MistralService and its derrived classes
* Address a number of TODO comments.

Closes-Bug: 1622534
Change-Id: I34ba6a8b8caf8bea17109e0f259085b373eb6d45
2016-11-30 17:42:15 +07:00
Luong Anh Tuan 1827e5434e Using sys.exit(main()) instead of main()
TrivialFix: Similar [1] in Kolla project
As we known, Exceptions are raised by the sys.exit() function. When they
are not handled, no stack traceback is printed in the Python interpreter.
Therefore, when using sys.exit(main()) instead of main()
may be more readable and reasonable.

[1] https://review.openstack.org/#/c/349353/

Change-Id: Ic71d4bfb1085c68ea9eb7e1382e7263a81634ad1
2016-09-26 10:52:42 +07:00
Dawid Deja cca13bb3db Fix for timeouting actions on run-action
Change-Id: I2aaed3c431134312942ab71ea8ad95f9b8529234
Partial-Bug: #1624284
2016-09-22 09:25:14 +00:00
Cao Xuan Hoang 72a34e8593 TrivialFix: Remove logging import unused
This patch removes logging import unused in
mistral/cmd/launch.py
mistral/engine/actions.py
mistral/utils/rest_utils.py

Change-Id: Ie742d1565f505404fdb58d4d9d0f013ae4475ff4
2016-08-26 17:19:06 +07:00
hardik 4adc3cfb56 Added support for SSL connection in mistra-api server
Change-Id: I4e114a11191821f4ef9dec967fb76b0a34179119
Implements: blueprint mistral-api-server-https
2016-08-04 19:03:53 +05:30
Lingxian Kong 2f3d897312 Add event engine service
This patch adds a new optional service in Mistral, event engine
service will listen to the message queue with exchange and topic users
configed, and trigger the corresponding workflow via rpc call to
mistral engine. The event engine service will receive rpc messages
from mistral api

Change-Id: I8bef24822575670ca90ef0479b9ce2e3e330cdeb
Implements: blueprint event-notification-trigger
Co-Authored-By: Lingxian Kong <anlin.kong@gmail.com>
2016-07-22 23:19:16 +12:00
Roman Dobosz 606059add8 Rename package mistral.engine.rpc to mistral.engine.rpc_backend
There is a small issue during changing branch from current to Mitaka, where
there was module called rpc.py, and currently there is a package rpc. Problem
is, if Mistral from master was already at least run once, and changed back to
Mitaka, there is a chance, that directory rpc still exists, and contains pyc
files. Trying to use old rpc module will fail with unable to find
get_transport method due remains of bytecode from master branch.

Simple solution for this situation is to rename rpc module to rpc_backend.

Change-Id: Ib5d05930a9715caf2bb5ce1dba4e520a503bc0cd
2016-07-14 15:15:50 +02:00
Nikolay Mahotkin 95e6b34b17 Integrating new RPC layer with Mistral
At this point all tests pass, but there is no test for kombu driver.
Komu driver must be consider as expermiental before new tests are introduced.

TODO (next commits):
 - tests

Partially implements blueprint mistral-alternative-rpc

Co-Authored-By: Dawid Deja <dawid.deja@intel.com>
Change-Id: I1f5ca1f1e8c741efcb61480ccbec8e50ad993cba
2016-07-05 11:30:02 +02:00
Nikolay Mahotkin fc6d712840 Make RPC implementation configurable
Partially implements blueprint mistral-alternative-rpc

Co-Authored-By: Dawid Deja <dawid.deja@intel.com>
Change-Id: I7308e75625e108b5b1395b6cfa6dc1d6321fe253
2016-07-05 11:30:02 +02:00
Nikolay Mahotkin 2cdd95aa29 Adding OsloRPC server and client
* Using new RPC interface OsloRPCServer and
   OsloRPCClient are created.

TODO (next commits):
 - integrate new RPC interface in Mistral
 - make RPC implementation configurable
 - unit tests

Partially implements blueprint mistral-alternative-rpc

Co-Authored-By: Dawid Deja <dawid.deja@intel.com>
Change-Id: I6c9770a5b84509529abc14dff2b4a9f6e3411951
2016-07-05 11:30:02 +02:00
Renat Akhmerov 967a8c09e6 Remove obsolete config option "use_mistral_rpc"
* "use_mistral_rpc" is not used anymore
* Style changes

Change-Id: I1f0f5a9306cd2ec7f5d4f2e0304cb4465a10afed
2016-07-04 18:16:52 +07:00
Winson Chan 45ac6d03b1 Enable osprofiler to measure performance
Add option to enable osprofiler to measure performance of workflow execution.

Change-Id: I98e66e3f721c134370848dc2b65fb37c49b0e8ee
Implements: blueprint mistral-osprofiler
2016-06-10 01:49:38 +00:00
Bertrand Lallau e697cc716d Remove unnecessary executable permissions
This removes executable permissions on launch module not requiring it.

Change-Id: Ie964bd76601b88c79ebd34e71e2a46e66e2d7a10
2016-05-23 17:30:14 +02:00
Nikolay Mahotkin b562989e78 Ack message after processing (oslo.messaging)
This patch fixes the HA gap when executor dies.
Now if executor dies, next executor pick up previous executor's task.

 * Currently it is almost impossible to write unit-test on
 this bug. For now, I created a new config option for whether
 to use this fix or to use original oslo.messaging. By default
 it is False.

 * For tests, need to wait creating of HA-gate.

Closes-Bug: #1502120

Change-Id: Ia6d25d039b1e8210b7e544540e4b527d28f6d394
2016-03-16 15:56:30 +00:00
cheneydc ab97276542 Fix the problem when parse config file
Mistral cannot parse configure file from command line
like this:
--config-file="/path/to/config_file"

So update the code to recognize configure file rightly.

Change-Id: I9f3c23d0437578c9714019c480785198c940d181
Closes-Bug: #1552146
2016-03-03 09:25:08 +08:00
Daryl Mowrer 984cec8547 Change for synchronous Mistral actions from CLI
This patch allows the Mistral actions to be run synchronously from
the Mistral CLI.

When a Mistral action is run synchronously using the Mistral CLI
'run-action' command, it does not complete successfully. This is due
to the Mistral API server handling requests on a single thread.

The 'run-action' command performs a REST POST call to the
ActionExecution RestController in the API server. That in turn calls
back into the python-mistralclient which then performs another REST
call back to the appropriate REST controller to actually run the
action requested. That call hangs since the requests are handled on
a single thread because the first POST has not completed yet.
Eventually the RPC call between the engine and the executor servers
times out, and the 'run-action' command fails.

This patch changes the Mistral API server so that requests are
handled in separate threads.

Added a new functional test to the tempest test package to test
synchronous action execution of a mistral action from within
mistral.

Change-Id: I8e06d3ef6aab4b2009a8fff4aa4d1acc118eee3f
Implements: blueprint mistral-mistral-actions
2016-02-23 13:49:10 -06:00
Anastasia Kuznetsova dbdc29531c Fix all H405 pep8 errors
Improved code style, fixed all H405 (Multi line docstring
summary not separated with an empty line) errors.

Change-Id: I6639a2e1a9dc5d3802cb1bda05c5bf9b302bc82f
2015-11-16 18:00:18 +03:00
Lingxian Kong ec28a305d9 Removing wait() when initializing notification listener
According to [1],the correct usage is to call stop() before wait(), so in
mistral case, we should never call stop() and wait() in initialization of
engine and executor service.

[1]:
http://docs.openstack.org/developer/oslo.messaging/notification_listener.html

Change-Id: Ic6a802bff35a6a6fdfadf38474782efd12fd34d8
Closes-Bug: #1514459
2015-11-12 11:44:28 +08:00
Jenkins 29ff8478d2 Merge "Add coordination feature to mistral service" 2015-08-12 04:38:38 +00:00
Guy Paz a8436dc5e2 Expiration policy for expired Executions
A mechanism that remove old executions (expired) according to Admin policy: how old they are, when executing this policy.
By the default this task is disable.

Change-Id: I15433d176d8e4a499a4466fc9324ceef60ddc4b9
Implements: blueprint expiration-policies-for-executions
2015-08-11 15:24:53 +00:00
LingxianKong 021824341b Add coordination feature to mistral service
Add mistrai-api, mistral-engine, mistral-executor services to their
service group respectively when they start up, so that we can retrieve
their information through service api which will be implemented in
subsequent patch.

Change-Id: Ibfcc17b35b9d2e408888b7d1cb64e63f2b32a90d
Partially-Implements: blueprint mistral-service-api
2015-08-11 21:54:33 +08:00
Doug Hellmann 506800208a Drop use of 'oslo' namespace package
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.

The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.

Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.

Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages

Change-Id: I73addc2c144c76c60f046e83c97e3b6ffe09d879
2015-06-22 20:02:59 +00:00
LingxianKong a674853d62 Make use of graduated oslo.log module
* use oslo.log for logging functionality
* get rid of the file /mistral/openstack/common/log.py

Implements: blueprint mistral-log-enhancement
Partial-Bug: #1459188

Change-Id: I6b62cffda0a20b6caf59cc30bcc33c21ae1e5898
2015-06-09 17:38:19 +08:00
Limor Stotland 379c6eb85b Fix command line arguments has lower priority than config file
*This bug happened because of how oslo is handling the order of command line

  *Make sure the command line parameter will always send last

Change-Id: Ia106ce2bd2994c494a6c25039d92358eeb0abdb9
Closes-Bug: #1454111
2015-05-27 07:24:43 +00:00
Renat Akhmerov 235a863c1b Renaming "engine1" to "engine"
Change-Id: I106704c26b4598f9f3dc2ed04213b1f565d00010
2015-04-09 17:47:36 +06:00
Renat Akhmerov b38328513a Deleting all v1 related stuff
* Old engine
* Endpoints
* Tests
* Fixed launch.cmd script
* Fixed the bug with incorrect RPC objects creation in tests

Change-Id: I8579d0878e8a295823124d3c3452f3c26151d43a
2015-04-07 20:07:24 +06:00
Nikolay Mahotkin 6f415eb2b2 Fixing version info in server title
Change-Id: I04e1c40199fa5663305f53a9117ed7d2de7009b1
2015-04-06 13:11:53 +03:00
Anastasia Kuznetsova 8919453e10 Fix H904 pep8 error
Change-Id: I6540c5547a68721cc40b46b96da3648be00fa766
2015-02-09 15:23:06 +04:00
Renat Akhmerov 2f8cc22235 Adding print out of server information into launch script
Change-Id: I57afa2c2340e209e10f4d4eeb2f8b1fd77d0dac3
2014-10-22 10:44:23 +07:00
Renat Akhmerov 74ebdf1122 Removing engine redundant parameter
Change-Id: Ief5a4d08187049e855da3b03fab521e69f0e08dd
2014-09-18 16:20:36 -07:00