This patch fixes 128 unit test failures under Python 3.12 when
building the Debian package under Unstable. Without this patch,
I'm getting:
Traceback (most recent call last):
File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/<<PKGBUILDDIR>>/mistral/services/legacy_scheduler.py", line 152, in _loop
random.Random().randint(0, self._random_delay * 1000) * 0.001
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/random.py", line 336, in randint
return self.randrange(a, b+1)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/random.py", line 312, in randrange
istop = _index(stop)
^^^^^^^^^^^^
TypeError: 'float' object cannot be interpreted as an integer
Change-Id: Ia85ea13923216d090ef1e4deee7840e70f9185b4
With complex parallel joins mistral had no mechanism to choose which
publish (left or right in terms of merge) should it use. It is a
common case when one branch updates existing value, but after merge
we see the old version.
This patch introduce context versioning feature, where every existing
key of mistral context has its version, and this version is used in
the context merge stage.
Change-Id: I604a9a8391150ac4801115b9892f781c33ecfdcb
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
hacking 3.0.x is too old.
Also remove the note about pip's behavior which was already fixed in
recent versions.
Change-Id: I65d350943649c3346ed5741631c01724ddd256ef
This option has never been used by actual logic. This deprecates
the ineffective option so that we can remove it in a future release.
Change-Id: I350e45fc9aef28db8790614ade7a5ad3071e574b
This feature introduces an enhanced error-handling
mechanism for workflows, allowing them to gracefully
handle issues within individual tasks without
causing a complete workflow failure. Previously,
when using subworkflow and passing an incomplete set
of parameters, the entire workflow would terminate.
With this feature, the workflow continues execution,
isolating errors at the task level. Consequently,
partial issues in one task no longer impact other
branches of workflow execution.
Implements blueprint partial-workflow-failure-handling
Change-Id: Id6a910c85c1d6953408682a2a724c4826333422f
We have no way to determine whether the workflow definition has
changed or not.
This complicates the work with Mistral for external services.
A 'checksum' field has been added to workflow_definitions_v2
to solve this issue.
Implements: blueprint mistral-add-checksum-field-to-wf-definition
Change-Id: I99d095c6c74c62f134aedfa3a63308a0712db38d
As is described in the doc[1], we should not iterate over sys.modules
directly because sys.modules can be changed during iterations.
[1] https://docs.python.org/3/library/sys.html#sys.modules
Change-Id: Ib6ab36f93b408fdae59c1c08146f2ca069ed4f96
In error cases join task could lose context of some branches
Change-Id: I58a94c4ebc5d860473c9b48df326f6ea29cba9fa
Closes-Bug: #2020370
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
The backend_url option can sometimes contain secrets.
For example when redis coordination backend is used and authentication
is enabled in redis, the plain redis password is put as an URL element.
[coordination]
backend_url=redis://:password@127.0.0.1:6379
Change-Id: Ie073d9ac8fd8580f7442370291814d99aad92066
After this patch mistral will run tasks using RPC which will distribute tasks amongst available engine threads. this will enhance performance in case of executing huge executions containing multiple tasks.
Implements: blueprint distribute-mistral-operations
Change-Id: I0b7202589eee68ba5560bf2aa60fbbd6118f3719
After this patch, user can update logging format to include root_execution_id in logs, which will be helpful to find and debug logs related to specific workflow execution.
- Logs about creation and status changes of Mistral entities(execution,
task, action execution, etc) are changed to INFO log level.
- User can update logging_context_format_string to include root_execution_id in logs.
Implements: Implements: blueprint improve-mistral-loggers
Change-Id: I54fe058e5451abba6ea7f69d03d498d78a90993e
This change introduces a new option, [healthcheck] enabled, which
enables the healthcheck middleware in mistral-api pipeline.
This middleware allows status check at /healthcheck path, which is
useful for load balancers or any monitoring services to validate health
of its backend services.
This change is created based on the same change proposed to ironic[1].
[1] 6f439414bdcef9fc02f844f475ec798d48d42558
Co-Authored-By: Jim Rollenhagen <jim@jimrollenhagen.com>
Change-Id: I9bf3de8a5ae6a8c9346285147732b773a3667f7e
after this patch user can choose option whether to replace or merge task data to the execution context.
ex: merge_strategy: replace/merge
Implements: blueprint merge-mistral-tasks-data
Change-Id: I3c96bab9953c4995f2b718ac48dff0f153872026
Opportunity to hide sensitive data from http action logs, such as:
* Request headers
* Request body
* Response body
Change-Id: I6d1b1844898343b8fa30f704761096e3d2936c4d
Implements: blueprint mistral-hide-sensitive-data-from-http-actions-logs
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
When building mistral context (using oslo.context) from environment, the
tenant extra key was given, which is not accepted by oslo.context.
This issue was detected when debugging a mistral-client test:
mistralclient.tests.functional.cli.v2.test_cli_v2.NegativeCLITests.test_target_action_execution
Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
Change-Id: I72ce8851ea5b379a8af75b32fb600691638836af
There was a logical flaw where we compared types
that were different causing a TypeError.
There was also a flaw in that we default to the
name of the action when sorting if the key does
not exist, to compare the data both value should
come from the same key as well otherwise they
should be treated as not equal since the data
cannot be.
Change-Id: Idcb276912582bb097dc5c1c9692facde63d5886b
After this patch, it will validate execution-id exists in the DB while fetching tasks for the execution-id So, GET /v2/executions/execution_id/tasks API will return 404 for absent execution id.
Closes-Bug: #1998210
Change-Id: I6ac05f4b87c470a68cd67f19d1bd95a4d6cebe59
This patch adds an ability to rerun failed workflow by
skipping failed tasks. Workflow behavior in skip case could
be configured by new fields in task definition:
* on-skip
* publish-on-skip
Change-Id: Ib802a1b54e69c29b4d0361f048c2b9c076a4c176
Implements: blueprint mistral-task-skipping-feature
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
For many various support reasons, Mistral should have a special
endpoint to store all necessary info data. This endpoint will read
json from created by admin info file. To configure this you should
use mistral configuration:
[api]
enable_info_endpoint = True
info_json_file_path = info.json
Change-Id: I6f344dc15a4ca5c69a6b21841544a31f95eb393f
Implements: blueprint mistral-info-endpoint
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
When creating a periodic workflow execution from a cron trigger, the
description field was filled with info like cron trigger ID, name, etc.
The cron trigger name can be set by user and is then injected in the
description field.
Beside the fact that this is not useful, it may also be dangerous.
The purpose of this patch is to remove the name from the description so
that the lenght of description field is not user dependant anymore.
Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
Change-Id: I326400a4a04fba71ccc90545909212a41d6b3b13
When getting a cron trigger (but also any other object) by name, pecan
core will try to guess the content_type from the extension.
While for most name, the guess will fail, it may succeed, e.g. when a
name is ending with '.pl'.
Pecan guess it as a perl file and then reject the call (with a 404
error).
The fix is to tell pecan to not guess any content_type from extension by
setting guess_content_type_from_ext to False on initialisation.
Closes-Bug: 1987418
Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
Change-Id: I1bc20b953787cae9b42d12bb5eb3c7a94ed4bbdd
A test is randomly failing due to a comparison between two dates:
created_at and started_at.
Most of the time these two dates are identical, but we can have one
second difference when the creation is done at the very end of a second
and the start is done at the beginning of the next one.
The fix is to make sure that we have less or equal than 1 sec between
started_at and created_at.
Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
Change-Id: I081bb659047f7639bed52dfac8b3858b10859404
Some tests which are cancelling workflows are also checking that only one task
was executed.
But the second task is delayed (wait-before: 3) so we can endup in a
situation when the second task appeared in the task_execs.
Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
Change-Id: I425b5ca0c1bedc22db1aa464f5f14aae69f9370e