Opportunity to hide sensitive data from http action logs, such as:
* Request headers
* Request body
* Response body
Change-Id: I6d1b1844898343b8fa30f704761096e3d2936c4d
Implements: blueprint mistral-hide-sensitive-data-from-http-actions-logs
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
* Reworked /code_sources and /dynamic_actions API endpoints to
simplify them. For now they don't work with multiple objects and
they are consistent with other endpoints. If needed, we'll add
support for multiple objects (i.e. adding multiple dynamic actions
with a single request) later in a backwards compatible manner.
* Simplified unit tests.
* Got rid of services/*.py modules since they didn't do anything
useful. They were just wrappers around DB API calls.
Change-Id: Ib5a53f1f1a185f0395ffae1ab0c401633fcdd0fc
* Style improvements to make sure the code is compliant with the
coding guidelines
* Fixing small but important things like the mismatch between the
sinatures of the methods find_all() in the DynamicActionProvider
class and the base ActionProvider interface.
* Improved the tests.
* Simplified the implementation of DynamicActionProvider.
Change-Id: Idbfb15b4c3bb415e7fa9c7ece27eabfe674b6059
* added dynamic actions:
these actions are created and modified in runtime,
each action needs a code source to be imported from and a
class name.
- there are 2 new endpoints:
- /v2/code_sources/:
used to add new code sources to mistral.
- /v2/dynamic_actions/:
used to add dynamic actions to mistral in runtime
- a new Action provider (DynamicActionProvider) was added:
it provides the actions created from the dynamic actions api.
Change-Id: I9fe8c28ffdef71016d9dc13aea60a288c8ebaa0a
Signed-off-by: ali <ali.abdelal@nokia.com>
* This module was always a weird entity in the system having just
one function that essentially creates a dynamic class. It was
created just because we didn't understand where else to put this
function. But now after the action provider refactoring we don't
need it anymore. Action instantiation is now a responsibility of
action descriptor classes.
Change-Id: Ic4b6a9a7ca2784a892d2998359edb220ff8c8911
* This patch refactors Mistral with the action provider concept
that is responsible for delivering actions to the system. So
it takes all the burden of managing action definitions w/o
having to spread that across multiple subsystems like Engine
and API and w/o having to assume that action definitions are
always stored in DB.
* Added LegacyActionProvider that represents the old way of
delivering action definitions to the system. It pretty much just
analyses what entries are configured in the entry point
"mistral.actions" in setup.cfg and build a collection of
corresponding Python action classes in memory accessible by names.
* The module mistral/services/actions.py is now renamed to
adhoc_actions.py because it's effectively responsible only for
ad-hoc actions (those defined in YAML).
* Added the new entry point in setup.cfg "mistral.action.providers"
to register action provider classes
* Added the module mistral/services/actions.py that will be a facade
for action providers. Engine and other subsystems will need to
work with it.
* Other small code changes.
Depends-On: I13033253d5098655a001135c8702d1b1d13e76d4
Depends-On: Ic9108c9293731b3576081c75f2786e1156ba0ccd
Change-Id: I8e826657acb12bbd705668180f7a3305e1e597e2
Remove six.moves Replace the following items with Python 3 style code.
- six.moves.urllib
- six.moves.queue
- six.moves.range
- six.moves.http_client
Subsequent patches will replace other six usages.
Change-Id: I80c713546fcc97391c64e95ef708830632e1ef32
* When YAQL output data conversion is disabled there's still
an issue caused by presence of not JSON-compatible types within
a YAQL result. The internal Mistral code is already able to deal
with that (due to the previous changes) by checking that and
converting them to what's needed. However, JSON serialization
may still not work if it's done via the standard "json" library.
The library simply doesn't handle those non-standard types and
raises an exception. We have a sanitizing function that all YAQL
results go through, however, it doesn't make sense to do recursive
sanitizing for performance reasons. It does make sense to convert
data as late as possible to avoid redundant data manipulations. So
the sanitizing function handles only the root object in the object
graph. The solution for this problem is to use our own utility
function based on the "oslo_serialization.jsonutils" that is able
to deal with at least part of the mentioned types, specifically
FrozenDict and iterators. Generators are still a problem and this
new function takes care of that separately, assuming that any
generator is just a special iterator and hence represents a
collection, i.e. a list in JSON terms. It works for all the cases
we've encountered so far working with YAQL.
* Used the new function "utils.to_json_str()" everywhere for JSON
serialization, including the action "std.http".
* Added necessary unit tests.
Closes-Bug: #1869168
Depends-On: I1081a44a6f305eb1dfe68a5bad30110385130725
Change-Id: I9e73ea7cbba215c3e1d174b5189be27c640c4d42
* Use inspector_url when creating a fake client for ironic inspector client
* Add a session and a url for designate fake client
Change-Id: I9cc78df13d0f0715538bbdb76c8ccad273bd2033
To simplify work with jsons and to avoid errors with json arrays
we should add support of request's json param alongside the data
param.
Change-Id: Id866ed13764b1d4db75cf1a819b53a7e8955b34a
Signed-off-by: Oleg Ovcharuk <vgvoleg@gmail.com>
They were removed from cinder api see https://review.opendev.org/#/c/658318/
also move to cinder v3 (v2 is deprecated)
Change-Id: I35dd5927465152bb70822638bbaf7573db1220f1
Using a dict is not guaranteed to work (and actually doesn't with
ironicclient 3.0.0, although a few other things are broken with it).
Change-Id: I59c113b22c60f04e89a631ade8039c4fa62933dc
* Module designateclient.v1 doesn't exist anymore after
python-designateclient 3.0.0 is out. The new client
requires a keystone session so all other parameters
were dropped. Since this service now requires a
a session the generator test now mocks the method
_get_fake_client() for this action.
* Minor style changes.
Change-Id: Ida722828e3f1481e08f52257405ddfa2175733fa
We can't do run senlin actions because have an error when
init client senlin. We need an other way to init client to
run client with cron trigger and manual.
Change-Id: I294d18b341a3c7dd0df9c24588540f9c94dd4562
Closes-Bug: #1843178
Some OpenStack services have a discover_version method [1]
that returned a most recent version supported by API
and client. Mistral should use this method rather than
hardcode API version (manilaclient was done).
[1] https://github.com/openstack/python-novaclient/blob/master/novaclient/api_versions.py#L250
Change-Id: I0459206be5cc390853b9c69e8c5002568d1efa60
The HTTPClient for Ironic is deprecated and will be removed in Stein.
Use the SessionClient for Ironic actions instead. Also uses
endpoint_override param instead of endpoint, as that is also
deprecated.
Change-Id: Ida3b502b25887ec9a7b51c4d6497699cc9466f05
This removes the parameter insecure in the client creation as this
no longer exists in the client. It also changes the client class
to be called as other clients in openstack using sessions
Change-Id: I3cb7ed4255f8996b8bfd9a3e1edba6de50f4e492
Closes-Bug: #1800819
This action is not included in the mapping.json and it is required
to retrieve secrets from barbican in the workflows
Change-Id: I68f2a75a30cbafba1dc5cc2ca222483c7b92dca1
Closes-Bug: #1800820
Using default mutable parameter is bad.
Default parameters are evaluated only once
if you mutate it you will get unexpected results.
Since we don't mutate here the default paramter, make
sure it is unmutable.
Change-Id: Ib5c451a8c8cad7b6c9a009369c1c039563023368
Reply to address is useful when sending email from an unmonitored email
address and to give user a place to respond in order to contact us.
Add Reply-to as described in section 3.6.2 of RFC5222
https://tools.ietf.org/html/rfc5322#section-3.6.2
Change-Id: Ib6b2bdc130e4f9e5170eb88760d69c3e08d2a1c7
This method allows to specify a private key and avoids its storage
in the filesystem of the executors. This can be used later in
combination of a secrets_retrieve to use keys stored in barbican.
Change-Id: Ide438a7f6d24c8bdc9eb2c82e935fd39a6acc2c6
Closes-Bug: #1806703
The new openstacksdk mechanism forces a keystone request to find info
about endpoints. We don't need this for fake client, so skip the
__init__ of the class.
Change-Id: I5b0d89ac57c14f982a6afa638f088d365e0e4ab8
The ssh error message can lead to information leak.
Removing the extra ssh message effects only the CLI call,
the full message is still being logged
Change-Id: I0b28e1cb17d4ce3ae711a25b6eaffb4ebf00ccd6
Closes-Bug: 1783708
Magnum client instantiation does not need the user_id parameter
coming from the context to do the operations.
Change-Id: I70070aee03671bf04ba4b933039b2c3fbf07c16f
Closes-Bug: #1786480
Adds support for cc and bcc addresses to send mails as copy to
administrators and also html formatting. If the html body is specified
the mail will be sent as multipart.
Closes-Bug: #1783349
Change-Id: I2b90354c33052c4b7ae3a98a08e7df1055524a25
remove invalid todo comment in std_actions.py, since there is no
need to implement this feature.
Change-Id: I500312bb039260853a4d96a54c3395992947b9d5
Related-Bug:#1676411
There are still some hardcoded v2 authentication in barbican actions.
This api has been deprecated and removed, so we can change it to use
instead v3. It also removes the version number from some helper methods.
Change-Id: I0390daf841463d11cb7c61653897949989b6e6eb
Closes-bug: #1783316
If the HTTP request fails, we need to fail the task. Returning the error
from the parent class will do this. While this means we also return the
success result it will be ignored by the Mistral engine.
Credit to @lijianying10 on GitHub for sending this fix via a pull
request. Tests were then added to verify the change.
Closes-Bug: #1779973
Change-Id: Ib8754c8de2d6677d71383b3793d0fa168be575f5
Oleg Ovcharuk