Commit Graph

420 Commits

Author SHA1 Message Date
David Moreau Simard 4a8711e73c
opendev: Remove content and leave an URL to the GitHub repository
Change-Id: I82a3238b6a8c21e6bb8056aca22ef71af9ea2538
2021-02-25 15:16:53 -05:00
David Moreau Simard f4ec13a5bf
ui: discard search arguments in links to index and playbooks
The intent when clicking on these is to return to the full unfiltered
index.

Change-Id: I1908acb6ab8d7a446b924bbe1d80627bf7901a35
2021-01-29 10:43:59 -05:00
David Moreau Simard fa8bfcb8bd
ui: Keep form arguments when filtering/sorting by date/duration
The links provided by the UI when filtering or sorting by date/duration
didn't carry existing search arguments so if you sorted or filtered, you
lost existing search queries.

The sorting and filtering is now handled by the same django form which
takes care of keeping our arguments around.

Fixes: https://github.com/ansible-community/ara/issues/201
Change-Id: I03af5baaadd7fd731133db3ad83dee74ed4813ba
2021-01-29 10:43:59 -05:00
Zuul 9144020d7d Merge "cli: add missing controller arg to playbook metrics" 2021-01-22 21:38:37 +00:00
David Moreau Simard 294d90fe8d
cli: add missing controller arg to playbook metrics
This provides the ability to get playbook metrics for a specific
controller.

Change-Id: I57abfc1bf6ef5e0ceb420da4f9a28094269a78ab
2021-01-22 11:21:31 -05:00
David Moreau Simard e3d358f99e
ui: stop filtering status to changed for successful playbooks
It's useful to filter status to failed/unreachable for failed playbooks
but filtering for changed results by default often leads to no results
being shown at all in the case of playbooks without changes.

I find myself clicking "clear filter" more times than the default filter
being useful so let's not do that anymore.

Change-Id: I7635a2282e1e12bbab4224960d36055a80be8f50
2021-01-22 11:16:03 -05:00
David Moreau Simard f9500e1b50 api/cli/ui: add support for searching by ansible_version
This allows to search playbooks for a specific ansible version.

Change-Id: Ie9e53d39993b1c6b7b2ff5ed06885116cb8c526a
2021-01-14 18:31:12 +00:00
David Moreau Simard 7b7849bb0f ui: fix syntax highlighting in result details
We were not including the pygments css file in the result details page
so despite the right css classes being set by the format_data filter, we
didn't have any highlighting.

Change-Id: Iae20c9ec82164bb24b69e4c056aec281b7ac40dc
2021-01-14 18:31:00 +00:00
David Moreau Simard f80ad9b2f2 ui: render nested results like regular results
Results for a task which involves a loop are typically stored in a
'results' list. This renders these results like normal ones inside the
'results' key instead of being raw json.

Change-Id: I3ec57d5f4fbe94809e7feef75b9f7f9e423a8a8b
2021-01-14 18:30:47 +00:00
David Moreau Simard 13837784e3
ui: Improve display for result diffs
Fixes: https://github.com/ansible-community/ara/issues/29

Change-Id: Id4442f0f9c4dcb5520f63970bb9fb17a03d5ec0d
2021-01-14 09:46:54 -05:00
David Moreau Simard f3da150e10 cli: Add 'ara playbook metrics'
This provides a first implementation in order to retrieve playbook
metrics through the CLI.

Change-Id: Id04359b798b86f5e2131aa772d4871c521acfc45
2020-12-18 21:36:29 +00:00
David Moreau Simard 4b5f09df87 cli: Add 'ara host metrics'
This provides a first implementation for retrieving host metrics through
the CLI.

Change-Id: Iaf2cb3960a60113b6068cc36c236b6ff733585b9
2020-12-18 21:36:21 +00:00
David Moreau Simard 7e6635da73 cli: Add 'ara task metrics'
This is a first implementation that provides a way to get metrics about
tasks.

Change-Id: Id0d14ab3c62a5c76fd8f7de1a447b8e16c80771a
2020-12-18 21:35:56 +00:00
David Moreau Simard 52b201dae2
Add support for recording controller fqdn
The controller fqdn that ran the playbook is now recorded and can be
searched for both in the UI and the CLI.

Fixes: https://github.com/ansible-community/ara/issues/193
Change-Id: I53e8d158fc3b6ba7a16582234aaa2542eab5fcdc
2020-12-18 16:35:40 -05:00
David Moreau Simard 4fe18c6492
callback: disable threading by default
Threading is prone to database lock exceptions when using the default
database backend (sqlite) and so disable it by default.

Users can enable threading when using mysql or postgresql by specifying
the ARA_CALLBACK_THREADS environment variable or callback_threads in
ansible.cfg.

Fixes: https://github.com/ansible-community/ara/issues/195
Change-Id: I80edfd6268684722c2783e01323355d791c19b9d
2020-12-18 10:25:01 -05:00
Zuul 1b848c7c36 Merge "ui: Add play name to task result details" 2020-11-14 02:13:07 +00:00
Matt Phillips 3e5649b0cd callback plugin: use utc timezone for now()
rather than a tz-naive stamp. ara drf should do the right thing with
timezones based on the now included iso tz.

Change-Id: I64e486e9c7d9ebe8ec48f9565ac42b3eecd9a788
2020-11-13 10:21:46 -05:00
David Moreau Simard 51053c72a6
ui: Add play name to task result details
Change-Id: I5b1b301a65b63b0c9b2254e36ebc9e780f0f0c08
2020-11-07 12:51:04 -05:00
David Moreau Simard bba0f8143c
callback: tweak threading based on api client
When using >=2 threads with the offline client, the playbook execution
can sometimes, but not always, lock up.

The issue doesn't appear to reproduce when using a single thread so do
that for now.

Otherwise, default to 4 threads as a more conservative default to
prevent hitting urllib's connectionpool max.

Change-Id: I2b0c57f784be36685619d80549fa626c385b0e04
2020-10-23 10:44:12 -04:00
David Moreau Simard 3837b3b6e2
callback: push non-blocking calls to threads
There are some parts of the callback that need to be synchronous because
they rely on an object that must be created before, for example.

However, for the parts that don't need to run synchronously, thread them
so we don't block Ansible from running.

This results in a dramatic performance increase.

Related: https://github.com/ansible-community/ara/issues/171
Change-Id: I4317d1d1175a5286704b04f57de9ef73fa911a3a
2020-10-16 15:34:19 -04:00
David Moreau Simard d301f9462f
ui: add amount of tasks in the playbook card and table
We had this data but we didn't include it for some reason.
It's at least as much relevant as the other numbers displayed so include
it.

Change-Id: I9b95a2cbe2bbee5bba1392e2d7d4afd6eebc6d03
2020-10-14 22:17:13 -04:00
David Moreau Simard 4fdf71d277
ui: order hosts by name in playbook report
Improve readability by alphabetically sorting the hosts by name instead
of a pseudo-random order defined by the order in which they were
created.

Change-Id: Ie745215d8acf9eb8a16c8d22cd91c35276d92ddd
2020-10-14 22:17:13 -04:00
David Moreau Simard 40894c7027
callback: record handler tasks too
It turns out that we had been missing out on handler tasks because we
need to use the hook "v2_playbook_on_handler_task_start" for those.

Fixes: https://github.com/ansible-community/ara/issues/178
Change-Id: Id05ba581fb22eee26270c7f4bccc3445e87b5d3b
2020-09-30 22:30:09 -04:00
Zuul 853a073940 Merge "http client: ensure timeout is an int" 2020-09-30 18:34:13 +00:00
David Moreau Simard d957ab9552
http client: ensure timeout is an int
It could otherwise lead to errors like:
    Timeout value connect was 15, but it must be an int, float or None

Change-Id: I7b426c88a09f27ca2d645fd05227c2cf106d8d1e
2020-09-30 11:08:07 -04:00
David Moreau Simard 23adca1575
API: Add support for searching tasks for handler
Tasks have a boolean "handler" field which indicates whether a task
originated from a handler or not.

This adds support for searching for them, ex:
    /api/v1/tasks?handler=true

Change-Id: I49fe9483c304e4133f710776c5663a34a206e3c8
2020-09-26 12:21:52 -04:00
David Moreau Simard 46a6123c1e
CLI: Add --resolve and --long to 'ara record list'
Re-order the columns to be consistent with the other list commands and
bring both --resolve and --long to it as well.

Change-Id: I31ffbbfb193c566f96704479d8d8e5a342efdd8a
2020-09-22 23:05:17 -04:00
David Moreau Simard cf84747717
CLI: truncate paths longer than 50 characters
This is the same logic currently used in the UI.

When printing paths or resolving paths, truncate them to 50 characters
at most unless --long is used.

Paths can easily take up over a hundred characters of width which made
the pretty table wrap too often.

Change-Id: I065db878c8bea73e19651d0981127242a2fa3b63
2020-09-22 23:01:09 -04:00
David Moreau Simard 724cd378d5
CLI: Improve display for 'ara play list'
Add columns for playbook, tasks and results.

Change-Id: Ib8530b6eb97109a84321bb0763d539d3903fbec1
2020-09-22 11:49:44 -04:00
David Moreau Simard c8b189770f
UI: fix static generation with 1.5 changes
1.5 moved the URLs to be pluralized and the playbook details no longer
return all of the results so they need to be queried individually.

Change-Id: I54878a6933abf9a3c032e9a869b34432fff61662
2020-09-22 09:59:56 -04:00
Zuul 645839c365 Merge "callback: Add argument_labels feature" 2020-09-19 13:43:32 +00:00
Zuul 76ca46a4e8 Merge "callback: improve accuracy of result durations with Ansible >=2.8" 2020-09-19 13:43:31 +00:00
David Moreau Simard 808b6b2ed1
callback: improve accuracy of result durations with Ansible >=2.8
Ansible >= 2.8 added the "v2_runner_on_start" callback hook which gives
us an accurate result start date instead of the task start date.

When running Ansible 2.7 and below, ara will fall back to the task start
date which was the original behavior.

In addition, recover the ended date for the result slightly sooner for
better accuracy.

Fixes: https://github.com/ansible-community/ara/issues/173
Change-Id: I43295fda19eb900a28df023bf6c4680e62a7f91e
2020-09-18 21:45:43 -04:00
David Moreau Simard fbe34695ff UI: Link playbooks to filtered results based on status
This will apply a default filter to playbook links based on its status.

When failed, it will search for results with the status "failed" or
"unreachable" -- otherwise, it will search for results that have
resulted in a change.

The objective is to highlight changed, failed and unreachable results in
an ocean of skipped and ok results.

Change-Id: I662682296be68e387a26da16be8313daa15584bf
2020-09-19 00:51:32 +00:00
David Moreau Simard fa3bc74eb5
callback: Add argument_labels feature
This new configuration option will label playbooks automatically
based on CLI arguments picked up by the callback.

For example, when "--check" is used, it would label the playbook with
the label "check:True" or "check:True" when it isn't used.

Related: https://github.com/ansible-community/ara/issues/148
Related: https://github.com/ansible-community/ara/issues/119
Change-Id: Ifd475875bf83a21ab35c9cf0ac0410520b98d804
2020-09-18 20:07:43 -04:00
David Moreau Simard 9ddce955e3
Revert "callback: set a more accurate start date for results"
This reverts commit dda3e43bc8.

It turns out that this doesn't have the intended purpose and was overly
optimistic.

Change-Id: Ie7f07c147fdf01b1ba1e48bfb4311b43e8264f44
Related: https://github.com/ansible-community/ara/issues/173
2020-09-18 16:17:08 -04:00
David Moreau Simard 6d893f7eef
UI: Don't display the unknown status in playbook search
The odds that a playbook ends up in this status are so small that it's
not worth displaying it.

Change-Id: Iad85a2f9c9f4a815bd6c1e5c3f397b69ae90f4f7
2020-09-17 10:45:18 -04:00
David Moreau Simard c639f438de
CLI: Add "ara expire" to expire old running objects
This new command will allow to change the status of objects that have
been running for more than 24 hours (by default) without being updated
and so we are expecting these to never finish.

Change-Id: Iedbac84188fc4202f51e405d2fae0a35d34b3a1d
Related: https://github.com/ansible-community/ara/issues/26
2020-09-17 10:45:12 -04:00
David Moreau Simard c3874e54f3
API: Add an "expired" status for playbooks, plays and tasks
This status can eventually be set from different contexts and for
different reasons but for now the main use case is to timeout
objects that have been in the running status for too long and will
never complete.

Change-Id: I13682a69ff2aa3d3609528583209007aa93ded0d
Related: https://github.com/ansible-community/ara/issues/26
2020-09-17 10:31:22 -04:00
David Moreau Simard dda3e43bc8
callback: set a more accurate start date for results
Because we always set the started date of results as the beginning of
the task, each host result for the task apparently took a little bit
longer than the previous to complete because they were running one after
the other.

By storing a timestamp at the end of a result, we can use it as the
start date of the new result. While imperfect, it will be far more
accurate than relying on the task started date to calculate the
duration of each result.

Fixes: https://github.com/ansible-community/ara/issues/173
Change-Id: Iade5dc3ba35272ac7993a114d06bbefcd096d677
2020-09-16 21:13:38 -04:00
David Moreau Simard a8c3345575
tests: unpin factory-boy and fix moved import
Fixes: https://github.com/ansible-community/ara/issues/155
Change-Id: If054525ed8d977ebd3ca12a3a6d0d7260b7c08ea
2020-09-16 11:53:03 -04:00
David Moreau Simard 389e7c996f
UI: Don't display result table if there are no results
Change-Id: I6c2172ebfa6f13b4a417ffaf955c12e769539243
2020-09-15 22:48:05 -04:00
David Moreau Simard db0548ad7d
UI: Improve hosts table to use new result search
- Move the link to host facts has to it's own column
- Link on the hostname queries results for the host's id
- Host result status numbers queries results for the host's id and status

Change-Id: I706d9b2d3b922db094271352f54cd1c9f9c8fbaf
2020-09-15 22:26:31 -04:00
David Moreau Simard afbdebb9cd
UI: Link form and 'clear filters' to the results anchor
The hosts table is automatically hidden when there is a search query.
Clicking 'clear filters' or submitting the form would lead back to the
top of the page so going back to the results would require scrolling.

Link to the task results anchor instead.

Change-Id: I3887fc9a8f23913332aad3a117b4ec226c61a935
2020-09-15 22:16:06 -04:00
David Moreau Simard 4344914ac0
UI: Move result search for 'changed' into it's own column
"changed" isn't a status, it's an attribute and so displaying it in the
same column as the statuses was confusing.

Change-Id: Ifb895cfc75e7b457f1cc6b31bbeec233efa40ea6
2020-09-15 22:02:43 -04:00
David Moreau Simard 08f5e324d0 UI: Add playbook result search, filtering and ordering
This is a first iteration that adds a form in the task results pane
which allows to search by:
- host id
- task id
- status
- changed

Ordering by date and duration is done by clicking on the respective
headers.

Change-Id: Iaa67fee1e182807e69d7df7c39bf2ee1d0b67bd4
2020-09-16 01:44:50 +00:00
David Moreau Simard 532ff7ddde
API: Remove "changed" and "ignored" statuses
The principal consumers of the API relied on a combination of the
status as well as the fields "changed" and "ignore_errors" to infer a
"changed" or "ignored" status.

This lead users to believe they could search for these statuses and it
wouldn't work. The consumers should expose the status differently but
that will be in another patch.

Change-Id: Ib34a65179d07186ab2d2efc65ce14c46e2596df0
2020-09-15 21:43:59 -04:00
David Moreau Simard 91c190837c UI: Add pagination for playbook results
This will allow larger playbooks to load much faster without needing to
return and render all of a playbook's results on a single page.

Fixes: https://github.com/ansible-community/ara/issues/168
Change-Id: I5fcf8bc411bc6da946fdd5ce23325b2b11a7197a
2020-09-06 15:57:50 +00:00
David Moreau Simard 74defc2273
API: Stop returning nested children resources
When querying the API for a playbook's detail, it would return all of
it's children (hosts, files, tasks, results) which could be very slow
when dealing with larger playbooks.

We no longer do that for playbooks as well as plays and tasks.
Instead, we can easily find a playbook's resources by searching for them
with the playbook id like so:

- /api/v1/plays?playbook=<id>
- /api/v1/tasks?playbook=<id>
- /api/v1/results?playbook=<id>
... and so on.

This commit adapts the built-in UI because it would've otherwise been
broken by the change.

Fixes: https://github.com/ansible-community/ara/issues/158
Change-Id: I442bff657e5da9d6a3916ebdbc5e66c0e670b00f
2020-09-06 11:57:32 -04:00
David Moreau Simard e2534a10fd
CLI: Add --resolve to 'ara host list'
This will return the playbook id in addition to it's path.

Change-Id: Ic8c912475d875b38b69eef9f2fcd7cef7f302e84
2020-08-31 18:53:43 -04:00