Commit Graph

88 Commits

Author SHA1 Message Date
Ghanshyam Mann d58ee7fb08 Retire openstack-health
openstack-health is QA repo and service to know the
job/test success/failure rate. But for a long (more than
a year), this code is broken. We do not have any maintainer
in QA to fix and maintain it. As it is broken, QA and infra team
agree to stop the service

- http://status.openstack.org/openstack-health/#/

In QA zed PTG, we decided to retire the repo also

- https://etherpad.opendev.org/p/qa-zed-ptg

Needed-By: https://review.opendev.org/c/openstack/governance/+/836706
Change-Id: Ie15aa8e469b0bb3aff47dcad422c0676fea640d2
2022-04-05 18:47:38 -05:00
Ghanshyam Mann b91c0054e4 Move openstack-health-nodejs10-run-test-browser to Firefox
openstack-health-nodejs10-run-test-browser job started failing on
chrome browser
- https://bugs.launchpad.net/stackviz/+bug/1909021

Horizon and its plugins also moved the nodejs browser jobs to
firefox now, let's move openstack-health jobs to firefox.

Closes-Bug: #1909021
Change-Id: I06476189c17323101b4904c3da52248ce04bfb83
2020-12-22 23:33:59 +00:00
mccasland, trevor (tm2086) 741a457f72 Add run time graph for jobs
Motivation for this change:
  By adding the run time graph to the job's page it will raise
  awareness to users of changes in run times among jobs.
  By adding the scatter graph to the job's page it can raise awareness
  of the run time deviation.

New behavior:
  A linear and scatter chart called Job Run Time is available in the jobs
  view. It gathers run_time data from the job_data objects provided by
  the timedelta in the response of /runs/key/<key>/<value> limited
  by the values in the resolution dropdown.
  The original response of this API call is wrapped in a 'data' property
  and the additional information to support drawing the scatter chart on
  the canvas is added to a new dict property called 'numeric'.

NOTE:
  * The methods in run_aggregator were derived from [1] and [2]
  * An experiment was done to see which implementation would
    result in the lowest load time and the results for the job
    'tempest-full' for 2 weeks and 1 month periods are as follows:
    - latest patch with 1 db call
        2.21s - 2 weeks
        3.40s - 1 month

    - older patch with 2 db calls
        7.86s - 2 weeks
        10.16s - 1 month

completes queens priority "Job duration graph in o-h" from:
  https://etherpad.openstack.org/p/qa-queens-priorities

[1] https://review.openstack.org/#/c/370913/4
[2] 4db9a61471/openstack_health/test_run_aggregator.py (L70-L90)
Change-Id: Ib5196d86b6b5efa0083d4aa4dd28f1fac3493560
2018-06-29 14:57:05 -05:00
mccasland, trevor (tm2086) fd387956f4 Add support for HeadlessChrome in karma
Motivation for this change:
  As a tester, I want to execute npm unit tests in a headless
  chrome environment.

Notes on implementation:
  * Running headless chrome requires verion chrome 59 or greater[1]
  * --no-sandbox flag is a workaround for running as root[2]
  * HeadlessChrome base was added in karma-chrome-launcher v2.1.0[3]
    - upgrading to this package caused a CHROME_BIN not set error[4].
      Setting CHROME_BIN to chromium-browser fixes the issue.
  * Using Chrome without the upgraded packages causes an error during
    the coverage report generation
    - TypeError: Cannot read property 'start' of undefined[5]

[1] https://github.com/karma-runner/karma-chrome-launcher#available-browsers
[2] https://github.com/karma-runner/karma-chrome-launcher/issues/158
[3] https://github.com/karma-runner/karma-chrome-launcher/releases/tag/v2.1.0
[4] http://paste.openstack.org/show/715433/
[5] https://review.openstack.org/#/c/456556/9

Change-Id: I907708d73b5fe5345f3a56d8d638c51e7d3aa6f7
2018-03-27 15:05:26 -05:00
Masayuki Igawa 96fab55909
Remove vendor-js
This commit removes vendor-js related code. We already didn't have the
vendor-js directory[1]. So this is just a cleaning up.

[1] Ibfefe310fbad1d80f4c7ec96bda083d97a09824e

Change-Id: Id5cb4bfb34e867cded9106bed9d13e3a8d383d37
2017-04-13 22:35:37 +09:00
Masayuki Igawa c72ab70202
Fix ratio to percent on failure rate graphs
This commit fixes the ratio values to percent on failure rate graphs. We
already use percents in many graphs and users are familiar with percents
rather than ratio.

Change-Id: I056b62b215230b4666401eabace42cac7e68201d
Closes-Bug: #1674066
2017-03-19 17:16:01 +08:00
Dong Ma da1186f592 Fix nodejs4-npm-run-test and docs failure
The angular.js 1.6.0 release have two breaking changes caused the
test failures, you can check the detail changelog[0].
JSONP requests now require a trusted resource URL[1]
JSONP callback must be specified by jsonpCallbackParam config[2]

[0] https://github.com/angular/angular.js/blob/master/CHANGELOG.md#160-rainbow-tsunami-2016-12-08
[1] 6476af83cd
[2] fb66341871

The docutils 0.13.1 release breaks some documentation jobs
when it tries to build docs with remote images. This has been reported
upstream[3]. The version matches the supported docutils versions for
sphinx. The requirements patch is here: [4]

[3] https://sourceforge.net/p/docutils/bugs/301/
[4] https://review.openstack.org/#/c/409529/

Change-Id: If2b24303a60f8452a8ae8b7b3be36fa9b2541880
2017-01-15 09:41:40 -08:00
Tim Buckley 7d87c32d86 Add canvas line chart
This commit adds a canvas-based line chart using the new canvas
charting directives, replacing the nvd3 line charts on the home,
grouped run, and job pages.

Change-Id: Ifd937196e7f9f1f3176ddf82fa54cdab49df4461
2016-10-02 19:01:07 -06:00
Tim Buckley e657b09726 Add karma-subunit-reporter to generate subunit results for unit tests
This adds a new plugin, 'karma-subunit-reporter', to generate subunit
streams for unit test output. These files should be compatible with
subunit2sql and (with appropriate log client configuration) could be
displayed in openstack-health or other tools like stackviz.

Change-Id: I7a72c0330852a9afd6bff35436978d9d1c266551
2016-06-10 09:19:55 -06:00
Matthew Treinish 1b1b3af054
Add elastic-recheck data querying
This commit is the start of adding support for elastic-recheck data
to openstack-health. This will get the bug numbers for the recent
failed runs it's looking at. To actually use this feature it requires
having elastic-recheck installed and a new api config option for the
location on disk of the query files. This does add some install time
complexity especially as eleastic-recheck is only available via git.
The elastic recheck support is optional and will only be activated if
it is present at import time and the config flag telling the api
server where the queries are located on disk is set.

In order to make the response times reasonable when elastic-recheck
support is enabled dogpile caching is used to keep a cached copy
of the recent results list and limit the amount we're querying
elastic search. By default this is configured to use a dbm file
and refresh the cache every 30mins.

Depends-On: Icf203668690419c38f6d6be6b6fe4af8462845f3
Change-Id: Iccd9ec6d62e2249ec7c09d42ec02ea27c71144cc
2016-05-20 10:25:16 -04:00
Tim a0561d5503 Disable Protractor tests and enable strictDi
This disables the Protractor E2E tests due to a high failure rate
related to a protractor/chrome bug [1] where ChromeDriver would
frequently hang. The E2E tests were mainly used to check for
dependency injection failures, but they didn't catch all (or even
most) failures. Instead, this enables angular's built-in 'strictDi'
mode, which will raise errors during normal development (i.e.
non-production builds) and during unit tests whenever a bad injection
is used.

[1] https://github.com/angular/protractor/issues/2419

Change-Id: I825986a3df457326a6c0c0c3b4d8584617a9d5e2
2016-04-02 14:27:08 -06:00
Tim Buckley 12016682bb Use prefix API on tests and tests detail pages
This uses the new prefix APIs to provide server-side processing for
listing tests by prefix and pagination. This removes the need for
users to download the entire list of all tests (~4 MB of JSON taking
15 seconds to generate and download), instead sending only a list of
prefixes which the user can select from (~6 KB and < 0.5 seconds to
load). Additionally, the tests detail page now loads a paginated list
of tests by prefix from the server, greatly reducing the required time
and memory needed to load the page.

Change-Id: I103f77c63c4cb8200db842f48c514b4a214ef9b7
2016-03-31 01:26:51 +00:00
Jenkins b0aa5e69e3 Merge "Introduce regex filter" 2016-03-16 18:18:28 +00:00
Jenkins 47e0a1f8aa Merge "Remove percentage filter" 2016-03-16 07:14:51 +00:00
Masayuki Igawa a85ccccaf5 Remove percentage filter
This commit removes the percentage filter. We don't use this now.

Change-Id: I057c78bcff2a53a43feed5e05ad26bd3143bbf23
2016-03-16 14:09:51 +09:00
Tim Buckley c98177513c Use new chart-gauge-horizontal chart on homepage
This commit replaces the existing horizontal bar charts on the
homepage with a new 'chart-gauge-horizontal' directive. This directive
renders the exact same horizontal bar chart using a standard canvas
element instead of an nvd3 svg, cutting memory usage of the home page
by around 33%. This also improves page loading time as well as
scrolling performance. Additionally, chart tooltips now make use of
faster angular-ui-bootstrap tooltips and cache generated HTML for an
additional (small) speed boost.

Change-Id: Ie2e575af39aa63a94a039a0136334382aafef332
2016-03-16 13:36:30 +09:00
Masayuki Igawa db137007f2 Introduce regex filter
This commit introduces a regex filter to the homepage. Users can specify
a filter with a regex string.
For example, "openstack/(nova$|glance$|cinder$)" show the result of only
openstack/nova, openstack/glance and openstack/cinder.

Change-Id: I608a33e34614b80bcf223f827cf7e57801b25d1f
2016-03-16 04:02:33 +00:00
Tim Buckley b67003a8fe Add TooltipService to generate custom nvd3 tooltips
This adds a new TooltipService with a method for generating custom
tooltip functions for nvd3 charts. These functions can include
arbitary information about the current datapoint rather than the
plain numeric value that nvd3 displays by default.

Change-Id: Ib769b76d6aaad0bb080116c55b5b8a8e0c30aa92
2016-03-16 03:00:49 +00:00
Masayuki Igawa 8a767161c7 Refactor contextual class name converter
This commit introduce a bootstrap contextual class name filter to
refactor the converter.

Change-Id: I85bd991dbbf5586c11d3698ea009f9b618bff8f9
2016-03-15 22:30:46 +00:00
Tim Buckley 567d9d2a24 Run unit tests in Chrome instead of PhantomJS
This commit adds karma-chrome-launcher and changes karma's settings
to use Chrome for all unit tests, removing PhantomJS. This change
is mainly to avoid downloading PhantomJS during gate tests, as the
phantomjs platform binaries would be downloaded from bitbucket and
their download servers have been returning rate limiting jobs and
returning 429 errors, leading to failed test runs. Since Chrome is
automatically installled from local mirrors, this should avoid the
problem entirely.

Change-Id: I5aba5480b34c4b4a6646f96c382c010495b3ea1b
2016-03-15 15:29:26 -06:00
Jenkins b2267c824e Merge "Split tests page into list and detail page" 2016-02-24 18:57:05 +00:00
Jenkins df5f92d6fc Merge "Add a list of recent failed tests to front page" 2016-02-23 20:23:47 +00:00
Masayuki Igawa 8a0ab1d425 Split tests page into list and detail page
This commit splits the tests page into a list page and its detail page
for improving the performance of showing the page.
Previously, we rendered numerous tests list in one tests page. So it
causes the performance issue. Splitting the page reduces the number of
showing test lists. This is not perfect solution but it's better than
before at least.

Change-Id: If1a97740d09359ae17d0f0b7df8d36256ac99f47
Closes-Bug: #1548354
2016-02-23 14:54:37 -05:00
Matthew Treinish bdbd42e9c7
Add a list of recent failed tests to front page
This commit adds a new table to the front page which contains a list
of the tests which failed as part of the 10 most recent failed runs.

Change-Id: I5ff8ed38742d31e0b1a972901b7907030ec99bea
2016-02-19 17:07:31 -05:00
Masayuki Igawa 393ccb0417 Improve projects list UI in the home page
This commit introduces bar graphs and a color classification by failure
rates to the home page project list.
In this commit, the width of the bar graphs is fixed to 300px because
auto widths cause unaligned widths. We should solve this issue in the
future but we don't have the solution right now.

Closes-Bug: #1543635
Change-Id: I448876d5342b1569ae7fe90c4a947dadab758a86
2016-02-19 13:33:17 +09:00
Jenkins a030483fdb Merge "Add user-selectable date and duration picker." 2016-01-31 21:21:10 +00:00
Tim Buckley 4473e6e6f6 Add user-selectable date and duration picker.
This adds a new date and duration picker to the crumb menu, along with
functionality in the view service to manage window size preferences
for views and users. Each controller can now specify allowed period
sizes and a preferred default which can be overridden by the user. The
user may also request an arbitrary end date, which defaults to the
current date.

A new library, moment, was added to help with the date caluclations,
as the builtin date utilities lack a huge amount of needed
functionality for computing date differences. Note that a run of
`npm install` may be needed to update local installations to reflect
the change.

Change-Id: I64c1629e527fcf5b4a167fba50c2bb3fe80a2e96
2016-01-29 10:24:19 -07:00
Masayuki Igawa 4a7dc0a4c1 Apply quotes rule for js files
This commit applies the quotes rule to keep the consistency. It seems we
and the Google style guide prefer single-quotes over double-quotes.
That's the reason why I choose single-quotes.

I don't want to stick this kind of things, actually. But I often confuse
that 'which quotes should I use?'. This commit avoids it.

Change-Id: Ic686d7e6bc91d8b75bba033c0b265b48a522df44
2016-01-27 17:04:58 +09:00
Masayuki Igawa 21eca6c9bb Fix sorting order on home page
This commit fixes the sorting order on the home page. We should have a
floating point value rounded to 4 digits after the decimal point at
least. For example, 0.3333 would be 33% with the original rounding.
However, this is not enough resolution for the home page. This should be
33.33% at least.

Closes-Bug: #1522200
Change-Id: I2e57fce318256fcd6e9aee892706d21d5269b81c
2016-01-26 00:11:36 +00:00
Tim Buckley b7a30358f0 Update expected encoded form of '/' in URLs.
Due to an update in angular-ui-router, literal '/' characters in URLs
are now encoded as '~2F' rather than '%252F', breaking E2E tests that
checked against these URLs. This updates the failing tests to use the
new style of URLs and should fix the failing E2E tests.

Change-Id: Iceca852e15a6d023b758de52800088ab1456d931
2016-01-25 16:56:51 -07:00
Matthew Treinish 8e338f9f53
Add recent runs table to the jobs page
This commit adds a new table to the jobs page listing up to the ten
most recent runs for that job and a link to the logs for the run.

Change-Id: I341e9a453d7d4597c06917b35cbe10ac353a7e2f
2016-01-13 01:23:05 -05:00
Matthew Treinish 0d1b306151
Add table for recent runs to grouped-runs page
This commit adds a new table to the grouped runs page that shows
recent runs for the group selected and links to the logs for the
failures.

Change-Id: I81089eb54bd0e03afda8e0de28073e98ab2b15a6
2016-01-13 01:17:55 -05:00
Jenkins c0e53ac836 Merge "Add global resolution and grouping key options." 2016-01-07 20:01:32 +00:00
Jenkins e468338934 Merge "Properly display missing data in run graphs" 2016-01-05 20:45:21 +00:00
Tim Buckley 74400a889b Add global resolution and grouping key options.
This commit enhances the breadcrumb menu to add global selectors
for resolution and grouping key into a single place on every page.
Existing pages make use of and share these values as users browse
between them. All existing UI elements for changing resolution or
grouping parameters have also been removed in favor of this
universal solution.

Change-Id: Ifdef72219392c3afb979251d13863ca0146dde03
Co-Authored-By: Matthew Treinish <mtreinish@kortar.org>
2016-01-04 12:39:10 -07:00
Matthew Treinish 9344ace136
Make the tests page useful
This commit attempts to fix the tests page and make it useful. The
first thing it does is to remove all the bar graphs. These were added
to try and show something visual on the page. But they didn't convey
any information to the user and instead confused people.

The second thing this does is change the grouping to be more logical.
Instead of grouping on the test classes (which was also incorrect and
making assumptions about the path depth) this groups by the top level
path which will be the package the tests were in. (ie like tempest,
manila_tempest, etc) This makes more sense and also future proofs the
dashboard for when non-tempest tests are in the subunit2sql db.

The third thing this patch does is to change the default sorting order
on the tables to put the highest failure percentage at the top. This
will mean that what most people are interested in will be at the top
of the list. The fourth aspect of this patch is to default the groups
to open instead of closed.

Change-Id: I4426ba951276ef5d9482d67d5dfca7cd3d46d800
2015-12-22 18:42:21 -05:00
Matthew Treinish 6cf4f2ae9f
Properly display missing data in run graphs
This commit fixes the display of missing run data on the run stat
graphs. Previously d3 would interpolate between any 2 consecutive
data points regardless if there were missing samples between them.
In an earlier commit we updated the rest api to return an empty list
for samples that have no data. This commit builds off of that and
treats those empty lists as 0 for all values in that timestamp.

Change-Id: I137e4cfa1f22aaf2d932aa3d3420ebd841abca43
2015-12-22 17:53:10 -05:00
Matthew Treinish ed67518fc0
Add skips data to the jobs page
This commit adds the skip data being returned from the rest api to the
jobs page. The skip information is sometimes necessary to see
especially in the table view. So there isn't any reason to not show it.

Change-Id: I1a0116b6cbd838342017b0fa0b0fad1be8888302
2015-12-21 11:27:09 -05:00
Glauco Oliveira 8ea97d4e57 Allow users to select information grouping
This commit allows users to select how they want to group information on
the dashboard page. It basically enumerates the existing run metadata
keys and let the user choose any of them. As up to today, we were only
grouping information by project.

Change-Id: I31514e75629483f91a35c8e7b622bc298db6a41f
Depends-On: Ibe591d9af94c1cd8d5c1d4956c1c82b4a764b5b7
Co-Authored-By: Tim Buckley <timothy.jas.buckley@hpe.com>
2015-12-17 01:06:24 +00:00
Tim 821601b16d Rename API endpoint for `get_runs_by_run_metadata_key`.
The route for `get_runs_by_run_metadata_key` conflicted with the
`get_test_runs_by_build_name` route when grouping by the
`build_name` metadata key. This moves the route into the
`/runs/key/` namespace to prevent it from colliding with other
endpoints.

Change-Id: I64280febe39eaecd7b694aea7f5d39df5c1bc45c
2015-12-16 18:04:56 -07:00
Jenkins 2b17af27d1 Merge "Update charts to format data correctly" 2015-12-08 20:49:04 +00:00
Jenkins d9b25cca8a Merge "Add test detail page" 2015-12-08 15:20:07 +00:00
Matthew Treinish adb7dd311f Add test detail page
This adds a new page for a single test's details. It displays a
runtime graph and a stacked area graph of test statuses for the
specified individual test across all runs over the past 30 days.

Change-Id: Ib1f5aaaaf49315e2b00d38407e07702922f3d96f
2015-12-07 23:58:45 +00:00
Matthew Treinish 58eb773fdb
Make date window for job pages longer for periodic jobs
Periodic jobs by their nature run far less frequently than pre-merge
ci jobs. As such the graphs showing the per job statistics aren't really
meaningful when looking at a single day. This commit makes the window
as big as the other pages when being run for a periodic job. The window
is still kept small for other jobs because of performance reasons.

Eventually we want to have all the graphs have a date window slider to
make the generated results as big or as small as required. This will
likely be made obsolete when this is implemented. But, in the meantime
lets make things at least moderately useful.

Change-Id: Icb32a8784ef70b2b7ff2e95f7aa12371b7dbaac1
2015-12-03 19:27:35 -05:00
Jenkins f2152f29f0 Merge "Refactor Home controller" 2015-12-03 20:52:55 +00:00
Jenkins 91170d4b11 Merge "Enable coverage reporting." 2015-12-03 20:47:20 +00:00
Jenkins cc48a54167 Merge "Add Project Service" 2015-12-03 20:47:07 +00:00
Jenkins 8e8c80c051 Merge "Add factory to create Project models" 2015-12-03 20:45:53 +00:00
Tim Buckley b4970b4238
Add a unique prefix to the `groupedRuns` route.
This adds a unique prefix to the `groupedRuns` route to ensure that
it doesn't conflict with any other routes. Currently its catchall
route mapping will prevent any lower-precedence routes from
loading; this prefixes all grouped run pages with '/g/` to prevent
conflicts while keeping the intended 'invalid route' behavior of
redirecting to '/'.

Change-Id: Ifdd72efa903c56a4367cbe3b95384208c7c8c187
2015-12-02 11:32:54 -05:00
Jenkins fef489e825 Merge "Add Metrics Service" 2015-11-18 19:42:24 +00:00