openstack-health-nodejs10-run-test-browser job started failing on
chrome browser
- https://bugs.launchpad.net/stackviz/+bug/1909021
Horizon and its plugins also moved the nodejs browser jobs to
firefox now, let's move openstack-health jobs to firefox.
Closes-Bug: #1909021
Change-Id: I06476189c17323101b4904c3da52248ce04bfb83
Motivation for this change:
By adding the run time graph to the job's page it will raise
awareness to users of changes in run times among jobs.
By adding the scatter graph to the job's page it can raise awareness
of the run time deviation.
New behavior:
A linear and scatter chart called Job Run Time is available in the jobs
view. It gathers run_time data from the job_data objects provided by
the timedelta in the response of /runs/key/<key>/<value> limited
by the values in the resolution dropdown.
The original response of this API call is wrapped in a 'data' property
and the additional information to support drawing the scatter chart on
the canvas is added to a new dict property called 'numeric'.
NOTE:
* The methods in run_aggregator were derived from [1] and [2]
* An experiment was done to see which implementation would
result in the lowest load time and the results for the job
'tempest-full' for 2 weeks and 1 month periods are as follows:
- latest patch with 1 db call
2.21s - 2 weeks
3.40s - 1 month
- older patch with 2 db calls
7.86s - 2 weeks
10.16s - 1 month
completes queens priority "Job duration graph in o-h" from:
https://etherpad.openstack.org/p/qa-queens-priorities
[1] https://review.openstack.org/#/c/370913/4
[2] 4db9a61471/openstack_health/test_run_aggregator.py (L70-L90)
Change-Id: Ib5196d86b6b5efa0083d4aa4dd28f1fac3493560
This commit removes vendor-js related code. We already didn't have the
vendor-js directory[1]. So this is just a cleaning up.
[1] Ibfefe310fbad1d80f4c7ec96bda083d97a09824e
Change-Id: Id5cb4bfb34e867cded9106bed9d13e3a8d383d37
This commit fixes the ratio values to percent on failure rate graphs. We
already use percents in many graphs and users are familiar with percents
rather than ratio.
Change-Id: I056b62b215230b4666401eabace42cac7e68201d
Closes-Bug: #1674066
This commit adds a canvas-based line chart using the new canvas
charting directives, replacing the nvd3 line charts on the home,
grouped run, and job pages.
Change-Id: Ifd937196e7f9f1f3176ddf82fa54cdab49df4461
This adds a new plugin, 'karma-subunit-reporter', to generate subunit
streams for unit test output. These files should be compatible with
subunit2sql and (with appropriate log client configuration) could be
displayed in openstack-health or other tools like stackviz.
Change-Id: I7a72c0330852a9afd6bff35436978d9d1c266551
This commit is the start of adding support for elastic-recheck data
to openstack-health. This will get the bug numbers for the recent
failed runs it's looking at. To actually use this feature it requires
having elastic-recheck installed and a new api config option for the
location on disk of the query files. This does add some install time
complexity especially as eleastic-recheck is only available via git.
The elastic recheck support is optional and will only be activated if
it is present at import time and the config flag telling the api
server where the queries are located on disk is set.
In order to make the response times reasonable when elastic-recheck
support is enabled dogpile caching is used to keep a cached copy
of the recent results list and limit the amount we're querying
elastic search. By default this is configured to use a dbm file
and refresh the cache every 30mins.
Depends-On: Icf203668690419c38f6d6be6b6fe4af8462845f3
Change-Id: Iccd9ec6d62e2249ec7c09d42ec02ea27c71144cc
This disables the Protractor E2E tests due to a high failure rate
related to a protractor/chrome bug [1] where ChromeDriver would
frequently hang. The E2E tests were mainly used to check for
dependency injection failures, but they didn't catch all (or even
most) failures. Instead, this enables angular's built-in 'strictDi'
mode, which will raise errors during normal development (i.e.
non-production builds) and during unit tests whenever a bad injection
is used.
[1] https://github.com/angular/protractor/issues/2419
Change-Id: I825986a3df457326a6c0c0c3b4d8584617a9d5e2
This uses the new prefix APIs to provide server-side processing for
listing tests by prefix and pagination. This removes the need for
users to download the entire list of all tests (~4 MB of JSON taking
15 seconds to generate and download), instead sending only a list of
prefixes which the user can select from (~6 KB and < 0.5 seconds to
load). Additionally, the tests detail page now loads a paginated list
of tests by prefix from the server, greatly reducing the required time
and memory needed to load the page.
Change-Id: I103f77c63c4cb8200db842f48c514b4a214ef9b7
This commit replaces the existing horizontal bar charts on the
homepage with a new 'chart-gauge-horizontal' directive. This directive
renders the exact same horizontal bar chart using a standard canvas
element instead of an nvd3 svg, cutting memory usage of the home page
by around 33%. This also improves page loading time as well as
scrolling performance. Additionally, chart tooltips now make use of
faster angular-ui-bootstrap tooltips and cache generated HTML for an
additional (small) speed boost.
Change-Id: Ie2e575af39aa63a94a039a0136334382aafef332
This commit introduces a regex filter to the homepage. Users can specify
a filter with a regex string.
For example, "openstack/(nova$|glance$|cinder$)" show the result of only
openstack/nova, openstack/glance and openstack/cinder.
Change-Id: I608a33e34614b80bcf223f827cf7e57801b25d1f
This adds a new TooltipService with a method for generating custom
tooltip functions for nvd3 charts. These functions can include
arbitary information about the current datapoint rather than the
plain numeric value that nvd3 displays by default.
Change-Id: Ib769b76d6aaad0bb080116c55b5b8a8e0c30aa92
This commit adds karma-chrome-launcher and changes karma's settings
to use Chrome for all unit tests, removing PhantomJS. This change
is mainly to avoid downloading PhantomJS during gate tests, as the
phantomjs platform binaries would be downloaded from bitbucket and
their download servers have been returning rate limiting jobs and
returning 429 errors, leading to failed test runs. Since Chrome is
automatically installled from local mirrors, this should avoid the
problem entirely.
Change-Id: I5aba5480b34c4b4a6646f96c382c010495b3ea1b
This commit splits the tests page into a list page and its detail page
for improving the performance of showing the page.
Previously, we rendered numerous tests list in one tests page. So it
causes the performance issue. Splitting the page reduces the number of
showing test lists. This is not perfect solution but it's better than
before at least.
Change-Id: If1a97740d09359ae17d0f0b7df8d36256ac99f47
Closes-Bug: #1548354
This commit adds a new table to the front page which contains a list
of the tests which failed as part of the 10 most recent failed runs.
Change-Id: I5ff8ed38742d31e0b1a972901b7907030ec99bea
This commit introduces bar graphs and a color classification by failure
rates to the home page project list.
In this commit, the width of the bar graphs is fixed to 300px because
auto widths cause unaligned widths. We should solve this issue in the
future but we don't have the solution right now.
Closes-Bug: #1543635
Change-Id: I448876d5342b1569ae7fe90c4a947dadab758a86
This adds a new date and duration picker to the crumb menu, along with
functionality in the view service to manage window size preferences
for views and users. Each controller can now specify allowed period
sizes and a preferred default which can be overridden by the user. The
user may also request an arbitrary end date, which defaults to the
current date.
A new library, moment, was added to help with the date caluclations,
as the builtin date utilities lack a huge amount of needed
functionality for computing date differences. Note that a run of
`npm install` may be needed to update local installations to reflect
the change.
Change-Id: I64c1629e527fcf5b4a167fba50c2bb3fe80a2e96
This commit applies the quotes rule to keep the consistency. It seems we
and the Google style guide prefer single-quotes over double-quotes.
That's the reason why I choose single-quotes.
I don't want to stick this kind of things, actually. But I often confuse
that 'which quotes should I use?'. This commit avoids it.
Change-Id: Ic686d7e6bc91d8b75bba033c0b265b48a522df44
This commit fixes the sorting order on the home page. We should have a
floating point value rounded to 4 digits after the decimal point at
least. For example, 0.3333 would be 33% with the original rounding.
However, this is not enough resolution for the home page. This should be
33.33% at least.
Closes-Bug: #1522200
Change-Id: I2e57fce318256fcd6e9aee892706d21d5269b81c
Due to an update in angular-ui-router, literal '/' characters in URLs
are now encoded as '~2F' rather than '%252F', breaking E2E tests that
checked against these URLs. This updates the failing tests to use the
new style of URLs and should fix the failing E2E tests.
Change-Id: Iceca852e15a6d023b758de52800088ab1456d931
This commit adds a new table to the jobs page listing up to the ten
most recent runs for that job and a link to the logs for the run.
Change-Id: I341e9a453d7d4597c06917b35cbe10ac353a7e2f
This commit adds a new table to the grouped runs page that shows
recent runs for the group selected and links to the logs for the
failures.
Change-Id: I81089eb54bd0e03afda8e0de28073e98ab2b15a6
This commit enhances the breadcrumb menu to add global selectors
for resolution and grouping key into a single place on every page.
Existing pages make use of and share these values as users browse
between them. All existing UI elements for changing resolution or
grouping parameters have also been removed in favor of this
universal solution.
Change-Id: Ifdef72219392c3afb979251d13863ca0146dde03
Co-Authored-By: Matthew Treinish <mtreinish@kortar.org>
This commit attempts to fix the tests page and make it useful. The
first thing it does is to remove all the bar graphs. These were added
to try and show something visual on the page. But they didn't convey
any information to the user and instead confused people.
The second thing this does is change the grouping to be more logical.
Instead of grouping on the test classes (which was also incorrect and
making assumptions about the path depth) this groups by the top level
path which will be the package the tests were in. (ie like tempest,
manila_tempest, etc) This makes more sense and also future proofs the
dashboard for when non-tempest tests are in the subunit2sql db.
The third thing this patch does is to change the default sorting order
on the tables to put the highest failure percentage at the top. This
will mean that what most people are interested in will be at the top
of the list. The fourth aspect of this patch is to default the groups
to open instead of closed.
Change-Id: I4426ba951276ef5d9482d67d5dfca7cd3d46d800
This commit fixes the display of missing run data on the run stat
graphs. Previously d3 would interpolate between any 2 consecutive
data points regardless if there were missing samples between them.
In an earlier commit we updated the rest api to return an empty list
for samples that have no data. This commit builds off of that and
treats those empty lists as 0 for all values in that timestamp.
Change-Id: I137e4cfa1f22aaf2d932aa3d3420ebd841abca43
This commit adds the skip data being returned from the rest api to the
jobs page. The skip information is sometimes necessary to see
especially in the table view. So there isn't any reason to not show it.
Change-Id: I1a0116b6cbd838342017b0fa0b0fad1be8888302
This commit allows users to select how they want to group information on
the dashboard page. It basically enumerates the existing run metadata
keys and let the user choose any of them. As up to today, we were only
grouping information by project.
Change-Id: I31514e75629483f91a35c8e7b622bc298db6a41f
Depends-On: Ibe591d9af94c1cd8d5c1d4956c1c82b4a764b5b7
Co-Authored-By: Tim Buckley <timothy.jas.buckley@hpe.com>
The route for `get_runs_by_run_metadata_key` conflicted with the
`get_test_runs_by_build_name` route when grouping by the
`build_name` metadata key. This moves the route into the
`/runs/key/` namespace to prevent it from colliding with other
endpoints.
Change-Id: I64280febe39eaecd7b694aea7f5d39df5c1bc45c
This adds a new page for a single test's details. It displays a
runtime graph and a stacked area graph of test statuses for the
specified individual test across all runs over the past 30 days.
Change-Id: Ib1f5aaaaf49315e2b00d38407e07702922f3d96f
Periodic jobs by their nature run far less frequently than pre-merge
ci jobs. As such the graphs showing the per job statistics aren't really
meaningful when looking at a single day. This commit makes the window
as big as the other pages when being run for a periodic job. The window
is still kept small for other jobs because of performance reasons.
Eventually we want to have all the graphs have a date window slider to
make the generated results as big or as small as required. This will
likely be made obsolete when this is implemented. But, in the meantime
lets make things at least moderately useful.
Change-Id: Icb32a8784ef70b2b7ff2e95f7aa12371b7dbaac1
This adds a unique prefix to the `groupedRuns` route to ensure that
it doesn't conflict with any other routes. Currently its catchall
route mapping will prevent any lower-precedence routes from
loading; this prefixes all grouped run pages with '/g/` to prevent
conflicts while keeping the intended 'invalid route' behavior of
redirecting to '/'.
Change-Id: Ifdd72efa903c56a4367cbe3b95384208c7c8c187