Commit Graph

35 Commits

Author SHA1 Message Date
Ghanshyam Mann d58ee7fb08 Retire openstack-health
openstack-health is QA repo and service to know the
job/test success/failure rate. But for a long (more than
a year), this code is broken. We do not have any maintainer
in QA to fix and maintain it. As it is broken, QA and infra team
agree to stop the service

- http://status.openstack.org/openstack-health/#/

In QA zed PTG, we decided to retire the repo also

- https://etherpad.opendev.org/p/qa-zed-ptg

Needed-By: https://review.opendev.org/c/openstack/governance/+/836706
Change-Id: Ie15aa8e469b0bb3aff47dcad422c0676fea640d2
2022-04-05 18:47:38 -05:00
Martin Kopec 23f9274a1a Cap pandas to < 1.2.0 to fix gates
pandas>=1.2.0 contains some changes in rounding numbers which makes
the following tests fail when executed by py38

 - test_get_numeric_data_diff_build_name
 - test_get_numeric_data_diff_build_name

with the following assert error

'2018-06-23T00:00:00': 4766.42225,
'2018-06-23T00:00:00': 4766.4222500000005,
                                 ++++++++
'2018-06-24T00:00:00': 4899.55725,
'2018-06-25T00:00:00': 5042.148499999999}}
                               ^^^^^^^^^

Change-Id: Icd33d84d94d58ba36b296d46f4397227fa36ef9c
2021-01-22 13:23:39 +00:00
Andreas Jaeger 40d322d283 Sync requirements to fix docs jobs
building of docs fails due to incompatibilities between sphinx and other
packages. Sync requirements with global requirements, this fixes the
build problems.

Change-Id: I2060945854515fa54a8c5e3e9598a9255e5493e4
2017-08-04 21:14:58 +02:00
Masayuki Igawa 8d9eb1da5d Fix RSS feed unavailable when no failure
This commit fixes RSS feed unavailable error to return zero length entry
list when there is specified run_metadata_key and value with no failure.

Closes-Bug: #1573630
Change-Id: I48659140314a2d8737611dd0a86097f10c1f3ac8
2016-08-11 15:09:47 -04:00
Masayuki Igawa 4db9a61471 Fix pandas functions warnings
This commit fixes pandas functions warnings. Some of pandas functions such as
rolling_mean(), resample(how=xx) are deprecated from v0.18.0.

Change-Id: I1465e50821af2aaa77d5458205469c4eec1dab58
Closes-Bug: #1580447
2016-06-30 17:06:50 +09:00
Jenkins 284f97f661 Merge "Fix changed imports in dogpile.cache 0.6.0" 2016-06-12 22:16:00 +00:00
Jenkins b662d1207c Merge "Stop using deprecated jsonpify import" 2016-06-07 17:29:36 +00:00
Matthew Treinish 2a447a8c5d
Fix changed imports in dogpile.cache 0.6.0
In the latest release of dogpile.cache, 0.6.0, a couple of backwards
incompatible import rejiggering broke our usage in openstack-health's
distributed dbm backend proxy. This commit updates the imports to work
with 0.6.0 and newer versions of dogpile.cache.

Change-Id: Iad50ab66c88a2164146fea98edf435a445e0ee6c
2016-06-07 11:52:08 -04:00
Tim Buckley 2a69274760 Cap dogpile.cache version to 0.5.7
The new dogpile.cache 0.6.0 release seems to be breaking our gate
tests. As a temporary fix, this caps the dogpile.cache version to
the previously-working 0.5.7.

Change-Id: I8f6f721f0b08a0501e711f2718baa91546c56f12
2016-06-06 16:21:10 -06:00
Matthew Treinish 62491309e6
Stop using deprecated jsonpify import
This commit updates the flask jsonpify extension import to stop using
the deprecated version. Starting In flask 0.11 when importing
jsonpify via flask.ext.jsonpify a deprecation warning is emitted to
update the import.

Change-Id: I266d0dcb0b71d97f4fb7edd0d5d7bf1cc8b36205
2016-06-06 16:24:51 -04:00
Matthew Treinish d5a542cc6e
Add DBM cache backend that uses memcached locking
This commit adds a new dogpile.cache backend that uses the memcached
distributed locking mechanism to enable async workers but still uses
the dbm file storage. This should enable us to cache large JSON blobs
without worrying about the memacached size constraints, but at the
same time reap the benefits of having an async worker update the
cache in the background. The tradeoff here is configuration complexity
because you still need to install memcached to leverage this.

Change-Id: Ied241ca1762c62a047bd366d7bd105028a884f30
2016-06-02 13:34:16 -04:00
Matthew Treinish 3076768c6b
Add elastic-recheck to status response
We recently added elastic-recheck/elastic-search as an additional data
source for openstack-health. However, adding the current configuration
was neglected in that patch. This commit fixes this oversight, but
including a field for elastic-recheck in the status response. It will
let API users know if elastic-recheck is installed, configured, and if
so what the health of the elastic-search cluster is.

Change-Id: Ia76a26de930b13a4a7cd90dc0ef45bbcecc714f6
2016-05-23 10:57:36 -04:00
Matthew Treinish 1b1b3af054
Add elastic-recheck data querying
This commit is the start of adding support for elastic-recheck data
to openstack-health. This will get the bug numbers for the recent
failed runs it's looking at. To actually use this feature it requires
having elastic-recheck installed and a new api config option for the
location on disk of the query files. This does add some install time
complexity especially as eleastic-recheck is only available via git.
The elastic recheck support is optional and will only be activated if
it is present at import time and the config flag telling the api
server where the queries are located on disk is set.

In order to make the response times reasonable when elastic-recheck
support is enabled dogpile caching is used to keep a cached copy
of the recent results list and limit the amount we're querying
elastic search. By default this is configured to use a dbm file
and refresh the cache every 30mins.

Depends-On: Icf203668690419c38f6d6be6b6fe4af8462845f3
Change-Id: Iccd9ec6d62e2249ec7c09d42ec02ea27c71144cc
2016-05-20 10:25:16 -04:00
Matthew Treinish 04610c054e
Remove storing xml files on each feed generation
This commit removes the file storing from the rss endpoint. The code
is buggy and also a potential sec issue. It also isn't actually needed
as most (if not all) rss reader will handle missing entries fine, as
long as the ids are unique it shouldn't matter. So even if we lose old
entries on a crash or restart it won't be an issue.

Change-Id: I595abc566880ae6c778b00affde12e2227c9ec35
2016-04-21 23:06:30 -04:00
Matthew Treinish 48f0b1f4cc
Add an rss api route to get a feed of failed tests
This commit adds a new api route to return an rss feed with the recent
failed test runs for all runs by a given key value run_metadata pair

Depends-On: Ic3463c1f9d9b4a20145467b3da1cdbf5344bbceb
Change-Id: Ib4583d2cf0e6890f86ee4216d07237be4bfce3b0
2016-04-19 10:23:04 -04:00
Tim Buckley 6850e8a487 Add REST API methods for test prefixes
This adds 2 new REST API methods for getting a list of test prefixes
('/tests/prefix') and getting a paginated list of tests for a given
prefix ('/tests/prefix/<prefix>').

Depends-On: I55e37c763f3f3ffc2840fddc8109d601291fcc0f
Change-Id: Ib6584a7a5600037dd33cce8fb4466495ed8e83d1
2016-03-30 19:26:35 -06:00
Masayuki Igawa 6c662aff40 Enable Python 3
This commit enables Python 3.

Change-Id: I769ec784f87a311eef52df09cd96fdbfed9cba1c
2016-02-24 19:56:46 +00:00
Matthew Treinish bdbd42e9c7
Add a list of recent failed tests to front page
This commit adds a new table to the front page which contains a list
of the tests which failed as part of the 10 most recent failed runs.

Change-Id: I5ff8ed38742d31e0b1a972901b7907030ec99bea
2016-02-19 17:07:31 -05:00
Masayuki Igawa 80ec3f76d1 Update requirements
This commit updates the requirements that comes from global
requirements. But some of libraries are not in global-requirements[1].
So I've added the licenses for them. And we may need to add them to
global-requirements later.

[1] flask-jsonpify, numpy, pandas, subunit2sql

Change-Id: I05a04406f966917e35c91aca3f8b0cd870e9a498
2016-01-29 11:46:18 +09:00
Matthew Treinish edeb4fc851
Add API method to get a list of recent runs by metadata
This commit adds a new rest api method to return a list of recent
runs for a given run_metadata key value pair. It will be useful in
constructing lists with links to recent runs.

Change-Id: Ic5557e42ab158b865c024dc539d291c061a69dce
Depends-On: I855c47e0056f459b1f71e919d0331c2e8ecbd3a8
2016-01-13 01:02:25 -05:00
Matthew Treinish 9b0b664517
Rework get_test_runs_for_test() rest api
This commit reworks the get_test_runs_for() rest api call and changes
it's basic behavior. The first thing done is the numeric data is split
from the non-numeric data in the response. This was done to enable
mean based resampling on the numeric data. This is necessary because
the other large change to the api is that a running mean and a running
std dev are added to the response. This was done to enable more
detailed graphs to the UI for the run_time of a single test.
To perform the running mean and std dev pandas is added to the
requirements for the rest api.

Change-Id: I3416dc0cad24c90405ac437df31c39b59f524d88
2015-12-07 18:57:25 -05:00
Matthew Treinish 4fd490d924
Add a rest api to get a test_runs time series by test
This commit adds a new REST API method to get a time series of
test_runs given the test_id for a test. The intent here is to enable
a per test view with a graph of success and failures over time, as
well as a test run_time graphed over time.

Change-Id: Id7fe36c3e1ca069d942fe246d688648f719d3168
2015-11-30 21:17:56 -05:00
Matthew Treinish 5cd6b14e7a
Revert "Add a temporary version cap on subunit2sql"
The release has happened, the migration worked, all's right with the
world.

This reverts commit 6b636200f5.

Change-Id: I4b19c5acc6c30f2379a30d2c68422057fdb83a90
2015-11-23 16:09:21 -05:00
Matthew Treinish 6b636200f5
Add a temporary version cap on subunit2sql
This commit adds a temporary version cap on subunit2sql to be < 1.0.0.
The 1.0.0 release includes a very large database migration which will
be slow to execute. The python DB api from >=v1.0.0 will not work with
a database that doesn't have the updated schema. So while the migration
is running let's cap the version we install to prevent everything from
breaking while the migration is running. (which might take days)

Change-Id: Ieadcc6847a1648bc247fd840ec95653f438be664
2015-11-23 15:55:45 -05:00
Matthew Treinish 4cc00e8999 Add REST API endpoint for second page view
This commit adds a rest api endpoint for the second page view. It
takes in a key value pair from the url and returns a time series dict
of all the runs which had that key value pair in it's respective run
metadata. The datetime resolution, start date and stop date are all
adjustable with url parameters.

The second page view will use this with the key being project and the
value being whatever project the page is being generated for.

Co-Authored-By: Glauco Oliveira <gvinici@thoughtworks.com>
Co-Authored-By: Moises Trovó <mtrovo@thoughtworks.com>

Change-Id: I7837073c9029014e03b2faca642f77f997ebdf82
2015-10-06 16:52:15 -03:00
Matthew Treinish b6cfad0d7c
Add rest api method for first page view
This commit adds a new rest endpoint /runs/group_by/metadata_key to
return a json dictionary of the runs stats for all the runs in a given
time resolution grouped by the metadata_key provided in the url.

This will be used for the first page view to return a json dict that
is grouped by project by making a call like: GET /runs/group_by/project
which will return all the runs grouped by project.

Change-Id: I73a9a040a8950b72b399d1866246422b08f2e60d
2015-09-28 18:33:14 -04:00
Caio Carrara ffcaa21eb7 Adds PyMySQL entry to requirements file
As PyMySQL is a dependency to run api, it should be in requirements
file.

Change-Id: I8c587235088897db96bf6c78238046a094dbe879
Co-Authored-By: Dhiana Deva <ddeva@thoughtworks.com>
2015-09-22 16:25:16 -03:00
Tim Buckley 7cee0cfb40 Add JSONP support to REST endpoints.
JSONP is required to allow cross-origin API requests from
JavaScript clients. This adds the `flask-jsonpify` library to
support this additional request method, and uses it in all current
routes.

Change-Id: Ifb7d5abaa16cf165bfc763fec1189fdfb48a4b88
2015-09-16 11:50:19 -06:00
Matthew Treinish c8011fd24b
Add python bits to the repo
This commit adds the basic python infrastructure to the repo. The
backend rest api on top of the data stores will live in the repo
but before we can add that we need to be able to support python code
in the repo.

Change-Id: I869f42e148c2f5c2369fb5613d43b4ec25aaa2db
2015-09-14 19:06:08 -04:00
David Lyle edaf92d16d removing python elements 2015-09-14 14:08:12 -06:00
Austin Clark ec1caea762 Add dependencies to requirements.txt 2015-08-20 13:55:21 -06:00
Austin Clark c32375e4bc Add upstream run data display page 2015-08-04 09:16:58 -06:00
Tim Buckley e3f652c0dd add DjangoRestless dependency 2015-07-15 16:40:28 -06:00
Tim Buckley d9ed54ce44 update requirements to add subunit and testrepository libraries 2015-07-15 11:40:03 -06:00
Tim Buckley 2d70634ff3 Initial import containing a barebones django / browserify project 2015-07-13 15:55:45 -06:00