By adding an 'include_attachments' flag to the get_test_runs_by_status
method we can join the Attachments table with the same structure as
the RunMetadata table allowing us to optionally include attachments or
metadata with test_runs filtered by a given status.
Notes on testing:
* As far as I could tell, logstash.o.o doesn't store attachments[1]
so to test this patch you will have to setup/use another subunit2sql
db with attachments stored in it
* you can find the openstack-health patch to render the attachments
in the Needed-By change of the footer of this commit message.
Notes on performance:
* on average loading attachments is about 13% slower when calling this
api with python[2]. The query times in the paste were produced by calling
the api method get_test_runs_by_status_for_run_ids 100x in a for loop with
49 test_runs being returned from the call, which is the number of failures
in the last 10 runs.
[1] http://paste.openstack.org/show/719192/
[2] http://paste.openstack.org/show/738662/
Needed-By: I3ce2fc50ada9462de3df29b5a8b76b12a548fd12
Change-Id: I31468c825cf259b8df62134e578f31d96af6ac75
This clarifies the release notes for the newly added delete by uuid
db api functions, as well as by the addition of the ability to pass
programatically the variables neccessary to process run results via
the shell module
Change-Id: Ie86d43ea7272b1b2cd86e8290461285a37539bed
This commit allows the 'run_at', 'artifacts', run_id',
'run_meta', and 'test_attr_prefix' to be passed in to the
process_results function, thus allowing for the programmatic
usage of subunit2sql without having to access and update
CONF for every run that gets processed.
Change-Id: Ie5db660a2b1a8162403b0475d13686e80b1b9cc6
These functions delete run and test_run entries associated to a
given run uuid. These functions are intended primarily for usage
in https://review.openstack.org/#/c/530681/ , but seemed like handy
utilities to have in general
Change-Id: I3a25b74875cd8664a7895b67f09870ea78c99e98
This commit adds a new flag, --dpi, to the subunit2sql-graph command.
This flag is used to specify a DPI for generated graph images.
Previously most graph commands hard coded the DPI to 900 (which is far
too high for most people) and others didn't specify one. This commit
changes that so it's user adjustable and consistent between all
commands.
Change-Id: I5c8c48bda18daff11d6069c35451d932ffe4e33b
This commit adds a new CLI option for allowing mixed subunit
content. This is desired for subunit.stream files with non-subunit
content mixed in but still want the subunit content converted to
sql. However, this commit only adds support for passing the argument
to ByteStreamToStreamResult. Nothing is added beyond that for managing
the non-subunit content.
Co-Authored-By: Matthew Treinish <mtreinish@kortar.org>
Change-Id: I68cbeb47e9093b98630f096fc5818a77932c8da4
Release notes are version independent, so remove version/release
values. We've found that projects now require the service package
to be installed in order to build release notes, and this is entirely
due to the current convention of pulling in the version information.
Release notes should not need installation in order to build, so this
unnecessary version setting needs to be removed.
This is needed for new release notes publishing, see
I56909152975f731a9d2c21b2825b972195e48ee8 and the discussion starting
at
http://lists.openstack.org/pipermail/openstack-dev/2017-November/124480.html
.
Change-Id: I9827b64eba24033388e82ab9023a640b0720e506
As part of the docs migration work[0] for Pike we need to switch to use the
openstackdocstheme.
[0] https://review.openstack.org/#/c/472275/
Change-Id: I9f5d44dfdfeeab87ced0936318d95741d62845ba
This commit adds a new graph type which is used to show run time
variance for individual tests. It will get all the run times for each
test and then generate a box and whiskers (gag) plot for this data.
Change-Id: Ia44ea36486296fb0db705b1c55cd2338c204dc91
This commit adds a new DB API function, get_tests_by_test_ids(), to get
multiple tests by test_id at once. There is already a function to do
this for an individual test, but if you need to get multiple tests by
tests looping that results in a lot of extra DB traffic when it can be
done in a single request.
Change-Id: I9db6c050720bd12693ff4fa29ee4f43fe60169be
This commit adds a new DB API function get_run_count() which, like it's
name implies, returns the number of runs in the DB.
Change-Id: I31507f2e8ee8e3cc9b4587573f3b44b3f10dffee
Releasenote translation publishing is being prepared. 'locale_dirs'
needs to be defined in conf.py to generate translated version of the
release notes.
Change-Id: Ic78152c2e83b525ac9dd6aadce47af6a62e1327f
This commit adds a new db API function,
get_run_times_time_series_grouped_by_run_metadata_key(), that will
return a time series dict of run_times grouped by a particular
run_metadata key.
Change-Id: I565aef69af1969bc0fb58a7c9459c9937658a19b
This commit adds a new pair of options to the DB API function
get_run_times_grouped_by_run_metadata_key() to optionally filter the
results to runs with a particular key value run_metadata pair. This is
useful if you want the results grouped by one run_metadata key but
limited to runs with particular key value metadata pair.
Change-Id: I6d499362eac158873ec982c7b6f2b91a96033a82
This commit adds a new CLI/config option for the subunit2sql cli command
to use the run wall time when populating the run_time column of the runs
table. This is often desirable because it gives you a better feeling
for the duration of the run. However, since using the sum of individual
tests executed during a run has been the default behavior so long we
can't just switch it for backwards compat reasons. This commit thus adds
an option to use the behavior if that's what you need to use.
Change-Id: I08fe16604454f5888d56573e5e919e40ac82efd1
The test_attr_prefix allows filtering test attributes for injection
into the test_metadata table. Such attributes only get a prefix for
the sake of filtering, so the prefix should be removed before they
are injected into the DB.
This is a change in behaviour, but I believe it's safe to do so,
since the CLI option was broken until now, which suggests no-one was
using it really.
Change-Id: I0cf3701a143f6fb71d03a0a7bdfb0a84c107ecbe
This commit adds a new function to the DB api for getting a list of
multiple runs from an input list of run_ids. This avoids the need to
loop over a list and call get_run_by_id().
Change-Id: I5df793ec5417245101a6e016def3bddfe7fb25e7
This commit adds a new option to the get_test_runs_by_test_id() DB API
function, most_recent_first, that when set True will order the response
list by date with the most recent being first. This is useful if you
want to look at a recent subset of the response list on the consumer
side.
Change-Id: Ia94f8066eb36deb33391162920b748707630e420
This commit adds a new function to the DB API,
get_recent_successful_runs_by_run_metadata(), which is used to return
a list of recent Run objects for all runs with a provided key value
run_metadata pair.
Change-Id: Id6ca7e7a4cd1f913d9f84efae5f48d247dd9ed64
This commit adds an optional start_date parameter to the get_recent*
methods that will only return results after a specified date.
Change-Id: Ib1c899701fdec6d86ced6edca7aa7a8385cefc92
This commit fixes an issue with the date formatting on the agg_count
subunit2sql-graph command to be consistent with the other commands
and actually use the ISO8601 format like the docs say it does. However,
because the actual str parsing and format were in the DB API (and for
several releases) we need to gracefully handle the switchover to
anything else. So this commit updates get_test_counts_in_date_range()
to also take in a datetime object and updates the docs to indicate
the previous str parameters are deprecated. It then updates the
agg_count command to own it's own string parsing and just pass a
datetime object to the DB api.
Change-Id: Ie7c67941e59c70285263611803f73b23b69ee06b
Story: #2000581
This commit adds a new option, include_run_id, to the
get_test_runs_by_status_for_run_ids() DB API method for including the
run uuid in the dictionaries returned by the method. This was an
oversight in the method and needed by users in certain cases, however
we can't just add it by default for backwards compat reasons. So a
bool flag is used to enable the additional field being returned.
Change-Id: Icf203668690419c38f6d6be6b6fe4af8462845f3
With the 1.3.0 release pending, this commit adds all the release
notes we neglected to add with the patches.
Change-Id: I0c5a579c550336dc68fb762ee31cf9bf6d95d56c
This commits all release notes missing for the pending 1.2.0 release.
These items should have been included with the commit that added the
change but was overlooked. We should be better about this in the
future. But, since the ship has sailed already this just ensures we
generate release notes before the release is pushed.
Change-Id: Ia7451e282bc1dbfa627bed0d21b9d8c6c689c5fc
This commit adds a new migration that will create some unique
constraints which will help with the performance of some larger
queries as well as better model the relationships of the
metadata tables.
Change-Id: Ie87fedd6567c7258fec9434b7cc3e23b601159cf
This commit adds support to the subunit2sql cli command to specify the
run_at field for a run being created. The db api has had support for
this for quite some time but it was never exposed through the CLI.
This resulted in the default behavior of using utcnow() to set the
run_at column for a created run. However, if you're trying to insert
old runs into the DB this isn't ideal because it will look like the
test results were recent. This commit enables users to specify a
different run_at time for these use cases.
Change-Id: I0c00ef71633e043c9d80adf0bc2df0157d1a11b3
This commit adds indexes which were previously implicitly added by the
use of foreign keys on the id columns in the metadata tables. However,
after the last migration to convert uuid primary keys to integers the
foreign keys were all dropped. This commit adds these indexes back to
fix the performance issues with queries involving the metadata tables.
Additionally, the index on the run_at column in the runs table was
overlooked, so this migration adds that back as well.
Change-Id: I162b6daec1116937cf499bd56fde92c44590f1d4
This commit adds a second page to the release notes dedicated to the
release notes for the 1.0.0 release.
Change-Id: I67b2473563deebdcdb35837aa172cf0ede0319cb
This commit adds support for using reno to generate the subunit2sql
release notes. This will make it much easier to keep up-to-date
and accurate release notes. Also doing it in tree will make the
notes reviewable which will help with that too.
Change-Id: I75502185a666945721dd84e4710e798962e612ee
mccasland, trevor (tm2086)