Commit Graph

65 Commits

Author SHA1 Message Date
Zuul 34d181d2ae Merge "Update URL links to opendev.org locations" 2022-04-14 22:50:08 +00:00
Zuul fe05f53545 Merge "Fix 'openreviews' script to work with Python3" 2022-04-14 21:18:02 +00:00
wangfaxin 4510c60135
Update URL links to opendev.org locations
Some links still pointed to out-of-date openstack.org URLs. This updates
references to point to the current opendev.org locations.

Change-Id: I03a006948bf3f8b056cc7787c5fcda51aa9373fb
2022-04-14 16:06:21 -05:00
Zuul c911f29a86 Merge "Fix 'reviewers' command to make with Python3" 2022-04-14 15:34:35 +00:00
Zuul c53796b31a Merge "Use unittest.mock instead of third party mock" 2020-07-22 13:29:30 +00:00
Ivan Kolodyazhny fac90aa416 Fix 'openreviews' script to work with Python3
Change-Id: Iadbbc00252bdbc3987d072e9064e80be7a8d01f2
2020-06-19 21:51:45 +03:00
Ivan Kolodyazhny 98e27f513d Fix 'reviewers' command to make with Python3
Change-Id: I9716ed3f629d54f1a4e82dc64bf65a0702af6a25
2020-06-19 21:48:35 +03:00
Andreas Jaeger 7be58d6af8 Cleanup repo
* Remove unused files babel.cfg and openstack-common.conf.
* Remove obsolete entries from setup.cfg.
* Remove six as requirement
* Switch to newer hacking version and fix all problems.
* Remove Babel requirement, update requirements for python3

Change-Id: I1fc46288e54a5b787afbd9ee31b32a0e7d28c6de
2020-06-09 10:45:08 +02:00
Sean McGinnis 9b9aabe74a
Use unittest.mock instead of third party mock
We can now use the standard library mock library instead of the third
party one previously needed for py27 compatibility.

Change-Id: I6b80b62d8e3c8bd9432f567e0725c191cef6f715
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-03-13 11:32:04 -05:00
Matt Riedemann 59f6c2e5ac reviewers: make --project and --stable mutually exclusive
When using the reviewers command to get stats on stable branch
reviews, I always think I can specify --stable and --project nova
to get nova stable reviews, but that's not how the command
works and the --project option is ignored resulting in a report
which I don't expect, i.e. there are non-nova reviewers in the
report.

This makes --project and --stable mutually exclusive to avoid that
confusion.

Change-Id: Ie9938f5e68f27b9a5a58635be2185365f041136f
2019-11-04 11:15:27 -05:00
Matt Riedemann 93f7506d54 Switch reviewers command to use argparse
optparse is deprecated in python 2.7 and not available
starting in python3. This changes the reviewers command
to use argparse instead.

Change-Id: I76f270b4e998e8cf2a56c12d7bf70c0651c69d16
2019-11-04 11:12:53 -05:00
Sean McGinnis 58f2b47814 Raise hacking version and fix pep8 errors
Various issues hit when switching to using stestr to run tests.
Presumably from other pulled in dependencies. This raises hacking to a
version that fixes a pyflakes issue [0] and addresses other pep8
violations that are then identified.

[0] https://github.com/PyCQA/pyflakes/pull/80

Change-Id: I9d8637d50067d6618aa408f4dbd9c736956099ed
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-08-01 17:01:07 +02:00
Ben Nemec 39b1e03047 Load subproject data from governance
Instead of manually curating a list of all the subprojects in a
project, we should use the governance repo that already includes
most, if not all, of this information.

For the moment, this change just adds the governance data to the
data already in the json files. This is because I'm not sure whether
there is a 1:1 relationship between what is in governance and what
is in the json files. It's possible we could eliminate the json files
entirely at some point, but for the moment this gets us updated
subproject lists for essentially free, since it's likely that many
projects' json files are missing subprojects added in the past few
years.

Change-Id: Ia3992dd30f83294a96ace155691e50838cd5543b
2019-08-01 17:01:07 +02:00
Ben Nemec ff938a9288 Don't fail when writing cache either
We handle errors loading the cache, but if we run out of memory while
writing the cache file it can also cause the run to fail unnecessarily.
If the cache is not written it just means data will have to be
retrieved from Gerrit.

Change-Id: I64ec3030000fc9d62fa8e55ba1524aeaf4f4095c
2019-05-17 20:28:08 +00:00
Ben Nemec 41422af648 Switch default server to opendev.org
review.openstack.org is now a redirect to review.opendev.org. We
should just point at the new location.

Change-Id: Ibae768aa25464e6133d4d65d9a59b919ee689fe4
2019-05-17 20:28:08 +00:00
Ben Nemec 1bbf631545 Handle all exceptions loading pickled data
The pickle file is just a local cache of Gerrit data. If we fail to
load it, it will just be regenerated from Gerrit. We should never
allow a failure in pickle.load to be fatal.

Change-Id: If433fa5f335db8db63cdd264919c0dbeb3a7a32e
2019-05-17 20:28:08 +00:00
Matt Riedemann 73d434b28c Add the ability to get reviewer stats across all stable branches
This adds a special "all" stable branch value handling for the
reviewers entrypoint. This allows one to do something like:

$ reviewers --stable all --days 90 --output ~/stable-all-reviewers-90

It would be nice to be able to specify a specific project, like nova,
with --stable but that is another change for another day.

Change-Id: I045e982e587c9fb2c53a9cb587f944f09c2c5793
2017-08-30 13:58:37 -04:00
Tony Breeds 389cd27fae Indicate the branch when generating stable checks
Changes from:
 Reviews for the last 120 days in stable
to:
 Reviews for the last 120 days in stable/liberty

Change-Id: I6adf3523b39a6621d657c569653bc605a7a911cc
2016-04-06 14:17:24 +10:00
Tony Breeds 67b5119896 Filter out members without usernames
It seems some groups (eg trove-stable-maint) contain members that lack a
'username' attribute.

This causes reviewstats to stack trace.

Fix that by filtering out these problematic users.

Change-Id: I5b5dd602a7ccf8be56e8227a3037f7d426087a06
2016-04-06 14:17:24 +10:00
Matt Riedemann ecaabf2146 Support passing just a single project name
This allows you to pass just the project name, like 'nova',
to the --project option so you don't have to actually know
the underlying file structure.

Change-Id: I9f0f7f54ff5f5c1d99f391fc0706c00ccbe8886f
2016-03-10 15:29:34 -05:00
Sean McGinnis e33cf702a1 Optimize core list retrieval in reviewers
The list of core team members was being retrieved for every review
for each patchset. The utils.get_core_team does some caching to make
sure an API call wasn't made for each of these calls, but since the
project never changes for the review we can save a few cycles by not
making the call if it's not necessary.

This moves the retrieval of the core list before the loop through
reviews to make sure it's only called once per patchset.

Change-Id: Iac9f3b6eb71131eccce58a60656865eeb77b8fda
2016-01-17 12:51:47 -06:00
Ben Nemec 8984d97d20 Fix --start with existing cache
It turns out len(changes) is only the correct number of changes to
skip the first time through with an empty cache.  On subsequent
runs it skips way too far ahead in the gerrit changes and thus
never gets any new ones.

This change keeps track of the number of new changes separately
from the length of changes, which seems to make the behavior with
a cache work correctly again.

Change-Id: I1e3509d2d54b514afba64fedecc6a9f5ca408235
2015-12-23 15:51:52 +00:00
Ben Nemec e8566158dd Gerrit 2.11 compatibility
Gerrit 2.11 no longer seems to support the sortKey method of
skipping changes.  Instead is provides a --start option that takes
a number of changes to skip.

See: https://review.openstack.org/Documentation/cmd-query.html

In addition, this required a change to the format of the changes
dict.  It turns out we were overwriting some changes if two had the
same id.  This is common in stable branch cherry picks.

In order to avoid this, I used a tuple of id, project, branch
as the key for the changes dict.  This does break compatibility
with existing cache files, so a check was added to clear old
caches.

Both of these changes were required together because otherwise we
end up with a smaller number of items in changes than we should
have, and the --start argument is incorrect so we start double
processing changes, which breaks the loop because it looks like
the change was already cached.
Change-Id: If5ab46ce3367b3790e0db1d8bb1ed18e39202ad9
2015-12-17 18:41:44 +00:00
Russell Bryant dd0211ab85 Stop hard coding core team member lists
All of the project json files had lists of the core review team
members.  Since this was originally written, gerrit has added a REST
API and this information is available through that API.  Get the info
from there instead when we can.

There are a few project files I didn't convert because the equivalent
gerrit team wasn't immediately obvious.

Change-Id: Ie7ae19d9a58a5148002676604fd6962ea1438786
Signed-off-by: Russell Bryant <rbryant@redhat.com>
2015-06-03 13:29:38 -04:00
Russell Bryant 8f5484e317 Create utils.get_core_team() function.
This just centralizes the code for getting the core team for a
project.  The next patch will add an alternate method for determining
the team by querying gerrit's REST API.

Change-Id: I016fd8378c5fb69ed7c9192899d8dcfe9bdb466e
2015-05-21 15:50:36 -07:00
Russell Bryant 4e0372fed1 Handle (dis|re)connect more appropriately.
This code was reconnecting via ssh on every iteration of the loop,
which is completely unnecessary.  It should use the existing
connection and only reconnect if there was an SSH error when exeuting
a command.  The code should also explicitly close the connection when
its done.

Change-Id: I2545a1ca5e0cd39a8b453445ea25927756b1fd19
2015-04-16 11:03:57 -04:00
Russell Bryant d0dd98b2de Make local cache more permanent.
With no cache, this code has to download the entire review history for
a project, which takes a really long time, not to mention it's
generally wasteful.

There was code to keep a local cache, but it expired after an hour.
It would make quick repeated runs go faster, but for the stats I keep
posted for OpenStack projects this didn't help.  I regenerate
everything less often.  My stats site is basically broken because of
how terribly inefficient this is.

This updated code keeps the cache around without expiring it.
Instead, it just updates the cache until it hits the first change that
hasn't changed since our cached version and stops.  This should make
things work much better and allow my stats site to keep working.

Change-Id: I784b99dfabeeca81c779dcfda15f1f20bb736957
2015-04-10 16:07:29 -04:00
Matthieu Huin 92f36be0bb Add --server option to openapproved script
Add the possibility to choose a gerrit server in this script, to
be consistent with the other command-line utilities.

Change-Id: If17f9dc3262ebc04cc8c8cabc3cdee04c0b14d5c
2015-03-26 17:38:17 +01:00
Robert Collins 2924696365 Make pdb debugging of openreviews possible.
This allows dropping into pdb while still using genresults-openreviews
which I found useful.

Change-Id: I2dbd2ce773b097248291c7c0897fa02539f1eddc
2014-09-03 12:43:07 +12:00
Jenkins 41a4857171 Merge "Add --server param to reviewers" 2014-08-10 14:52:59 +00:00
Paul Bourke 8f76f64551 Add --server param to reviewers
Add --server param to reviewers to make it consistent with the
openreviews command.

Change-Id: I10f48237d15e2a31b1546ded0a6bf5d8479c2231
2014-07-30 11:22:48 +01:00
Paul Bourke a56bdbfc6b Add missing spaces to --output usage message
Change-Id: Icefe34ca0dd4814e4e9ae07d1d19bc8503f97df4
2014-07-30 11:19:44 +01:00
Timothy Chavez 3c3b7c6bea Fix small grammatical errors
I think it should be '1st quartile' rather than '1rd quartile', but I'm no statistician.

Change-Id: I8d41bda6438c22d7af4b105bd96a8784875f8bda
2014-06-17 00:34:03 +00:00
Ben Nemec 8e363b81e0 Encode to utf-8 before printing
When redirecting the output of openreviews to a file, if there are
utf-8 characters in, for example, a commit message, openreviews
will raise an exception because it tries to encode as ascii by
default.

This change explicitly encodes basestring output to utf-8 so it
doesn't matter what the encoding of the output stream is.

Change-Id: I581c1a98bdbbbfb17ccf33c59295d6fbf8729ab9
2014-06-10 14:38:24 +00:00
Ben Nemec f6a80e3be4 Properly support WIP in Gerrit 2.8
Since WIP is just a workflow vote now, the old method of filtering
out those patches is no longer working.  This change should take
care of that.

Change-Id: I8a028de992399df324927a4b0c191e7a5b684773
2014-05-21 14:53:15 +00:00
Ben Nemec d6ec27547f Update for Gerrit 2.8
Some of the review types returned in the Gerrit JSON data have
changed, so reviewstats wasn't finding any reviews.  This also
adds support for the new Workflow votes so new approvals will
show up correctly.

Note that the output from reviewstats still seems to be rather
bogus because the review timestamps appear to have all been updated
in the upgrade, but I think reviewstats is processing the data as
intended with this change.

Change-Id: I6cccf427a0ff875fdbc59f8c8d83a37db567f786
2014-05-01 16:48:55 +00:00
Russell Bryant 36002723b2 Add date and time to reviewers output
Change-Id: Ic989a68ef23a408a244717605d192a6bd038ead4
2014-04-22 14:31:01 -04:00
Romain Chantereau 5713bd9df5 Documented module functions.
Change-Id: I1d05337ca1296b3ea21533ba6ea9ea7c2c4e83b3
2014-03-28 11:13:26 +01:00
Russell Bryant ceb72af13a Retry ssh connect a few times
I'm seeing random ssh connection failures that are messing up my
periodic runs of these scripts.  Add a few retires for the connect
attempt.

Change-Id: I171328813e29a2d6141c90866d50486b500ff61a
2014-03-25 06:01:57 -04:00
Russell Bryant 6687a12e4e automatically retry connect with allow_agent=False
This seems to be a common reason a connect failure happens.
Explicitly trying again with allow_agent=False may make it work.

Change-Id: If14a9f9eb0d6f707e4f939cb96ab5f6c4844a541
2014-03-25 05:12:21 -04:00
Russell Bryant 86a3870682 Fix another div by zero error when there are no reviews
Change-Id: I81e5f6c6d51d54f1eb377846e5695a68998612cb
2014-03-15 07:10:22 -04:00
Russell Bryant 537e9d5aba Fix div by zero exception if there are no reviewers
Change-Id: I56f2e63733bd37fa66324f14e3643a0ab31b534b
2014-03-15 06:58:37 -04:00
Russell Bryant 904e55d76c Don't blow up on a MemoryError
My VM that re-generates the stats is now breaking on a MemoryError when trying
load the local cache.  Ignore this for now since it will fall back to querying
gerrit.

Change-Id: I151fae89d9fb9a45d72e72c5aade4cb3342b4267
2014-02-24 14:58:59 -05:00
Jenkins 614ac99826 Merge "Support reviewers stats for stable branches" 2014-02-17 16:14:09 +00:00
Thierry Carrez 4bb58b50f5 Support reviewers stats for stable branches
Adds a "-s" option that lets you specify the name of a stable branch
(for example "-s havana") instead of specifying a project (-p) or all
projects (-a).

reviewers.py will then use a specific stable.json file that defines the
set of projects stable-maint-core has authority on, as well as the
contents of that core team.

Note: -s disables result caching since the search is made on
specific branches, which would pollute the general purpose cache.

Change-Id: Idd03f51c98e006972bc93e3ce11e052cd1cc26aa
2014-02-17 13:50:44 +01:00
Russell Bryant df69834e9a Change description of oldest reviews
Make it more clear that the list of oldest reviews is just that, and not call it
a variant of calculating the longest waiting reviews.

Change-Id: I890f07519ceccfc57a5a73293817105a3b8c9f53
2014-02-14 13:24:56 -05:00
Russell Bryant 2d8b51d3a2 Don't blow up on an empty core team list
Change-Id: Ie75315a95b6be6e4efd6106f40eff83e303ed936
2014-01-28 15:09:12 -05:00
Russell Bryant 617b8f5ea2 Add --csv-rows option
Add an option to include only --csv-rows <rows> number of rows in the
CSV output.

Change-Id: Ib6d8bf20561b42afef119533b571d7f1f32739a9
2013-12-16 11:27:22 -05:00
Robert Collins 046cb6f7b3 Add a bug statistics tool.
This is a first draft at a timeseries generator to extract an overview
of bug activity in Launchpad. I'm entirely sure that this can be
improved, mainly I wanted it up before the Nova meeting.

Change-Id: I887db503b8a0ce274f3819d90877ef14f13cbeff
2013-12-06 08:13:37 +13:00
Jenkins c58961b9c3 Merge "Use six to handle cpickle." 2013-12-05 14:22:50 +00:00