Commit Graph

403 Commits

Author SHA1 Message Date
Jeremy Stanley 8fd3fda132 Revert "update_bug.py: Check project along with series"
This reverts commit 5db1d05b70.

I should have thought about this more before approving, it naively
assumes project short names match between Gerrit and Launchpad, but
the gerrit/projects.yaml file in openstack/project-config allows to
remap projects in Gerrit to different names in LP. See the
process_bugtask function for an example of dereferencing this.

Change-Id: I3af965d705f032ffad30f65e4bd5e16c926dda14
2023-07-12 18:26:19 +00:00
Zuul d9885155de Merge "update_bug.py: Check project along with series" 2023-05-19 20:44:50 +00:00
melanie witt 5db1d05b70 update_bug.py: Check project along with series
Noticed recently that update_bug.py can end up updating the wrong
project's task to Fix Committed on a bug affecting multiple projects
because only the series is being checked.

This adds an additional check to make sure the project also matches
before setting the task to Fix Committed.

Change-Id: I817535bd84f76780be44ad21a7e4bb4abbda0135
Depends-On: https://review.opendev.org/883664
2023-05-19 14:12:16 +00:00
Clark Boylan e6653ea2aa Raise an error if acl pushes fail
Previously it was possible for an acl push to fail then we would record
successful acl push via our project cache's sha entry for that acl file.
The reason for this is despite success or failure of the acl push we
continue on without "problems". Unfortunately this can lead to projects
failing to update without any indication this has happened.

Update jeepyb to raise an exception if there is a failure pushing. THis
should prevent the project cache file from being updated and will cause
manage-projects to try again later. The downside to this is we will
short circuit our project handling if there are many updates to perform.
THis seems preferable to silently "succeeding".

Note that we special case read only projects by detecting the error
pushing an acl to a read only project and treating this as success. This
will allow us to delete our cache file and update everythin while
passing over read only projects as is.

Change-Id: I0d1c1160f2a28fc023fd71c8010ddcf6555efa14
2023-01-24 13:41:08 -08:00
Ian Wienand 72661fc4d8
update_blueprint: handle recent gerrit arguments
This is a port of 6eca4077d0 to the
update_blueprint tool, to handle the new arguments passed by later
Gerrits.

Change-Id: Ic47d998089c320e8e4ca371b8fb4b338c5fd903a
2022-12-01 13:46:51 +11:00
Ian Wienand 16604d687e
Fix minor pep8 issues
pep8 now wants this without ().  The dependent change should fix the
gerritlib-jeepyb-integration which is also currently broken.

Depends-On: https://review.opendev.org/c/opendev/gerritlib/+/866241
Change-Id: Iad46d49f9e825b2d95f7318748b84aed90f60556
2022-12-01 13:46:11 +11:00
Ian Wienand 54e2e56ed5 hound: add detect-ref config option
Hound has the ability to query the remote origin and detect the HEAD,
which is useful for projects that don't have the default "master"
branch but have switched to "main".  Add this to the config.

Change-Id: I4bb72b8d65f418b1a0f2f67ab64a8267cd25f2cf
2022-02-25 12:54:20 +11:00
melanie witt faca72ce59 Convert update_blueprint to use the Gerrit REST API
This calls the Gerrit REST API to retrieve subject and topic for a
change instead of querying the Gerrit database. The ReviewDb was
removed in Gerrit 3.0 [1], so we need to use the REST API instead.

This also uses the Gerrit API to get the change commit message instead
of running git commands directly in the git directory.

[1] https://www.gerritcodereview.com/releases-readme.html#30-eol

Change-Id: I25b67745d3943786767d6c8960ff19cdc51b5769
2021-06-17 21:43:21 +00:00
Zuul 6b1c39df5d Merge "Set default branch in .gitreview files when creating project" 2021-04-22 23:41:08 +00:00
Jeremy Stanley 864e17f9d1 Correct set_in_progress parameters
Update the callers of update_bug.set_in_progress() to drop the
uploader parameter which was removed previously in change
I1e08ed0dfa62a1d59084a19d87e3af317dabad04.

Change-Id: I966ecfdf634009e8c2ba89bb5c52d9e4df1e73e4
2021-04-13 20:15:06 +00:00
Jeremy Stanley 6eca4077d0 Update Gerrit hook command-line parameters
As of Gerrit 3.2, some hook command lines now include username
parameters of various sorts in addition to the IDs. Also
patchset-created no longer passes an --is-draft parameter, as that
is now no longer a Gerrit feature. Add the missing new parameters
and remove the defunct one.

Change-Id: I5808e1b3398926810ba0e54bf132a0adeac5d2ac
2021-04-13 19:51:12 +00:00
Jeremy Stanley c2cda826e7 Stop trying to assign Launchpad bugs
Gerrit 2.15 and later no longer stores account information in an
RDBMS, so querying it to determine potential mapping to Launchpad
accounts is not possible. As a stopgap, just stop trying to assign
bugs when setting their state to in-progress, but still perform the
other expected tasks.

This functionality should be considered deprecated, and would be
better replaced with a Zuul job for future maintainability and
extensibility.

Change-Id: I1e08ed0dfa62a1d59084a19d87e3af317dabad04
Depends-On: https://review.opendev.org/782603
2021-03-24 01:25:02 +00:00
Zuul ca156fe7ef Merge "Just use flake8 instead of hacking" 2020-12-03 18:57:45 +00:00
Zuul 255a6e6f2a Merge "Make local git dir creation optional" 2020-11-17 21:15:24 +00:00
Zuul 14270d487a Merge "Update OpenDev Manual URL in new contributor intro" 2020-11-12 23:00:04 +00:00
Clark Boylan 454e31c4d7 Make local git dir creation optional
manage-projects unconditionally set up a local git dir even if the
config option was unset. We change that behavior because new gerrit url
paths make a local git dir option less straight forward. If you've set a
git dir path we'll continue to honor it, otherwise this step is skipped.

Depends-On: https://review.opendev.org/762196
Change-Id: I39ef5ba7cd8747d52487cb0252a9b12878d6247b
2020-11-10 19:02:06 +00:00
Clark Boylan b7292d1a59 Set default branch in .gitreview files when creating project
When jeepyb creates a project we need to set the defaultbranch value in
.gitreview as it may not be the default that git review expects which is
currently master. In the future git review may have a different default
than the gerrit server as well. This covers all the bases by setting
it always.

Change-Id: Ib7ebe658f2d37bbc3ac8eb6054a1fc6d27618475
2020-10-16 09:25:16 -07:00
Clark Boylan 6efdfe9f87 Set repo HEAD on gerrit project creation
By default we set repo HEAD to master but add configuration that allows
us to set it to some other value for new projects. Note this shouldn't
be used until tools for other systems like Gitea are ready.

We explicitly set the default to master as we rely on Gitea and Gerrit
and Git to all be in sync with the default branch. That is the case
today but may not be going forward. Being explicit allows us to avoid
early unexpected change but would also allow us to swap before all tools
update their defaults as well.

Note that we configure our local operating copy with the default head as
well to ensure there isn't additional confusion between the local repo
used to create .gitreview files and the gerrit canonical copy.

Depends-On: https://review.opendev.org/741277
Change-Id: Ie064056ba1722b1bca309dd9f96b61aae0a2fef9
2020-07-16 17:13:56 -07:00
Jeremy Stanley bc7087425a Update OpenDev Manual URL in new contributor intro
The OpenDev Manual (formerly OpenStack Infra Manual) has moved to
opendev.org, so update the link for it in the welcome message
accordingly. Do the same for the pull request closer utility, and
also reword it for better generality (a bit of de-OpenStacking).

While we're here, update outdated URLs for other content and
services which have moved to opendev.org, refresh some configuration
examples, and correct some lingering OpenStack LLC copyrights to
OpenStack Foundation per the guidance at
https://wiki.openstack.org/wiki/LegalIssuesFAQ#Copyright_Headers
(though I did not touch scripts we no longer use, like create
cgitrepos or openstackwatch).

Change-Id: Ibb34d1ab8ab58dee653c51141a871834b93cd54b
Depends-On: https://review.opendev.org/729362
2020-05-19 19:31:03 +00:00
Zuul 4b0999f64d Merge "Inspect all configs in manage-projects" 2020-05-04 15:43:38 +00:00
Monty Taylor 9d733a9ded Decode utf-8 from subprocess.Popen
We use subprocess.Popen directly in these scripts rather than
run_command. We need to decode the output so that we get strings
back and not bytes.

Change-Id: Ie967ceac77ecb898cab24827644b981eed22e141
2020-04-11 09:58:36 -05:00
Monty Taylor 5567a02437 Fix issues from rolling out containers
We need to be able to set the location of GERRIT_GIT_DIR via
env vars. And we need to turn off strict for config parser.

Change-Id: I9d511caa6db6f958a215f131692e185c717b66d5
2020-04-10 13:01:46 -05:00
Monty Taylor 77bed1ae7a Username is on the connection objet
We're manipulating a Gerrit object not a GerritConnection object.
The username property is found on the Gerrit object.

Change-Id: I9b4c4cc0dacf456c1cb71c3798141ac726b1db71
2020-03-30 08:03:44 -05:00
Jeremy Stanley ce25e06d8f Catch exceptions when checking for groups
Work around gerritlib raising a generic "Exception" exception when
listGroup() finds no group.

Change-Id: I63de5aac729b29366b37f1304369419328f4a051
2020-03-29 20:44:24 +00:00
Monty Taylor ca1945dc63 More python3 fixes
Return strings from run_command

jeepyb is written expecting strings. Decode the bytes into a
string.

Fix fsck_command for python3

We fixed run_command to return strings, but fsck_command was
looking for bytes.

Filter returns an interator not a list in python3, wrap in a
list.

Change-Id: Ibb4d540a987711bd6b4c804a3f6b7cb2ccfe1baa
2020-03-27 16:22:21 -05:00
Monty Taylor 80b1e15236 Encode the input to hashlib
Another python3-ism.

Change-Id: Ide54c16fd75b1e06b45a926b771d65575d50ca2e
2020-03-25 14:47:56 -05:00
Monty Taylor b6401b0f2f Update StringIO to use six
It's StringIO.StringIO in python2 and io.StringIO in python3,
but six.StringIO everywhere.

Change-Id: Ief0b091b716e3ea6224c0ac374a6066f0270bebc
2020-03-24 18:50:19 -05:00
Monty Taylor 96ca93b2e3 Just use flake8 instead of hacking
We don't use hacking in Infra. Use flake8 instead.

Change-Id: I04ac08a0c4218398b43e906a6d65f75b3ff104c3
2020-03-24 15:00:12 -05:00
Zuul fd310cf1e7 Merge "Clean up some more python3 things" 2020-03-24 15:08:48 +00:00
Monty Taylor 086a9bf6e3 Clean up some more python3 things
Because system commands are bytes not strings. Because yay.

Change-Id: Ib54b5717c2e2051addc2d7208a70b82a66ca8ccb
2020-03-23 13:36:48 -05:00
Zuul 7e539bd34f Merge "Update ConfigParser imports for python3" 2020-03-22 14:17:56 +00:00
Zuul c7e501b57e Merge "Log errors when committing and pushing acls" 2020-03-21 21:44:42 +00:00
Zuul 0ec183dd56 Merge "Don't use the db in manage-projects" 2020-03-21 21:44:41 +00:00
Monty Taylor 20808ddc5d Update ConfigParser imports for python3
We're starting to run jeepyb under python3 in the new gerrit image.

Change-Id: Ia71cb13f3b45055bbb9a311807fa65a89c8be62a
2020-03-21 16:42:29 -05:00
Jeremy Stanley 891047afa0 Overhaul default welcome message for OpenDev
The "Welcome, new contributor!" hook bot should leave a more generic
message relevant to most OpenDev projects, and not something
OpenStack-specific (as this could easily confuse new contributors to
non-OpenStack projects in OpenDev).

Change-Id: Ifc41e0639adf27a476025e66728d547a71ad99d5
2020-02-20 21:50:25 +00:00
Clark Boylan 369d025428 Log errors when committing and pushing acls
For some reason we don't log the git output so failures are opaque to
us. Fix that by adding additonal logging.

Change-Id: If212ae6115c8266e1352431588c04547c897e982
2020-01-28 10:46:45 -08:00
Clark Boylan cef1dde44f Don't use the db in manage-projects
Direct access to the db restricts where we can run manage-projects from
and complicates testing. Instead rely on the list-groups ssh command to
get group uuid info. Note that this should be tested using new
integration testing before we merge it.

Change-Id: I6e3f0881398da8efeab5dfef370ad8d35163a7be
2020-01-28 09:52:51 -08:00
Monty Taylor 837515213f Don't index retired projects
While they should just have a README, it's still a waste of energy
for codesearch to index them in the first place.

Remove the special condition for deb- repos - they are all retired.

Change-Id: I60d90fccb607fcd48de42739188af5fb241ded7f
2019-12-18 10:55:16 -05:00
Monty Taylor ccb152d532 Fix create_hound_config for python3
pep8 did a business!

Change-Id: I1ffc99e30a0b5fc41c1f2485546b10553c525915
2019-12-18 10:55:16 -05:00
Ian Wienand 12bd0e0f35 Hound config: disambiguate key names for projects in new namespaces
Because we use os.path.basename() for the key in the config
dictionary, openstack/project-config and zuul/project-config map to
the same key and overwrite each other, thus we only get indexing on
one or the other.

Use the full project name as the key instead.

Change-Id: I8a32ddc2e24211cc9300a2f90df19e1354251ae5
2019-05-13 09:35:46 +10:00
Monty Taylor 1d08f4e920 Use opendev and https by default
We are currently defaulting the hound config to using git://.
Change that to https and change the git host to opendev.org.

Also, change the path and anchor so that they will be correct
for gitea.

Change-Id: Icd1eef1a5476fbb7e0fff97e0d401b79e371fb36
2019-04-23 13:29:08 +00:00
Jeremy Stanley c132a30732 Correct blueprint topic links for modern Gerrit
The Gerrit query URL pattern has changed, so correct how we form it
in the update_blueprint hook to match expectations of current Gerrit
versions.

Change-Id: I368d0d30451051ab88a7b07dc6a2fbe7c4ddb886
2019-04-09 12:39:24 +00:00
Clark Boylan f9771e88ca Inspect all configs in manage-projects
We were skipping the setting of retired project acls in manage-projects
because we were iterating over projects that weren't retired. In
manage-projects we don't want to skip over retired projects so that we
can retire the projects properly.

Address this by adding a new registry property that returns all projects
retired or not then iterate over that in manage-projects.

Change-Id: I7aa89b53696fb0fca262fc7068e5e2ad2b7ad8fc
2019-03-18 15:03:46 -07:00
azvyagintsev dcd2739069 manage_projects: exit with error code
* Currently, manage_projects always perform
   `exit 0`, even if some errors were.
 * Patch would add `exit 1` beh., in
   case any issues were during processing.
   It would affect only exit code , and will not
   stop processing.
 * Misc: perform auto-ident

Change-Id: I50d425e5a9e02441a50f7b1d32219485f80e6632
2018-12-21 16:43:12 +02:00
Andreas Jaeger 0673abe338 Use https for links
Use https instead of http when link to docs.openstack.org.

Change-Id: Iaf643a1c110aac53cd036537a583d58eebbf817d
2018-10-29 13:19:02 +01:00
Christian Berendt f21a8fef14 Remove leading whitespaces from the Gerrit welcome message
Change-Id: I02f02b361ded18d21013c2d8c58986340a6e5dfb
2018-05-18 08:30:31 +00:00
James E. Blair 6dfb888f4e Support cgit alias sites
This allows creation of new top-level sites and aliasing of
existing repos into those sites (under arbitrary names).

For example, this will let us list only the zuul projects under
git.zuul-ci.org.  This scheme does not extend to the git protocol,
which we will start to deprecate.

Change-Id: I4c759f02d8d7e77439984d8ad1f012bd381fae59
Story: 2001382
Task: 6092
2018-03-23 15:04:20 -07:00
Clark Boylan 5b68b2227f Handle newer gerrit in update blueprint script
Newer gerrit provides the project name and the branch the change is
proposed against as well as the change id itself in the --change
argument to patchset created hooks. This is a behavior change that we
have to handle as old gerrit passed only the change id.

We do this by going to the old behavior of the script by splitting off
the change id from the new string and using only that. Note that this
may not be strictly correct as multiple changes can share a change id
(likely why gerrit made this change in the first place). We can worry
about properly correct behavior in future updates.

Change-Id: Idef56e98ed6c753a58b766024295b2f5147e3aea
2017-09-23 12:21:13 -07:00
Jenkins 8b50df9ead Merge "Remove self from group after group creation" 2017-09-22 21:37:32 +00:00
Clark Boylan 87b466c0c5 Remove self from group after group creation
Newer gerrit automatically adds the user creating a group to that group.
We don't actually want this behavior as the user used to manage this is
used for nothing else. Check if the current user was added to the group
and if so remove it.

Depends-On: Id5d8915f2c0bf6f6057b414785b8ec1b18e810f2
Change-Id: Ia0ca5faaaa466ae7a930bbbfb78e2975771f3af9
2017-09-22 10:27:21 -07:00