Commit Graph

496 Commits

Author SHA1 Message Date
Jeremy Stanley 8fd3fda132 Revert "update_bug.py: Check project along with series"
This reverts commit 5db1d05b70.

I should have thought about this more before approving, it naively
assumes project short names match between Gerrit and Launchpad, but
the gerrit/projects.yaml file in openstack/project-config allows to
remap projects in Gerrit to different names in LP. See the
process_bugtask function for an example of dereferencing this.

Change-Id: I3af965d705f032ffad30f65e4bd5e16c926dda14
2023-07-12 18:26:19 +00:00
Zuul d9885155de Merge "update_bug.py: Check project along with series" 2023-05-19 20:44:50 +00:00
Zuul b2e6d65fbd Merge "Switch from tox to nox" 2023-05-19 19:54:40 +00:00
Zuul f2c2926a08 Merge "Raise an error if acl pushes fail" 2023-05-19 19:27:32 +00:00
Clark Boylan c7f4d61b82 Switch from tox to nox
Recent tox releases have put us on a config treadmill. Avoid these
issues entirely by using nox. Nox is a tox alternative that uses
standard tools like pip and should be simpler to use for us.

Change-Id: Ifce288808f535b5ee071dddc597ef54a6d3d03bb
Depends-On: https://review.opendev.org/883664
2023-05-19 17:15:47 +00:00
melanie witt 5db1d05b70 update_bug.py: Check project along with series
Noticed recently that update_bug.py can end up updating the wrong
project's task to Fix Committed on a bug affecting multiple projects
because only the series is being checked.

This adds an additional check to make sure the project also matches
before setting the task to Fix Committed.

Change-Id: I817535bd84f76780be44ad21a7e4bb4abbda0135
Depends-On: https://review.opendev.org/883664
2023-05-19 14:12:16 +00:00
Clark Boylan e6653ea2aa Raise an error if acl pushes fail
Previously it was possible for an acl push to fail then we would record
successful acl push via our project cache's sha entry for that acl file.
The reason for this is despite success or failure of the acl push we
continue on without "problems". Unfortunately this can lead to projects
failing to update without any indication this has happened.

Update jeepyb to raise an exception if there is a failure pushing. THis
should prevent the project cache file from being updated and will cause
manage-projects to try again later. The downside to this is we will
short circuit our project handling if there are many updates to perform.
THis seems preferable to silently "succeeding".

Note that we special case read only projects by detecting the error
pushing an acl to a read only project and treating this as success. This
will allow us to delete our cache file and update everythin while
passing over read only projects as is.

Change-Id: I0d1c1160f2a28fc023fd71c8010ddcf6555efa14
2023-01-24 13:41:08 -08:00
Clark Boylan f6757510c1 Add launchpadlib as a dep
This adds launchpadlib as a dep to jeepyb so that our assemble script
can install it alongside all the other jeepyb deps when building Gerrit
images. This way we don't need to install python3-launchpadlib anymore
and can use our base python images with Gerrit.

Depends-On: https://review.opendev.org/c/opendev/system-config/+/870118
Change-Id: I94fb9c7f16aa6c12840b54dac5e272950b4dea0c
2023-01-17 15:21:45 -08:00
Ian Wienand 72661fc4d8
update_blueprint: handle recent gerrit arguments
This is a port of 6eca4077d0 to the
update_blueprint tool, to handle the new arguments passed by later
Gerrits.

Change-Id: Ic47d998089c320e8e4ca371b8fb4b338c5fd903a
2022-12-01 13:46:51 +11:00
Ian Wienand 16604d687e
Fix minor pep8 issues
pep8 now wants this without ().  The dependent change should fix the
gerritlib-jeepyb-integration which is also currently broken.

Depends-On: https://review.opendev.org/c/opendev/gerritlib/+/866241
Change-Id: Iad46d49f9e825b2d95f7318748b84aed90f60556
2022-12-01 13:46:11 +11:00
Ian Wienand 54e2e56ed5 hound: add detect-ref config option
Hound has the ability to query the remote origin and detect the HEAD,
which is useful for projects that don't have the default "master"
branch but have switched to "main".  Add this to the config.

Change-Id: I4bb72b8d65f418b1a0f2f67ab64a8267cd25f2cf
2022-02-25 12:54:20 +11:00
melanie witt faca72ce59 Convert update_blueprint to use the Gerrit REST API
This calls the Gerrit REST API to retrieve subject and topic for a
change instead of querying the Gerrit database. The ReviewDb was
removed in Gerrit 3.0 [1], so we need to use the REST API instead.

This also uses the Gerrit API to get the change commit message instead
of running git commands directly in the git directory.

[1] https://www.gerritcodereview.com/releases-readme.html#30-eol

Change-Id: I25b67745d3943786767d6c8960ff19cdc51b5769
2021-06-17 21:43:21 +00:00
Zuul 6b1c39df5d Merge "Set default branch in .gitreview files when creating project" 2021-04-22 23:41:08 +00:00
Zuul cc25f50c72 Merge "Bump gerritlib requirement to 0.10.0" 2021-04-22 23:27:59 +00:00
Jeremy Stanley 864e17f9d1 Correct set_in_progress parameters
Update the callers of update_bug.set_in_progress() to drop the
uploader parameter which was removed previously in change
I1e08ed0dfa62a1d59084a19d87e3af317dabad04.

Change-Id: I966ecfdf634009e8c2ba89bb5c52d9e4df1e73e4
2021-04-13 20:15:06 +00:00
Jeremy Stanley 6eca4077d0 Update Gerrit hook command-line parameters
As of Gerrit 3.2, some hook command lines now include username
parameters of various sorts in addition to the IDs. Also
patchset-created no longer passes an --is-draft parameter, as that
is now no longer a Gerrit feature. Add the missing new parameters
and remove the defunct one.

Change-Id: I5808e1b3398926810ba0e54bf132a0adeac5d2ac
2021-04-13 19:51:12 +00:00
Jeremy Stanley db3a718fcc Bump gerritlib requirement to 0.10.0
With the recent upgrade of OpenDev's Gerrit deployment to 3.2, it
became apparent that group creation/detection in manage-projects
stopped working due to a behavior change with the Gerrit
command-line API. This was fixed in gerritlib 0.10.0 so we now
require at least that version.

Change-Id: I34dec9b661d08b9987000d41008367d03be74465
Depends-On: https://review.opendev.org/782603
2021-03-26 12:44:46 +00:00
Jeremy Stanley c2cda826e7 Stop trying to assign Launchpad bugs
Gerrit 2.15 and later no longer stores account information in an
RDBMS, so querying it to determine potential mapping to Launchpad
accounts is not possible. As a stopgap, just stop trying to assign
bugs when setting their state to in-progress, but still perform the
other expected tasks.

This functionality should be considered deprecated, and would be
better replaced with a Zuul job for future maintainability and
extensibility.

Change-Id: I1e08ed0dfa62a1d59084a19d87e3af317dabad04
Depends-On: https://review.opendev.org/782603
2021-03-24 01:25:02 +00:00
Zuul ca156fe7ef Merge "Just use flake8 instead of hacking" 2020-12-03 18:57:45 +00:00
Zuul 255a6e6f2a Merge "Make local git dir creation optional" 2020-11-17 21:15:24 +00:00
Zuul 14270d487a Merge "Update OpenDev Manual URL in new contributor intro" 2020-11-12 23:00:04 +00:00
Clark Boylan 454e31c4d7 Make local git dir creation optional
manage-projects unconditionally set up a local git dir even if the
config option was unset. We change that behavior because new gerrit url
paths make a local git dir option less straight forward. If you've set a
git dir path we'll continue to honor it, otherwise this step is skipped.

Depends-On: https://review.opendev.org/762196
Change-Id: I39ef5ba7cd8747d52487cb0252a9b12878d6247b
2020-11-10 19:02:06 +00:00
Clark Boylan b7292d1a59 Set default branch in .gitreview files when creating project
When jeepyb creates a project we need to set the defaultbranch value in
.gitreview as it may not be the default that git review expects which is
currently master. In the future git review may have a different default
than the gerrit server as well. This covers all the bases by setting
it always.

Change-Id: Ib7ebe658f2d37bbc3ac8eb6054a1fc6d27618475
2020-10-16 09:25:16 -07:00
Clark Boylan 6efdfe9f87 Set repo HEAD on gerrit project creation
By default we set repo HEAD to master but add configuration that allows
us to set it to some other value for new projects. Note this shouldn't
be used until tools for other systems like Gitea are ready.

We explicitly set the default to master as we rely on Gitea and Gerrit
and Git to all be in sync with the default branch. That is the case
today but may not be going forward. Being explicit allows us to avoid
early unexpected change but would also allow us to swap before all tools
update their defaults as well.

Note that we configure our local operating copy with the default head as
well to ensure there isn't additional confusion between the local repo
used to create .gitreview files and the gerrit canonical copy.

Depends-On: https://review.opendev.org/741277
Change-Id: Ie064056ba1722b1bca309dd9f96b61aae0a2fef9
2020-07-16 17:13:56 -07:00
Jeremy Stanley bc7087425a Update OpenDev Manual URL in new contributor intro
The OpenDev Manual (formerly OpenStack Infra Manual) has moved to
opendev.org, so update the link for it in the welcome message
accordingly. Do the same for the pull request closer utility, and
also reword it for better generality (a bit of de-OpenStacking).

While we're here, update outdated URLs for other content and
services which have moved to opendev.org, refresh some configuration
examples, and correct some lingering OpenStack LLC copyrights to
OpenStack Foundation per the guidance at
https://wiki.openstack.org/wiki/LegalIssuesFAQ#Copyright_Headers
(though I did not touch scripts we no longer use, like create
cgitrepos or openstackwatch).

Change-Id: Ibb34d1ab8ab58dee653c51141a871834b93cd54b
Depends-On: https://review.opendev.org/729362
2020-05-19 19:31:03 +00:00
Zuul 4b0999f64d Merge "Inspect all configs in manage-projects" 2020-05-04 15:43:38 +00:00
Monty Taylor 9d733a9ded Decode utf-8 from subprocess.Popen
We use subprocess.Popen directly in these scripts rather than
run_command. We need to decode the output so that we get strings
back and not bytes.

Change-Id: Ie967ceac77ecb898cab24827644b981eed22e141
2020-04-11 09:58:36 -05:00
Monty Taylor 5567a02437 Fix issues from rolling out containers
We need to be able to set the location of GERRIT_GIT_DIR via
env vars. And we need to turn off strict for config parser.

Change-Id: I9d511caa6db6f958a215f131692e185c717b66d5
2020-04-10 13:01:46 -05:00
Monty Taylor 77bed1ae7a Username is on the connection objet
We're manipulating a Gerrit object not a GerritConnection object.
The username property is found on the Gerrit object.

Change-Id: I9b4c4cc0dacf456c1cb71c3798141ac726b1db71
2020-03-30 08:03:44 -05:00
Jeremy Stanley ce25e06d8f Catch exceptions when checking for groups
Work around gerritlib raising a generic "Exception" exception when
listGroup() finds no group.

Change-Id: I63de5aac729b29366b37f1304369419328f4a051
2020-03-29 20:44:24 +00:00
Monty Taylor ca1945dc63 More python3 fixes
Return strings from run_command

jeepyb is written expecting strings. Decode the bytes into a
string.

Fix fsck_command for python3

We fixed run_command to return strings, but fsck_command was
looking for bytes.

Filter returns an interator not a list in python3, wrap in a
list.

Change-Id: Ibb4d540a987711bd6b4c804a3f6b7cb2ccfe1baa
2020-03-27 16:22:21 -05:00
Monty Taylor 80b1e15236 Encode the input to hashlib
Another python3-ism.

Change-Id: Ide54c16fd75b1e06b45a926b771d65575d50ca2e
2020-03-25 14:47:56 -05:00
Monty Taylor b6401b0f2f Update StringIO to use six
It's StringIO.StringIO in python2 and io.StringIO in python3,
but six.StringIO everywhere.

Change-Id: Ief0b091b716e3ea6224c0ac374a6066f0270bebc
2020-03-24 18:50:19 -05:00
Monty Taylor 3611248789 Trigger new images
Change-Id: I428b46af2c04867582f170b48c64fc6ff087052c
2020-03-24 15:04:01 -05:00
Monty Taylor 96ca93b2e3 Just use flake8 instead of hacking
We don't use hacking in Infra. Use flake8 instead.

Change-Id: I04ac08a0c4218398b43e906a6d65f75b3ff104c3
2020-03-24 15:00:12 -05:00
Zuul fd310cf1e7 Merge "Clean up some more python3 things" 2020-03-24 15:08:48 +00:00
Zuul 4a4e291735 Merge "Run pep8 job in-repo" 2020-03-23 19:21:34 +00:00
Monty Taylor 086a9bf6e3 Clean up some more python3 things
Because system commands are bytes not strings. Because yay.

Change-Id: Ib54b5717c2e2051addc2d7208a70b82a66ca8ccb
2020-03-23 13:36:48 -05:00
Monty Taylor 90b98698c3 Run pep8 job in-repo
We removed the pep8 job from project-config, let's add
it back to our local .zuul.yaml.

Depends-On: https://review.opendev.org/714516
Change-Id: I5a9a65569e4103914e64a09c47104d0e3d7c5fe1
2020-03-23 13:36:45 -05:00
Zuul 7e539bd34f Merge "Update ConfigParser imports for python3" 2020-03-22 14:17:56 +00:00
Zuul d62bca8a03 Merge "Run gerritlib and Gerrit integration test" 2020-03-22 14:17:56 +00:00
Zuul c7e501b57e Merge "Log errors when committing and pushing acls" 2020-03-21 21:44:42 +00:00
Zuul 0ec183dd56 Merge "Don't use the db in manage-projects" 2020-03-21 21:44:41 +00:00
Monty Taylor 20808ddc5d Update ConfigParser imports for python3
We're starting to run jeepyb under python3 in the new gerrit image.

Change-Id: Ia71cb13f3b45055bbb9a311807fa65a89c8be62a
2020-03-21 16:42:29 -05:00
Jeremy Stanley 891047afa0 Overhaul default welcome message for OpenDev
The "Welcome, new contributor!" hook bot should leave a more generic
message relevant to most OpenDev projects, and not something
OpenStack-specific (as this could easily confuse new contributors to
non-OpenStack projects in OpenDev).

Change-Id: Ifc41e0639adf27a476025e66728d547a71ad99d5
2020-02-20 21:50:25 +00:00
Clark Boylan 2bb53ddb7f Run gerritlib and Gerrit integration test
This gives us functional integration testing of gerritlib and gerrit
with manage-projects. This could be expanded further to test more
functionality too.

Depends-On: https://review.opendev.org/#/c/704623/
Change-Id: I739807b26ccd89f9bf97b3706bcf567a9d790dd7
2020-01-28 11:25:23 -08:00
Clark Boylan 369d025428 Log errors when committing and pushing acls
For some reason we don't log the git output so failures are opaque to
us. Fix that by adding additonal logging.

Change-Id: If212ae6115c8266e1352431588c04547c897e982
2020-01-28 10:46:45 -08:00
Clark Boylan cef1dde44f Don't use the db in manage-projects
Direct access to the db restricts where we can run manage-projects from
and complicates testing. Instead rely on the list-groups ssh command to
get group uuid info. Note that this should be tested using new
integration testing before we merge it.

Change-Id: I6e3f0881398da8efeab5dfef370ad8d35163a7be
2020-01-28 09:52:51 -08:00
Monty Taylor 837515213f Don't index retired projects
While they should just have a README, it's still a waste of energy
for codesearch to index them in the first place.

Remove the special condition for deb- repos - they are all retired.

Change-Id: I60d90fccb607fcd48de42739188af5fb241ded7f
2019-12-18 10:55:16 -05:00
Monty Taylor ccb152d532 Fix create_hound_config for python3
pep8 did a business!

Change-Id: I1ffc99e30a0b5fc41c1f2485546b10553c525915
2019-12-18 10:55:16 -05:00