Commit Graph

119 Commits

Author SHA1 Message Date
Clark Boylan 0b7c438455 Switch from tox to nox
Recent tox releases have put us on a config treadmill. Avoid these
issues entirely by using nox. Nox is a tox alternative that uses
standard tools like pip and should be simpler to use for us.

Change-Id: I24277512cf542bac7a8852d50009c6f08db7fa37
2023-05-19 17:14:09 +00:00
Jeremy Stanley 9ae9f7936e Pin python requests when installing docker-compose
New urllib3 and python requests break python docker-compose. Pin
requests back which should use an older urllib3. Hopefully this
fixes docker-compose.

Note this should probably only be used as a temporary workaround. A
more permanent fix should change the way we install docker-compose.

Also stop relying on six since it's no longer available and we
dropped Python 2.7 support quite some time back anyway.

Clean up some things in tox.ini which regressed in Tox v4.

Co-Authored-By: Clark Boylan <cboylan@sapwetik.org>
Change-Id: I69a7ba09f737130a403f0657708877ce986f751c
2023-05-19 13:42:55 +00:00
Ian Wienand 0c5c5871fe
Drop Python < 3.8, add Python 3.10 testing
Also update the integration job to Jammy, and update to the latest
ansible in Zuul.

Change-Id: Idec1941f6879a0fabee8fac637bc1a43e0acd793
2022-12-01 12:22:10 +11:00
Clark Boylan aed4e49286 Add python39 testing
We deploy gerritlib via jeepyb on our gerrit images which have python39
now. Add testing for that version of python to help cover that setup.

We also drop the python3.5 default in tox.ini as we don't test that
version of python anymore.

Change-Id: I4be713a6fe67b6524eb3a463c9835851bc7f90e5
2021-10-12 15:24:38 -07:00
Clark Boylan 2e08509b16 Rename Gerrit Non-Interactive Users to Service Users
Gerrit made this change in versions 3.3 upstream. Reflect that here.

Change-Id: I580a4309b52a4a9c355624842057146a1922f5d1
2021-10-11 09:20:19 -07:00
Clark Boylan cdf19298e3 Fix CI setup
We need to assign nodesets that can run the requested python versions
when running tox-pyXY jobs. Also remove the Sphinx version cap as old
sphinx cannot install with modern setuptools. We don't run CI that
includes sphinx so this should be fine.

If this breaks docs builds we can sort that out later.

Change-Id: I577d67912fe6feb68ae50ff34c2622f9461dd122
2021-10-11 09:20:19 -07:00
Zuul cf83939469 Merge "Update documentation" 2021-05-07 01:40:09 +00:00
Zuul d3dfc491c2 Merge "Add ignore events filter" 2021-05-07 01:40:06 +00:00
Zuul 1aa6969c7e Merge "docker-compose: update header syntax" 2021-05-07 01:40:04 +00:00
Tristan Cacqueray d30b133a89 Add ignore events filter
This change enables GerritWatcher user to ignore noisy events
such as replicated-ref event.

Change-Id: If95d560a3dbb20e530e9ea1e94c1250aa083c9ea
2021-04-07 19:57:19 +00:00
Jeremy Stanley 874d4776ee Run gerritlib-jeepyb-integration on ubuntu-focal
Recent versions of pyca/crytpography, which jeepyb requires, need a
new enough pip to support ABI3 wheels and pyptoject.toml files. The
version on ubuntu-bionic nodes is too old for this, so use
ubuntu-focal for the job.

Also switch to referring to the Gerrit SSH API by IP address
(127.0.0.1) since it listens on 0.0.0.0 (all IPv4 addresses) but
ssh-keygen seems to resolve the name "localhost" to ::1 (the IPv6
loopback).

Change-Id: I9b09943bd6bb283a83497bc42e9ffdc9955eb01d
2021-03-24 01:08:49 +00:00
Brian Rosmaita f34c110fd9 Update documentation
Update various URLs in the docs, add .eggs to .gitignore, and correct
a pep8 problem in gerrit.py

Change-Id: Ie86ff3a7cb6e9c742eac0eab243cc374ead42fb6
2021-01-04 22:19:38 -05:00
Ian Wienand 6705931678 docker-compose: update header syntax
The HEADER_ format argument has been deprecated for some time, but
now removed from Ansible.  Update it.

Change-Id: I80256978f676e58416c939efd78003df0d08dd56
2020-12-08 12:51:51 +11:00
Ian Wienand 99136e1e16 integration test: run manage-projects with debug
This should have gerritlib log the commands it is running in the logs,
which helps with verification.

Change-Id: Iad3c710d73a95d152689d8a60a7ef4051129b0f9
2020-12-03 11:34:28 +11:00
Clark Boylan 3ff5407988 Test manage-projects repos harder
The previous change assumed that all projects would have their acl file
paths updated via ansible's lineinfile module. But lineinfile only
updates the last match. Since we've got two config lines we want to
match and update we run the lineinfile task twice which should end up
updating both of them.

Change-Id: I714281d86569f248f4ae3d909b019f060fe6c520
2020-12-02 15:28:32 -08:00
Clark Boylan a7f00a4d9b Handle newer Gerrit ls-groups command syntax
In Gerrit 2.14 the -q flag to ls-groups was deprecated in favor of -g.
At some point between 2.14 and 3.2 the -q support was removed entirely.
This change checks the gerrit version then uses the appropriate flag for
querying a group based on that.

We also add more robust testing to check creation of groups with spaces
in their names. We also check that if an acl is updated without group
changes that we don't try to recreate the group and get the group uuid
successfully.

Change-Id: I58c0aa2b496bcd792c8f4bc7db89b08cefa3af3d
2020-12-02 14:36:20 -08:00
Zuul 05d2786986 Merge "Avoid crash due to POLLIN event check" 2020-12-02 14:29:33 +00:00
Ian Wienand 194ddff30a Handle empty reads as closed connections
When the new Opendev 3.2 Gerrit closes the SSH connection, gerritlib
goes into a hard loop.  Either this has never really worked, or our
new version of gerrit closes the SSH connection differently and
triggers this.

Currently the _read() function in this code interprets a 0 read as
data and continues the event loop.  This means the _listen() function
never exits.  The poll() keeps returning that there is a POLLIN event
and the _read() function keeps reading the empty FD.

Modify this to set the state to DISCONNECTED for a blank read.
Additionally improve the error message if we see invalid data (which
shouldn't happen).  Insert a short-circuit in the listen to check if
the state has switched to DISCONNECTED and exit the loop in that case.

Change-Id: I46849bcbfc3a0d409baa654867018994ce717481
2020-11-24 11:58:48 +11:00
Sorin Sbarnea 12e395e8ea Avoid crash due to POLLIN event check
Fixed bug which prevented use on newer version of python as we
incorrectly checked event value against the bitmask.

Also improves exception by included value of the event in it.

Change-Id: I49f41d2555f53f2377c32c7626e1350c4f42b8de
2020-11-03 09:27:08 +00:00
Zuul 73005984a9 Merge "Decouple linters from unit testing" 2020-10-01 03:07:04 +00:00
Zuul a32237cc7d Merge "Fix package metadata" 2020-10-01 03:02:36 +00:00
Sorin Sbarnea b317b5737c Fix package metadata
Updates outdated urls, mailing lists and team names in metadata.

Change-Id: I69a88c3bdeb5b5b363d606648b1ae034912fc4c3
2020-09-28 16:33:54 +00:00
Sorin Sbarnea ac82273984 Allow custom retries on gerrit connection
Previous code was not allowing use of Gerrit without hardcoded infinite
retries inside Gerrit constructor, meaning that wrong credentials
would create an endless loop which gerritlib consumer could not
prevent.

This allows client to specify number of attempts and delay before
a failure is returned.

Change-Id: I5bd14a14539a2705babb9f9922c800383031c6ac
2020-09-28 16:50:00 +01:00
Sorin Sbarnea 8ceeaa5a20 Decouple linters from unit testing
Avoids conflicts between unit testing and linter and avoids
installing unneeded dependencies on both of them.

Change-Id: Id384a998bea298612ec65eaa329f85a38afee3f6
2020-09-28 16:45:59 +01:00
Arun S A G 40858f3678 Add support for TCP keep alive in gerritlib
Clients using gerritlib behind NAT/firewall
can be stuck when the NAT/firewall discard
packets belong to the TCP connection after extended
silent periods thinking the connection is dead.

This change allows clients using gerritlib to set
keep alive interval to a non zero value. This will
enable TCP keep alive packets to be sent in
configured intervals keeping the TCP connections alive.

Change-Id: I588706bb51ce41810cd2243f4969763d193d1d55
2020-09-09 17:45:39 -07:00
Clark Boylan 6058d994b8 Add branches arg to createProject
This new functiona argument will allow callers to set the default
project branch on repo creation. It takes a list, all branches will be
created with the first being repo HEAD.

Change-Id: Iae5a71662740678e663bcd73d6ac660559db5d9a
2020-07-15 11:11:44 -07:00
Sorin Sbarnea 95e17ddf05 Fixed linting issues
Newer versions of flake8 identify these valid violations.
N802 added as skip because the entire library uses camelcase.

Change-Id: Ia687b05bb8d0590eb30a970a710a2639548dbc60
2020-05-21 17:12:40 +01:00
Sorin Sbarnea 2f666c7b35 Replace testrepository with stestr
Replace outdated testrepository with its newer sibling, which
is maintained.

Change-Id: I9755b82d8a2a82f146e71fdc0f32e3f6002e804d
2020-05-20 20:07:15 +01:00
Sorin Sbarnea a0a5f260ea Enable py36-py38 testing
- updates tox.ini, also fixing a typo
- enables new jobs, which would at least test ability to install
  package under these versions.
- updates Trove metadata
- replace fake sole-survivor unittest with a bit more practical one

Change-Id: Iae224a19850489ba3563ad5a40d358002dacdb97
2020-05-20 19:38:05 +01:00
Sorin Sbarnea c49a8e0e5f Added missing retry_delay instance variable
Found exception at line 179, where self.retry_delay was references
without being assigned.

Change-Id: I6d00fe9478ed3467120e0441bed3c32f68e20bca
2020-05-20 18:13:21 +01:00
vass ad26169a4d Use ensure-* roles
The install-* roles in zuul-jobs have been renamed to ensure-*, follow
the rename.

Change-Id: Ia5dffd38399142d9f17a1123b3e4a46bdbca7662
2020-04-13 09:36:36 +02:00
Zuul 04537022f9 Merge "Return lists from listing functions" 2020-03-30 17:35:16 +00:00
Monty Taylor 833fce6eba Run gerritlib integration tests with python3
Now that we're running manage-projects in a python3 container,
we should switch our integration tests to use python3.

Depends-On: https://review.opendev.org/715527
Change-Id: I33a6daf92906d9087c4a31f11966e5037445cd68
2020-03-27 16:22:57 -05:00
Clark Boylan 17380a3e71 Return lists from listing functions
We use filter to filter out entries when listing items like groups. This
returns a list in python2 but an iterable in python3. Return a
consistent list via forced coersion.

Change-Id: Iad84f6f383fb85363f3ef51c4bd1da05b8069e43
2020-03-27 14:18:05 -07:00
Zuul cfceeb50e0 Merge "Add integration testing with jeepyb and gerrit" 2020-03-22 14:06:04 +00:00
Clark Boylan fa0a23a751 Pass Gerrit not GerritConnection to GerritWatcher
We want to call addEvent on the 'gerrit' arg passed to GerritWatcher.
For that to work it needs to be of type Gerrit not GerritConnection.

Update the location of username, port, etc data to the connection
details in the gerrit object.

Change-Id: Ic739ddc970f6761f5587ab8fcba3e619a98a4379
2020-01-28 14:14:20 -08:00
Clark Boylan 2b81068c01 Add integration testing with jeepyb and gerrit
This adds integration testing to gerritlib with jeepyb and gerrit.

Depends-On: https://review.opendev.org/704660
Change-Id: I1815e63693656d52c10a9cb9bf60f999e036dfed
2020-01-28 11:01:01 -08:00
Zuul d244a41362 Merge "Convert gerrit output to utf8 unicode" 2020-01-28 02:28:47 +00:00
Zuul 0b438471c8 Merge "Use newer paramiko" 2020-01-28 02:00:56 +00:00
Clark Boylan 945d2088a7 Convert gerrit output to utf8 unicode
We get back bytestrings from paramiko when it talks to gerrit. Make this
a bit more user friendly for python3 and decode these bytestrings into
utf8 unicode.

Change-Id: I1866de53d0978f805faef0933ef37e68454ff5f3
2020-01-27 16:13:13 -08:00
Clark Boylan c234e11b51 Use newer paramiko
Old paramiko uses pycrypto which is out of date and insecure. Switch to
new paramiko and cryptography instead.

Change-Id: I651244d549d10a848643d1e3b96438833a5954bd
2020-01-27 16:13:11 -08:00
Darragh Bailey a731676fd1 Support querying for single group info
Allow querying to check if a single group entry exists, which when
combined with 'verbose=True' will return the UUID for a single group.

Change-Id: I851ab29999c0d059c9cb71bd38a821c035ae98f7
2020-01-23 12:11:34 +00:00
Darragh Bailey 500c8d2cc9 Reuse same paramiko connection recreating on errors
Move connection related code to a common object to allow reuse of the
connection setup code. This helps avoiding unnecessary recreation of
the connection where it is being used multiple times within the same
process for multiple calls.

Change-Id: Idf318ce5c36100e550182e773ce55da69fe35063
2020-01-23 12:01:52 +00:00
OpenDev Sysadmins 609deaf9a5 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:26:01 +00:00
Ian Wienand 1d63855d0d Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I194eaf19074655822776b2e533519607622738c8
2019-03-24 20:33:40 +00:00
Zuul f7d1832b45 Merge "Updated list of Python versions in tox" 2018-06-28 17:01:33 +00:00
Clark Boylan 2de58d560f Close ssh connections when done with them
Newer paramiko leaks connections if they are not explicitly closed. Add
a finally handler to always close the ssh connections when we are done
with it to avoid leaking these connections.

Change-Id: Ia2e53998d362683a42bda074d82e3a3a75f380b4
2018-05-03 13:39:57 -07:00
Sorin Sbarnea ffbc080f1a Updated list of Python versions in tox
Change-Id: I40118820d7eb0956dc012556aa1d5767f40d652e
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2018-01-27 14:50:51 +00:00
Clark Boylan 01a634014e Add group membership management functions
Add functions to list group members and add and remove users from
groups.

Change-Id: Id5d8915f2c0bf6f6057b414785b8ec1b18e810f2
2017-09-22 10:24:53 -07:00
Clark Boylan eb6588734c Handle different gerrit versions with create-project
Gerrit < 2.12 used a --name argument on create-project. Newer Gerrit
simply takes the project name as an arg. Check the Gerrit version before
constructing the create-project command.

Change-Id: I59ca051627d54a11ef6524cce9b5a4414fd2193d
2017-09-19 16:25:39 -07:00