Commit Graph

56 Commits

Author SHA1 Message Date
Zuul d7ee8cd996 Merge "reno: Update master for unmaintained/wallaby" 2024-03-12 14:21:12 +00:00
Zuul 05f7736160 Merge "reno: Update master for unmaintained/victoria" 2024-03-12 13:15:58 +00:00
Takashi Kajinami 3bce8e1dca Make authentication/SSL for redis sentinel optional
Change 4954e284b9 updated the redis
sentinel driver to apply auth/ssl settings for redis sentinel, based
on ones of redis, but this change broke the existing usage in kolla
deployments, which require redis with authentication enabled and
sentinel with authentication DISABLED.

This restores the old behavior, which do not enable authentication and
ssl for sentinel even when these for redis is enabled.

Closes-Bug: #2056656
Change-Id: I3047c80359df3dad64be041db6f4a3a6180479d6
2024-03-11 10:28:06 +09:00
OpenStack Release Bot f039107925 reno: Update master for unmaintained/wallaby
Update the wallaby release notes configuration to build from
unmaintained/wallaby.

Change-Id: I7d324b3ddb89bf7f970f066abce5e9acee4219be
2024-03-07 14:24:09 +00:00
OpenStack Release Bot b039a92768 reno: Update master for unmaintained/victoria
Update the victoria release notes configuration to build from
unmaintained/victoria.

Change-Id: Id40c905697cf174690d78e82c57587a3989b55c0
2024-03-07 14:19:48 +00:00
Zuul 1d0ece27a4 Merge "redis: Fix wrong type used to parse socket_keepalive" 2024-02-16 18:30:15 +00:00
Takashi Kajinami b950cf8cf4 redis: Fix wrong type used to parse socket_keepalive
The socket_keepalive option in RedisClient is not an integer but
a boolean. This fixes the type to parse the options accordingly.

Note this change still keeps support for socket_keepalive=1 or
socket_keepalive=0 , but would remove support for the other integer
values such as 2 or -1.

Change-Id: I47b8628b9676f37ab5bb82b0fa9e45fcff679893
2024-02-14 11:46:47 +09:00
Takashi Kajinami 593b328c2f redis: support healthcheck interval
Redis client supports health checks using PING/PONG since 3.3.0[1] but
this feature was not configurable in tooz.

This adds the new health_check_interval option, which is forwarded to
Redis client, to enable that feature.

[1] f60b2b07ca

Change-Id: Icba79be1929ec460d3cb17363e701fbee7de7d76
2024-02-14 11:29:29 +09:00
Takashi Kajinami 4954e284b9 Redis: Fix missing ssl/auth options for sentinel
This ensures options for redis connections are replicated to sentinel
clients, so that users can enable SSL or authentication in Redis
sentinel while they also enable these in Redis.

Closes-Bug: #2052372
Change-Id: I78727387cf8287554549ff5a99a80f3317cbd59b
2024-02-04 19:31:35 +09:00
Takashi Kajinami a267b2811b redis: Add username
Redis introduced ACL feature in 4.0.0, and this feature is supported by
redis-py since 3.4.0[1]. When ACL is enabled, authentication requires
username in addition to password.

[1] 8df8cd54d1

Closes-Bug: #2052371
Change-Id: I9b53c3a314dcffbe49535799ba6d7a1fdddb8d84
2024-02-04 08:11:23 +00:00
Zuul 770894333f Merge "Redis: Allow customizing ssl ca certificates path" 2024-02-01 19:45:04 +00:00
Zuul ef898ba118 Merge "etcd3gw: Discover API version automatically" 2023-12-19 18:56:07 +00:00
Takashi Kajinami 9acbc53915 Deprecate zake driver because of unmaintained library
The zake library[1] was already archived a few years ago. It has had
no release for 7 years.

[1] https://github.com/yahoo/Zake
[2] https://pypi.org/project/zake/#history

Change-Id: I82d6cdeec2be4f69d325270f4cd74f2cafe9db55
2023-12-18 06:08:41 +00:00
Takashi Kajinami f180411f42 etcd3gw: Discover API version automatically
etcd changed API version from v3beta to v3alpha, then v3, and because
of transition available api versions are different according to
the etcd server available.

This change implements the mechanism to detect the current api version
according to the etcd version obtained via the version API. So that
the url without api version works regardless of the etcd server used.

Closes-Bug: #2043810
Depends-on: https://review.opendev.org/c/openstack/etcd3gw/+/901244
Change-Id: Ib81390eb73feafe08d007db1ced4d0ceb3bde212
2023-12-15 09:51:40 +09:00
Takashi Kajinami 38523acdc6 Redis: Allow customizing ssl ca certificates path
This allows users to give ssl_ca_certs argument of redis client so that
a specific file, instead of the system global certificates, can be
used.

Closes-Bug: #2012226
Change-Id: Id7e76c35c5ce7b22a8261026fe76557d509616b3
2023-11-27 10:13:03 +09:00
Takashi Kajinami 6bc02cda5b Remove etcd3 drvier
This driver was deprecated in the 2.11.0 release[1]. The 3.2.0 release
which contains this deprecation is used in upper-constraints of
stable/2023.1 so we are ready to remove this feature now.

[1] 7ee2780af8

Change-Id: I5ca2fe43cb25b4687ace6bd21a866b74f289d628
2023-11-17 23:34:26 +09:00
Jan Gutter 738931d026 Update the default etcd3gw endpoint to v3
* Recent versions of etcd no longer support the /v3alpha/ endpoint
* It's probably better to skip directly to the /v3/ endpoint
  as a default.
* Note that this probably means a new minor version of tooz, while
  the behaviour can be worked around in config, this is enough to
  break existing systems on upgrade.
* Updated the test to test backwards compatibility.
* Updated the tests to use etcd 3.3.27, unfortunately we need
  a newer version of pifpaf that supports 3.4 before the default
  api can be used.
* The etcd backend tests have been removed: the deprecated backend
  uses the v2 api which is no longer provided - a future commit
  should remove the deprecated backend. The etcd3gw backend is the
  replacement.

This fix is required in pifpaf, but it's not in any release:
7496e5e1c3

Depends-On: https://review.opendev.org/c/openstack/devstack/+/891353
Change-Id: Ibb5e587f6b1348794b1bf4d6405bc4a32a653c2c
2023-09-27 09:26:52 +00:00
Tobias Urdin 378566dab8 Change StrictRedis usage to Redis
The StrictRedis class is only an alias for
Redis in >= 3.0.0

Change-Id: I89146dcf63cf984410f29b31ac2492f5ccaf3468
2023-01-18 09:55:44 +00:00
Dmitriy Rabotyagov f11666b66c Allow to pass ssl-related args for zookeeper
Zookeeper does support TLS encryption and authentication for client
connections. There's no reason not to pass these arguments to the kazoo
to allow encrypted connections.

We bump minimum kazoo version to 2.6.0 since change implementin SSL support
has been merged with [1] and was first released with 2.6.0 tag.

[1] 35ce10669a

Change-Id: Ied29512989f477a19753afcb789e5588877fd688
2022-12-08 19:54:11 +00:00
Zuul d145b503ca Merge "Add TLS support for MySQL driver" 2022-07-15 21:00:16 +00:00
Zuul a10db5d089 Merge "[etcd3gw] create new lease if expired." 2022-07-15 20:20:25 +00:00
zhen 6b2216ebcc Fix inappropriate logic in memcachedlock.release()
Whether 'was_deleted' was 'TRUE' or not, eventually we have to remove
self from '_acquired_locks'.
For example:
1. App #1 with coordinator 'A' wants to release lock "b"
2. 'self.coord.client.delete()' failed for some reason(.e.g,
BrokenPipeError,MemcacheUnexpectedCloseError)
3. According to the former logic,lock "b" will not remove
from "_acquired_locks", so "self.heartbeat()" will make it alive
forever until App #1 was down or lock "b" turned expired.
4. Now App #1 with coordinator 'A' wants to acquire lock "c", who
have the same lock-name with lock "b",It is clear that this will
fail and prevent the locked program from continuing to execute.

Change-Id: I6fc33b8e0a88510027bcfc30d1504489d2a91b4e
2022-05-26 17:23:41 +08:00
Mitya_Eremeev ae5adcf387 [etcd3gw] create new lease if expired.
It can happen membership lease can be expired
to the moment of lease refreshing.
In this case heartbeat will try to refresh
expired lease forever without success.
The patch checks if lease expired and create new one.

Closes-Bug: 1975524
Depends-On: https://review.opendev.org/c/openstack/etcd3gw/+/843003
Change-Id: I5017724fd0134f3f0a51f059640e641dd9853ff2
2022-05-25 20:21:42 +03:00
Tobias Urdin 7ee2780af8 Deprecate the etcd3 driver
The etcd3 driver depends on the python-etcd3 [1]
library that is currently stale and no has no
active maintenance [2] with a non-working CI
that blocks all PRs.

It's also currently blocking Tooz from supporting
newer versions of tenacity [3] that a lot of
projects is now pinning because newer version of
tenacity dropped old legacy compatibility code
that has been in there for years.

We currently have etcd3gw as an alternative driver
where the library the driver depends on is a
deliverable that we maintain.

[1] https://github.com/kragniz/python-etcd3
[2] https://github.com/kragniz/python-etcd3/issues/1719
[3] https://review.opendev.org/c/openstack/tooz/+/829412

Change-Id: Iff0cd3b51cfc45ecbddc831c55267e80e9c79cac
2022-03-15 10:35:09 +00:00
Pavlo Shchelokovskyy 81c629676c Add TLS support for MySQL driver
use pymysql TLS-related arguments as query params in DB connection URL.

Change-Id: I3136df2a9ac4e05daa46f33e47c1438e3b02a321
2021-11-08 19:41:16 +02:00
Pavlo Shchelokovskyy 47c4d56e44 Enable retries in redis driver
this is followup to Iaab5ce609c0dcf7085f5dd43efbd37eb4b88f17b

actually retry for specified number of retries instead of raising
error on first ConnectionError

Change-Id: Ibca3f568b65dfea252da4b67f6d5105ba7f1ecb1
2021-06-16 19:44:38 +03:00
Pierre Riteau 608ea7979d Fix formatting of release list
Change-Id: I5b599bcc101446df96c2aebcc66962263709ad49
2021-04-16 14:23:46 +02:00
OpenStack Release Bot 420059e69a Update master for stable/wallaby
Add file to the reno documentation build to show release notes for
stable/wallaby.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/wallaby.

Sem-Ver: feature
Change-Id: I84eb9877fd221a1c14bf02b00f38ed77c67ed7a9
2021-03-18 11:34:29 +00:00
Hervé Beraud c3d8c2ed96 Adding pre-commit
Introduced changes:
- pre-commit config and rules.
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.

Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.

pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker
  (check-byte-order-marker);
- Checks that non-binary executables have a proper
  shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings
  (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
  calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)

For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks

Change-Id: I7a2386a823fdd43b138956894bd915c0e11289a9
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
2020-10-09 15:35:34 +00:00
OpenStack Release Bot b7fe722e97 Update master for stable/victoria
Add file to the reno documentation build to show release notes for
stable/victoria.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/victoria.

Change-Id: I28b00085199c59047b66d2c13960ba518941937b
Sem-Ver: feature
2020-09-11 21:04:20 +00:00
Zuul fca43df897 Merge "hashring: allow choosing hash function" 2020-07-21 12:12:11 +00:00
Dmitry Tantsur d59b283440 Fix breakage with PyMySQL 0.10.0
In this version a Connection is no longer a context manager. Fix
it by simply getting a Cursor out of it (locks don't seem to
interact with transactions, at least according to MariaDB docs).

Change-Id: I5ea06ebd2b976465ff82f10a74e140f30e9e803f
2020-07-20 15:52:32 +02:00
Dmitry Tantsur f5d32480d9 hashring: allow choosing hash function
In FIPS mode it's not possible to use md5 for any purposes.

Change-Id: Ia3d9035abb645e82083fe9a796d7d7df7cf4d7a7
2020-07-12 14:56:17 +02:00
Andreas Jaeger 69fc39083f Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems

Update Sphinx version as well.

Disable openstackdocs_auto_name to use 'project' variable as name.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I6254b8b17eea752a5b1d6031c89de47ecc865a49
2020-06-02 13:55:46 +02:00
Zuul 94bf257392 Merge "Update master for stable/ussuri" 2020-05-25 12:41:06 +00:00
JP Bourget 08b2d93226 Add support for Consul ACL token parameter.
When running Consul with ACLs enabled, the best practice is to
default deny all. In order to support applying a policy to the
tooz coordination KV store path, passing through an ACL is needed.

Closes-Bug: #1752205
Change-Id: I98fc96468b21368ce66365e3fc38c495b1f2918a
2020-05-14 21:48:49 -04:00
OpenStack Release Bot b11456a00d Update master for stable/ussuri
Add file to the reno documentation build to show release notes for
stable/ussuri.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/ussuri.

Change-Id: I5e9b7258593e10ff1323ab408acaf2f075fdc111
Sem-Ver: feature
2020-04-14 10:20:27 +00:00
Zuul 1880be2c00 Merge "Add TLS support in etcd3 and etcd3gw drivers" 2020-04-01 18:27:03 +00:00
Alan Bishop a598cce62b Add TLS support in etcd3 and etcd3gw drivers
The etcd3 and etcd3gw drivers parse CA, key and cert options from
the coordination URL, and pass them on to the backend clients. The
etcd3gw driver implements the "etcd3+https" scheme.

Change-Id: I78d8ca0583f883f7f746791f82fbcc116458ce2c
2020-04-01 06:56:51 -07:00
Nick Maludy 6e750b6921
Adds heartbeating to the consul driver.
Change-Id: Ie22e944e270dfd37988724f76823f6fd38d19b21
Closes-Bug: #1752225
2020-03-03 15:43:00 -05:00
Andreas Jaeger ba27954b06 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Parent tox py36/py37 jobs to openstack-tox-py36/37 so that the correct
interpreter is installed.

Change-Id: I14a01f0a9989195e30c3ed0dadec35490f41bd48
2020-02-03 22:19:47 +01:00
OpenStack Release Bot c430dbaa69 Update master for stable/train
Add file to the reno documentation build to show release notes for
stable/train.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/train.

Change-Id: I42bd5786c06043c21010ef7260184b5d9d4de00f
Sem-Ver: feature
2019-09-20 17:45:17 +00:00
OpenStack Release Bot d2dec45f28 Update master for stable/stein
Add file to the reno documentation build to show release notes for
stable/stein.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/stein.

Change-Id: Ice93b3895ae4c9699e3656c6e493b1b368badc0f
Sem-Ver: feature
2019-05-23 15:47:47 +00:00
OpenStack Release Bot 22083baafd Update reno for stable/rocky
Change-Id: I2bd0a2be48eb68ecb95da210cc3b393ff6c8d31e
2018-07-26 12:43:11 +00:00
Thomas Herve 6ab8c380c8 Implement group support for etcd3gw
This adds the various methods to support groups operations on the etc3gw
driver.

Change-Id: I8acb2f617f6044449fb3a91a9c3de73ef0061d36
2018-02-28 22:08:55 +01:00
OpenStack Release Bot e173ac10d1 Update reno for stable/queens
Change-Id: Id73fa795a6236ed349564a8ea823b8c63c1edd8c
2018-01-24 18:11:38 +00:00
Andreas Jaeger 7c1bfe579f Remove setting of version/release from releasenotes
Release notes are version independent, so remove version/release
values. We've found that projects now require the service package
to be installed in order to build release notes, and this is entirely
due to the current convention of pulling in the version information.

Release notes should not need installation in order to build, so this
unnecessary version setting needs to be removed.

This is needed for new release notes publishing, see
I56909152975f731a9d2c21b2825b972195e48ee8 and the discussion starting
at
http://lists.openstack.org/pipermail/openstack-dev/2017-November/124480.html
.

Change-Id: I17194b0c5fabbed79cb2082d650b29857f92072c
2017-11-16 22:31:28 +01:00
Jenkins 4a8f74700e Merge "Add etcd3 group support" 2017-08-23 16:52:10 +00:00
OpenStack Release Bot 1b321bcfc1 Update reno for stable/pike
Change-Id: Ie2e38752b4955e21c6618d21c6cd8444c93368e2
2017-07-28 21:09:02 +00:00
Akihiro Motoki 38bcf7bbdb Switch from oslosphinx to openstackdocstheme
Change-Id: Id5e2952676e89f768e13b4ffc2f238eb012ec324
2017-07-07 02:54:25 +00:00