Commit Graph

62 Commits

Author SHA1 Message Date
Takashi Kajinami 6bc02cda5b Remove etcd3 drvier
This driver was deprecated in the 2.11.0 release[1]. The 3.2.0 release
which contains this deprecation is used in upper-constraints of
stable/2023.1 so we are ready to remove this feature now.

[1] 7ee2780af8

Change-Id: I5ca2fe43cb25b4687ace6bd21a866b74f289d628
2023-11-17 23:34:26 +09:00
Dmitriy Rabotyagov 7d50893625 Loosen protobuf version that's used for docs/renos
protobuf package is present in upper-constraints, thus should not be
specified in requirements to version that is smaller that one on u-c,
otherwise pip will fail due to conflict between requirements and
constraints.
We do that only for docs and releasenotes, since used workaround
decreases performance.

The issue is mainly raised by etcd3 module that we rely on, so not
to affect runtime things a left as is for now.

Change-Id: Ieeba8d53a8311ed6cc228f444c831d148ee29025
2022-11-30 18:16:48 +01:00
Christian Rohmann 0c145b4567 Add grouping support in etcd to compatibility matrix.
Support has been added with
6ab8c380c8

Closes-Bug: 1995125
Change-Id: I507301618c2bb32fa989db3e5634016c8e15f305
2022-10-28 17:28:21 +02:00
jiaqi07 00972f3b09 remove unicode from code
Change-Id: Ie63625d79dff284ce985e1fad8c487e5281fbbb4
2022-08-02 16:11:32 +08:00
Stephen Finucane e3de0e634a Cap protobuf < 4.x
tooz doesn't use upper-constraints so we get the newest version of all
dependencies. It seems one of these, etcd3, is incompatible with recent
versions of protobuf.

  Failed to import test module: tooz.tests.drivers.test_etcd3
  Traceback (most recent call last):
    File "/usr/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
      module = self._get_module_from_name(name)
    File "/usr/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
      __import__(name)
    File "/home/zuul/src/opendev.org/openstack/tooz/tooz/tests/drivers/test_etcd3.py", line 22, in <module>
      import tooz.drivers.etcd3 as etcd3_driver
    File "/home/zuul/src/opendev.org/openstack/tooz/tooz/drivers/etcd3.py", line 18, in <module>
      import etcd3
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/__init__.py", line 3, in <module>
      import etcd3.etcdrpc as etcdrpc
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/__init__.py", line 1, in <module>
      from .rpc_pb2 import *
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/rpc_pb2.py", line 16, in <module>
      from etcd3.etcdrpc import kv_pb2 as kv__pb2
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/kv_pb2.py", line 36, in <module>
      type=None),
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 755, in __new__
      _message.Message._CheckCalledFromGeneratedFile()
  TypeError: Descriptors cannot not be created directly.
  If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
  If you cannot immediately regenerate your protos, some other possible workarounds are:
   1. Downgrade the protobuf package to 3.20.x or lower.
   2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

The etcd3 dependency is only used for the etcd3 driver which was
deprecated in change Iff0cd3b51cfc45ecbddc831c55267e80e9c79cac. The
protobuf dependency is only used for etcd3. We can't outright remove the
etcd3 driver yet as it hasn't been long enough since we deprecated it.
That will happen soon enough though and until then we can simply cap
protobuf to an older version that is compatible with the most recent
etcd3 release.

While we're here, we also sync the lower boundaries in the docs
requirments and add some comments indicating where they come from/what
they're for.

Change-Id: I8d6647118be22b0ce55e01b7e5451612ebe30e73
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2022-06-28 15:11:35 +01:00
Hervé Beraud c3d8c2ed96 Adding pre-commit
Introduced changes:
- pre-commit config and rules.
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.

Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.

pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker
  (check-byte-order-marker);
- Checks that non-binary executables have a proper
  shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings
  (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
  calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)

For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks

Change-Id: I7a2386a823fdd43b138956894bd915c0e11289a9
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
2020-10-09 15:35:34 +00:00
Nick Maludy 80f379bfa2
Implements Group API for the Consul driver
Also fixes tox env for py38-consul tests
Update zuul script to use consul 1.7.4 (2020) vs 0.6.3 (2016)

Adds ACL tokens to all session management calls.

Change-Id: Iaddf21f14c434129541e7c9ec7134e0661f7be52
2020-06-15 11:24:49 -05:00
Zuul 130df13e18 Merge "Add release notes links to doc index" 2020-06-03 09:08:46 +00:00
Andreas Jaeger 69fc39083f Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems

Update Sphinx version as well.

Disable openstackdocs_auto_name to use 'project' variable as name.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I6254b8b17eea752a5b1d6031c89de47ecc865a49
2020-06-02 13:55:46 +02:00
Hervé Beraud a5abf9ef08 Add release notes links to doc index
Change-Id: Ibd20fc5776b789bb869cdb1f7b1e58acc4c2c207
2020-04-16 17:54:15 +02:00
Andreas Jaeger ba27954b06 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Parent tox py36/py37 jobs to openstack-tox-py36/37 so that the correct
interpreter is installed.

Change-Id: I14a01f0a9989195e30c3ed0dadec35490f41bd48
2020-02-03 22:19:47 +01:00
ushen 771a3e99f8 Blacklist sphinx 2.1.0
See https://github.com/sphinx-doc/sphinx/issues/6440 for upstream details

Change-Id: I9a292920cc05c296ad28d1f45c818ee679685f07
Depend-On: https://review.opendev.org/#/c/663060/
2019-07-04 09:32:20 +08:00
Hervé Beraud 89b61cddb2 Update Sphinx requirement and uncap grpcio
Sphinx 2.0 no longer works on python 2.7, so we need to start capping it there.

grpcio-1.16.0 introduced a bug that is breaking the tooz unit tests.
cf. I8b026a0680a91a68281e354a59012565f4db1b95 for more info. 1.18.0
fixes the problem, so bump our minimum requirement to that version.

Change-Id: I4b672563f4a80ce0e0478c867b7f85842ac3b0d3
2019-06-04 09:24:49 +02:00
caoyuan 96633f3b92 Replace git.openstack.org URLs with opendev.org URLs
1. Replace git.openstack.org URLs with opendev.org URLs
2. Remove unncessary "=="

Change-Id: I788df8332b620cd914c9c5cffdd88e2dcf31e045
2019-05-22 14:58:13 +08:00
Ben Nemec e76235b675 More explicitly document driver connection strings
If you're not intimately familiar with RFC 1808 it can be a bit tricky to
write tooz connection strings. In addition, some drivers do not document the
options that they recognize in a connection string. This can force a user to
read the driver code in order to figure out how to configure tooz to use said
driver, which should not be the case.

This change more explicitly documents the format of the connection string,
including which parts of an RFC 1808 URI it recognizes, and the options that
can be passed. It also clarifies which parts of the connection string are
optional (possibly because they have defaults) and which must always be
included.

Note that not every possible configuration is documented. For example, the
postgres driver supports multiple methods of configuring the connection host
and port. For consistency, I only documented the one that matches the other
drivers. This should be sufficient to allow a user to write a working
connection string.

This change also wires the etcd3 and etcd3gw drivers into the doc index since
they were missing before.

Change-Id: I66e54433115f27fb54badc2173f8f6044c49aacb
2018-12-17 22:37:55 +00:00
Ben Nemec 19c0a0b468 Switch to autodoc_default_options
autodoc_default_flags was deprecated and is now triggering a warning
that breaks our builds. Switch to the equivalent autodoc_default_options
configuration, which essentially turns the flags into dict keys that
map to None[1]: 'Setting None is equivalent to giving the option name
in the list format (i.e. it means “yes/true/on”).'

Change-Id: I573f19a054e535a90db8bb16a02b2c7f28d8e6d1
1: http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_default_options
2018-09-18 00:37:30 +00:00
Doug Hellmann bb234731d3 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Decode the output of subprocess in doc/source/conf.py so that we always
have a string.

Change-Id: I17601d56b6e6d9d300bb07d8982a7bbde637822e
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-06 16:18:21 -04:00
Andreas Jaeger e7afb739bf Add doc/requirements.txt
Fix doc building with new PTI changes and job updates.

Add a doc/requirements.txt file with all needed dependencies.

Change-Id: Ic7963520aeb0b9c463c4f9b5675589308720d6f8
2017-12-20 12:14:08 +01:00
Akihiro Motoki ee6f82d9d5 doc: use list-table for driver support tables
list-table is easier to be maintained.

In addition, we avoid too long lines.
We no longer have too long lines which doc8 complains,
so doc8 ignore_path can be dropped.

Change-Id: I574391295288ef9e33744aa608c2f2930a6e46f8
2017-07-10 12:44:29 +00:00
Akihiro Motoki 2ca15fedbe rearrange existing documentation to fit the new standard layout
Change-Id: If03b33fc1393c3a04d7d155b31e8377dc9e09360
2017-07-10 12:41:52 +00:00
Akihiro Motoki 38bcf7bbdb Switch from oslosphinx to openstackdocstheme
Change-Id: Id5e2952676e89f768e13b4ffc2f238eb012ec324
2017-07-07 02:54:25 +00:00
Akihiro Motoki cf12457896 Turn on warning-is-error in doc build
- Update sphinx so that warning-is-error option is available
- Fix sphinx warnings
- Install python modules required for automodule into docs env
- Fix docstring warning in tooz/coordination.py
- pbr warnerrors is no longer needed in favor of sphinx warning-is-error

Change-Id: I5d54c452522b6ee00884d7d17105af22b4c95189
2017-07-07 02:52:30 +00:00
Julien Danjou ba62597183 doc: update heartbeat doc to use start_heart=True
Change-Id: I5560b1ec1da7922586e5544d55b5e508c70a33fc
2017-04-27 16:40:57 +02:00
Julien Danjou 9ad740fa1e Add partitioner implementation
Add a new object called `Partitioner' which is based on the consistent
hashring implementation of tooz.hashring. The coordinator API allows to
create such an object and to use it to dispatch objects ownership across
several workers.

Change-Id: Iae8c861d5b6c15b714f1ec3bd7090d15dd468421
2017-01-01 19:03:40 +01:00
Julien Danjou f3dddcffdd Stop making tooz.utils depending on tooz.coordination
Change-Id: I5fdb1174ae1a2a84412f3b3159ad5835b2f7359f
2017-01-01 18:47:16 +01:00
Julien Danjou e29ca79ed9 Add a hashring implementation
This is a consistent hash ring implementation based on the one that can
be found in Nova, Ironic, Ceilometer and Aodh.

Change-Id: I5d2f4efcd354a187747fa645482db2029a1a14b7
2016-12-07 23:40:35 +01:00
Julien Danjou 04866ab9c3 Move ToozError to root module
Change-Id: I73b1c5a37b75991f79d8a5fb383c605689503333
2016-12-07 23:40:35 +01:00
Joshua Harlow 170680e93c Add docs for new consul driver
Change-Id: Ica007008d95cf2c9e5917b7258b793179ed63e1d
2016-06-13 14:16:03 -07:00
Jenkins 6daa485eeb Merge "Ensure etcd is in developer and driver docs" 2016-06-09 07:16:02 +00:00
Victor Morales 0f4e1198fd Fix coordinator typo
s/coordinataor/coordinator

Change-Id: I42e679663fca9577a0fa48962460083bcf4954c5
2016-05-24 14:33:12 -05:00
Joshua Harlow 601fb264c5 Ensure etcd is in developer and driver docs
Change-Id: I2da690a7ea93646e38d86f90168571abaab69067
2016-05-12 22:32:30 -07:00
Jenkins 349834f39c Merge "Add programatic introspection of drivers characteristic(s)" 2016-01-08 17:02:01 +00:00
Joshua Harlow a3f22fe90c Add programatic introspection of drivers characteristic(s)
Change-Id: Icf187db9ba7d076b1a0f3cc9343acb0eaa1862cb
2016-01-07 12:38:15 -08:00
Jenkins dbde7f4787 Merge "etcd: driver with lock support" 2016-01-04 14:54:23 +00:00
Julien Danjou d2529173ec etcd: driver with lock support
Change-Id: Ibac90b9b2a751eb4f502e2f8b723e5608dcaad18
2015-12-29 10:02:05 +01:00
Joshua Harlow ed7cf36f5b Remove tested under 2.6 from docs
We haven't tested under 2.6 in a while so remove
it from being mentioned in the docs.

Change-Id: I2bba68f1b2a2c7d51ecaac1a75808ec957bb0db1
2015-12-19 17:40:57 -08:00
Joshua Harlow 24621a5d20 Spice up the driver summary/info page
Start to organize this page with a more helpful structure
that will allow people to read it and figure out which driver
best fits there usage (and to help people decide which ones
don't).

Change-Id: I871d2eb7e7493a6633f7738ff063042d39690364
2015-12-17 13:06:17 -08:00
Joshua Harlow 8482fad4ff Include changelog/history in docs
Change-Id: Ic68fc583338c48f829cd36c21e1c2fe654022bf5
2015-09-24 15:06:05 -07:00
Jay Clark 3fca529ea3 Changes to add driver list to the documentation.
Change-Id: Ic3c8273813c5147ca9203e761d4ade43d3c2ad31
2015-08-26 15:01:41 +00:00
Joshua Harlow 5dc4f3d107 Update compatibility matrix due to file drivers new abilities
This also fixes the redis driver matrix since it now does have
leadership support.

Change-Id: I8f2f075a3d70df75b649dc3f5715ab22953d659f
2015-06-27 09:46:54 -07:00
Vilobh Meshram a5e7d57c1e Typo in Locking doc
Fixes typo in the developer doc for lock.

Change-Id: I5bc682d162e9a5c648168ff39e8fffe78b99e04f
2015-04-28 11:28:53 -07:00
Joshua Harlow a890d34265 Add exception docs to developer docs
Change-Id: Ie58db0660f5dad1b5bf14edf4512226648829980
2015-04-15 08:56:47 -07:00
Joshua Harlow 6569329e9b Beef up the docstrings on the various drivers
Change-Id: I1ca05629cc5f28c1248767bb18384a29801bece2
2015-04-08 12:35:07 -07:00
Joshua Harlow 5dc82132ed Denote that 2.6 testing is still happening
Since we run 2.6 CI tests (and the setup.cfg
classifiers so we support 2.6); we might as well mention
this in the docs.

Change-Id: I9b59378f717f7df280c429a53d0a83c129327481
2015-04-01 14:31:29 -07:00
Joshua Harlow bb00a5beee Add a driver feature compatibility matrix
Change-Id: I8c5e58fbfa6da53a334ca43829d02d765f85a3fd
2015-03-09 18:34:00 -07:00
Julien Danjou 405dfec672 Add a file based driver
Change-Id: Ie299a8a27045526c27907cdf97b8a240325d908c
2015-01-16 10:31:13 +01:00
Chris Dent 886aa622b0 Update sentinel support to allow multiple sentinel hosts
Multiple sentinel hosts are now allowed by listing the additional
hosts as multiple 'sentinel_fallback' parameters. These are combined
with the first sentinel host to create a list that is provided to
Sentinel class constructor.

This provides safety in the case when the first (or any other) sentinel
host goes down.

Tests are run twice: once with fallbacks, once without.

Change-Id: Ie0a134f02617e8552a87c8bcd49996daaba07f4e
2015-01-12 14:18:53 +00:00
Chris Dent a2216e383e Add support for an optional redis-sentinel
If sentinel support is desired then the connection URI should take
the form:

   redis://<sentinel host>:<sentinel port>?sentinel=<master name>

This will be parsed to connect to a Sentinel server to discover a
master. The response will be used to make a connection to the
current master.

The setup-sentinel-env.sh script will exit with an error if either
the redis-server or sentinel cannot be found. When the script is
done it will clean up the mess it makes.

Change-Id: I205ed88c72587d9f5f916d344c44192bee2aa41a
2015-01-07 12:16:59 +01:00
Jenkins 7debf06f04 Merge "Add create/join/leave group support in IPC driver" 2014-12-26 10:45:50 +00:00
Joshua Harlow 4c2996525f A few more documentation tweaks
- Capitalize tooz where appropriate.
- Adjust contents to be an actual header.
- Add a footnote that states tooz can be generally useful
  for anyone (and not just openstack projects); this was taken
  from taskflows similar footnote.

Change-Id: I9b4e0304483e2c7155f4d5107fd2967ed2c8d1a4
2014-12-18 13:19:30 -08:00