Commit Graph

650 Commits

Author SHA1 Message Date
Tony Breeds e13f99b619 Retire Packaging Deb project repos
This commit is part of a series to retire the Packaging Deb
project. Step 2 is to remove all content from the project
repos, replacing it with a README notification where to find
ongoing work, and how to recover the repo if needed at some
future point (as in
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project).

Change-Id: I00c02762c4c6dfa3a0cfa1dc2262f2ed98256ace
2017-09-12 16:12:51 -06:00
ChangBo Guo(gcb) c2167820c8 Update URLs in documents according to document migration
Change-Id: I579b62a46f484e0129f88b12d1e16018cc9c7684
2017-07-13 12:09:43 +08:00
Akihiro Motoki ee6f82d9d5 doc: use list-table for driver support tables
list-table is easier to be maintained.

In addition, we avoid too long lines.
We no longer have too long lines which doc8 complains,
so doc8 ignore_path can be dropped.

Change-Id: I574391295288ef9e33744aa608c2f2930a6e46f8
2017-07-10 12:44:29 +00:00
Akihiro Motoki 2ca15fedbe rearrange existing documentation to fit the new standard layout
Change-Id: If03b33fc1393c3a04d7d155b31e8377dc9e09360
2017-07-10 12:41:52 +00:00
Akihiro Motoki 38bcf7bbdb Switch from oslosphinx to openstackdocstheme
Change-Id: Id5e2952676e89f768e13b4ffc2f238eb012ec324
2017-07-07 02:54:25 +00:00
Akihiro Motoki cf12457896 Turn on warning-is-error in doc build
- Update sphinx so that warning-is-error option is available
- Fix sphinx warnings
- Install python modules required for automodule into docs env
- Fix docstring warning in tooz/coordination.py
- pbr warnerrors is no longer needed in favor of sphinx warning-is-error

Change-Id: I5d54c452522b6ee00884d7d17105af22b4c95189
2017-07-07 02:52:30 +00:00
Jenkins f947d1f377 Merge "Simplify env list and test running" 2017-07-05 12:50:22 +00:00
Jenkins 03dbf70f70 Merge "Make sure Lock.heartbeat() returns True/False" 2017-07-05 12:50:16 +00:00
Jenkins d88f0f4c84 Merge "etcd3: replace custom lock code by more recent etcd3 lock code" 2017-07-05 12:36:22 +00:00
Jenkins e6274bdd5b Merge "test: leverage existing helper method in test_partitioner" 2017-06-29 11:35:07 +00:00
Julien Danjou 7987f4455a Make sure Lock.heartbeat() returns True/False
- add a test to be sure that no lock refresh is made on non-acquired locks
- add thread safety for etcd3 drivers on heartbeat() (like Redis)

Change-Id: I350ea10163d59a06712c22c4c5be4dfcb6885bf8
2017-06-28 10:56:13 +02:00
Jenkins fd08ee972c Merge "coordination: remove double serialization of capabilities" 2017-06-27 10:37:43 +00:00
Jenkins 635f822da6 Merge "tests: fix missing .get() on some group operations" 2017-06-26 19:11:33 +00:00
Julien Danjou 965fb4dc4a etcd3: skip ProcessPool based test
The other test is already skipped, but this one should also be skipped for the
same sad reason.

Change-Id: I101ef55dd3e83c0a30cc9dba12b224743071db34
2017-06-26 14:28:07 +02:00
Jenkins 1c6b3506fd Merge "Mutualize executor code in a mixin class" 2017-06-24 06:40:36 +00:00
Julien Danjou c94b2a39b3 etcd3: replace custom lock code by more recent etcd3 lock code
The code in python-etcd3 for locks has been adjusted to reflect how tooz wants
his lock to work. That's amazing because now we can use them directly and not
implement our own locking mechanism!

Change-Id: Ia5741bae83cddd90f9bb78fba9173dca8509e6d9
2017-06-23 17:41:26 +02:00
Jenkins 743603a07c Merge "pgsql: fix self._conn.close() being called without connection" 2017-06-23 00:10:05 +00:00
Julien Danjou 90a56b0ab6 pgsql: fix self._conn.close() being called without connection
If an error occurs on connection establishment, self._conn is None so it can't
be closed.

Change-Id: If6fce24cb27aa7090b5165ca4a895c846c40d2dd
2017-06-22 12:48:06 +02:00
Julien Danjou 361406e595 test: leverage existing helper method in test_partitioner
Change-Id: Ib90ee990618cec712d7082e3c0498d92fe31946b
2017-06-21 17:43:26 +02:00
Julien Danjou 15f6de65e0 coordination: remove double serialization of capabilities
The capabilities was meant to be a byte string only, but since now all drivers
have been enhanced to serialize the data in a way or another, it accepts
anything. There's no need to pass a byte string anymore.

Change-Id: Ib732dccd18d67edd9e39e990b8ca3a8da7ab5663
2017-06-21 17:43:26 +02:00
Julien Danjou a0590a7f7b tests: fix missing .get() on some group operations
Change-Id: Iaf009d7640865a107f36008e59c4ed5c6ae1a63f
2017-06-21 17:43:26 +02:00
Julien Danjou 13e23e3087 Mutualize executor code in a mixin class
Change-Id: I364b0ca5a6361a818a084bbcb83c9e7855cb3583
2017-06-21 17:43:26 +02:00
Julien Danjou cc09f3f6ec coordination: fix reversed fiels for __repr__ for events
Change-Id: I6bcadea3ee998db82853228818668ad98951dcab
2017-06-21 17:31:27 +02:00
Julien Danjou 413527c181 Fix docstring for group and member id
What is really expected is bytes with ascii inside.

Change-Id: I6f2c4c6841558a73a129f4e1a39e896957c35911
2017-06-20 16:34:45 +02:00
Julien Danjou ee66b6e4e4 {my,pg}sql: close connections when out of retry
The connection is supposedly closed when blocking is False and the lock is not
acquired. Or when an exception happens in the locking code.

But if blocking is e.g. 5 seconds, then RetryError is raised but without
closing any connection at the end. This fixes that.

Change-Id: I7b40e466fe5fc01ee3f8c012f765aafca9ce716a
2017-06-16 11:40:42 +00:00
Jenkins 5625106703 Merge "Disable test_get_lock_serial_locking_two_lock_process for etcd3" 2017-06-16 09:45:41 +00:00
Julien Danjou ebd405056f Disable test_get_lock_serial_locking_two_lock_process for etcd3
gRPC (used by python-etcd3) is not able to work correctly with fork. One can
only use it in the child or master process, but not both. Therefore this test
will fail or hang forever.

Change-Id: I32453d2d968faefeffcff1f5b70b46daec36c300
2017-06-12 17:00:35 +02:00
Jenkins 42d7e2b27b Merge "Factorize tox envlist for better readability" 2017-06-12 11:27:36 +00:00
Julien Danjou 6d1e1b55cb Simplify env list and test running
This should make it easy to run any Python version test, e.g. Python 3.6,
without any further change.

Change-Id: I91a47b736371549e50b7c7fd60e46e0f87c357ae
2017-06-09 18:09:31 +02:00
Julien Danjou 0fa14dd60e Factorize tox envlist for better readability
Also add missing etcd.

Change-Id: I99dc4d6d2a62459652904dceb1d5047d1eb20077
2017-06-08 09:27:38 +02:00
Julien Danjou 18fdfa9de7 consul: remove unused executor
Change-Id: I98d7790fdc74a86a067fdbc0665fbd1abd7de77c
2017-06-06 18:31:02 +02:00
Davanum Srinivas 6ec73a0bed Separate etcd3gw driver that uses the etcd3 grpc gateway
Using grpc to directly access etcd3 does not work well with
eventlet based services. So we need to use the grpc HTTP
gateway API (/v3alpha).
https://coreos.com/etcd/docs/latest/dev-guide/api_grpc_gateway.html

For this we use the etcd3gw package:
https://pypi.python.org/pypi/etcd3gw

The structure of the code is very similar to the one we
already have that uses the etcd3 package.

Change-Id: I97bd7ffb05a7e40cb08c9b9d62cc45236ad292aa
2017-05-22 14:33:44 +02:00
Julien Danjou 24101cf264 etcd3: use discard() rather than remove()
Change-Id: Icdf2ecc7e3500ef6c2efdc1f61d2431b13831141
2017-05-22 14:27:09 +02:00
Julien Danjou ae1e42274b etcd: fix blocking argument
The value should be passed, not only a boolean. No need for conversion.

Change-Id: I73a8a1b377d9eb7e4a8b6433f0b09832756c7188
2017-05-22 14:25:40 +02:00
Julien Danjou 1a57bbcc24 etcd: fix acquire(blocking=True) on request exception
If requests raises an exception and blocking is True, then there's no watch and
nothing is returned. The problem is that then the reply variable is checked but
it is actually not assigned, so a NameError is raised.

This makes sure False is returned if an error occurs and blocking is True.

Change-Id: I078826a894ee2cb8754259c8db9ea5ae19114910
2017-05-19 10:50:06 +02:00
Julien Danjou 9eb820bfde etcd3: fix test run
pifpaf always exports the URL as being etcd:// which always loads the etcd
driver, and never the etcd3 driver. Therefore, it's not currently tested.

This fixes that by exporting a variable in tox that will allow us to replace
the URL correctly.

Change-Id: Id4f54bb1f729498a39e9bf58da8ee01562c4344b
2017-05-09 21:31:52 -04:00
Julien Danjou cae8421fd9 coordination: factorize common async result futures code
Many drivers leverage concurrent.futures to return result, so let's just
factorize all of that in a same place.

Change-Id: Ibdec36912b553eb4b76c9f99619c171bc41766bb
2017-05-09 21:30:33 -04:00
Jenkins 17620203cb Merge "Fix psycopg2 connection argument" 2017-05-09 19:44:41 +00:00
Julien Danjou c9421c6269 Fix psycopg2 connection argument
commit 6eea97c3ef modified the kwarg from "user"
to "username" by mistake.

Change-Id: Idfe8425d08c065ef19e8952e8fcb205779243a99
Closes-Bug: #1689211
2017-05-08 17:30:04 -04:00
Jenkins 8ebc10ace8 Merge "doc: update heartbeat doc to use start_heart=True" 2017-04-27 20:27:42 +00:00
Jenkins ed15f7cc1c Merge "sql: close connection for lock if not used" 2017-04-27 20:16:25 +00:00
Jenkins 58219aca5a Merge "simplify hashring node lookup" 2017-04-27 15:27:36 +00:00
Julien Danjou ba62597183 doc: update heartbeat doc to use start_heart=True
Change-Id: I5560b1ec1da7922586e5544d55b5e508c70a33fc
2017-04-27 16:40:57 +02:00
gord chung 6e3f5ceb75 sql: close connection for lock if not used
each lock creates it's own connection and only gets released when
garbage collection kicks in because of limitations

change logic so if we don't have lock, we don't have connection.

Change-Id: Ibf6eaf3ff3031424917e9fdb6920bf2fc1808b7c
2017-04-27 14:27:30 +00:00
fuzk 5946fef119 http->https for security
Change-Id: I7eb7d3aff82970913cb7dd618754863ac661a447
2017-04-19 19:10:58 -07:00
Jenkins 4364f7a839 Merge "etcd3: add etcd3 coordination driver" 2017-04-18 01:20:30 +00:00
Jay Pipes 9ba92a88d0 etcd3: add etcd3 coordination driver
This patch adds support to Tooz for taking advantage of the etcd3 gRPC-based
API (instead of the etcd2 HTTP/REST-based API) via the python-etcd3 library.

Change-Id: Ic7c3d9be42a9912fcb09c43e7637270db4011c4a
2017-04-17 11:47:15 -04:00
mengalong 92c83b2b2e Implement heartbeat for FileDriver
As the title said this patch just implement the heartbeat api for
FileDriver.
1. You can delivery the parameter 'timeout' via the connect url like
   this: file:///tmp/test_coord?timeout=10. Otherwise the default timeout
   is 10secs.
2. We can call the function: heartbeat(self) periodic
   to make sure the specific member is alive.
3. When you call get_members(), it will compare the last modify time of
   the file with current time, if the delta_seconds greater than
   the timeout, it will mark the member status is dead and will not return
   the member_id.

Change-Id: I8617e17fc1a74e70d7eb5b6f8e12dc9d9a66c755
Closes-Bug: #1664559
2017-04-11 20:04:48 +08:00
gord chung 1917f5ea27 simplify hashring node lookup
current code uses a nested loop which makes additional _get_node
calls needlessly when replicas > 1. this patch replaces it with a
simpler single loop.

Change-Id: I6bb5753ba1cef489648d7d1a75374cbb63ac5656
2017-04-02 17:45:30 +00:00
Jenkins 9f87cf158d Merge "redis: fix concurrent access on acquire()" 2017-03-29 18:12:00 +00:00