Blueprint remove-namespace-packages
Depends-on: I2eeef93ee2e61a721c69f62add819f93f62f077d
for openstack/ceilometer
Depends-on: I26390dd908769be5f1a5b70be22d3b98e3a45563
for openstack/ceilometermiddleware
Depends-on: Ifa8baab33cdb3e606cf175a8c29c3a4ef6c44480
for openstack/glance
Depends-on: I029c3260051aa48cfaae428c096c1ac7b43b2bd2
for openstack/ceilometermiddleware
Change-Id: I8c5595bbafa82db33f62fa47d214f5cb756a2639
Some of the heat unit tests import code from a private part of
oslo.messaging. With the namespace package change, that private part was
no longer being exposed. This change introduces the needed module so we
can release oslo.messaging with the namespace changes without breaking
heat, to give the heat team time to update their tests.
Partial-Bug: #1410196
Change-Id: Ib3ea106646f75b17665877b08ccbd382c5698714
Move the public API out of oslo.messaging to oslo_messaging. Retain
the ability to import from the old namespace package for backwards
compatibility for this release cycle.
bp/drop-namespace-packages
Co-authored-by: Mehdi Abaakouk <mehdi.abaakouk@enovance.com>
Change-Id: Ia562010c152a214f1c0fed767c82022c7c2c52e7
When the duration of the timeout timer used in the rabbit driver.is
None and we want that the timer return a maximum of N secs it return None
(infinite) instead of N.
This change fixes that.
Closes-bug: #1408370
Change-Id: I7f4cb3075f776c63aa7dc497173677f92b68c16d
oslo.utils.strutils.mask_password take too much time on
big payload (nova-conductor can receive payload ~ 66k in
largeops jobs for example).
This change removes this logging until we make mask_password
more efficient or we have a smarted the oslo.messaging logging to
not log everything.
Change-Id: Ib1f1d70c5cb820e8ff2de10e6064037808ea1f3a
Closes-bug: #1408362
* add i18n support to some exception messages
* remove the return value of DecayingTimer.start()
Relates to review: I898a236a384b3466147026abc7a1ee21801e8ca1
Change-Id: I7adf5478732f1f46db1009b059b66ff8af6ecdc3
The concurrent.futures module is one of the ways
that async activities can be done in the future,
so we should try to work on getting to that future
by using more futures. To enable this (as well as
to enable getting off eventlet), add a thread pool
based executor which will process incoming messages
using the pool.
Also begins adding according docs as well for the
different types of executors that are available.
Change-Id: I1482fd70abbf69f4e2994597c5e95d91fecb815e
Sanitizes password fields found in lists of dicts for messages
before logging.
This change uses oslo.utils.strutils.mask_password to do it.
Change-Id: I7cd1e53e2ced7ebf9c5942b7a0dbbeb991acab4d
Closes-Bug: #1268459
Basic functional and unit tests for zmq driver.
Note as the zmq driver is directly dependent on eventlet, this
change also updates the notify logger tests to remove the
direct dependency on threading which was being monkey patched,
causing test failures.
As the zmq driver has a direct dependency on eventlet, tests are
skipped under py3.
Co-Authored-By: Kapil Thangavelu <kapil.thangavelu@canonical.com>
Co-Authored-By: Edward Hope-Morley <edward.hope-morley@canonical.com>
Change-Id: I93b8b2e92d0f2a353d3357a5e61f6d472ec84944
Partial-bug: #1302941
It's up to the driver to set a suitable timeout for polling the broker,
this one can be different that the one requested by the driver
caller as long as the caller timeout is respected.
This change also adds a new driver listener API, to be able
to stop it cleanly, specially in case of timeout=None.
Closes bug: #1400268
Closes bug: #1399257
Change-Id: I674c0def1efb420c293897d49683593a0b10e291
The qpid driver must always honor the timeout passed the iterconsume
method, this change fixes that.
Related bug: #1400268
Related bug: #1399257
Change-Id: I8f267fc8b5a7abc852f0caf84d1e7c2c342ba951
The iterconsume always set the timeout of kombu to 1 second
even the requested timeout more precise or < 1 second.
This change fixes that.
Related bug: #1400268
Related bug: #1399257
Change-Id: I157dab80cdb4afcf9a5f26fa900f96f0696db502
This change reintroduces the fake_rabbit only for backward compatibility,
but mark it as deprecated.
Now, to use the kombu in-memory driver (that is not thread safe) we must
set the transport_url config option to 'kombu+memory:////" or the
rpc_backend to kombu+memory.
Or we can use the fake driver of oslo.messaging by setting the
transport_url to 'fake:///' or the rpc_backend to 'fake'
This is effectively reverting commit bcb3b23b8f.
Closes-bug: #1399085
Change-Id: I7b6fb3811fc6f695f75ecd350e04e69afd26c428
Add the missing 's' for the message format and
just print the number queues instead of print a list
full of '<object object at 0xXXXXXXXXXX>'.
Change-Id: Idaab4057bc6a41523a1944ae0c8e15e5e885b390
The iterconsume method of the rabbit driver must
honor timeout parameter when reconnection to the broker
occurs.
Change-Id: I666d818449750c6bae9dde02f519842687a8e4fa
This removes a TODO and also fixes a issue due to the
global state of oslo.config.cfg.CONF.
Closes bug: #1397339
Change-Id: Ib366f35678f899fda93821e6f07897baf8f631b4
Each driver instance must use it's own connection pool.
This removes the last global state of qpid and rabbitmq driver
Make the relation between classes more simple.
The previous behavior was not very safe, as explained in the bug report.
And also, this is a first step to replace this custom connection pool
handling by the kombu one.
Closes bug: #1397925
Partial bug: #1397339
Change-Id: Iecd2b39c76417d9ac081d46810f72eb6e38edfda
Currently it appears the timeout provided to the wait()
method will be reused X number of times (where X may be
indeterminate depending on reconnections, loss of messages
and so-on) so instead of reusing it which can potentially
result in a infinite number of calls have a new object be
used that will cause the subsequent timeouts used elsewhere
in the wait function to actually decay correctly.
Closes-bug: #1379394
Change-Id: I12c4ea1eef6b857d12246db0483adaf7c87e740c
This change warns the library consumer when the process if forked and
we can't be sure that the library work as expected.
This also add some documentation about forking oslo.messaging Transport
object.
Change-Id: I2938421775aa72866adac198d70214856d45e165
Related-bug: #1330199
Retry Queue creation to workaround race condition
that may happen when both the client and broker race over
exchange creation and deletion respectively which happen only
when the Queue/Exchange were created with auto-delete flag.
Queues/Exchange declared with auto-delete instruct the Broker to
delete the Queue when the last Consumer disconnect from it, and
the Exchange when the last Queue is deleted from this Exchange.
Now in a RabbitMQ cluster setup, if the cluster node that we are
connected to go down, 2 things will happen:
1. From RabbitMQ side, the Queues w/ auto-delete will be deleted
from the other cluster nodes and then the Exchanges that the
Queues are bind to if they were also created w/ auto-delete.
2. From client side, client will reconnect to another cluster
node and call queue.declare() which create Exchanges then
Queues then Binding in that order.
Now in a happy path the queues/exchanges will be deleted from the
broker before client start re-creating them again, but it also
possible that the client first start by creating queues/exchange
as part of the queue.declare() call, which are no-op operations
b/c they alreay existed, but before it could bind Queue to
Exchange, RabbitMQ nodes just received the 'signal' that the
queue doesn't have any consumer so it should be delete, and the
same with exchanges, which will lead to binding fail with
NotFound error.
Illustration of the time line from Client and RabbitMQ cluster
respectively when the race condition happen:
e-declare(E) q-declare(Q) q-bind(Q, E)
-----+------------------+----------------+----------->
e-delete(E)
------------------------------+---------------------->
Change-Id: Ideb73af6f246a8282780cdb204d675d5d4555bf0
Closes-Bug: #1318721
This patch attempts to deal with applications that have forked the
process after connecting to the broker. First, the creation of the
connection is delayed until the application attempts to perform a
messaging operation. Second, each time the application performs a
messaging operation the current process id is checked against the id
of the process that created the connection. If the process ids do not
match, the application has called os.fork(). The new child process
discards the existing connection and creates a new one.
Change-Id: I5455cb0f8d380d6b65f1268b34a91355cbb14aa2
Closes-Bug: #1392868
The PROTOCOL_SSLv3 should not be used, as it can be exploited with
a protocol downgrade attack. Also, its support has been removed in
Debian, so it simply doesn't work at all now in Sid.
This patch removes PROTOCOL_SSLv3 from one of the possible protocols
used by oslo.messaging.
Closes-Bug: #1395095
Change-Id: I2c1977c3bfc1923bcb03744e909f2e70c7fdb14c
The rabbit driver have a custom code to reconnect to a broker,
to change the broker in HA configuration, to retry a to send
a message, to handle the interval between reconnection.
But all of that exists in kombu, so just use it.
Using the kombu Connection object with the url make also the rabbit
driver more generic.
Futher patches can rename rabbit* oslo.config options to a more generic
name and add a new driver entry_point 'kombu' to allow to use this driver
with any borker supported by kombu.
Change-Id: Id6b89d5448126ca652b46fe6ce5a9b3ed5839795
ZeroMQ Context is a singleton and thus is created only once. This leads
to problems when there is more than one process working with it.
For example, while Neutron server starts, it firstly loads core
plugin and service plugins, which start message handling server,
and only then forks to create api-workers. As a result, all child
processes get the same copy of the context.
Creating new Context for each socket will prevent such situations
from happening and will guarantee that each process works with its
own Context.
Change-Id: I56912e39b119c20f6f23311fc2c7c4b9e9e480d0
Closes-Bug: #1364814
The ConnectionContext depends on the ConnectionPool.
A ConnectionPool already known the connection classes,
the configuration object and the url needed to create a new connection.
But we pass again thoses informations when we create a ConnectionContext.
This is unuseful, we can reuse thoses in the connection pool even we
want a not pooled connection.
This change removes the unuseful ConnectionContext parameters,
this also ensures that connection created with or without the pool
are created in the same ways and only at one place (the create method
of the connection pool).
Change-Id: I4bd43d202fa2774ad5dcb0f8dd05e58ba60c6009