Move the public API out of oslo.messaging to oslo_messaging. Retain
the ability to import from the old namespace package for backwards
compatibility for this release cycle.
bp/drop-namespace-packages
Co-authored-by: Mehdi Abaakouk <mehdi.abaakouk@enovance.com>
Change-Id: Ia562010c152a214f1c0fed767c82022c7c2c52e7
The concurrent.futures module is one of the ways
that async activities can be done in the future,
so we should try to work on getting to that future
by using more futures. To enable this (as well as
to enable getting off eventlet), add a thread pool
based executor which will process incoming messages
using the pool.
Also begins adding according docs as well for the
different types of executors that are available.
Change-Id: I1482fd70abbf69f4e2994597c5e95d91fecb815e
Basic functional and unit tests for zmq driver.
Note as the zmq driver is directly dependent on eventlet, this
change also updates the notify logger tests to remove the
direct dependency on threading which was being monkey patched,
causing test failures.
As the zmq driver has a direct dependency on eventlet, tests are
skipped under py3.
Co-Authored-By: Kapil Thangavelu <kapil.thangavelu@canonical.com>
Co-Authored-By: Edward Hope-Morley <edward.hope-morley@canonical.com>
Change-Id: I93b8b2e92d0f2a353d3357a5e61f6d472ec84944
Partial-bug: #1302941
ZeroMQ Context is a singleton and thus is created only once. This leads
to problems when there is more than one process working with it.
For example, while Neutron server starts, it firstly loads core
plugin and service plugins, which start message handling server,
and only then forks to create api-workers. As a result, all child
processes get the same copy of the context.
Creating new Context for each socket will prevent such situations
from happening and will guarantee that each process works with its
own Context.
Change-Id: I56912e39b119c20f6f23311fc2c7c4b9e9e480d0
Closes-Bug: #1364814
We can now set the pool name of a notification listener
to create multiple groups/pools of listeners consuming notifications
and that each group/pool only receives one copy of each notification.
The AMQP implementation of that is to set queue_name with the pool name.
Implements blueprint notification-listener-pools
Closes-bug: #1356226
Change-Id: I8dc0549f5550f684a261c78c58737b798fcdd656
The ZmqListener constructor only accepts a driver as an argument.
Drop surplus 'None' in listen_for_notifications method.
Change-Id: I6aec91122bb9406e387728eb2a2380f2c2094595
Closes-Bug: #1368154
zmq: Send message to correct endpoint for directed messages
If set the zmq driver needs to use the topic.server
attribute to ensure that messages directed to a
specific endpoint are sent to the correct server rather
than being randomly sent to endpoints advertising the
required topic.
Change-Id: If63235640e0b8e6ec8690a4edaefa1b303705ac6
Closes-bug: #1301723
oslo-config-generator will use the sample_default for the sample
config file rather than the hostname of the system it happens to
be running on.
Change-Id: I9c7bf24cfe5c66155d00d9a1b390894d1ccb6bd8
Instead of the lambda method _ the method should be imported
from oslo.messaging.openstack.common.gettextutils.
Change-Id: I373337cc1227b0d7b7bf93603b53a64de83721a9
Passing mutable objects as default args is a known Python pitfall.
We'd better avoid this.
Change-Id: I67cc0774a65886ef9fce0b72e52157b622248a85
Closes-Bug: #1327473
There are files containing string format arguments inside
logging messages. Using logging function parameters should
be preferred.
Change-Id: I4a7ce6916aefb2f2d445f0ebd977c824b1c51e24
Partial-Bug: #1321274
allowed_remote_exmods which is passed to _multi_send method in
impl_zmq.py is not passed further to _call, _cast, etc.
Due to that, all remote exceptions, that are supposed to be
expected, are wrapped into messaging.RemoteError exception with
all details of the remote exception.
To fix this allowed_remote_exmods needs to be passed as a
parameter in a called method().
Change-Id: If1dc2f85eab6f542bc21961b9d5c0bd0ab64add2
Closes-Bug: 1330460
When a rpc client try to make a RPC call and the server is unreachable
The rpc call hang until the server come back.
In most case this is the desired behavior.
But sometimes, we can prefer that the library raise an exception after a
certain number of retries.
For example in ceilometer, when publishing a
storage.objects.incoming.bytes sample from the Swift middleware to an
AMQP topic, you might not want to block the Swift client if the AMQP broker
is unavailable - instead, you might have a queueing policy whereby
if a single reconection attempt fails we queue the sample in memory and
try again when another sample is to be published.
This patch is the oslo.messaging part that allow this.
Closes bug #1282639
Co-Authored-By: Ala Rezmerita <ala.rezmerita@cloudwatt.com>
Change-Id: I32086d0abf141c368343bf225d4b021da496c020
For asynchronous programming, a timeout parameter is required on the listener
to allow to stop it at exit. poll() returns None on timeout.
It plan to use it in my new asyncio (Trollius) executor:
https://review.openstack.org/#/c/70983/
See also the related blueprint for the rationale:
https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio
Change-Id: I918ae3c267743a0eaed1d6a210c79fb4a0eb8733
According to the OpenStack translation policy available at
https://wiki.openstack.org/wiki/LoggingStandards debug messages
should not be translated. Like mentioned in several changes in
Nova by garyk this is to help prioritize log translation.
Change-Id: I4af4a45a56b1364a2f5196b75cff299d607ab393
Partial-Bug: #1317950
The blockless fanout doesn't take effect and it
only processes the FIRST one by returning after the first loop.
Change-Id: Iad071ab455e36be7b26637b286ca9f83fea4a019
Closes-Bug: #1300539
This patch removes some common helper method/class.
Because they are not used anymore by current amqpdriver.
Change-Id: I183750e158e05cf1d0b5e37725676d4882e0c043
In oslo-incubator, we default to not serializing messages using a
message envelope:
def _cast(.., envelope=False, ...)
except using the v2 notifyer which explicitly passes envelope=True.
However, in oslo.messaging, we accidentally switched to using a
message envelope:
def _send(self, ..., envelope=True):
This was not intentional and may be disruptive to users.
This issue was identified because the envelope=True path just doesn't
work! We don't have any zmq tests in oslo.messaging, but even in
oslo-incubator the code path isn't tested and has been broken since
I73aad7697cf83ad4aabb3c2058b7cc4f53f783c2
Partial-Bug: #1301132
Change-Id: I6aa2e2ac3cebf443719d64ebcd9eaadc0a9328b8
In Icehouse b3 and rc1, when we use impl_zmq as
rpc_backend in nova, nova cannot send any messages
throught impl_zmq to oslo-messaging-zmq-receiver.
The reason is we send two params to serialize_msg
method but it only can recive one.To fix this
problem, I removed the second param which not
used when we call the method serialize_msg.
Change-Id: I2e7546147affd9493f425fd879169932ef206d48
Closes-bug: #1301132
In commit d8d2ad9 we added support for notification listener endpoint
methods to return REQUEUE, but if a driver does not support this we
raise NotImplementedError when the application attempts to requeue
a message.
This requeuing behaviour might only be used by an application in
unusual, exceptional circumstances and catch users by surprise.
Instead, let's require the application to assert that it needs this
feature in advance and raise NotImplementError at that point if the
driver doesn't support it.
Change-Id: Id0bb0e57d2dcc1ec7d752e98c9b1e8e48d99f35c
This patch allow to requeue the notification received by the
notification listener.
Partial implements blueprint notification-subscriber-server
Change-Id: I49c4ba91224c280e479edb19289ccb337a2ab843
Follow oslo.config style guide for help strings better to create
consistent help strings:
* Capitalize first word of each help string
* Finish help strings with "."
Change-Id: Ia08fa09593661e6e5b834d98bbd92689c2674075
This patch allow the fake driver to comsume multiple topics
with one listener.
Partial implements blueprint notification-subscriber-server
Change-Id: Ib52dc181e10b487854fbb398eda9f758232a1251
In Python 3, some data structures' attribute is different in Python 2.
See http://pythonhosted.org/six/#object-model-compatibility
This is change mapping:
six Python 2 Python 3
six.next(it) it.next() next(it)
six.iterkeys(dict) dict.iterkeys() dict.keys()
six.itervalues(dict) dict.itervalues() dict.values()
Implements: blueprint make-python3-compatible
Change-Id: Ida48f39ff230860feee7305b93b134c625a21663
Doug Hellmann