The purpose of this patch is to add an endpoint directly in RPC
dispatcher, so this endpoint will always be available, in a cross
project manner, without the need for projects to manage it by themself.
This endpoint stay disabled by default, so this change is harmless
without a specific configuration option.
To enable this ping endpoint, an operator will just have to add a new
parameter in the [DEFAULT] section, alongside with rpc_response_timeout
[DEFAULT]
rpc_ping_enabled=true # default is false
The purpose of this new endpoint is to help operators do a RPC call (a
ping) toward a specific RPC callback (e.g. a nova-compute, or a
neutron-agent).
This is helping a lot for monitoring agents (for example, if agents are
deployed in a kubernetes pod).
The endpoint is named oslo_rpc_server_ping.
Change-Id: I51cf67e060f240e6eb82260e70a057fe599f9063
Signed-off-by: Arnaud Morin <arnaud.morin@corp.ovh.com>
Users of the oslo.messaging RPC communications service must use
the rabbit ("rabbit://...") or AMQP 1.0 ("amqp://...") drivers.
Change-Id: If3474142f1fe99d41d7b4466061ed0e23ca38549
Closes-Bug: 1789259
It is recommended that all users of the Pika driver transition to
using the Rabbit driver instead. Typically this is done by changing
the prefix of the transport_url configuration option from "pika://..."
to "rabbit://...". There are no changes required to the RabbitMQ
server configuration.
Change-Id: I52ea5ccb7e7c247abd95e2d8d50dac4c4ad11246
Closes-Bug: #1744741
Currently Kafka driver for an oslo.messaging uses kafka-python==0.9.5
and mostly broken. This package version supports only low level Kafka
producer and consumer API which are marked as deprecated now [1]. Using
of these interfaces bring a big concern to the message processing,
because current KafkaConsumer has not any consuming coordination. This
fact causes a message duplication for the several consumers of one
topic. This behavior is specific to Ceilometer and services which read
and process notifications from other services.
New version of kafka-python allows to use async thread safe message
producers and coordinated consumers [1].
[1] http://kafka-python.readthedocs.io/en/master/changelog.html#feb-15-2016
The driver is currently experimental, python-kafka<1.0.0 API have major
issue described above that can't make the oslo.messaging driver works,
so we prefer having a working driver with a non-synced dependencies, that the
reverse.
Co-Authored-By: Mehdi Abaakouk <sileht@redhat.com>
Change-Id: I29862ed7bf56b9d8878fa8e9fb1cbd9d643908a4
This patch makes message serialization in the pika driver
configurable.
Change-Id: Ib86cb91110aa89663a54afdd9932d60f6f45f585
Depends-On: Idb12666255a990dfc8f8ff6b43e941b3481b9c1c
Each time driver couldn't update from Redis
warning message is being logged and work continues
with cached targets state and on existing connections.
Change-Id: I459532f1f60fab1d4c9926e02cdc3c3c6a162047
Closes-Bug: #1610958
ZeroMQ driver options are current stored into the DEFAULT group.
This change makes the zmq configuration clearer by putting its
options into oslo_messaging_zmq group.
Change-Id: Ia00fda005b1664750d2646f8c82ebdf295b156fb
Closes-bug: #1417040
Co-Authored-By: Oleksii Zamiatin <ozamiatin@mirantis.com>
This patch simply moves the existing AMQP 1.0 driver files into a
directory layout that is consistent with the other drivers. The
driver unit tests are also moved to the proper test directory. There
are no changes in driver functionality or API.
Change-Id: I83a5d5433be8c3b317597100af69192ec5be81f1
Closes-Bug: #1579823
new - create new connection each times
single - use single connection for whole transport
read_write - use two connections for whole transport
(one for listening and one for sending)
Change-Id: I464c83beb498453b6df2237e7b8022d47ca3fa14
Now we have situation when openstack projects like Mistral needs
extra oslo.messaging functionality.
But it is too complicated now to to implement something new and
integrate it with current code because there is a little bit mess.
1) Executor should be responsible for how to run jobs
(but now also has code with server logic)
2) Dispatcher should be responsible for routing message to the
target endpoint for processing (but it also has serialisation, sending replies,
executing some executor's callbacks etc)
3) Server should do all server specific logic, we need to have different
implementation of servers for RPC and notification, not different implementations
of dispatchers
This patch fixes 1-st point
Change-Id: Ib6408f408889bb7b7056722be636a5547b1a780d
The configuration options for matchmaker_redis appeared in two groups:
[DEFAULT] and [match_redis]. This removes the configs from the incorrect
group [DEFAULT].
Change-Id: I17d8d84592de3435ddbcba42361fc1258dd62b21
Closes-Bug: #1543305
The group is still DEFAULT at the moment, but the correct group is
oslo_messaging_notifications. Sample configurations files generated with
the oslo.config tools are invalid due to this incorrect group.
Change-Id: Idc6c52f4b15dfff6e8d6b4a2ef9f5598bb3468cd
Closes-Bug: #1531219
Back in liberty we marked this driver as deprecated. This patch removes
it from the tree. The patch also removes tests, options and other
references in the documentation. Note that one script is being kept
because it's required by the amqp driver.
Depends-On: If4b1773334e424d1f4a4e112bd1f10aca62682a9
Change-Id: I4a9cba314c4a2f24307504fa7b5427424268b114
This change introduces to possibility for the consumer application
to change the default oslo.messaging config options.
Same API interface as oslo.db have been taken.
The first option is executor_thread_pool_size.
This option was called rpc_thread_pool_size, but because it's used by
notification and rpc 'executor_thread_pool_size' looks a better name.
Changing executor_thread_pool_size default will be useful for ceilometer
the default of 64 is really to small to consume many notifications at
once for batching them. When is clearly sufficient for rpc stuffs.
Change-Id: Iea0d7a72e38d27c600403c815258aa5eee0d0c8c
This patch replaces the old outdated matchmakers and replace it into the
new ones.
Call/Cast test_specific_server() functional tests passes now.
Change-Id: I8635396110d30d26812f39b242fbbabd1a0feaaa
The context manager in the executor fit only for the blocking executor.
Even the dispatcher needs to run code before and after the application
callback, eventlet and future executors have to run the pre/post code
into the main thread and can run the callback into an other thread,
and that force them to run __enter__ and __exit__ manually and deal
the exception path.
This change adds a helper object instead of the context manager.
It is designed to be explicit on what must be executed
before and after the callback and what can be done in a thread or not.
All the executor code is now in the impl_pooledexecutor.py
and use the futures "PoolExecutor" API.
This use futurist to provide a eventlet and aioeventlet futures friendly
object.
Change-Id: I8cd7640f36beeda47560e3c82671bad3530e38d1
To avoid creating a new ZMQ connection for every message sent
to a remote broker, implement pooling and re-use of ZmqClient
objects and associated ZMQ context.
A pool is created for each remote endpoint (keyed by address);
the size of each pool is configured using rpc_conn_pool_size.
All outbound message client connections are pooled.
Closes-Bug: 1384113
Change-Id: Ia55d5c310a56e51df5e2f5d39e561a4da3fe4d83
All drivers options are current stored into the DEFAULT group.
This change makes the configuration clearer by putting driver options
into a group named oslo_messaging_<driver>.
Closes-bug: #1417040
Change-Id: I96a9682afe7eb0caf1fbf47bbb0291833aec245b
Move the public API out of oslo.messaging to oslo_messaging. Retain
the ability to import from the old namespace package for backwards
compatibility for this release cycle.
bp/drop-namespace-packages
Co-authored-by: Mehdi Abaakouk <mehdi.abaakouk@enovance.com>
Change-Id: Ia562010c152a214f1c0fed767c82022c7c2c52e7
Arnaud Morin