We currently do not support overriding the class being
instantiated in the RPC helper functions, this adds that
support so that projects that define their own classes
that inherit from oslo.messaging can use the helpers.
For example neutron utilizes code from neutron-lib that
has it's own RPCClient implementation that inherits from
oslo.messaging, in order for them to use for example
the get_rpc_client helper they need support to override
the class being returned. The alternative would be to
modify the internal _manual_load variable which seems
counter-productive to extending the API provided to
consumers.
Change-Id: Ie22f2ee47a4ca3f28a71272ee1ffdb88aaeb7758
We already expose functions to handle the instantiation
of classes such as RPCServer and RPCTransport but the
same was never done for RPCClient so the API is
inconsistent in its enforcement.
This adds a get_rpc_client function that should be used
instead of instatiating the RPCClient class directly to
be more consistent.
This also allows to handle more logic inside the function
in the future such as if implementations for an async client
is implemented, as investigation in [1] has shown.
[1] https://review.opendev.org/c/openstack/oslo.messaging/+/858936
Change-Id: Ia4d1f0497b9e2728bde02f4ff05fdc175ddffe66
This commit added support to send rpc metrics to oslo.metrics.
Changes includes:
- Adding client wrapper for oslo.metrics to process metrics information
and send to oslo.metrics socket
- Modify rpc client to send metric when certain rpc events happens
For more information on oslo.metrics
https://opendev.org/openstack/oslo.metrics
Change-Id: Idf8cc0e52ced1f697ac4048655eff4c956fd5c79
The purpose of this patch is to add an endpoint directly in RPC
dispatcher, so this endpoint will always be available, in a cross
project manner, without the need for projects to manage it by themself.
This endpoint stay disabled by default, so this change is harmless
without a specific configuration option.
To enable this ping endpoint, an operator will just have to add a new
parameter in the [DEFAULT] section, alongside with rpc_response_timeout
[DEFAULT]
rpc_ping_enabled=true # default is false
The purpose of this new endpoint is to help operators do a RPC call (a
ping) toward a specific RPC callback (e.g. a nova-compute, or a
neutron-agent).
This is helping a lot for monitoring agents (for example, if agents are
deployed in a kubernetes pod).
The endpoint is named oslo_rpc_server_ping.
Change-Id: I51cf67e060f240e6eb82260e70a057fe599f9063
Signed-off-by: Arnaud Morin <arnaud.morin@corp.ovh.com>
Remove six, the python 2/3 compatibility library. It's not needed
anymore since the repo is python3 only.
Remove a now unneeded hacking test.
Change-Id: I40522c4accb4aaf8115d11fee8b081e2d991cb4d
The blocking executor has been deprecated in Pike and marked for removal
in Rocky, but some user like Mistral asked us to wait before. We decided
to remove this executor for Train or next cycle, now we are in the
Ussuri and after some researchs on usage I think we can go ahead.
This patch drop the deprecation warnings, related unit tests and
set the server with the threading executor is the default executor.
Change-Id: If07bab61ee2b148658b88be98b12f8539f274efe
Closes-Bug: #1715141
Such non-existing methods are used as health probes.
Please see bug for details.
Change-Id: I8f6b4a669ec11721f76cf03abcd7d802c3c11eb5
Closes-Bug: #1857319
Waiting on a threading.Event with eventlet can cause busy looping via
epoll_wait, see related bug for more details.
Change-Id: I007613058a2d21d1712c02fa6d1602b63705c1ab
Related-bug: #1518430
Typically a simple log will not narrow down the
performance, but give us more information about
the service status.
Change-Id: I51c8f2743dd39cccd3d1d021d3c50dc09f70cd97
Closes-Bug: #1847747
With this feature, the server will raise and log a Message Undeliverable
exception. So it is possible to log immediately an error in case the
reply queue does not exist for some reason.
This is part of blueprint transport-options
The blueprint link is [1]
Please follow the link [2] to use and test the feature.
1-
https://blueprints.launchpad.net/oslo.messaging/+spec/transport-options
2- https://github.com/Gsantomaggio/rabbitmq-utils/
tree/master/openstack/mandatory_test
Change-Id: Iac7474c06ef425a2afe5bcd912e51510ba1c8fb3
With this new parameter is possible to pass other parameters
from the client to the drivers.
So it is possible to tune the driver behavior.
For example can be used to send the mandatory flag in RabbitMQ
Note:
- The transport_options parameter is not actually used (yet).
- This part of blueprint transport-options (first part)
Implements: blueprint transport-options
The blueprint link is
https://blueprints.launchpad.net/oslo.messaging/+spec/transport-options
Change-Id: Iff23a9344c2c87259cf36b0d14c0a9fc075a2a72
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.
Change-Id: I9ddb6595fc52e46ed8844e39d2fa71029c90b65c
Closes-Bug: #1674567
The patch for https://review.openstack.org/#/c/436958/ fixed a
threading problem by moving the ack back to the polling
thread. However the RPC server expects to catch any failures of the
ACK and abort the request. This patch adds the ACK error handling
back to the polling thread.
This patch is based heavily off the original work done by Mehdi
Abaakouk (sileht).
Change-Id: I708c3d6676b974d8daac6817c15f596cdf35817b
Closes-Bug: #1695746
This adds an optional call_monitor_timeout parameter to the RPC client,
which if specified, will enable heartbeating of long-running calls by
the server. This enables the user to increase the regular timeout to
a much larger value, allowing calls to take a very long time, but
with heartbeating to indicate that they are still running on the server
side. If the server stops heartbeating, then the call_monitor_timeout
takes over and we fail with the usual MessagingTimeout instead of waiting
for the longer overall timeout to expire.
Change-Id: I60334aaf019f177a984583528b71d00859d31f84
Since RPC and notifications can have different backends, it is
useful to warn users if they use a notification transport in RPC
and vice versa. This patch introduces RPCTransport and
NotificationTransport subclasses of Transport, so it's easier to
add different behavior for them if need be.
Related-Bug: #1680192
Change-Id: Iab60544d69053c8e74c28a2d5c84665be749013f
An endpoint can have an optional 'target' attribute which is used to
filter the callable endpoint by the target.version or .namespace
attributes. Therefore 'target' is reserved and attempting to use an
endpoint that overrides the target attribute (say with a function
call) should fail with a TypeError.
Change-Id: I0bbf9fca0ecbe71efa87c9613ffd32eb718f2c0e
Closes-Bug: #1709131
Using blocking executor is not recommended for application. But it looks
like some use it just because it's the default and are not aware their
should change it despite of documentation and logging message.
Choosing the application threading model is an important step
of an application.
This change deprecates it, in the future we will just make executor
mandatory. This will ensure that application make a choice.
Also this will reduce headache of oslo.messaging developers to
make the driver code working in a sync and async.
And to finish test coverage of blocking executor is 0%...
This rework some tests to remove logging.captureWarnings() that can
catch unwanted warning of other tests. Tests mocks warning instead.
Related-bug: #694728
Change-Id: Ic67164d12e7a9bed76d6e64ca2ced12e3984ff5f
Be specific regarding which characters can be used in the 'username'
and 'password' fields of the URL.
Change-Id: I9a2a801b3dd86fceffb05f4d52c715748c88d0cb
Closes-Bug: #1693967
The get_rpc_transport wraps get_transport to unify the API in
anticipation of comprehensive separation of RPC and Notification
messaging backends.
Related-Bug: 1680192
Change-Id: Ic6af07b98ff43806c2af38a3ba129991f1e0ec86
oslo.messaging supports three types of executors: blocking, eventlet
and threading, so it's good to list all of them in docstring.
Change-Id: Ief14d54b5f029cf3e9c79334e405dca5cf2dc13d
debtcollector's decorator updated_kwarg_default_value is used to indicate
default value will be changed in future version. We introduced parameter
access_policy and add itfor constructor method of RPCDispatcher in
d3a8f280eb. Other services usually call
get_rpc_server method instead of calling constructor method of
RPCDispatcher. get_rpc_server method constructs RPCDispatcher instance
in [1], then FutureWarning will be generated even we set parameter
access_policy explicitly. Need move the decorator updated_kwarg_default_value
to method get_rec_server to avoid the case.
Closes-Bug: #1661572
[1] f1efaf1b6e/oslo_messaging/rpc/server.py (L200).
Change-Id: I3214045856d0bbd05a0000eeafb769e4697cd34c
1.As mentioned in [1], we should avoid using six.iteritems to achieve iterators.
We can use dict.items instead, as it will return iterators in PY3 as well.
And dict.items/keys will more readable. 2.In py2, the performance about
list should be negligible, see the link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
Change-Id: Ia235afc3532f62f265f91ca46d2306c72fc2a2a2
Rework the inline documentation for executors, RPC servers and
clients, notifiers and notification listeners for clarity and flow.
Change-Id: If4f1db853a7fc85340177fd2c9c43a479d72459d
Implements access_policy for dispatcher to restrict endpoint methods.
Implements the following access policies:
* LegacyRPCAccessPolicy
* DefaultRPCAccessPolicy
* ExplicitRPCAccessPolicy
* Implement decorator @rpc.expose for use with the
ExplicitRPCAccessPolicy
* Modify get_rpc_server to allow optional access_policy argument
* Set default access_policy to LegacyRPCAccessPolicy (Nova exposes
_associate_floating_ip in tempest tests). Added debtcollector
notification.
* Add test cases for access_policy=None
* Clarify documentation
Change-Id: I42239e6c8a8be158ddf5c3b1773463b7dc93e881
Closes-Bug: 1194279
Closes-Bug: 1555845
The patch fixes some problems arised after merging the previous
patch which this patch depends on.
Change-Id: If38edec19f9bffc8ac54818b2536243a9edaf7e8
Depends-On: I608c828123b196099933d16801f8ce7d9dd3a079
Closes-Bug: #1584889
This patch removes log_failure argument from the function
serialize_remote_exception and from driver implementations
using it (because it is never used and always defaults to True)
and prevents error logging in this function (because these errors
are already logged by servers while processing incoming messages).
Change-Id: Ic01bb11d6c4f018a17f3219cdbd07ef4d30fa434
Closes-Bug: 1580352
This patch fixes the bug in the RPCClient class when a client's
version_cap is set, but target.version is unset. The code does not
check this case, which results in unhandled exceptions.
Change-Id: I623c14b74b9101bb4ab199dff6609fab44388c4a
Closes-Bug: 1574615
The RPCClient class is fully based on the _CallContext class.
I think it is redundant to have the same attributes in the RPCClient
as in the _CallContext just in order to pass instances with these
attributes to the _CallContext._prepare method (which is not so obious
because it can work with both _CallContext and RPCCLient classes,
though these classes are not in the same class hierarchy).
This patch tries to solve all these issues by including both classes
in a hierarchy and to make a little optimization by eliminating
dummy _CallContext instances creation on each cast/call in RPCClient.
Change-Id: I608c828123b196099933d16801f8ce7d9dd3a079
1) try to catch all possible exceptions during message acknowledging
and dispatching to send ack/nack when we can to do it
2) improve logging in case of exceptions during message acknowledging
and dispatching
Depends-On: I2d230d49e5aff6ab4d84ab8c3d2834f85e3405eb
Change-Id: I41a768c5624fa2212257ce20bf9a67d09de0c4ab
1) Add MessageHandler base interface for on_incoming_callback replacement
2) Move message_handler parameter form Listener's __init__() to start()
3) Remove wait method from listener
Change-Id: Id414446817e3d2ff67b815074d042a9ce637ec24
Current Listener interface has poll() method which return messages
To use it we need have poller thread which is located in MessageHandlerServer
But my investigations of existing driver's code shows that some implemetations have
its own thread inside for processing connection event loop. This event loop received
messages and store in queue object. And then our poller's thread reads this queue
This situation can be improved. we can remove poller's thread, remove queue object
and just call on_message server's callback from connection eventloop thread
This path provide posibility to do this for one of drivers and leave as is other drivers
Change-Id: I3e3d4369d8fdadcecf079d10af58b1e4f5616047
Dispatcher should be responsible for routing message to the
callback method of endpoint object and returning result back to the
server only. But now it is also responsible for sending reply,
ack/reque messages etc.
Also this patch makes small improvements:
1) Notification dispatcher now requeue message if endpoint raises exception
2) unstable behaviour of test_mask_passwords test is fixed
Change-Id: I5f23e23644e90919cb67f81fc306ee85c5e09974
Tobias Urdin