This patch cleans up the current rpc
implementation by moving the seralizer
back to the rpc module, this is more in
line with other projects; such as Nova.
- Moved _init_serializer back into rpc.
- Added back unit-tests for profiler.
Change-Id: Ia148b2d3bc352e96e7633f7af82ecd26b5f35e35
This patch changes the rpc server to re-use
the transport in the same manner as the rpc client.
Story: 2008494
Task: 41752
Change-Id: I93eecacbe45d19c4f73e9a974d60e642e87bbdf0
We are currently creating a new transport for each api
call. This patch changes that so that each worker
can re-use the same transport for multiple requests.
Story: 2008494
Task: 41544
Change-Id: I11a24f035a9d66a536e5e58328084ee08f0c6285
The periodic jobs are currently getting registered per each worker
which means that in cases with large number of workers, the APIs
for services such as Heat and Keystone will be hit very hard.
This patch resolves this issue by registering the jobs only to the
main process, ensuring that they run once per instance (or group
of workers).
Closes-Bug: #1702349
Change-Id: If9e13effc14fd35e646d02bb4f568e79786aa958
oslo.messaging allow dispatcher to restrict endpoint methods
since 5.11.0 in d3a8f280ebd6fd12865fd20c4d772774e39aefa2, set with
DefaultRPCAccessPolicy to fix FutureWarning like:
FutureWarning: The access_policy argument is changing its default
value to <class 'oslo_messaging.rpc.dispatcher.DefaultRPCAccessPolicy'>
in version '?', please update the code to explicitly set None as the
value: access_policy defaults to LegacyRPCAccessPolicy which exposes
private methods. Explicitly set access_policy to DefaultRPCAccessPolicy
or ExplicitRPCAccessPolicy.
Change-Id: Ie3dc84840fd39a3a55d6c1dad720d998fadd3c2e
With the added get_rpc_transport() function in oslo.messaging [1]
and the recent version bump of oslo.messaging to 5.24.2, we can
safely replace get_transport() with get_rpc_transport() which is much
cleaner.
[1] https://review.openstack.org/#/c/454194/
Related-Bug: #1680192
Change-Id: I3a3e88f0fc85a03c0ee9b82edd165436f136a4d0
The default executor of o.m is 'blocking', but after change [1],
o.m with blocking is broken. 'eventlet' and 'non-blocking' work
fine and most services use 'eventlet', so this is why it was
noticed by magnum which used the default one.
Use eventlet to not be affected by bug #1694728 , but more
importantly, the oslo team suggests to not use blocking which
has 0% test coverage and it is going to be deprecated and unset
from the default configuration.
[1] https://review.openstack.org/#/c/463673/
Change-Id: I47da73787456c97f7d84fd4440404b551ff62528
Closes-Bug: #1694728
Those are remnants from the oslo-incubator times. Also, oslo.messaging
deprecated [1] transport aliases since 5.2.0+ that is the minimal
version supported for stable/newton. The patch that bumped the minimal
version for Magnum landed 3 months+ ago, so we can proceed ripping
those aliases from the code base.
[1] I314cefa5fb1803fa7e21e3e34300e5ced31bba89
Closes-Bug: #1424728
Change-Id: I7c35f71f601f3191de4a97866d52db343e5735b1
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who wants to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds first wsgi trace point, with info about HTTP request
* Add initialization of osprofiler at start of service
Currently that includes oslo.messaging notifer instance creation
to send Ceilometer backend notifications.
* Traces HTTP/RPC/DB API calls
Demo: https://hieulq.github.io/cluster-create-false-new-html.html
Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com>
Implements: blueprint osprofiler-support-in-magnum
Change-Id: I7d68995aab81d365433950aada078ef1fcd5469b
This is patch 3 of 3 to change the internal usage of the terms
Bay and BayModel. This patch updates Bay to Cluster in DB and
Object as well as all the usages. No functionality should be
changed by this patch, just naming and db updates.
Change-Id: Ife04b0f944ded03ca932d70e09e6766d09cf5d9f
Implements: blueprint rename-bay-to-cluster
The periodic task unneccessarily lists Heat stacks in the
global tenant (across all tenants) which the Magnum service
user may lack permission for. Also, the most restrictive way
to let it use global stack-list is chose a Keystone role and
open that operation to any user in any project holding that
role.
This commit substitutes a direct lookup of all bays' stack_id
attributes for this global stack list. This direct lookup will
yield the same net result. In order to get the neccessary
permissions it will use each bay's stored Keystone trust to
act on behalf of the bay's creating user.
Co-Authored-By: Jiri Suchomel <jiri.suchomel@suse.com>
Closes-Bug: #1589955
Change-Id: I67b176c137c463e37e037970cc4e468d51db30c9
Several commands take much longer to complete after that commit.
Therefore, we should revert that specific commit first, and
investigate root cause before merging it back.
This reverts commit 873214b6e2.
Closes-Bug: #1540646
Change-Id: I0eb12f76e93fb1675c78bf60367db534061aceb1
The connection to amqp was not getting cleaned up, even after the
communication to conductor across amqp was complete, for a given
request. Thus, sockets were leaking with each communication and finally
led to a hang situation, where no more fds were available.
Change-Id: I1deabdbce6ba448fe4c25d7694aabe5e5fec7b5a
Closes-Bug: #1510776
Liberty tech-debt-bug
Magnum service was initially utilizing periodic task within magnum
conductor. This commit lays out independent periodic framework for
magnum service, by decoupling from periodic task.
Change-Id: Ie0e725b75069309a9fa783f77c1a454f9dae549e
Partial-Bug: 1500998
To support 'magnum service-list' after 'nova service-list', we need to
introduce periodic status update functionality for internal services.
Change-Id: Ia0c09222405c87cb61e5de4a43ba345ae3405b50
Partially-Implements: blueprint magnum-service-list
Closes-bug: #1492501
This patch adds periodic timer to conductor service.
Any new periodic tasks need to be added to magnum/service/periodic.py
We add 2 default configure options to support periodic task:
'periodic_enable', default=True
'periodic_interval_max', default=60
These 2 options are for rpc_service.
PS:
We don't need to supervisor timers because timer itself won't have any
exception, and it is not a thread, which means can not be killed.
Partial-Implements: blueprint add-periodic-task
Co-Authored-By: ShaoHe Feng <shaohe.feng@intel.com>
Change-Id: I132d9cc89b26baf249b62b713ccd3776f0078d7d
Currently, conductor service doesn't handle interrupts, so it will show
lots of call trace if gets KeyboardInterrupt.
This patch improves this by puting in service.launcher, which already has
better interrupts handling.
Closes-Bug: #1465472
Change-Id: I6a6948c9942cc7c84fcf4ce22b029e7b6727c84e
rpc_service.py is already importing rpc.py so it should reuse
RequestContextSerializer rather than redefining it.
Change-Id: Ie8a6deca7e7c238b8076a8bbbe9598a025222aa1
Closes-Bug: #1459090
The conductor listener RPC API and its handler provide a way for
conductors to confirm that other conductors are still alive. This
will be used during acquisition of a lock. In particular, if a conductor
wants to perform an operation on a bay and the bay was locked, the
conductor will attempt to communicate with the conductor who owns that lock.
If the conductor responds affirmatively, the lock will be considered active.
Change-Id: I17a6f9cdc2e8b40d997c1b4e1964658d8e372297
Partial-Implements: blueprint horizontal-scale
Whenever any remote commands raise an exception,
that exception is returned as RemoteError to the
controller rather than being from the magnum
exceptions. Then controller further returns this
to user causing 500 InternalServerError status.
This was so because magnum exceptions were not
listed in allow_remote_exmods.
This patch registers all magnum exceptions in
allow_remote_exmods list. Thus helping to send
proper response to user.
Partially implements blueprint exception-handling-in-magnum
Closes bug: #1424904
Change-Id: Ia90b74798ac070ea514c66a4f66db939dbd1d2a9
Oslo team is recommending everyone to use the direct imports and
not use the Oslo namespaces. So switch all our code to use oslo_*
instead of "from oslo." or "import oslo" or "from oslo"
NOTE: some of the tests still have mocks referring to oslo.utils
@mock.patch('oslo.utils.timeutils.utcnow')
as the tests break otherwise. We should do this later.
Closes-bug: #1419385
Change-Id: I8e3fbeb833cddc3f55674a0e781ffe69d5033ad4
This uses a simplified RPC model while integrating with the
versioned objects serializer.
bay-create is handled by the backend.
bay-list is handled by the RPC service.
Change-Id: I384994584ad7f928df9366a2e6f677951a450e40
Erik Olof Gunnar Andersson