The monotonic package was needed for monotonic time operations when
running under Python runtimes older than 3.3. Since we now only support
versions higher than this, this third party package requirement can now
be removed.
Change-Id: Ie99411882822a5db47462fcb2748c6e978587b8c
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: Ic18f03681c14131217e3025e2cb0cca46ada84ee
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Instead of having a copy-pasted version in this project, let's just
use the original directly. It is added to the public API of
oslo.utils in the dependency.
Depends-On: https://review.openstack.org/614806
Change-Id: If0dfac2505d097c117ef94c99399b1614f1e1f8f
Patch [1] for switched to use eventlet Event for loopingcall events.
It may now happen that stop event is sent when other event was
already sent and loopingcall is already not running.
That cause AssertionError in eventlet.event module.
To avoid that, we should check if if loopingcall is
running before sending _abort.set().
[1] https://review.openstack.org/#/c/611807/
Closes-Bug #1800879
Change-Id: I28ad3bdb51a20350c90dee4420058c30946897e5
This fixes broken logic for finding out how to use green threads. Using
threading.Event with eventlet creates useless load related to timers.
Change-Id: I62e9f1a7cde8846be368fbec58b8e0825ce02079
Closes-Bug: #1798774
Eventlet has used at least three different clock implementations by
default over the past couple of years (time.time, vendored monotonic,
and plain monotonic). Since we're forcing use of monotonic anyway,
this is all irrelevant to us outside of the fact that once we depend
on a new enough version of eventlet we can remove our monkey patching
of the hub. Asserting on the particular module that eventlet uses by
default is just breaking our unit tests on newer eventlet for no
good reason.
Change-Id: I0f5135567a16dde4c34cee35baa2054091dce728
Partial-Bug: 1788959
This reverts commit 5975da493b.
Added code to support the case where unit tests are not being
monkey patched (example heat).
Change-Id: If715fbe21ac085e4f5c83cef0729dbca8dcb19ca
Some of the openstack services implement worker tasks that are based on
the oslo-service LoopingCallBase objects. They do this as a way to have
a task that runs periodically as a greenthread within a child worker
process. For example, the neutron-server runs AgentStatusCheckWorker()
objects as base service workers in its child worker processes.
When the parent server process handles a SIGTERM signal it attempts to
stop all services launched on each of the child worker processes (i.e.,
ProcessLauncher.stop()). That results in a stop() being called on each
of the underlying base services and then a wait() to ensure that they
complete before shutdown.
If any service that is implemented on a LoopingCallBase related object
is suspended on a greenthread.sleep() the previous call to stop() will
have no effect and so the wait() will block until the sleep() finishes.
For tasks that either have a frequent FixedLoopingBase interface or a
short initial_delay this may not be a problem, but for those with a long
delay this could mean that the wait() blocks for minutes before the
process is allowed to shutdown.
To solve this the LoopingCallBase calls to greenthread.sleep() are being
replaced with a threading.Event() object's wait() method. This allows a
caller of stop() to interrupt the sleep and expedite the shutdown.
Closes-Bug: #1660210
Change-Id: I5835f9595826df5349e4cc8b1da8529bb960ee04
Signed-off-by: Allain Legacy <allain.legacy@windriver.com>
The backoff timer has a few issues that can cause it to get stuck
in an infinite loop and never time out.
1. The random.gauss() function used to generate random jitter can
return negative values, so when it does, it makes the elapsed time
self._error_time go "backward."
2. The random jitter is used as a multiplier for the self._interval,
so self._interval can deviate far away from the mean over time and
walk to zero, causing self._interval to be 0, which will prevent
the timer from making progress from that point on because idle
will always evaluate to zero and the elapsed time won't increase.
3. The evaluated interval doesn't have a lower bound, so over time
it can get extremely small if jitter (the mean) < 0.5.
This adds a min_interval keyword argument to the BackOffLoopingCall
start() function that defaults to 0.001s and uses it to lower bound
the interval calculations. We'll also take the absolute value of the
return from random.gauss() to prevent elapsed time going backward, and
we'll calculate the running self._interval separately to make it track
the desired growth rate of the backoff and not let it drift with the
random.gauss() values.
Closes-Bug: #1686159
Change-Id: Id17668a34d5cedbe870c9056350a7e9c7196faa7
Currently when using FixedIntervalLoopingCall, folks need to
add timeout checking logic in their function if they need it.
Adding a new class FixedIntervalWithTimeoutLoopingCall to
provide timeout checking support will save those effort.
Change-Id: I78bfb9e259c2394137d7efbc0ee96bb18a6dc5e7
Many assertEqual sentences don't follow assertEqual(expected, actual),
this patch fix all of them.
Closes-Bug: 1604213
Change-Id: Ice72233f41d2ee1a32ef1be7ff65a8a2ea19c508
eventlet's hub mechanism defaults to using time.time for figuring
out how long to sleep, when to wake up etc:
https://github.com/eventlet/eventlet/blob/master/eventlet/hubs/epolls.py#L37https://github.com/eventlet/eventlet/blob/master/eventlet/hubs/hub.py#L344
So if you mess with the system time, anything using loopingcall
will stop working. Best example as described in the bug is
`nova-manage service list` which fails for some services when
the clock is advanced.
If we use a monotonic timer, then we can get past this problem.
In this review we use the EVENTLET_HUB hook to specify
a custom Hub. In our custom Hub, we just delegate to the
existing hub mechanism (get_default_hub()) to create an
instance of the Hub and override the clock with one based
on monotonic.
Note that use_hub() mechanism will not work because that sets
the Hub just for the current thread and will not work for other
threads that may be started later. In order to set this
environment variable early, we set it in the __init__.py so
anyone who uses oslo service will enable this support just
by a simple import referencing anything in oslo.service
Co-Authored-By: Victor Stinner <vstinner@redhat.com>
Closes-Bug: 1510234
Change-Id: I4c1cf223e9f6faa1e6dcaf432be1aa709072a343
If the periodic task for dynamic looping call returns no suggestion for
delay, then we should use periodic_interval_max. If
periodic_interval_max is not specified, we should raise RuntimeError.
Otherwise, passing 'None' as a value, causes Exception at eventlet's
sleep() call.
Change-Id: Ida3e75bc64132d6b5920fa94657aa962e2fe9f53
Closes-bug: #1489998
In some cases we don't want LoopingCall to iterrupt if there is
an uncaught exception in the callee. In case of exception LoopingCall
needs to wait similarly as if a call succeeded.
Co-Authored-By: Eugene Nikanorov <enikanorov@mirantis.com>
Related-Bug: #1458119
Change-Id: I58da03017b923cf5ec2223ecfc61c766726bd1de
RetryDecorator from oslo.vmware needs a better home. It allows
users to specify specific exceptions upon which the request can
be retried.
http://git.openstack.org/cgit/openstack/oslo.vmware/tree/oslo_vmware/api.py#n48
Depends-On: I0f07858e96ea3baf46f8a453e253b9ed29c7f7e2
Depends-On: I33bd2d9dff9cb7dc1a50177db7286b7317966784
Closes-Bug: #1465629
Change-Id: Ic5d57fcf769a4af53cd1cf82a3ca93142dbdb03f
This ensures that the looping call is not running after the
test has finished (which is/would be bad).
Change-Id: Ica296938d4316c5b16aa6545da792af1f589a518
Monotonic time and oslo.utils stopwatches will not go backwards
and therefore will not cause periodicity + misc. problems where tasks
are ran again or at the wrong times in a system that is having its
clock altered (via ntpd or other).
Related-Bug: #1450438
Change-Id: I90881842185c607eb6c8ea5bb7326a37e1bc3742
Takashi Natsume