All files use UTF-8 by default in Python 3.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I7acf3a0ad369371aa4075962b3656467a6a37c58
The monotonic package was needed for monotonic time operations when
running under Python runtimes older than 3.3. Since we now only support
versions higher than this, this third party package requirement can now
be removed.
Change-Id: I0e5bf6090ed85db2659b50096122a9abda9dde6a
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I2b8d8d6cf4ac29dc58cd8fe418d613445d65c3be
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
PrettyTable as a depend can have a tendency to conflict with other
things that want to use it, which is a shame for a general purpose
library like futurist. Make it optional to reduce the transitive
dependency burden for consumers.
Change-Id: Ie976f7f4e544f273fccf0cf579b82b659cbd48d1
monotonic is a backport of time.monotonic from python3. Only install it
for python2.
Depends-On: https://review.openstack.org/615441
Change-Id: Id27cd748e883d54dd93dac2e6bd8caee6728f7e1
This makes it easier to use the other various
functions in futures or executors that will raise
this when they timeout.
Change-Id: If18d55fc20b2cb6a919c9a1a98a818f4068bd3ec
This change allows simple enable/disable periodic and set spacing
via one configuration option, like
@periodics.periodic(spacing=CONF.task_interval,
enabled=CONF.task_interval > 0)
Change-Id: I1f00242b49c7646421390c0ff26777aba6d5b6d9
Closes-Bug: #1654035
Allow functions to raise a never again exception to remove
themselves from running
It also introduces a new param named ``auto_stop_when_empty``
to ``start()`` method. When the provided periodic functions have
all exited and this param is set to false then the thread
responsible for executing those methods will just spin/idle
waiting for a new periodic function to be added; switching it to
true will make this idling not happen (and instead when no more
periodic work exists then the calling thread will just return).
Co-Authored-By: Debayan Ray <debayan.ray@gmail.com>
Change-Id: If7c5e1a407fb72af8e0f7f3a1017a123590cffb2
SystemExit exception is usually raised by signal handlers
and then caught in main thread to handle signal. Signal
handler executes in main threads but if we use eventlet
we have ony one thread and signal handler can interrupt
any greenlet. Therefore we need to re-raise this exception
to catch it in main greenlet at last.
Also it is necessary to keep executor working and continue
processing scheduled tasks because signal handling logic may
wait for some job results
Change-Id: Ia920922d8444c8216b442577c37710538ed8e919
Due to eventlet's bug eventlet.patcher.is_monkey_patched('threading')
always returns False, so it is better to replace 'threading' with
'thread', which seems to work properly.
Change-Id: I849e427c312e41b2d518a088e3f91c92d7574bd7
This needs further improvement as it does not appear
to be working as expected so for now just revert it back
to the naive (and more expected) behavior for starting up
new workers (and adding them to the worker pool).
Change-Id: Ia9cc4e44c55ea39bc6270ed42b375ca495bf9ec1
To be able to add extended functionality without having
to (just yet) upstream those changes into the stdlib, especially
around thread pool execution and tracking start to add our own
thread workers and the needed mechanics to clean them up
appropriately.
Change-Id: If9af0d905009435d91e7d8ac00005f4ca30bd987
Ensures it is used, and exceptions from it are
handled, and that it is type checked correctly before
being used.
Closes-Bug: #1531530
Change-Id: I0843f363ae70d3b6475cddeebb7564204a10a04d
Instead of using time functions to delay submissions
to validate that rejection fails use an event instead
and use it to force the needed threads to block up so
that rejection will always happen (even with time
changes).
Closes-bug: #1552232
Change-Id: Ib688dccd8eda8c11b71b4aa36904ee67d2f93889
Have a smaller helper class that unifies the retaining
or not retaining of failure tracebacks instead of having
two nearly equivalent functions to do the same.
Change-Id: I54bb087bbe35f5597c6658ba8aa823795f560045
Seems like the wrong function signature was being decorated
so this fixes that and adds an initial test to make sure it
continues to work as expected.
Also fixes the invalid usage of random in the jitter
decorator.
Change-Id: If30fcc1430b84078dc5fc104aa28f06471e4c358
The rest of the code single quotes it so this code might
as well to (to follow the existing logging style).
Change-Id: Ibd2a8b49d11dcd32d9168a96950e08616ac1c54d
When testing Ironic with Futurist I found out that whe number of tasks
seriously exceed number of possible workers, the conductor goes into loop
retrying them and mostly stops reacting. Insert a short random delay before
retrying, so that the executor has chances of processing tasks before we
through more on it. Also recalculate the "now" value, as submitting a task
could take some time.
Change-Id: I8e45ddf7c87cf130028fc9fe937691968db17ee9
We say we raise when backlog goes past max size, but actually we raise when
backlogs tries to reach the max size. This patch fixes reject_when_reached to
actually allow max size to be reached, but no more.
Also adding missing unit test for reject_when_reached.
Change-Id: Ide835b6e5fd15171b9bebc7ca112b84423b39fab
Now that shutdown can be skipped (which would previously
wait for all submitted futures to be finished) we need to have
a way to know that all submitted futures have been finished
before our run method returns.
A simple countdown barrier/latch does this so ensure that we use
it in the periodic run loop to be able to only return from the
run loop when no active futures are in progress.
Change-Id: Ia3dd84ebfe2416060aaf5113cc8310a23919a3f9
The current behaviour of always shutting down the executor makes it troublesome
to use a precreated executor and makes completely impossible to use the same
executor for running PeriodicWorker itself (which is what I'm doing for Ironic).
This change adds a base class ExecutorFactory with 'shutdown' attribute
defaulting to True. It also adds a new class ExistingExecutor that takes an
existing executor and sets shutdown to False.
Change-Id: I6a574ba77439aeee46b9e154aa059098ec087308
When a future is cancelled and the result is fetched
it will raise a CancelledError which is quite useful to
be able to catch.
So to enable catching that specific exception expose it
in futurist __init__.py so that people can easily catch
it just by catching futurist.CancelledError and handling
the outcome of that.
Change-Id: I7bc1838aadcbb8cd0d635da83bb1e1b1ec761e89
Sometimes it might be required to pass the same arguments to all
tasks collected by create (e.g. ironic passes context).
Change-Id: I7992c59e76b8b7b9101501d75cb7a53d982d07fd
Currently PeriodicWorker fails if executor raises RejectedSubmission.
This change makes it catch RejectedSubmission and RuntimeError from
executor.submit and reschedule the task as soon as possible.
Change-Id: Ifff7850c9b44ee076f428ed3a613a13e182a549d
Closes-Bug: #1532255
People implementing their own periodic tasks collection (as opposed
to using PeriodicWorker.create) need to know if something is
a periodic task without inspecting its private attributes.
This function returning True means something is safe to add
to a PeriodicWorker (will not raise ValueError).
Change-Id: Iee8da601b1bc994188c58ca459574a466270bf63
It can be quite useful to replace the running of work
function in the sync executor with a function that logs
or with a function that shows whats running, or other
so enable that replacement by having a new sync executor
parameter that can be provided to give alternate functions
that run work items.
Change-Id: I8f4a806ed698370b4f5a300d74ee507cbe4eebc5
Avoid having different repr styles and just unify the executor
stats one to look like the one used in the watcher object, which
is the style more commonly used in other libraries also.
Change-Id: Ibd98094371f57ecb8bca78b8446ca28727da84e9
When a workers watchers are iterated over and
say printed to stderr/stdout the default '__repr__'
doesn't provide much value; so instead of the default
provide our own that adds slightly more value.
Change-Id: I8f82a732ad3d9f0ce0ec5857d5d883b3b5f2ff1d
In the internal '_run' instead of having a loop that
has various cases with quite a bit of code per case instead
have the loop call into smaller helper functions that handle
the different types of processing the loop does.
Also adds a note to the 'reset' method that explains why
the metrics keys are overwritten vs being copied/replaced
with a new copy.
Change-Id: I5ac1794d5803850120be4e67b85efe0449aa1891
Instead of having the green executor(s) helper classes
be inline with the rest of the other executors move its
helper classes/objects into its own module and use them
from there instead.
This also moves the work item helper into '_utils.py' for
similar/equivalent reasons.
Change-Id: I028738bf7d64f97320450d63debbd5116b505a72
This change makes @periodic more compatible with periodic tasks from
oslo.service, and also enables the following pattern found in ironic:
@periodic(enabled=CONF.periodic_task_foo_enabled)
def foo(...)
Change-Id: I390197158522a59145420bd7988ffa74e6e5a391
There really is no need to give a green worker
a reference to its creator and it hinders/makes it
harder for python to garbage collect the executor
and its workers.
It is also not used, so let's make the python
garbage collectors life easier by not even passing
it in.
Change-Id: I0f9d27663c3ca5ca908ef6f90f3d1d0f6b5f9104