Introduced changes:
- pre-commit config and rules
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.
Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.
pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker
(check-byte-order-marker);
- Checks that non-binary executables have a proper
shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings
(check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)
For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks
Change-Id: Ib18f38dbec90c62e870307bf22a8b4f193237bce
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: Ic18f03681c14131217e3025e2cb0cca46ada84ee
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
If user set the type of worker to str or others, there will
be no error here, but a lot of processes will be created,
which may cause the system to crash.
I recommend adding a type check here.
Change-Id: I3ad0f7ec59f29a3106d23b057327c3dfef19a98f
The intention of this test was to wait 5s after sending SIGHUP to a
child process to make sure that it doesn't exit. However, due to a logic
error, it just stopped checking and declared success immediately. Fix
the logic so that we have a better chance of seeing if SIGHUP
incorrectly kills the process.
Change-Id: I1f320a8dfdd7a922b461d070491ad53e6cd2b20d
Related-Bug: #1803731
The patch 2ee3894f49 broke the original
fix cad75e4e13 that ensured eventlet could
be interrupted while sleeping after PEP475 was implemented in Python
3.5. Eventlet monkey-patches the signal module with its own version, so
we have to look up the original module to determine whether the
underlying OS actually supports the poll() function.
Change-Id: Ia712c9a83d8081bf0b5e6fe36f169f9028aae3dc
Closes-Bug: #1803731
Related-Bug: #1788022
Related-Bug: #1705047
Recent versions of eventlet close idle connections, so just opening
a connection is no longer sufficient to trigger graceful shutdown
behavior. This change sends a request and adds a delay on the server
side so the connection will be active throughout the test.
Change-Id: I4e908c1fb1a61d8a57769e2aa85c02a21722367b
Closes-Bug: 1788959
Setting up signal handling fails on Windows due to missing select
module attributes. This patch skips the setup on Windows.
Change-Id: I75db453ec907efb01e1bb411690dadd92a48314d
Closes-Bug: #1788022
The order that kill() is called on the child processes of a service is
determined by iterating over a dict, so don't hard-code a particular
expected order in the test. Doing that caused the test to be broken on
Python 3.
Change-Id: I4ca85cc8f559985b133e523cd00f297e5576d00a
There's a short window where sending a second SIGHUP will cause the
process to exit rather than reload config. This causes some confusion
to operators/testers and is hazardous for use with a config management
system. It would be easy to write plays that accidentally, even
intermittently, cause the service to exit.
Disallow this for SIGHUP. Graceful/fast shutdowns can still be
achieved with SIGTERM/SIGINT.
Change-Id: Ia2146b25b857217d498bf785c17bd0b629174c86
Many assertEqual sentences don't follow assertEqual(expected, actual),
this patch fix all of them.
Closes-Bug: 1604213
Change-Id: Ice72233f41d2ee1a32ef1be7ff65a8a2ea19c508
This patch adds a 'restart_method' parameter to Launcher. When set to
'reload' (the default), behavior is as today. If a service chooses
'mutate' then when a SIGHUP is received, mutate_config_files is called
instead of reload_config_files. This means the 'mutable' flag on
options will be respected.
Change-Id: Icec3e664f3fe72614e373b2938e8dee53cf8bc5e
Depends-On: I9bb2ff39dc1ed0a63bd7e7635704b34c53d32d79
I need to tweak the launcher used, as it stands this isn't possible.
Also I find it ugly muddling the service arguments with the
_spawn_service arguments.
Pass in constructors for each of these. This allows us to pass
whatever arguments, set up extra things, whatever we want. It's
important that at least the launcher is created in the child process
so we can't pass in objects.
This patch also fixes Launcher.launch_service to accept a workers
argument (which must be 1 or a ValueError will be raised). This makes
ServiceLauncher.launch_service compatible with
ProcessLauncher.launch_service, avoiding awkward "if isinstance" or
"if workers == 1" logic when you're not sure of the type of the
launcher.
Change-Id: Ia309828c0c7e7599d4b82994dd36165106dc6513
SignalHandler._handle_signal() is not protected against reentrant
calls. If a signal is received during the execution of
_handle_signal(), _handle_signal() is called again. The problem is
that some functions are not reentrant, especially
SignalHandler.clear().
If SignalHandler._handle_signal() gets a second SIGTERM signal while
it is currently running the SIGTERM signal handler, there is a risk
of calling twice _sigterm() method. This is unexpected: _sigterm() is
supposed to only be called once, that's why its first instruction is
to unregister *all* signal handlers.
This patch changes how Python callbacks are executed. Instead of
calling them directly in the Python signal handler registered with
signal.signal(), the call to the final Python callback is delayed
using eventlet.spawn().
The goal is to avoid reentrant calls to the final Python callback to
avoid complex race conditions.
Rename also _handle_signals() to _handle_signal() since it handles a
single signal (but it may call multiple Python callbacks).
Related-Bug: 1538204
Change-Id: Iaccd4aa0135c77793eec6a2689da6dbff5498031
When exception is raised in service.start() call service.wait()
is not unblocked. This patch fix this problem.
Change-Id: I30959fad3a5fe089e25c9191abe2ad6df5a87ed3
Signal handlers were changed:
SIGTERM - causes graceful shutdown,
SIGINT - instantaneous shutdown.
Tests for handling signals by ServiceLauncher were added.
Closes-Bug: #1446583
Change-Id: Idccee004547b0eb905c484538d6510c8315c210f
The run() method was using much more code than required. Simplify this.
Add a workers argument to run() method.
Make port_queue argument mandatory as without this we fall over.
Change-Id: I3618dec41009606269d267969bff36906d79e620
Signature of method wait in ServiceLauncher and ProcessLauncher must
be the same because if user uses service.launch() function and
workers number is taken from config file user couldn't know what
kind of launcher will be instantiated.
Change-Id: Ie25a3349e15eb2c10b5b054c005b32f471dbafa1
* use assertIsNone(val) instead of assertEqual(None, val).
* use assertTrue(val) instead of assertEqual(True, val).
* use assertFalse(val) instead of assertEqual(False, val).
Change-Id: I7e7512328c82de84f4e0ed2e5fabecd96bfc8ef7
Option graceful_shutdown_timeout is responsible for interruption of
graceful shutdown if graceful_shutdown_timeout is exceeded.
DocImpact
Closes-Bug: #1446583
Change-Id: I0dfbf2d0a4943337da24c0904a1ed6f0cdccd77b
Reloading of application can be performed by sending HUP signal to
master process. But it is not useful right now because there should
be implemented many lines of code to support real reloading for each
of config options, and of course it is not implemented.
The easiest way to achieve reloading and applying configuration is
gracefully stop children and then respawn them.
When master process receives HUP signal it reloads it own
configuration, sends TERM signal to children to shutdown them
gracefully and respawns new children with updated config.
There is no impact on the user because sockets are still listening
in master process. So new requests will be put in queue.
Change-Id: I3e7264a1efcbf66a9afc69d8ed20f600c985c296
1. Modules in oslo_service uses eventlet, but they are not monkey_patched.
This patch add monkey_patch to all unittest code.
The original unittest code does not really work, it muddles through.
2. According to what Elena Ezhova said, there is a bug about patching os module on
windows [1]. As a result, the monkey patch bypass os module on windows.
3. subprocess module is not monkey_patched by default, we should use
eventlet.green.subprocess explicitly.
4. test_stop_immediately() does not work as expected, the reason it passes is because
foo() never get a change to run, if foo get a change to run, the blocking version
time.sleep() in it will block the main thread. We should give it a chance to run
so that the test counts.
[1] https://bitbucket.org/eventlet/eventlet/issues/132/eventletmonkey_patch-breaks
Change-Id: Ic586be571b78fc096f77aadc6c9f3c85e4f26c80
Signed-off-by: apporc <appleorchard2000@gmail.com>
Use string representations of signals when adding a handler to
avoid AttribureError in case a signal is not supported on a
certain platform.
Fixed a check for sighup support in add_handler method.
Added the add_handlers method that allows to add several handlers
at once and remove this functionality from add_handler to prevent
recursive calls in case we pass a tuple of strings.
Change-Id: Id3783160381f4a89e241134ef13c4521a30dfa94
Closes-Bug: #1481607
Singleton class Signal Handler is responsible for interacting
with signal module: allow to have several handlers assigned
to distinct signal. Added correct handling SIGINT signal.
Closes-Bug: #1477102
Change-Id: I4bc4c98645e9afeb18635576d50a442fb7ce8425
LauncherTest contains a number of tests that actually test
eventlet_backdoor while unnecessarily creating and launching
services. Moreover some of these tests duplicate existing tests
from test_eventlet-backdoor.
This change moves such tests to test_eventlet_backdoor, removes
mox usage from them and deletes duplicated ones.
Change-Id: Ib7533400115700bcb36ef403f9ccad9c8796e455
In order to determine what to do about this module it
is quite useful to provide for a way to run it as its own
main program (and interact with it from another telnet
session).
This adds a simple `main` function and uses it.
This can then be used like:
$ python oslo_service/eventlet_backdoor.py --backdoor_port 8000
Then from another terminal:
$ telnet localhost 8000
Change-Id: I8c2b93dfad328929e1a106c5883903fe0bfb188a
The check introduced by I79b38a354dd85f58a7c16b34661bff02d9170a9f
is supposed to verify that a service that is being launched is an
instance of service.ServiceBase abstract base class.
However in some projects services are launched by instantiating a
launcher (ServiceLauncher or ProcessLauncher) and then directly
calling its launch_service method. In this case the check never executes.
This change moves the base class check into a separate function that is
being called by launch_service methods of the both launcher classes.
Closes-Bug: #1467439
Change-Id: I39e9070fbedd769c2c1bd3141bbe35769e889476
Currently, termination of WSGI application or RPC server immediately stops
service and so interrupts in-progress request.
Graceful handler for SIGTERM signal was added.
SIGINT signal handler was removed to allow instantaneous termination of
service.
DocImpact: graceful termination of process can be done by sending SIGTERM
signal to parent WSGI process. Graceful termination is not instantaneous.
To force instantaneous termination SIGINT signal must be sent.
Change-Id: If25f9565d832d1c36642ec3b1921563ef02890aa
Closes-bug: #1382390
Co-Author: Tiantian Gao <gtt116@gmail.com>
Change the API of oslo.service to accept conf object and
remove usage of global CONF object throughout the library.
List of functions/classes that accept conf as a parameter:
1. eventlet_backdoor.py
* initialize_if_enabled
2. periodic_task.py
* PeriodicTasks
3. service.py
* launch
* ServiceLauncher
* ProcessLauncher
4. sslutils.py
* is_enabled
* wrap
Fixed the unit tests.
APIImpact
bp graduate-oslo-service
Change-Id: Icbd5f9e386b0eee2f7c60a65c076f53ba5f0f637
Service object that is passed to any launcher must implement
the following methods:
* start
* stop
* wait
* reset
Introducing abstract base class from which all openstack
services that use common.service code can subclass guarantees it.
Added documentation for launcher classes in service.py
Change-Id: I79b38a354dd85f58a7c16b34661bff02d9170a9f
Closes-Bug: #1447965
dengzhaosen