Commit Graph

112 Commits

Author SHA1 Message Date
Takashi Natsume 845fc45fab Fix misuse of assertTrue
Replace assertTrue with assertEqual.

Change-Id: I918930a6fd3980297ee7d68dd6c9397416fb3335
Closes-Bug: 1988308
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
2022-09-19 01:12:30 +00:00
dengzhaosen aea7627eb0 Remove references to 'sys.version_info'
We support Python 3.6 as a minimum now, making these checks no-ops.

Change-Id: Ibaa2cdb7cc27f541aeb778782813354df2911156
2021-04-29 09:10:24 +08:00
Hervé Beraud 10ce88ed2b Adding pre-commit
Introduced changes:
- pre-commit config and rules
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.

Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.

pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker
  (check-byte-order-marker);
- Checks that non-binary executables have a proper
  shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings
  (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
  calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)

For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks

Change-Id: Ib18f38dbec90c62e870307bf22a8b4f193237bce
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
2020-09-08 14:26:37 +02:00
Zuul 585768b94d Merge "Fix wsgi SSL tests for wsgi module under python 3" 2020-06-23 18:30:37 +00:00
Zuul 8fd3ce22da Merge "Reactivate wsgi test related to socket option under python 3" 2020-06-23 18:30:36 +00:00
Zuul ce6262437d Merge "Align tests with monkey patch original current_thread _active" 2020-06-22 17:22:56 +00:00
Hervé Beraud caf66be763 Fix wsgi SSL tests for wsgi module under python 3
Previously some tests were ignored under python 3 environment, this was
due to some design changes introduced by python 3.7 [1] in the SSL
module of the stdlib. These changes reactivate some of them (some other
are still skipped and needs further works).

Indeed, when we try to use requests with SSL in a monkey patched
environment we faced the following issue:

```
TypeError: wrap_socket() got an unexpected keyword argument '_context'
```

This is due to the fact that we are in a monkey patched environment
where `requests` is monkey patched too.

We don't need `request` for our needs. Indeed we can easily send
http requests through low level socket. Our main goal is to test
our wsgi server and not to test the `requests` library, and `requests`
was just used to make the code more simpler.

In our case we can implement a code dedicated to send request to our green
server, unlock our tests and move away from this bug/side effect.

Also this reactivated test will check WSGI server with and without SSL,
so these changes add changes that allow us to submit a request without
wrapping the socket with SSL.

These changes move away from `requests` which is badly monkey patched by
eventlet [1]. Now we use monkey patched socket and ssl to
request the green server which is executed in background. Low level
(monkey patched) modules could help us to skirt layers that are possibly
badly monkey patched on higher level modules (urllib, requests, etc...).

[1] https://github.com/eventlet/eventlet/issues/526#issuecomment-482694279
[2] https://github.com/eventlet/eventlet/issues/526

Change-Id: I3a018d507d102266c1e2fc9b6732a9c09fa2bb49
Closes-Bug: #1482633
2020-06-17 12:23:42 +02:00
Hervé Beraud fb1ed4085b Reactivate wsgi test related to socket option under python 3
Previously these tests was ignored surely because the WSGI tests
with SSL were broken due to some design changes introduced by python 3.7 [1]
in the SSL module of the stdlib.

However, unlike the other WSGI/SSL tests these changes don't use
the `requests` module or high level modules which are broken by
the monkey patching of the stdlib.

I suppose previous developers simply turned off all of tests related to
WSGI/SSL without much more details than "SSL tests with python 3 and eventlet
is broken" but some of them don't need much more work than just reactivate
them to be ran successfully.

This test seems to work without change, and it run successfully locally.

Let reactivate this test.

Change-Id: Ie0257af10a9439de8ec124a698ca75778bdf90b5
Partial-Bug: #1482633
2020-06-16 14:34:30 +02:00
Hervé Beraud ebc2d8a42f Fix wsgi/SSL/ipv6 tests for wsgi module under python 3
Previously some tests were ignored under python 3 environment, this was
due to some design changes introduced by python 3.7 [1] in the SSL
module of the stdlib. These changes reactivate some of them (some other
are still skipped and needs further works).

Indeed, when we try to use requests with SSL in a monkey patched
environment we faced the following issue:

```
TypeError: wrap_socket() got an unexpected keyword argument '_context'
```

This is due to the fact that we are in a monkey patched environment
where `requests` is monkey patched too.

We don't need `request` for our needs. Indeed we can easily send
http requests through low level socket. Our main goal is to test
our wsgi server and not to test the `requests` library, and `requests`
was just used to make the code more simpler.

In our case we can implement a code dedicated to send request to our green
server, unlock our tests and move away from this bug/side effect.

These changes move away from `requests` which is badly monkey patched by
eventlet [1]. Now we use monkey patched socket and ssl to
request the green server which is executed in background. Low level
(monkey patched) modules could help us to skirt layers that are possibly
badly monkey patched on higher level modules (urllib, requests, etc...).

[1] https://github.com/eventlet/eventlet/issues/526#issuecomment-482694279
[2] https://github.com/eventlet/eventlet/issues/526

Change-Id: Iacdde51d2d923bafa3263fa3dc12de8d501d471a
Partial-Bug: #1482633
2020-06-16 13:35:48 +02:00
Hervé Beraud bfc8172942 Fix some SSL tests for wsgi module under python 3
Previously some tests were ignored under python 3 environment, this was
due to some design changes introduced by python 3.7 [1] in the SSL
module of the stdlib. These changes reactivate some of them (some other
are still skipped and needs further works).

Indeed, when we try to use requests with SSL in a monkey patched
environment we faced the following issue:

```
TypeError: wrap_socket() got an unexpected keyword argument '_context'
```

This is due to the fact that we are in a monkey patched environment
where `requests` is monkey patched too.

We don't need `request` for our needs. Indeed we can easily send
http requests through low level socket. Our main goal is to test
our wsgi server and not to test the `requests` library, and `requests`
was just used to make the code more simpler.

In our case we can implement a code dedicated to send request to our green
server, unlock our tests and move away from this bug/side effect.

These changes move away from `requests` which is badly monkey patched by
eventlet [1]. Now we use monkey patched socket and ssl to
request the green server which is executed in background. Low level
(monkey patched) modules could help us to skirt layers that are possibly
badly monkey patched on higher level modules (urllib, requests, etc...).

[1] https://github.com/eventlet/eventlet/issues/526#issuecomment-482694279
[2] https://github.com/eventlet/eventlet/issues/526

Change-Id: Id44ad12a1cf3fd7090a67bb6e8e42bfdc47502cf
Partial-Bug: #1482633
2020-06-16 12:58:13 +02:00
Hervé Beraud d6f72ec70b Stop to use the __future__ module.
The __future__ module [1] was used in this context to ensure compatibility
between python 2 and python 3.

We previously dropped the support of python 2.7 [2] and now we only support
python 3 so we don't need to continue to use this module and the imports
listed below.

Imports commonly used and their related PEPs:
- `division` is related to PEP 238 [3]
- `print_function` is related to PEP 3105 [4]
- `unicode_literals` is related to PEP 3112 [5]
- `with_statement` is related to PEP 343 [6]
- `absolute_import` is related to PEP 328 [7]

[1] https://docs.python.org/3/library/__future__.html
[2] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html
[3] https://www.python.org/dev/peps/pep-0238
[4] https://www.python.org/dev/peps/pep-3105
[5] https://www.python.org/dev/peps/pep-3112
[6] https://www.python.org/dev/peps/pep-0343
[7] https://www.python.org/dev/peps/pep-0328

Change-Id: I42e2fc25eaec2e79540c3ab60d8752141637f7b8
2020-06-02 20:40:55 +02:00
Hervé Beraud d08727effa Drop six usages
Support of python 2.7 was dropped during the Ussuri cycle so we can now
drop six usages too.

Change-Id: I56a9c0641fd9bce613943ff938d33178d263eedf
2020-05-26 15:20:01 +02:00
Hervé Beraud b9f8224006 Align tests with monkey patch original current_thread _active
Recent changes [1] introduced an eventlet fix to monkey patch original
current_thread _active.

The goal was to monkey patch the original current_thread to use the
up-to-date _active global variable. This solution is based on that
documented at: eventlet/eventlet#592

I think we need this patch on unit test too to ensure a consistent
behavior, so these changes doing that.

[1] https://review.opendev.org/#/c/725359/

Change-Id: I7b6cca86e44261bf2f953be74e9738ac09507649
2020-05-11 15:58:22 +02:00
Sean McGinnis 6793fe15b3
Remove monotonic usage
The monotonic package was needed for monotonic time operations when
running under Python runtimes older than 3.3. Since we now only support
versions higher than this, this third party package requirement can now
be removed.

Change-Id: Ie99411882822a5db47462fcb2748c6e978587b8c
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-05 09:37:02 -05:00
Sean McGinnis 7d3578827d
Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: Ic18f03681c14131217e3025e2cb0cca46ada84ee
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-20 09:29:01 -05:00
Bernhard M. Wiedemann af8e8c2593 Extend test cert validity to 2049
to make tests pass after 2023

Background:
As part of my work on reproducible builds for openSUSE, I check
that software still gives identical build results in the future.
The usual offset is +15 years, because that is how long I expect
some software will be used in some places.
This showed up failing tests in our package build.

See https://reproducible-builds.org/ for why this matters.

Note: I tested that i586 builds still pass tests on 2037-12-01
so this change introduces no year2038 problem.

Change-Id: I0110914432c4b557dcabcac0fad1ba54bcd8be85
2019-10-10 10:26:00 +02:00
Zuul ac29c82699 Merge "Make PID availabe as formatstring in backdoor path" 2019-05-30 01:29:43 +00:00
Sebastian Lohff dd174fbfde Make PID availabe as formatstring in backdoor path
When multiple processes are spawned with the same configuration
each process has the same backdoor_socket path configured and
only the first process able to bind to the socket can later be
accessed via the backdoor. To give each process a unique socket path
we now expose the PID of the process as a format string argument,
which can then be used like this:

backdoor_socket = /var/lib/neutron/backdoor-{pid}

Change-Id: I3f86f4867eb0cd5010abadf68620aa3450d3e64d
2019-05-22 16:21:44 +02:00
chenke 60e160367d Add workers' type check before launching the services
If user set the type of worker to str or others, there will
be no error here, but a lot of processes will be created,
which may cause the system to crash.

I recommend adding a type check here.

Change-Id: I3ad0f7ec59f29a3106d23b057327c3dfef19a98f
2019-05-09 18:23:07 +02:00
Zuul 3c645a77ec Merge "Avoid eventlet_backdoor listing on same port" 2019-01-17 13:52:22 +00:00
Ben Nemec 4c0d4490e8 Use eventletutils Event class
Instead of having a copy-pasted version in this project, let's just
use the original directly. It is added to the public API of
oslo.utils in the dependency.

Depends-On: https://review.openstack.org/614806
Change-Id: If0dfac2505d097c117ef94c99399b1614f1e1f8f
2019-01-10 22:06:11 +00:00
venkata anil 811650783d Avoid eventlet_backdoor listing on same port
Oslo.service is binding to same port when we provide a port range
as eventlet is internally setting SO_REUSEPORT flag (starting from
eventlet version v0.20). And there is a flag (reuse_port)
introduced in v0.22 to give control to user to avoid SO_REUSEPORT.
In this patch, first we try passing reuse_port=False, if this fails
then directly open socket and listen instead of eventlist.listen.

Closes-Bug: #1810280
Change-Id: Idc842acc7e430199c76fe12785b0bf0e7a58e121
2019-01-03 23:41:30 -05:00
Zuul d987a4a84c Merge "Actually test child SIGHUP signal" 2018-12-24 10:42:49 +00:00
Zuul 230aa89477 Merge "Add stop_on_exception to TG timers" 2018-12-19 14:15:34 +00:00
Zuul b7e697ae4b Merge "Add better timer APIs to ThreadGroup" 2018-12-19 14:15:34 +00:00
Zuul 2c44e7fe43 Merge "Restore correct signal handling in Python3" 2018-12-19 14:10:30 +00:00
Zane Bitter 130e49feac Actually test child SIGHUP signal
The intention of this test was to wait 5s after sending SIGHUP to a
child process to make sure that it doesn't exit. However, due to a logic
error, it just stopped checking and declared success immediately. Fix
the logic so that we have a better chance of seeing if SIGHUP
incorrectly kills the process.

Change-Id: I1f320a8dfdd7a922b461d070491ad53e6cd2b20d
Related-Bug: #1803731
2018-12-10 20:46:28 +13:00
Zane Bitter 159ef2e1d2 Restore correct signal handling in Python3
The patch 2ee3894f49 broke the original
fix cad75e4e13 that ensured eventlet could
be interrupted while sleeping after PEP475 was implemented in Python
3.5. Eventlet monkey-patches the signal module with its own version, so
we have to look up the original module to determine whether the
underlying OS actually supports the poll() function.

Change-Id: Ia712c9a83d8081bf0b5e6fe36f169f9028aae3dc
Closes-Bug: #1803731
Related-Bug: #1788022
Related-Bug: #1705047
2018-12-10 20:46:28 +13:00
Duc Truong 48c51fe9cd Add stop_on_exception to TG timers
ThreadGroup add_dynamic_timer_args and add_timer_args use
DynamicLoopingCall and FixedIntervalLoopingCall respectively.  Both
classes have support for stop_on_exception, but this parameter was not
exposed in ThreadGroup functions to create timers.

This change adds the missing stop_on_exception to the timer functions so
that ThreadGroup timers can continue on exceptions if the user chooses
to do so.

Change-Id: If03276f290e86e95ddc0b1d749b7460ed752b8ef
Co-Authored-By: Zane Bitter <zbitter@redhat.com>
2018-12-10 11:13:00 +13:00
Zane Bitter 750b51caaa Add better timer APIs to ThreadGroup
The ThreadGroup.add_timer() API is unintuitive, inflexible, and all
around pretty terrible. By allowing the caller to pass *args and
**kwargs, it strongly implies that you can write a wrapper like:

    def add_timer(self, interval, func, *args, **kwargs):
        self.group.add_timer(interval, func, *args, **kwargs)

and in fact at least 6 projects have done so (probably copying and
pasting from each other). But this is wrong, and will result in the
first positional arg intended for the callback function to be treated as
the initial_delay parameter and dropped from the list of arguments to be
passed to the callback when it is run. When called like this, the
initial_delay argument not only must be passed (preventing the caller
from relying on the default), interspersed between the callback function
and its arguments, but it must be passed as a positional and not a
keyword argument (preventing the caller from effectively documenting
what is going on):

        self.group.add_timer(interval, func, None, *args, **kwargs)

We are also unable to add any further options without breaking existing
consumers.

To improve the situation in the future, add a new add_timer_args() API
that takes the args and kwargs as individual sequence/mapping parameters
rather than as variadic parameters. The above call would become:

        self.group.add_timer_args(interval, func, args, kwargs)

and any optional parameters can be passed as keyword arguments. Any new
keyword arguments we might want to add can be safely added to this
method.

Calling the original add_timer() method with arguments (either
positional or keyword) intended for the callback function is now
deprecated. Those parameters could be removed in a future major release.

Change-Id: Ib2791342263e2b88c045bcc92adc8160f57a0ed6
2018-12-10 11:09:48 +13:00
Eric Fried d9855f0ec4 Use SleepFixture in looping call test suite
A recent change [1] created a SleepFixture tailored for consumers of the
loopingcall module to be able to avoid incurring real sleeps while
testing. This patch makes use of the fixture in a couple of
previously-long-running tests in test_loopingcall.

NB: We carefully use the fixture only in specific test cases rather than
over the entire test class because tests in this module may (now and in
the future) be testing things below the mock.

=======
BEFORE:
=======
======
Totals
======
Ran: 31 tests in 13.0000 sec.
 - Passed: 31
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 14.4619 sec.

==============
Worker Balance
==============
 - Worker 0 (1 tests) => 0:00:00.306611
 - Worker 1 (1 tests) => 0:00:00.036168
 - Worker 2 (1 tests) => 0:00:00.029992
 - Worker 3 (4 tests) => 0:00:00.007588
 - Worker 4 (5 tests) => 0:00:00.019243
 - Worker 5 (6 tests) => 0:00:00.017683
 - Worker 6 (7 tests) => 0:00:12.036425
 - Worker 7 (6 tests) => 0:00:02.019783
Test id                                                                                          Runtime (s)
-----------------------------------------------------------------------------------------------  -----------
oslo_service.tests.test_loopingcall.DynamicLoopingCallTestCase.test_repeat                       12.019
oslo_service.tests.test_loopingcall.LoopingCallTestCase.test_do_not_stop_on_exception             1.005
oslo_service.tests.test_loopingcall.LoopingCallTestCase.test_repeat                               1.004
oslo_service.tests.test_loopingcall.LoopingCallTestCase.test_looping_call_timed_out               0.307
oslo_service.tests.test_loopingcall.RetryDecoratorTest.test_retry_with_expected_exceptions        0.036
oslo_service.tests.test_loopingcall.LoopingCallTestCase.test_interval_adjustment                  0.030
oslo_service.tests.test_loopingcall.DynamicLoopingCallTestCase.test_do_not_stop_on_exception      0.007
oslo_service.tests.test_loopingcall.TestBackOffLoopingCall.test_exponential_backoff               0.005
oslo_service.tests.test_loopingcall.DynamicLoopingCallTestCase.test_no_double_start               0.005
oslo_service.tests.test_loopingcall.DynamicLoopingCallTestCase.test_timeout_task_without_return   0.005
_____________________________________________________________________ summary _____________________________________________________________________

======
AFTER:
======
======
Totals
======
Ran: 31 tests in 1.0000 sec.
 - Passed: 31
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 0.4553 sec.

==============
Worker Balance
==============
 - Worker 0 (1 tests) => 0:00:00.005381
 - Worker 1 (1 tests) => 0:00:00.006639
 - Worker 2 (1 tests) => 0:00:00.005145
 - Worker 3 (1 tests) => 0:00:00.308991
 - Worker 4 (1 tests) => 0:00:00.037721
 - Worker 5 (2 tests) => 0:00:00.031293
 - Worker 6 (12 tests) => 0:00:00.034781
 - Worker 7 (12 tests) => 0:00:00.031908
Test id                                                                                             Runtime (s)
--------------------------------------------------------------------------------------------------  -----------
oslo_service.tests.test_loopingcall.LoopingCallTestCase.test_looping_call_timed_out                 0.309
oslo_service.tests.test_loopingcall.RetryDecoratorTest.test_retry_with_expected_exceptions          0.038
oslo_service.tests.test_loopingcall.LoopingCallTestCase.test_interval_adjustment                    0.028
oslo_service.tests.test_loopingcall.DynamicLoopingCallTestCase.test_initial_delay                   0.013
oslo_service.tests.test_loopingcall.LoopingCallTestCase.test_do_not_stop_on_exception               0.007
oslo_service.tests.test_loopingcall.DynamicLoopingCallTestCase.test_do_not_stop_on_exception        0.007
oslo_service.tests.test_loopingcall.DynamicLoopingCallTestCase.test_repeat                          0.005
oslo_service.tests.test_loopingcall.TestBackOffLoopingCall.test_exponential_backoff_negative_value  0.005
oslo_service.tests.test_loopingcall.LoopingCallTestCase.test_repeat                                 0.005
oslo_service.tests.test_loopingcall.RetryDecoratorTest.test_retry_with_max_retries                  0.004
_____________________________________________________________________ summary _____________________________________________________________________

[1] I0089c7778957456db66599abffaaad3a5332243c

Change-Id: I7035abef8d089c9d4cf664daee0af0fde5a6e19c
2018-11-07 17:21:43 -06:00
Eric Fried b85d9353fb Fixture to mock loopingcall wait()
Consumers of loopingcall may wish to exercise their code in test cases
without incurring actual wall clock sleep time in their test runs.
Heretofore, this required digging into the internals of the loopingcall
module and mocking something private (or something hidden by something
private).

This patch exposes a public oslo_service.fixture.SleepFixture for this
purpose. It can be maintained opaquely as internals change without
affecting its consumers.

See [1] for (one example of) the motivation behind this change.

[1] https://review.openstack.org/#/c/615724/

Change-Id: I0089c7778957456db66599abffaaad3a5332243c
2018-11-07 09:34:51 -06:00
Slawek Kaplonski 3e08f3375e Fix stop of loopingcall
Patch [1] for switched to use eventlet Event for loopingcall events.

It may now happen that stop event is sent when other event was
already sent and loopingcall is already not running.
That cause AssertionError in eventlet.event module.

To avoid that, we should check if if loopingcall is
running before sending _abort.set().

[1] https://review.openstack.org/#/c/611807/

Closes-Bug #1800879

Change-Id: I28ad3bdb51a20350c90dee4420058c30946897e5
2018-10-31 23:19:17 +01:00
Thomas Herve c2463470f5 Use eventlet Event for loopingcall events
This fixes broken logic for finding out how to use green threads. Using
threading.Event with eventlet creates useless load related to timers.

Change-Id: I62e9f1a7cde8846be368fbec58b8e0825ce02079
Closes-Bug: #1798774
2018-10-22 16:36:40 +02:00
Ben Nemec 2705800cd3 Ensure connection is active in graceful shutdown tests
Recent versions of eventlet close idle connections, so just opening
a connection is no longer sufficient to trigger graceful shutdown
behavior. This change sends a request and adds a delay on the server
side so the connection will be active throughout the test.

Change-Id: I4e908c1fb1a61d8a57769e2aa85c02a21722367b
Closes-Bug: 1788959
2018-08-31 17:08:14 +00:00
Ben Nemec a850fda322 Stop asserting on Eventlet internals
Eventlet has used at least three different clock implementations by
default over the past couple of years (time.time, vendored monotonic,
and plain monotonic). Since we're forcing use of monotonic anyway,
this is all irrelevant to us outside of the fact that once we depend
on a new enough version of eventlet we can remove our monkey patching
of the hub. Asserting on the particular module that eventlet uses by
default is just breaking our unit tests on newer eventlet for no
good reason.

Change-Id: I0f5135567a16dde4c34cee35baa2054091dce728
Partial-Bug: 1788959
2018-08-30 22:33:47 +00:00
Claudiu Belu 2ee3894f49 Skips signal handling on Windows
Setting up signal handling fails on Windows due to missing select
module attributes. This patch skips the setup on Windows.

Change-Id: I75db453ec907efb01e1bb411690dadd92a48314d
Closes-Bug: #1788022
2018-08-23 17:00:02 -07:00
Charles Short 135ab8f83a Remove moxstubout
Newer versions of oslo.service will have moxstubout
deprecated, so get out in front of the curve and remove it.

Change-Id: I74369b90c3d9352e1c59062bda8410a09675b40c
Signed-off-by: Charles Short <zulcss@gmail.com>
2018-05-19 14:26:50 +00:00
TommyLike 20a2178171 [ThreadGroup] Don't remove timer when stop timers
Graceful exit is not working since we removed the timers
before waiting them to complete.

Closes-Bug: #1769825
Change-Id: I2e52b680a09d337d9b598a4826132df8c7b0b4d9
2018-05-09 00:54:54 +00:00
Zane Bitter 753253772a Python 3: Fix non-deterministic test
The order that kill() is called on the child processes of a service is
determined by iterating over a dict, so don't hard-code a particular
expected order in the test. Doing that caused the test to be broken on
Python 3.

Change-Id: I4ca85cc8f559985b133e523cd00f297e5576d00a
2018-05-03 17:18:22 -04:00
Davanum Srinivas (dims) 9091986228 Revert "Revert "Permit aborting loopingcall while sleeping""
This reverts commit 5975da493b.

Added code to support the case where unit tests are not being
monkey patched (example heat).

Change-Id: If715fbe21ac085e4f5c83cef0729dbca8dcb19ca
2018-04-13 13:53:28 -04:00
Davanum Srinivas 94bc9ceef1 Fixup certificates and skip failing test
Using cfssl to regenerate fresh certificates:
rm *
echo '{"CN":"CA","key":{"algo":"rsa","size":2048}}' | cfssl gencert
-initca - | cfssljson -bare ca -
echo '{"signing":{"default":{"expiry":"43800h","usages":["signing","key
encipherment","server auth","client auth"]}}}' > ca-config.json
export ADDRESS=127.0.0.1,::1,localhost
export NAME=server
echo
'{"CN":"'$NAME'","hosts":["127.0.0.1"],"key":{"algo":"rsa","size":2048}}'
| cfssl gencert -config=ca-config.json -ca=ca.pem -ca-key=ca-key.pem
-hostname="$ADDRESS" - | cfssljson -bare $NAME
export ADDRESS=
export NAME=client
echo
'{"CN":"'$NAME'","hosts":["127.0.0.1"],"key":{"algo":"rsa","size":2048}}'
| cfssl gencert -config=ca-config.json -ca=ca.pem -ca-key=ca-key.pem
-hostname="$ADDRESS" - | cfssljson -bare $NAME
cp ca.pem ca.crt ;
cp ca-key.pem ca.key;
cp client.pem certificate.crt ;
cp client-key.pem privatekey.key ;

Also the results from the related bug show that there is currently no
solution to make self-signed certificates work in conjunction with
raw IPv6 addresses consistently, so we skip the failing test.

Related-Bug: 1656329
Change-Id: I160108059e17122035d8d914c5157b7ce3ada3a5
2018-04-10 14:47:19 +00:00
ChangBo Guo(gcb) 5975da493b Revert "Permit aborting loopingcall while sleeping"
This reverts commit ba28d511e0.

The original change breaks trove and vitrage.

Change-Id: Icb39a334b82834b592932facf93be0687563c316
2017-12-15 08:32:24 +00:00
Allain Legacy ba28d511e0 Permit aborting loopingcall while sleeping
Some of the openstack services implement worker tasks that are based on
the oslo-service LoopingCallBase objects.  They do this as a way to have
a task that runs periodically as a greenthread within a child worker
process.  For example, the neutron-server runs AgentStatusCheckWorker()
objects as base service workers in its child worker processes.

When the parent server process handles a SIGTERM signal it attempts to
stop all services launched on each of the child worker processes (i.e.,
ProcessLauncher.stop()).  That results in a stop() being called on each
of the underlying base services and then a wait() to ensure that they
complete before shutdown.

If any service that is implemented on a LoopingCallBase related object
is suspended on a greenthread.sleep() the previous call to stop() will
have no effect and so the wait() will block until the sleep() finishes.
For tasks that either have a frequent FixedLoopingBase interface or a
short initial_delay this may not be a problem, but for those with a long
delay this could mean that the wait() blocks for minutes before the
process is allowed to shutdown.

To solve this the LoopingCallBase calls to greenthread.sleep() are being
replaced with a threading.Event() object's wait() method.  This allows a
caller of stop() to interrupt the sleep and expedite the shutdown.

Closes-Bug: #1660210

Change-Id: I5835f9595826df5349e4cc8b1da8529bb960ee04
Signed-off-by: Allain Legacy <allain.legacy@windriver.com>
2017-06-02 14:18:07 -04:00
melanie witt 8f67ed7d53 Add min_interval to BackOffLoopingCall
The backoff timer has a few issues that can cause it to get stuck
in an infinite loop and never time out.

  1. The random.gauss() function used to generate random jitter can
     return negative values, so when it does, it makes the elapsed time
     self._error_time go "backward."
  2. The random jitter is used as a multiplier for the self._interval,
     so self._interval can deviate far away from the mean over time and
     walk to zero, causing self._interval to be 0, which will prevent
     the timer from making progress from that point on because idle
     will always evaluate to zero and the elapsed time won't increase.
  3. The evaluated interval doesn't have a lower bound, so over time
     it can get extremely small if jitter (the mean) < 0.5.

This adds a min_interval keyword argument to the BackOffLoopingCall
start() function that defaults to 0.001s and uses it to lower bound
the interval calculations. We'll also take the absolute value of the
return from random.gauss() to prevent elapsed time going backward, and
we'll calculate the running self._interval separately to make it track
the desired growth rate of the backoff and not let it drift with the
random.gauss() values.

Closes-Bug: #1686159

Change-Id: Id17668a34d5cedbe870c9056350a7e9c7196faa7
2017-04-25 17:13:46 +00:00
Duan Jiong add1b72780 Fix some reST field lists in docstrings
Probably the most common format for documenting arguments is reST field
lists [1]. This change updates some docstrings to comply with the field
lists syntax.

[1] http://sphinx-doc.org/domains.html#info-field-lists

Change-Id: I242b83c5713cc1faa30dcdffa2d989fb9a63b3ef
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
2017-03-22 13:24:21 +00:00
ChangBo Guo(gcb) 4ca23440a1 Remove duplicated register_opts call
We have registered all config options in method setUp of
ServiceBaseTestCase, don't need repeat them again in subclass.

Change-Id: Iad4452dc3e7d353a7e963547aee3a356afdf5ad5
2017-01-28 22:24:01 +08:00
Wenzhi Yu d05e086b8d Add FixedIntervalWithTimeoutLoopingCall
Currently when using FixedIntervalLoopingCall, folks need to
add timeout checking logic in their function if they need it.
Adding a new class FixedIntervalWithTimeoutLoopingCall to
provide timeout checking support will save those effort.

Change-Id: I78bfb9e259c2394137d7efbc0ee96bb18a6dc5e7
2016-12-29 14:20:29 +08:00
Jenkins 42adde23f7 Merge "Stay alive on double SIGHUP" 2016-10-17 03:39:16 +00:00
gecong1973 48bebc751f Replace 'MagicMock' with 'Mock'
MagicMock magically allow things to pass. Any usage should
be replaced with the Mock class and explicit attributes
should be set on it. [1]
[1] http://stackoverflow.com/questions/17181687/mock-vs-magicmock

Change-Id: I09866096a9b8f02b61153c127a11afed7196f40a
2016-10-11 09:38:45 +08:00