This change formalises locking in MessageHandlingServer, which closes
several bugs:
* It adds locking for internal state when using the blocking executor,
which closes a number of races.
* It does not hold a lock while executing server functions,
which removes a potential cause of deadlock if the server does its
own locking.
* It fixes a regression introduced in change
gI3cfbe1bf02d451e379b1dcc23dacb0139c03be76. If multiple threads
called wait() simultaneously, only 1 of them would wait and the
others would return immediately, despite message handling not having
completed. With this change only 1 will call the underlying wait,
but all will wait on its completion.
Additionally, it introduces some new functionality:
* It allows the user to make calls in any order and it will ensure,
with locking, that these will be reordered appropriately.
* The caller can pass a `timeout` argument to any server method, which
will cause it to raise an exception if it waits too long.
* The caller can pass a `log_after` argument to any server method,
which will cause it to raise a log message if it waits too long. It
can also be used to disable logging when waiting is intentional.
We remove DummyCondition as it no longer has any users.
This change was originally committed as change
I9d516b208446963dcd80b75e2d5a2cecb1187efa, but was reverted as it
caused a hang in a Nova test. This was caused by the locking behaviour
for handling restarting a previously stopped server. The original
patch caused the state to 'wrap' immediately after the user called
wait(). This caused a hang in tests which redundantly called stop()
and wait() multiple times. This new patch only wraps when the user
calls start() again. Callers who do not restart a server will
therefore not be affected by the wrapping behaviour. Callers who do
restart a server will be no worse than before. We add a deprecation
warning on restart, as this operation is inherently racy with this api
and there is a simple, safe alternative.
This new version has been successfully tested against the unit and
functional tests of nova, cinder, glance, and ceilometer.
Change-Id: Ic79f87e7b069c1f62d6121486fd6cafd732fdde7
00d07f5205