Update git submodules
* Update masakari-monitors from branch 'master' to 6e84289a530b86007550e447172b9bdb0f9c149b - Merge "Fix hostmonitor hanging forever after certain exceptions" - Fix hostmonitor hanging forever after certain exceptions The hostmonitor, like other Masakari monitors, starts as an Oslo service (based on eventlet). The main thread is supposed to run a loop that has an internal wait mechanism (instead of reusing periodic_tasks from oslo_service). However, the loop could be broken, if an unexpected exception appeared, and it never ran again but the process was still alive (due to oslo_service not stopping). The example mentioned in the bug report is about unavailability of the Masakari API (and/or Keystone API) before notification sending. This exception is not caught early because SendNotification._make_client is called outside of the try block (unlike the actual notification sending). The exception bubbles up and stops the main loop, leaving a useless hostmonitor process. The user is unaware unless they notice the logs are no longer growing. While the general design begs for a revamp (we might get away with that by using Consul in the first place), the easy fix is to prevent exceptions breaking the loop completely so that the hostmonitor can continue to work and try to regain health. At the very least it will keep posting ERROR messages in the log which is more likely to be spotted in comparison to lack of logs (which is, unfortunately, less commonly considered an alerting situation). This change also fixes, adapts and robustifies the two relevant unit tests. Closes-Bug: #1930361 Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com> Change-Id: I7e3447dcddc7998e3e3c30f4f0019d91a99c79ce
This commit is contained in:
parent
f64ee9a90d
commit
5cc836389a
|
@ -1 +1 @@
|
|||
Subproject commit 1244861394a29549ec17f0b31fbd5a4aee39863f
|
||||
Subproject commit 6e84289a530b86007550e447172b9bdb0f9c149b
|
Loading…
Reference in New Issue