In the abstract DNSService's _dns_handle_tcp method, error handling
is broken in a way that stops the main loop for handling TCP
connections.
Because socket.timeout is a subclass of socket.error, the error
handling block for socket.timeout is never reached.
Because of this, error handling of a TCP timeout is sent to the
socket.error block. Due to the way eventlet hijacks these errors,
the errorcode is not available and a KeyError is raised. This
KeyError interferes with the main loop because it is not caught.
Further improvement may include ensuring that these main loops
can never die due to unexpected exceptions.
Many thanks to Erik Andersson for pointing out the issue, which
was seemingly innocuous but ended up being the cause of our
problems.
Closes-bug: 1549980
Change-Id: I47e1260a0818cc42cbd56e4d296e083f8fcbbae5
d5d0706705