Increase rabbitmq tcp backlog

From https://bugzilla.redhat.com/show_bug.cgi?id=1778428

We need to tune the default rabbitmq tcp listen backlog.  Currently it defaults
to 128, but here's what happens:
Say we have 1500 total rabbitmq client connections spread across a 3 node
cluster, evenly distributed so each node has 500 clients.

Then, we stop rabbitmq on one of the nodes.

Now those 500 client connections all immediately fail over to the other two
node.  Assume roughly even split, and each gets 250 connections simultaneously.
Since the tcp listen backlog is only 128, a large number of the failover
connections cannot connect and get ECONNREFUSED because the kernel just drops
them.

Eventually things retry and the backlog clears, but it just makes things noisy
in the logs and makes failover take a little bit longer.

Upstream docs discuss here:
https://www.rabbitmq.com/networking.html#tuning-for-large-number-of-connections-connection-backlog

Suggested-By: John Eckersberg <jeckersb@redhat.com>
Closes-Bug: #1854704

Change-Id: If6da4aff016db9a72e1cb9dfc9731f06e062f64d
(cherry picked from commit 9f4832fcc4)
This commit is contained in:
Michele Baldessari 2019-12-02 09:03:06 +01:00
parent 6978a62d40
commit e788989198
1 changed files with 1 additions and 0 deletions

View File

@ -126,6 +126,7 @@ outputs:
rabbitmq::wipe_db_on_cookie_change: true
rabbitmq::port: 5672
rabbitmq::loopback_users: []
rabbitmq::tcp_backlog: 4096
rabbitmq::package_provider: yum
rabbitmq::package_source: undef
rabbitmq::repos_ensure: false