openstack
/
puppet-gnocchi
Code Issues Proposed changes

Provide information related to haproxy and mysql 

When using HA Proxy to front MySQL, the issue becomes that the way
the client/server handle the connection it will cause HA Proxy to
leave ports in TIME_WAIT and then exhaust all available TCP ports.
The information provided will give insight on the issue and also ways
to tune HA Proxy to handle the high connections to MySQL.

Change-Id: I427fa5ff8249387362911dd2637b4c5002e7177d
This commit is contained in:
Matthew J. Black 2016-09-13 10:37:06 -04:00
parent bbd0b178f4
commit ace794b5fb
1 changed files with 14 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
README.md
View File

@ -71,6 +71,20 @@ If value is equal to ensure_absent_val then the resource will behave as if `ensu
Limitations
-----------
### Load balancing Gnocchi MySQL with HAProxy
#### Issue
MySQL client/server interaction causes an issue where the HAProxy server will keep a connection in TIME_WAIT. When Gnocchi is processing data it will generate a lot of connections to MySQL and [exhaust all available tcp ports](http://blog.haproxy.com/2012/12/12/haproxy-high-mysql-request-rate-and-tcp-source-port-exhaustion/) for a given IP address. If the HAProxy VIP is shared with other components, this can cause them to be unavailable too. Tuning of HAProxy instance is essential when using Gnocchi with a MySQL behind an HAProxy.
The sysctl parameters need tuning.
* net.ipv4.tcp_tw_reuse = 1
* net.core.somaxconn = 4096
* net.ipv4.tcp_max_syn_backlog = 60000
Additionally, HAProxy can be configured to use different source IP addresses on each backend to help further mitigate the issue.
Development
-----------