Commit Graph

103 Commits

Author SHA1 Message Date
Dmitriy Rabotyagov 373b9bb0f2 Do not resolve all host_vars when haproxy_backend_node is a mapping
We do allow to supply haproxy_backend_nodes as list of mappings rather
the regular list, which supports `ip_addr`, `name` and `backend_port` keys.

However, we do verify hostvars[host_name] and try to set ip_addr regardless
if this needed or not.

During hostvars[host_name] request Ansible tries to fetch all host variables
and resolve some of them, which not always can be possible or preffered
in some scenarios.
Good example of that would be Mozilla SOPS [1] encrypted variables for
specific host or group, which can not be decrypted by some operators.
In the meanwhile they can be eligible to configure haproxy frontend/backend
for this service. So we should have a way to avoid asking for specific
hostvars when it's not needed, and backend_nodes are already contain
all required information.

[1] https://docs.ansible.com/ansible/latest/collections/community/sops/docsite/guide.html

Change-Id: I17a7f2421cd31b37bbda4f9c85971b1825e54891
2024-03-22 12:36:45 +01:00
Dmitriy Rabotyagov 9a1c483381 Imporove Jinja indentation for service templates
At the moment service templates are hardly readable, partially due to
complex logic, but incosistent presence of indetnation makes things
way worse, as there's no way to know if you're under some cycle
or condition for sure.

This patch aims to make indents correct which should improve template
readability overall.

Change-Id: Ie60ca87c044281104fbc8334d7254ac351d3d912
2024-03-21 20:05:08 +01:00
Dmitriy Rabotyagov 16ab20815f Add httpchk option when httpcheck_options are defined
In order for http-check to work, option httpchk must be loaded first. Otherwise
regular L4 check will be issued and all `http-check` will be simply ignored.

Closes-Bug: #2046223
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/903488
Change-Id: Ie9ed322ab9c4a04d42cab4456567ac5d1f5c966b
2023-12-12 11:20:24 +00:00
Dmitriy Rabotyagov 2cc2fceaf6 Fix haproxy_stats SSL path defenition
Neither `vip_interface` nor `vip_address` are defined or available in
the context they're being used.
Thus we need to refer to available variables in order to render base config
properly

Current version fail with "AnsibleUndefinedVariable: 'vip_interface' is undefined"
on "Drop base haproxy config" task.

This fix the issue that was introduced with [1] and backported back to Zed

[1] https://review.opendev.org/q/Ib8be6b7fc3dada9d20905b0f07d90ddce0335605

Change-Id: I4e52378d8c5b3eaa6863ecaf0d04554d082e3dc0
2023-11-27 18:23:11 +01:00
Christian Rohmann 04a8f8532a Enable stats to use dedicated server certificate and allow for client cert auth
Some environments use a dedicated PKI for monitoring and metric collection.
This change allows to configure the serving certificate for stats independently
by setting `haproxy_stats_ssl_cert_path`, the default is to use the same cert.

Also client certificate authentication for stats can now be enabled by defining
a CA cert via `haproxy_stats_ssl_client_cert_ca`.

Change-Id: Ib8be6b7fc3dada9d20905b0f07d90ddce0335605
2023-09-28 09:32:22 +00:00
Dmitriy Rabotyagov 67e19ebccd Add HTTP/2 support for frontends/backends
This patch implements extra variables/keys that can be used to
enable HTTP/2 protocol for frontends and backends.

With that patch does not add HTTP/2 support for any redirect frontends
since they can not be configured to use TLS and this it will
cause such redirect backends to be HTTP/2 only, which might break old
clients.

With that regular frontends, that are not terminating TLS can be
configured to be HTTP/2 only as well as TCP backends.

Change-Id: Ib14f031f3c61f31bf7aaf345a3ba635ca5fb9ff8
2023-08-23 13:24:43 +00:00
Zuul 2a60a55cee Merge "Add possibility to override haproxy_ssl_path" 2023-07-19 08:34:17 +00:00
Marc Gariepy 1d83177575 Add possibility to override haproxy_ssl_path
It's now possible to set ssl cert path in case you want to bind to
specific hostname via ``haproxy_bind`` and want to share a common
certificate. set ``haproxy_ssl_path`` to override per service.

Change-Id: Ib517f52c0edbc4ac8d0df2a2ae078c9138141aae
2023-07-13 15:10:46 -04:00
Marc Gariepy 4513bc84ae Add ability to have different backend port.
Add the possibility to have multiple backend services running on
differents ports.

Change-Id: I1748bfc15bdf879f78aa06c385af7b6c45bde7ff
2023-07-13 13:18:19 -04:00
Damian Dabrowski c1be49a95c Fix service-redirect.j2 template
This change fixes service-redirect.j2 template that was not working so
far, mainly by replacing:
- 'vip_bind' with 'vip_addres'
- 'item' with 'service'

Additionally, I removed `haproxy_tcp_upgrade_backend` support because
it's not really needed after haproxy separated service config was
implemented.

I also changed variable name `haproxy_tcp_upgrade_frontend` to
`haproxy_accept_both_protocols` to better describe what exactly it does.
Release note is not needed as ``haproxy_tcp_upgrade_frontend` was not
working properly before.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/884445

Change-Id: Iba9156c5b909f7b18599638db4471bab12794f0e
2023-05-26 00:37:21 +02:00
Zuul d721633081 Merge "Fix use of haproxy_backend_ssl when haproxy_backend_ca is not defined" 2023-05-19 21:01:15 +00:00
Damian Dabrowski 8168af6635 Deprecate certbot-auto
Certbot-auto is deprecated since 2020[1] and it is no longer available
under https://dl.eff.org/certbot-auto.
This change removes certbot-auto from haproxy_server role leaving
distro method as the only available option.

[1] https://community.letsencrypt.org/t/certbot-auto-deprecated-explanation-and-solutions/139821

Change-Id: Ibe0f13fc7308359d337fb382cb72998befb90d84
2023-04-26 16:47:58 +02:00
Jonathan Rosser 79bef8773a Fix use of haproxy_backend_ssl when haproxy_backend_ca is not defined
For certificates from widely trusted CA there is no need to provide
a specific CA file for an ssl backend, but the code fails with
undefined variable if only haproxy_backend_ssl is enabled.

A workaround is to set `haproxy_backend_ca: false` but this patch
allows haproxy_backend_ssl to be used on it's own.

Change-Id: I7c87317433acb4ed73070a2252240737b22dccfc
2023-04-04 13:56:46 +01:00
Damian Dabrowski a5f285c51e Simplify haproxy_service_configs structure
For historical reasons the ``haproxy_service_configs`` variable was
a list of nested mappings with only single valid key for the top
level mapping.

There have been no use-cases for extra keys, so this patch simplifies
the code by removing one level of nesting.

Change-Id: I50c17b7020a459ab8a88b004cc8828cac857f1c9
2023-03-16 14:19:22 +01:00
Jonathan Rosser d548b7e5ff Add support for haproxy map files
HAProxy supports the use of map files for selecting backends, or
a number of other functions. See [1] and [2].

This patch adds the key `maps` for each service definition allowing
fragments of a complete map to be defined across all the services,
with each service contributing some elements to the overall map file.

The service enabled/disabled and state flags are observed to add and
remove entries from the map file, and individual map entries can also
be marked as present/absent to make inclusion conditional.

[1] https://www.haproxy.com/blog/introduction-to-haproxy-maps/
[2] https://www.haproxy.com/documentation/hapee/latest/configuration/map-files/syntax/

Change-Id: I755c18a4d33ee69c42d68a50daa63614a2b2feb7
2023-03-16 13:17:39 +01:00
Jonathan Rosser 42d80464af Allow default_backend to be specified
Currently default_backend for a service is always set to the
haproxy_service_name for a service, but this might not be what is
required for some configurations.

This patch allows haproxy_default_backend to be configured for
a service to customise the default_backend setting.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/876436
Change-Id: I9e2be37cb27a33350577a93f23b69e560493b320
2023-03-06 11:56:51 +00:00
Zuul 044d65e9bb Merge "Accept both HTTP and HTTPS also for external VIP during upgrade" 2023-03-01 00:27:41 +00:00
Damian Dabrowski a30ecbee08 Accept both HTTP and HTTPS also for external VIP during upgrade
In change [1] we have added functionality to accept both HTTP and
HTTPS during an upgrade.
However it's only limited to internal VIP. I see no reason not to
implement this also for external VIP. Some people may find it useful.

[1] https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/829899

Change-Id: I672016b75d4b514d87dbb47119ff549bbc4e923e
2023-02-21 15:23:40 +00:00
Jonathan Rosser f7c87fd118 Add a variable to allow extra raw config to be applied to all frontends
Currently this must be configured on a per-frontend basis through
service.haproxy_frontend_raw. This patch adds a new role default
variable haproxy_frontend_extra_raw which will be combined with all
per service raw config lines.

Change-Id: I506d46d64df93bbb9e6d1ebfa3d3caa44c80fdd5
2023-02-14 18:55:00 +00:00
Andrew Bonney 445b15f9c3 Fix dict object key error when haproxy interfaces not defined
The ternary options appear to be getting evaluated whether they
are used or not, so item['interface'] is always accessed.

This patch aims to check for the key's presence before performing
ternary operations, or use Ansible variables to postpone evaluation
until absolutely necessary.

Change-Id: Ib1462c04d1a0820a37998f989e2ed16566f71f54
2023-01-11 11:03:31 +00:00
Zuul fd3ba428d9 Merge "Fix warnings in haproxy config" 2022-12-10 12:16:23 +00:00
Dmitriy Rabotyagov 6532898a3c Make use of haproxy_rise and haproxy_fall variables
At the moment for some reason we're not taking into account default
variables haproxy_rise/haproxy_fall but instead trying to count
based on amount of backends. This makes quite little sense to
depend amount of backend rechecks on amount of backends overall,
so we're chaning behaviour to pre-defined variables that already exist.

Change-Id: I1e53a997f6f443718ea2c6bdfbe8a0b98c44896d
2022-12-09 15:44:29 +01:00
Damian Dabrowski 34ac0710c5 Fix warnings in haproxy config
Haproxy config check(/usr/sbin/haproxy -c -f /etc/haproxy/haproxy.cfg)
returns 3 warnings:

1. keyword 'forceclose' is deprecated in favor of 'httpclose', and will
not be supported by future versions.
2. backend 'galera-back' : 'option tcplog' directive is ignored in
backends.
3. 'http-request' rules ignored for backend 'galera-back' as they
require HTTP mode.

This change fixes 1. and 2.
Fixing 3. will be a bit more tricky as it's a part of
`openstack_haproxy_stick_table` defined in
/opt/openstack-ansible/inventory/group_vars/haproxy/haproxy.yml

Change-Id: Idaa4b5580039857435f90416924dee26a702deba
2022-12-01 16:41:04 +00:00
Dmitriy Rabotyagov 6025eaac36 Allow do disable SSL only for stats frontend
Currently there is no way of disabling SSL connection for stats frontend
as it implies more global variable. However, for some systems consuming
self-signed root certificate might be not an option and disabling
SSL verification tricky. Thus, we introduce new variable that allows to
nicely control if SSL should be served for stats frontend or not.

Change-Id: Ic4bc4393ec89469876e9e95b12bb9c4069972713
2022-10-05 11:29:50 +00:00
Zuul a502817a89 Merge "Allow haproxy to bind on the interface" 2022-09-27 21:11:49 +00:00
Zuul c1df0a5b56 Merge "Add variable for setting certbot `domains` option" 2022-09-20 16:03:26 +00:00
Danila Balagansky 1664c993b6 Add variable for setting certbot `domains` option
Add `haproxy_ssl_letsencrypt_domains` variable, which
contains a list (defaults to `external_lb_vip_address`)
for `--domains` certbot option.

Change-Id: I2ebfff9eeb5279a3964b8578a6e66aa132d763f5
2022-09-15 08:03:07 +00:00
Dmitriy Rabotyagov 901523ddbb Allow haproxy to bind on the interface
In some user scenarious (like implementing DNS RR) it might be useful to
bind on 0.0.0.0 but at the same time do not conflict with other services
that are binded to the same ports. For that, we can specify a specific
interface, on which haproxy will be binded to 0.0.0.0.

In netstat it would be represented like `0.0.0.0%br-mgmt:5000`.

With that we also allow to fully override `vip_binds` if assumtions
that role make are not valid for some reason.

Change-Id: Ic4c58ef53abc5f454b6fbebbd87292a932d173ae
2022-08-09 18:47:29 +00:00
Jonathan Rosser 06e76706c7 Allow customisation of stick-tables for each service.
A new variable "haproxy_stick_table" is added which allows a custom
stick-table to be supplied that is used as the default stick-table
for all haproxy back-ends.

In addition, the variable service.haproxy_stick_table can be defined
for each service to allow a unique stick-table to be supplied for
a particular service.

The old default stick-table definition is removed as there was no
use case defined for it in this role before. An example is added
to defaults/main.yml to show how the custom stick-table can be used
to rate-limit requests that generate 4xx responses which commonly
occur during vulnerability scanning or credential stuffing attacks.
There are many other uses for stick-tables, consult the HAProxy
documentation for details.

Change-Id: I50daba08c10f071157d6450ea2fa97df448f99ec
2022-07-11 14:54:01 +00:00
Dmitriy Rabotyagov be9a66c280 Don't restrict haproxy tunable options
Instead of hardcoding specific supported tunable options, we
just pass key as an option to haproxy config.

This change might break deployments during upgrades, since format of
values in variable has changed, but appropriate release note was written

We also increase maxrewrite by default, as otherwise usage of CSP leads
to 500 error.

Change-Id: I949960420ed5dbd6d58f0de7dae0ac629a85b7fc
Related-Bug: https://github.com/haproxy/haproxy/issues/1597
Needed-By: https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/844815
2022-06-20 08:27:40 +02:00
JamesGibo d30bb2e6d1 Add functionality to accept both HTTP and HTTPS during upgrade
Enable TLS on internal communication has 2 parts
* Enabling TLS on the internal VIPs for haproxy frontends
* Enabling TLS on the service backends
Haproxy has support for enabling TLS on frontends and backends,
but doing so would cause downtime.

In the case of upgrading frontends, enabling TLS would prevent
openstack services from working until their config is changed
from http to https, as they do not follow redirects.

In the case of backends haproxy would mark each backend as down
because if could not initiate a HTTPS connection to the backend
until the backend is updated.

This patch fixes this and allows haproxy to accept both HTTP and
HTTPS on the same well known port for each service. It also
allows for both HTTP and HTTPS backends.

Support for HTTP and HTTPS on the frontend is enabled by setting
haproxy_tcp_upgrade_frontend: true

Support for HTTP and HTTPS on the backend is enabled by setting
haproxy_tcp_upgrade_backend: true

This is a temporary patch and will be removed once instances have
been upgraded to HTTPS for internal communications in a future
release of OSA.

Change-Id: I4279005d5b4e6133cf85ba43379b51149c838f17
2022-02-18 14:40:14 +00:00
Zuul ab0c91f810 Merge "Adjust default configuration to support TLS v1.3" 2022-01-28 18:56:35 +00:00
Andrew Bonney 0aeaeb590a Adjust default configuration to support TLS v1.3
This adds TLS v1.3 support to the HAProxy role by default, along
with a new variable to manage cipher suites.

The old variable for TLS v1.2 and below ciphers is renamed for
consistency, but is still supported as a default where overridden
by deployments.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/823943
Change-Id: Iaf9709ac5f5ac8db281a9ec7278cef274186ba15
2022-01-10 08:57:40 +00:00
James Gibson 92dac326a9 Add default CA store to use when haproxy_backend_ca is true
If haproxy_backend_ca set to true, default to using system CA
so you dont have to specify the exact CA to use.

Change-Id: I536c32a0b152a2b754787e07574472ecfaebd7e7
2021-12-15 14:06:25 +00:00
James Gibson 33dbb82bbb Add option to force encryption of all health checks over SSL
This is required if a server only speaks HTTPS

Change-Id: Ib99eed929dfded2bbf11bc1a54c4184edafe8452
2021-12-06 17:22:43 +00:00
Zuul f8b7ce18a2 Merge "Fix inconsistency in haproxy_frontend_raw naming" 2021-11-22 18:30:50 +00:00
James Gibson 9b1fd69588 Remove httplog directive from backends as ignored
Option httplog directive is ignored in backends
http://cbonte.github.io/haproxy-dconv/2.0/configuration.html#4-option%20httplog

Change-Id: I792f22b8e20b3ca34be78a9c87ca0f1050f9225c
2021-11-12 10:46:47 +00:00
Dmitriy Rabotyagov afa12fbc97 Fix inconsistency in haproxy_frontend_raw naming
We have announced and documented haproxy_frontend_raw variable
while in fact introduced haproxy_raw. Since announced variable name
better reflects its purpose and it was announced,
we rename it to haproxy_frontend_raw in template generation.

Change-Id: I7ba9588b599f42dddad395df1a2e18ccfe6b3fe3
2021-11-12 12:29:57 +02:00
Dmitriy Rabotyagov 1195355b43 Fix haproxy Let's Encrypt SSL path
With releasing PKI role we broke Let's Encrypt option because of
changing directories where certs should be located
and not reflecting these changes for let's encrypt. At the same time
we should not generate self-signed cert when let's encrypt path is used.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/811742
Closes-Bug: #1938961
Change-Id: I1a6701b171782528373bc1d0a39e70e6d1ef20ab
2021-10-05 15:08:40 +01:00
Andrew Bonney 964a33558b Add configuration option for native prometheus exporter
HAProxy added native Prometheus support from v2.0. This can be
enabled using the existing stats endpoint via an additional
/metrics path.

Change-Id: If9528969c7915db06138c0746dc419d8302f0e7c
2021-07-30 09:07:50 +01:00
Zuul 1dc4fa621c Merge "Generate self-signed SSL per listen IP" 2021-06-25 17:21:17 +00:00
Dmitriy Rabotyagov f14ba91798 Generate self-signed SSL per listen IP
We're providing an option to have an IP address per VIP
address. Currently it's used only for creating self-signed
SSLs signed with internal CA per each VIP. With follow-up
patches that will also allow to provide user certificates
per VIP, making possible to cover internal and external
endpoints with different non-wildcard certs.

Change-Id: I0a9eb7689eb42b50daf5c94c874bb7429b271efe
2021-06-25 13:30:25 +00:00
James Gibson 3139772435 Add variable to disable stick-table
In some use cases you may want to define your own stick-table and
rules, this can be done using the backend_arguments variables.
As you can have only one stick-table per backend or frontend
the default stick-table needs to be disabled.

I am also not convinved the default stick-table is used for anything,
it just logs requests and never uses the logs, i think it could be
removed.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/797819

Change-Id: I54307c00673ababb277257f2bb0e456e3e011ac4
2021-06-24 11:53:32 +01:00
Dmitriy Rabotyagov f058cf8d61 Replace whitelist with allowlist naming
Change-Id: I568273d0ef1d5ee391a42981e66cc9895b9d71b6
2021-06-16 14:40:19 +03:00
Jonathan Rosser ca76349e9f Use http-request set-header instead of reqadd
This is needed for HAProxy 2.1 which is in Debian Bullseye

Change-Id: I912c6d810acc137c3b3e73dc40160d6376cb3884
2021-05-13 16:08:50 +00:00
Andrew Bonney 357daad5c8 Allow HAProxy stats to be pinned to one or more processes
When HAProxy is run in multi-process mode, the single stats page
shows metrics for one of the processes at a time, with a random
selection made on page reload.

Whilst a more complete solution may be to enable a stats page for
each process, this is a little cumbersome. This addition allows
the stats page to be pinned to one process, providing a partial
snapshot of the state of the instance.

Change-Id: Id9314e5b267aafeaf34c82874eb8bfe0713dfac3
2021-03-25 14:08:15 +00:00
Marc Gariepy ca2c011cf2 Add haproxy_frontend_only and haproxy_raw feature.
you can add prometheus metric exposed directly via haproxy if your
version is recent enough.
https://www.haproxy.com/blog/haproxy-exposes-a-prometheus-metrics-endpoint/

Change-Id: I10e7220071290301a85409a1f74fcbad2743d19d
2020-12-14 19:10:29 +00:00
Danny Meloy 8120c9a3c5 Add haproxy_backend_only flag to service template
Addition of a `haproxy_backend_only` flag to the service template
for instances in which a frontend service uses ACLs to propgate
requests to multiple backend services, and not all backend services
require a corresponding frontend. This should prevent the
proliferation of spurious frontend services.

Change-Id: I8c419be82cffd289ffcc5086afac923d6eb1a78a
2020-09-01 11:14:55 +00:00
Zuul 1dc605c4aa Merge "Allow backend healthcheck interval to be overridden per service" 2020-05-07 18:54:54 +00:00
Zuul 1d65b887c6 Merge "Use a certbot pre-hook to ensure haproxy backend is up before renewal" 2020-05-07 18:54:51 +00:00