Commit Graph

156 Commits

Author SHA1 Message Date
Takashi Kajinami 4342c31ae2 dib: Remove Ubuntu Forcal support
Ubuntu Focal is no longer part of the tested environments, because of
newer LTS available now (Jammy).

Change-Id: I7a6df974762abdd94784416609304618ce702b6e
2024-03-13 12:16:59 +09:00
Takashi Kajinami 02a2195e54 dib: Remove remaining logic for CentOS/RHEL 8
... because CentOS 8 and RHEL 8 is no longer supported.

Change-Id: I90e5b85827a324c0a258fd30cf94b3e8ac8c841f
2024-03-13 12:14:36 +09:00
Zuul 33fed53043 Merge "Add support for Rocky Linux" 2023-09-20 18:27:36 +00:00
Gregory Thiemonge 294bd406f3 Fix TCP HMs on UDP pools with SELinux
SELinux denied some specific TCP ports when using TCP-based HMs in UDP
pools (keepalived).
Enable a SELinux boolean keepalived_connect_any which allows keepalived
to connect to any port.

Closes-Bug: #2023751
Change-Id: Ie611ba9fde7b399989d847dd0c61dd3a158652bc
2023-06-14 04:32:08 -04:00
Gregory Thiemonge 4e6e8f0c0e Add support for Rocky Linux
* Added support for Rocky Linux in the amphora-agent
* Amphora images for Rocky can be built when setting
  OCTAVIA_AMP_BASE_OS="rocky"
* Fixed the devstack plugin for Rocky Linux hosts

Change-Id: I41f7e2341332b9cb74b4a59fedb6eed1af3c8062
2023-05-23 09:49:25 +02:00
Gregory Thiemonge 8c6f1a0ae5 Fix focal amphora image build
Purge packages based on the ditro name and release
Add a testing job for building a jammy amphora image

Change-Id: Ic1ec002062f91b7d3a43669e86380a2ffbdd7b18
2023-03-20 08:18:31 +01:00
Michael Johnson 764cbf014f Purge some dev tools from the amphora image
Some of the image build time tools are not being uninstalled from the amphora
image during cleanup. This patch adds a package uninstall callout for those
packages.

Change-Id: I556c89dbc3aa73415d084187eea21d1790decbd5
2023-03-10 22:28:48 +00:00
Zuul 5f90b0bc5c Merge "Fix image creation for stable branches" 2023-03-01 15:00:51 +00:00
Zuul b4b6e07fe0 Merge "Fix typo in lvs-masquerade.sh" 2023-02-17 00:26:40 +00:00
Zuul 38ec0d6673 Merge "Fix PING health-monitor with recent Centos releases" 2023-01-23 20:18:38 +00:00
Zuul d90ea57eb8 Merge "Set nohz_full in cpu-pinning element" 2023-01-12 10:47:22 +00:00
Zuul 250b54aa15 Merge "Add cpu-pinning element" 2023-01-12 10:01:49 +00:00
Zuul c47bc0d90c Merge "Remove i386 amphora build support" 2023-01-04 12:00:44 +00:00
Tom Weininger baefc3b464 Set nohz_full in cpu-pinning element
nohz_full (or full dynticks) is a Linux kernel setting that should reduce
kernel noise on configured CPUs to a minimum. This change activates this
setting on all vCPUs except the first one.

Change-Id: Ie009aff52d572873da0bb5cc43fedd7f3325dccc
2022-12-08 11:59:32 +01:00
Tom Weininger c1ebe18e2a Add cpu-pinning element
The new "cpu-pinning" element optimizes the amphora image for better
vertical scaling. When an amphora flavor with multiple vCPUs is
configured it will configure the kernel to isolate (isolcpus)
all vCPUs except the first one.
Furthermore, it uninstalls irqbalance and sets the IRQ affinity to the
first CPU. That way the other CPUs are free to be used by HAProxy
exclusively. A new customized TuneD profile applies some more tweaks
for improving network latency.

This new feature is disabled by default in diskimage-create.sh.

Story: 2010236
Task: 46042
Change-Id: I1a0591de79be867483a044705e866b2368b2a567
2022-12-08 11:59:32 +01:00
Gregory Thiemonge 344d6d76ce Fix prometheus-proxy service name in Red Hat-based distros
The prometheus-proxy service is called octavia-prometheus-proxy in RDO's
RPMs [0].
Add a svc-map for the service name to fix the amphora-agent element for
these distros.

[0] https://review.rdoproject.org/r/c/openstack/octavia-distgit/+/44152

Partial-Bug: 1992668

Change-Id: Ie739cd582103d1aab3021c56654e395b5919358f
2022-10-13 09:48:17 +02:00
Michael Johnson b22e07262d Fix image creation for stable branches
A previous patch[1] renamed the udp-masquerade.sh file in the amphora-agent element to lvs-masquerade.sh as it is now also being used for the SCTP protocol in addition to UDP.
Unfortunately this broke cross-branch image building because the DIB install-static element will copy the file from the current checked out Octavia elements.
So if a user tries to build an image for stable/victoria using the "-g stable/victoria" flag, they will get a victoria amphora-agent, but the static files from the checked out version of Octavia.
This patch adds a symbolic link from the old file name to the new one to facilitate backwards compatibility with the older amphora-agents.

[1] https://review.opendev.org/c/openstack/octavia/+/753247

Story: 2010313
Task: 46379
Change-Id: I8b2862cf67d815309af3d13a68bab65e770775aa
2022-09-21 21:54:37 +00:00
Gregory Thiemonge d9ee63f561 Allow multiple VIPs per LB
User can specify additional subnet_id/ip_address pairs to bring up on
the VIP port. This will allow for situations like having an LB with both
IPv4+IPv6 or being exposed on both public and a private network.

For UDP/SCTP loadbalancers, mixing IPv4 VIP and IPv6 members is not
supported (IPv6 VIP and IPv4 members as well). It's still possible to
use IPv4 and IPv6 VIPs at the same time in the same loadbalancer but an
IPv4 VIP can only communicate with IPv4 members.

Thanks Michael for help with validating/fixing the templates!
Thanks Gregory for help with the centos networking!

Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Co-Authored-By: Gregory Thiemonge <gthiemon@redhat.com>
Co-Authored-By: Brian Haley <haleyb.dev@gmail.com>
Story: 2005608
Task: 30847
Change-Id: Id7153dbf33b9616d7af685fcf13ad9a79793c06b
2022-08-31 17:08:35 +02:00
Gregory Thiemonge 55e6001ea4 Fix typo in lvs-masquerade.sh
A typo in the lvs-masquerade script prevented the deletion of the
'ip6-udp-masq' nftables chain when stopping a network interface.

The name of the chain should be 'ip6-udp-masq' instead of 'ip-udp-masq'

The bug had no impacts on the loadbalancers but it displayed an error
message in the logs.

Change-Id: Iba07010b63771aafa1db224dbd0e0c97925d6173
2022-08-19 17:28:37 +02:00
Zuul b1d83eedcf Merge "Apply openstack-selinux policies in Centos amphorae" 2022-08-18 17:31:57 +00:00
Zuul 550a978d70 Merge "Fix centos 8 stream haproxy repository" 2022-08-04 15:39:03 +00:00
Zuul 1876b99dd7 Merge "Fix prometheus-proxy systemd service" 2022-07-25 21:22:51 +00:00
Gregory Thiemonge 3115589f21 Apply openstack-selinux policies in Centos amphorae
When enforcing SELinux, the commit enables the following SELinux
booleans:
- os_haproxy_enable_nsfs: it allows haproxy to run in a network
  namespace.
- os_haproxy_ping: it allows calls to the ping command from haproxy (for
  PING HMs)
- cluster_use_execmem

Change-Id: I4b5199d13d122b56619c4a2d21e86a06bd0dc3bf
2022-07-22 17:14:40 +00:00
Gregory Thiemonge e4c76108eb Add explicitly vim-minimal in CentOS-based images
CentOS-9-Stream doesn't include any editor in the base image, map
vim-tiny to vim-minimal so we get at least one editor (and the terminfo
package that is helpful when using top).

Change-Id: Ibf85f279a9d0440dd6134e58f4d4f718407a277f
2022-05-17 10:26:29 +02:00
Tom Weininger 72bdc0f88d Remove unneeded sudo in lvs-masquerade.sh
Fixes issue with SELinux and the lvs-masquerade.sh script on the
amphora.
The script already runs with root permissions, so the use of sudo
inside the script is unneeded.

Change-Id: I63474acbcea5106ea702d21cb238aa57cef5d96d
2022-05-03 07:57:11 +02:00
Gregory Thiemonge cbf79c5627 Fix PING health-monitor with recent Centos releases
Centos 8 Stream removed the capabilities to open raw sockets from the
ping binary and added new sysctl default to allow an unprivileged user
to send ping request.  This sysctl was not inherited by the
amphora-haproxy namespace. Add it explicitly to our sysctl list.

Story 2009953
Task 44911

Change-Id: I499bdd5edc23b602495a9b2b9f22baf9547261d5
2022-04-01 13:49:07 +02:00
Zuul 328ffbab5a Merge "Remove ureadahead package" 2022-03-01 19:41:41 +00:00
Michael Johnson 7afe2fb243 Fix prometheus-proxy systemd service
This patch changes the inter-dependency between the prometheus-proxy
systemd service unit and the amphora-agent service. This will stop
systemd from shutting down the prometheus proxy should the amphora
agent be stopped.

Change-Id: I30c6c5809cc8568159feca193f38f62ca09874fd
2022-02-25 22:48:42 +00:00
Zuul c5561a3ecb Merge "Use centos amphora image in the FIPS jobs" 2022-02-23 01:42:56 +00:00
Michael Johnson 0d9674bd87 Add the PROMETHEUS protocol to listeners
This patch adds a new protocol for listeners called "PROMETHEUS" that exposes
a Prometheus endpoint. This allows detailed metrics collection from Octavia
load balancers.

Change-Id: I3e27e4e57ad955bcd7728426c91f05171a46ef7f
2022-02-22 01:57:53 +00:00
Gregory Thiemonge cc18c1e1b5 Fix centos 8 stream haproxy repository
Centos 8 stream amphora images were incorrectly using haproxy from
Centos 8 NFV network-extras repo. This repo was deleted and was replaced
by a Centos 8 stream repo.

Change-Id: I5b50ebc5e9443f1a0edd987069d482e03d85e726
2022-02-18 14:47:43 +01:00
Pooja Jadhav c523b19f9c Fix amphora-agent elements for RHEL9
Update elements for RHEL9.

Ref: https://review.opendev.org/c/openstack/octavia/+/816370

Change-Id: I913cc99520f2eb6a73c77c3778842124e4cff126
2022-02-17 11:16:31 +05:30
Gregory Thiemonge 74a7cbe122 Use centos amphora image in the FIPS jobs
The FIPS jobs use centos-8-stream controllers but the image is still
based on ubuntu, this commit updates the amphora images to
centos-8-stream and enable FIPS inside the amphora.

Change-Id: I8916796ed6727a103907a33d3c14e99e1d3734e6
2022-02-11 17:21:48 +00:00
Gregory Thiemonge 791fbfce6c Fix devstack on centos-9-stream
debootstrap doesn't exist on centos-9-stream, disable it for
rhel9-based distribution.
Also remove a previous unsuccessful attempt to remove it.

Change-Id: I8fac328cdda6e1015a145a1f8a497e007ac6ac02
2022-02-07 21:20:53 +01:00
Gregory Thiemonge 0a9f3a8e6f Add support for Centos 9 Stream
Update elements for c9s

Change-Id: I3f9d08fd4ce038208b8c2d160e6abecae8c8383d
2022-02-04 10:27:52 +01:00
Frode Nordahl 4ce30395ef Remove ureadahead package
The 'ureadahead' package was removed from Ubuntu at cosmic. The
last Debian release to distribute the 'readahead' was jessie. The
pkg-map already removes the package for redhat family of
distributions, so let's just remove it altogether.

Related-Bug: #1690891
Change-Id: I0e2df43384f327928f171665b3ce0bb5d60e3560
2022-02-02 13:34:26 +00:00
Gregory Thiemonge 4791cfe93c Fix nft command line with negative priority values
When using nft with negative priority values it is recommended to use --
to prevent the parser to interpret the value as an argument.

Story 2009710
Task 44065

Change-Id: Ia6bc6eee3df30bfb3c0acccf902267fd2a4d37f6
2021-11-25 09:08:58 +01:00
Zuul 74f3242c8e Merge "Workaround to fix amphora-agent on centos" 2021-11-15 16:07:58 +00:00
Zuul 0f7eb0079c Merge "fix amphora-agent-install script, typo, heath -> health" 2021-11-15 16:03:35 +00:00
Gregory Thiemonge 2043788473 Workaround to fix amphora-agent on centos
dnspython now requires to have at least one nameserver in
/etc/resolv.conf.
The amphora image doesn't need and doesn't use any dns servers, add a
dummy dns server as a workaround on centos (on ubuntu, resolv.conf
contains the systemd resolver).
We will revert this patch when we find a more convenient fix for this
issue

Story: 2009281
Task: 43552

Change-Id: I81af508e7df5ffef343f1614335ffe1a98540912
2021-10-12 18:49:42 +02:00
Michael Johnson 421eecd1e5 Increase TCP buffer maximum and MTU discovery
This patch increases the maximum allowed TCP buffer memory to 32MB
and enabled MTU probe discovery when an ICMP black hole is detected.

This should improve reliability in some situations when the amphora are
provisioned on hosts with 10gbps or faster network adapters.

Change-Id: If7dc46ee18a46e3bd15756839f60ea5b51c9288b
2021-10-07 15:45:00 +00:00
Zuul b8ddbc94a1 Merge "Disable conntrack for TCP flows in the amphora" 2021-09-15 10:36:34 +00:00
Gregory Thiemonge 5dd7ad9ad8 Add generic network interface management in the amphora
Handle network configuration using Octavia tools.

amphora-interface configures network interfaces inside the amphora
using pyroute2 and a set of json files for persistent configuration in
the /etc/octavia/interfaces/ directory.

Story: 2005235
Task: 30019

Depends-On: https://review.opendev.org/806558

Change-Id: I5360c8246cd39f90eb7104a883f87c0042d146c4
2021-08-31 17:21:37 +02:00
Gregory Thiemonge 4174f4a5a4 Update nr_open limit value in the amphora
Using haproxy 2.x, the ideal rlimit value for nr_open is close to
connection_limit * 2.5 (see compute_ideal_* in src/haproxy.c).
Set this limit to 2,600,000 in the amphora to support a loadbalancer
with maxconn 1M.

This prevents the following warning messages when launching/reloading
haproxy:

* "Cannot raise FD limit to 2375058, limit is 2097152."
* "FD limit (2097152) too low for maxconn=950000/maxsock=2375058. Please
  raise 'ulimit-n' to 2375058 or more to avoid any trouble."

Change-Id: I6251cd17bd6fa9faf5109e50c2190dda3614908d
2021-08-18 14:21:24 +00:00
Bernd Mueller 06a1862af1 fix amphora-agent-install script, typo, heath -> health
Signed-off-by: Bernd Mueller <mueller@b1-systems.de>
Change-Id: Ida88bac532b1cc247d7c8751c6ebfeaa38de2144
2021-07-15 16:23:23 +02:00
Gregory Thiemonge b3f59eeb73 Disable conntrack for TCP flows in the amphora
conntrack is used in the amphora for LVS-based listeners (UDP and
SCTP) and it's enabled by default.
Add nft/iptable rules to disable conntrack for TCP flows, it reduces
memory usage and prevents getting "nf_conntrack: table full, dropping
packet" messages.

Story: 2008979
Task: 42623

Change-Id: Ia15a110c6c5a9b83a29bab56538fa425d994d477
2021-06-16 10:24:43 +02:00
Carlos Goncalves 4333881cc0 Remove i386 amphora build support
Diskimage-builder has removed support to building i386 images, and major
Linux distributions have already end-of-lifed 32-bit host support.

Depends-On: https://review.opendev.org/c/openstack/diskimage-builder/+/790878
Change-Id: I258b9b49b4a876a2bf9772b19f2100187ea83007
2021-05-12 09:30:49 +02:00
Carlos Goncalves e99d7b9389 Add aarch64/amd64 support to amphora and devstack
The disk image create tool can now build aarch64/arm64 amphora images.
The devstack plugin will facilitate image builds and upload to Glance
for this CPU architecture.

Change-Id: I1cebd8a3da58dc56ebbfac22f7802ab7f52585e1
2021-03-10 18:54:50 +01:00
Zuul de644d398b Merge "Add SCTP support in Amphora" 2021-02-17 16:12:28 +00:00
Gregory Thiemonge a518cefda1 Disable auto configuration of new interfaces in NetworkManager
NetworkManager in Centos images configures new network devices as soon
as they appear in the default namespace, it means that we might have
conflicts between the management interface's routes and address and the
new VIP or member interfaces' routes and addresses during a small period
of time before they are moved to the amphora-haproxy namespace.

Now, the "no-auto-default=*" option is enabled in NetworkManager, it
disables the configuration of new interfaces, while the management
interface is still enabled/configured through cloud-init.

Story 2008599
Task 41773

Change-Id: I6dd8e99b07ff557674871cb503dece96a9df3ada
2021-02-06 22:26:07 +01:00