Commit Graph

286 Commits

Author SHA1 Message Date
Takashi Kajinami 16f6b2e8f6 redis: Add username
Redis introduced ACL feature in 4.0.0, and this feature is supported by
redis-py since 3.4.0[1]. When ACL is enabled, authentication requires
username in addition to password.

Also this removes the default password string because it can confuse
underlying libraries in case a more strict check such as 'is None' is
implemented there.

[1] 8df8cd54d1

Depends-on: https://review.opendev.org/c/openstack/taskflow/+/907667
Change-Id: Ie85589ab4e02046f54864a10b9b8adce6996d82a
2024-03-01 01:57:35 +09:00
Takashi Kajinami d1d7fe7197 Use cryptography to load PKCS12 certificates
... because implementation in PyOpenSSL has been derprecated, according
to the following warning.

```
DeprecationWarning: PKCS#12 support in pyOpenSSL is deprecated. You
should use the APIs in cryptography.
```

Closes-Bug: #2042787
Change-Id: Ic81e98c54c4bce100e3f44ff1a2fe6ce7b7f4256
2023-11-06 21:58:06 +09:00
Zuul a904f35304 Merge "Add support for HTTP Strict Transport Security" 2023-08-23 20:23:51 +00:00
Tom Weininger c907547512 Add support for HTTP Strict Transport Security
Closes-Bug: #2017972
Depends-on: https://review.opendev.org/c/openstack/octavia-lib/+/880821
Change-Id: I0f2f2ff6b8c430b2dd06d707097af74bb608dcc9
2023-08-23 18:18:02 +02:00
Tom Weininger 3e6fd13bfb Remove python-neutronclient
This patch replaces the deprecated library with openstacksdk. It also
deprecates the use of the [service_auth] section of the configuration
for authentication with Neutron. In a future release these settings
need to be part of the [neutron] configuration section.

Update needed on OVN provider side:
https://review.opendev.org/c/openstack/ovn-octavia-provider/+/870514

Story: 2010509
Task: 47104
Change-Id: I686cfdef78de927fa4bc1921c15e8d5853fd2ef9
2023-04-14 13:11:38 +02:00
Tobias Urdin 244f44ae6c Use new get_rpc_client API from oslo.messaging
Use the new API that is consistent with
the existing API instead of instantiating the client
class directly.

This was introduced in release 14.1.0 here [1] and
added into oslo.messaging here [2]

[1] https://review.opendev.org/c/openstack/requirements/+/869340
[2] https://review.opendev.org/c/openstack/oslo.messaging/+/862419

Change-Id: I33d55df17fcf9ca69b4a3dc1906b440b9cb87b46
2023-02-10 15:51:14 +01:00
Takashi Kajinami abf7b9efcf Add conf file for oslo-config-generator
This change introduces the config file for the oslo-config-generator
command, so that users can easily generate octavia.conf.example without
tox.

Note this change adds parameters of oslo.policy and oslo.middleware
which were missing previously.

Change-Id: I5ea921cf8d63b28c5143f95dbb47802d5018d7a4
2022-09-04 05:14:16 +09:00
Adam Harwell 1d19b702b1 Failover stop threshold / circuit breaker
Stop failovers if the count of simultaneously failed
amphora reaches the number configured in the new
failover_threshold option.
This may prevent large scale accidental failover events,
like in the case of network failures or read-only
database issues.

Story: 2005604
Task: 30837
Co-Authored-By: Tatsuma Matsuki <matsuki.tatsuma@jp.fujitsu.com>
Co-Authored-By: Tom Weininger <tweining@redhat.com>

Change-Id: I0d2c332fa72e47e70d594579ab819a6ece094cdd
2022-09-02 17:15:59 +02:00
Michael Johnson 085cf1c2e3 Remove netaddr module requirement
This patch removes the 'netaddr' module from the Octavia requirements list.
It replaces the use of netaddr in Octavia with the python standard library 'ipaddress' module.
It also increase unit test coverage for the impacted functions.

Change-Id: I9ec023116417e5c8850395d4c06db63e5be1c022
2022-07-20 23:46:49 +00:00
Gregory Thiemonge 38cb7b6fe8 Drop lower-constraints.txt and its testing
As discussed in TC PTG[1] and TC resolution[2], we are
dropping the lower-constraints.txt file and its testing.
We will keep lower bounds in the requirements.txt file but
with a note that these are not tested lower bounds and we
try our best to keep them updated.

[1] https://etherpad.opendev.org/p/tc-zed-ptg#L326
[2] https://governance.openstack.org/tc/resolutions/20220414-drop-lower-constraints.html#proposal

Change-Id: Ib929fbe28241bd6d3c3bd352b2d900289efd6d49
2022-07-06 18:44:21 +02:00
Michael Johnson 0d9674bd87 Add the PROMETHEUS protocol to listeners
This patch adds a new protocol for listeners called "PROMETHEUS" that exposes
a Prometheus endpoint. This allows detailed metrics collection from Octavia
load balancers.

Change-Id: I3e27e4e57ad955bcd7728426c91f05171a46ef7f
2022-02-22 01:57:53 +00:00
Zuul f144dc7e87 Merge "Replace md5 for fips" 2021-08-19 18:03:11 +00:00
Ghanshyam Mann 7e264e1f21 Fix oslo policy DeprecatedRule warnings
Since 3.7.0, oslo policy started the DeprecationWarning[1] if
deprecated_reason and deprecated_since param are not passed
in DeprecatedRule or they are passed in RuleDefault object.

These warnings are logged for every test which increase the
log size and sometime can full the log buffer and fail the
job.

[1] https://github.com/openstack/oslo.policy/blob/3.7.0/oslo_policy/policy.py#L1538

Change-Id: Ifa8c620fe76517b1c040af5905e6a6c1cdb4c922
2021-07-12 13:10:27 -05:00
Ade Lee db7a633a4f Replace md5 for fips
md5 is not an approved algorithm in FIPS mode, and trying to
instantiate a hashlib.md5() will fail when the system is running in
FIPS mode.

md5 is allowed when in a non-security context.  There is a plan to
add a keyword parameter (usedforsecurity) to hashlib.md5() to annotate
whether or not the instance is being used in a security context.

In the case where it is not, the instantiation of md5 will be allowed.
See https://bugs.python.org/issue9216 for more details.

Some downstream python versions already support this parameter.  To
support these versions, a new encapsulation of md5() has been added to
oslo_utils.  See https://review.opendev.org/#/c/750031/

In this case, md5 is used to generate etags and to check file integrity when
uploading certs. fingerprints when ssh keys are
being generated and imported.  Without this patch, these operations
fail on FIPS enabled systems.

Change-Id: Ib189c6f67946851d37c31a6a8d657460c15f491e
2021-07-07 18:35:41 +00:00
Ghanshyam Mann c5e59b5e9e [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: I8b78c7b640ab18ddfc809cb4603decc739d494d1
2021-03-10 22:09:35 -06:00
Michael Johnson a5f142c566 Add support for scoped tokens and default roles
This patch is the base patch to enable support for Keystone
scoped tokens[1] and default roles[2] in the Octavia API.

It strives to maintain backward compatibility and support for
Octavia Advanced RBAC roles.

[1] https://docs.openstack.org/keystone/latest/admin/tokens-overview.html#authorization-scopes
[2] https://docs.openstack.org/keystone/latest/admin/service-api-protection.html

Change-Id: I4443d4531dc97d14f8277024baa11ab43e87fb39
2021-03-08 19:33:35 +00:00
Lance Bragstad a822f30eb1 Bump oslo.policy version to 2.1.0
The secure RBAC work requires the of oslo.policy's scope_types argument,
which was made available in 1.32.0. This commit updates to version 2.1.0
so that we're using something more relevant.

Required oslo.context bump.

Change-Id: I48e8882f629886d1b8abadff6e60aad91d1169c7
2021-02-08 17:18:55 +00:00
Lance Bragstad 7d8b0db3c8 Bump oslo.log version to 4.3.0
This allows us to use the Wallaby release marker in versionutils for
deprecations.

Required some other bumps to satisfy lower-constraints.

Change-Id: I3c2be8a5189dcda24c42ed7ab4d8fa33a03c5d3e
2021-02-08 17:17:15 +00:00
Carlos Goncalves d2d5fc80f8 Add ALPN support for TLS-enabled pools
ALPN is a TLS extension for application-layer protocol negotiation
within the TLS handshake [1].

This patch extends the Pool API to include a new 'alpn_protocols'
parameter. With this parameter, users can set an ALPN preference list
(descending order of preference) to be advertised by load balancer to
members.

This patch also adds HTTP/2 over TLS support to TLS-enabled pools to the
Amphora provider driver, although default the pool ALPN protocol list
configuration setting has HTTP/2 disabled similarly to the default
listener ALPN protocol list value added in Victoria release.

[1] https://tools.ietf.org/html/rfc7301

Change-Id: I91924486bab22601c15c538c8a5282ad8bc54700
2021-01-28 14:42:48 +01:00
Brian Haley dc876eed9d Fix gate failure
Bumped a bunch of constraints and requirements to fix
a gate failure with recent pip update.

Change-Id: I7af382a993bd16ce4e32a9052337f006d9aa58aa
2020-12-15 14:33:09 -05:00
Zuul e45ecc2111 Merge "Fix requirements.txt for Victoria" 2020-09-18 18:56:34 +00:00
Michael Johnson 6c8c4f18fd Fix requirements.txt for Victoria
There are a couple of extra requirements no longer used and one
that was missing. This patch corrects that.

Change-Id: I5f45e6f0f15bca201c3ef63f2e42a183510f5977
2020-09-17 21:45:18 +00:00
Corey Bryant 59c4ad4832 Drop diskimage-builder from root requirements.txt
diskimage-builder is already listed in diskimage-create/requirements.txt,
and it isn't really a run-time dependency for octavia so it tends to be
confusing downstream as we generally add run-time dependencies based on
the root requirements.txt.

Change-Id: I612ea1c583090897bd44453b867d75929a01b7fc
Story: 2007934
Task: 40389
2020-09-11 10:10:54 -07:00
Michael Johnson a446c04f77 Revert pyroute2 workaround
The pyroute2 module had a regression in 0.5.13, but neutron needed a fix
that was included in 0.5.13. It was easier for us to work around it than
for neutron to revert the update to 0.5.13, so we hacked a workaround[1].

A new pyroute2 release, 0.5.14, has been published, so this patch reverts
the hack and bumps our minimum version of pyroute2 to 0.5.14.

[1] https://review.opendev.org/#/c/744045

Change-Id: Ia4357a4f5db07e1a6b0beb11b58ce125215eeef1
2020-09-10 16:57:02 -07:00
Ann Taraday befa073495 Bump taskflow version
Taskflow 4.4.0 contains essential fix for jobboard behaviour on
Storage failure [1].

Also add jobboard_redis_sentinel parameter to allow usage sentinel
for Redis jobboard. Support of this also appear in 4.4.0 version.

[1] - https://docs.openstack.org/releasenotes/taskflow/unreleased.html#bug-fixes

Change-Id: I48245b3322b0f2e5f2c11594a15632501a7e4086
2020-09-10 14:12:39 +00:00
Michael Johnson df3f58903b Fix the lower-constraints job for focal
The move to focal for the tox jobs has raised issues with package
versions supporting python 3.8.
This patch bumps the version for the following packages:
PyMySQL==0.8.0
pyroute2==0.5.13
SQLAlchemy==1.2.19
cffi==1.14.0
cryptography==3.0
pyOpenSSL==19.1.0
oslo.db==8.30
oslo.messaging==12.4.0

This patch makes the grenade job non-voting as there is a chain
of gate fix patches required to fully fix all of the issues.

Change-Id: I95cc182a2dbcb2ea872e822ce9372bf5d7df63bd
2020-09-09 22:48:42 -07:00
Carlos Goncalves a5f0524fd0 Add ALPN support for TLS-terminated HTTPS LBs
ALPN is a TLS extension for application-layer protocol negotiation
within the TLS handshake [1].

This patch extends the Listener API to include a new 'alpn_protocols'
parameter. With this parameter, users can set an ALPN preference list
(descending order of preference).

Presently, the amphora provider driver is limited to http/1.0 and
http/1.1 ALPN protocol IDs. Support for "h2" (HTTP/2 over TLS) depends
on HAProxy 2.0 or newer.

[1] https://tools.ietf.org/html/rfc7301

Change-Id: If08a8169498cdfaa75440e8971ba0caff45ac4c4
2020-08-27 13:19:52 +02:00
Hervé Beraud 8119b21452 Cap jsonschema 3.2.0 as the minimal version
Previous versions of jsonschema (<3.2.0) doesn't support python 3.8 [1].
Python 3.8 is part of the victoria supported runtimes [2] so we now force
to use jsonschema version 3.2.0 to avoid issues, remove ambiguity and ensure
that everything works with python 3 in general.

[1] https://github.com/Julian/jsonschema/pull/627
[2] https://governance.openstack.org/tc/reference/runtimes/victoria.html#python-runtimes-for-victoria

Change-Id: I2816360b59f82c996038929ed6e90b744feb3f75
2020-06-12 19:23:10 +02:00
Zuul b6e0221ca8 Merge "Remove all deprecated driver code that moved to octavia-lib" 2020-06-11 22:02:48 +00:00
Brian Haley de69b2c7ff Remove all deprecated driver code that moved to octavia-lib
In https://review.opendev.org/#/c/613709/ octavia was
changed to use octavia-lib for a lot of API driver-related
code and deprecation warnings put in place. Now that
we're in Victoria remove all the deprecation shims and
use octavia-lib exclusively.

Change-Id: If92988150479a7daf465af5f8df22818664a0fce
2020-06-08 14:41:00 -04:00
Michael Johnson a720f76c84 Remove Babel requirement
Babel is no longer a requirement for OpenStack projects.
Let's remove it and save some disk space.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014227.html

Change-Id: Ib4944fb44fea53c8dd1908b7f264d45a84884639
2020-04-17 09:46:14 -07:00
Ann Taraday 9c13499d21 Jobboard based controller
Introduce TaskFlowServiceController which uses taskflow
jobboard feature and saves jobs info into persistence backend.

Jobboard could be operated via RedisTaskFlowDriver or
ZookeeperTaskFlowDriver, that could be set via the config.
RedisTaskFlowDriver is intoduced as default backend for jobboard.
Usage of jobboard allows to resume jobs in case of restart/stop
of Octavia controller services.

Persistence backend saves state of flow tasks that required in
case of resuming job. SQLAlchemy backend is used here.

Bump taskflow version to 3.7.1 and add dependency to
SQLAlchemy-Utils (required for taskflow sqlalchemy
backend support).

Story: 2005072
Task: 30806
Task: 30816
Task: 30817

Change-Id: I92ee4e879e98e4718d2e9aba56486341223a9157
2020-04-08 19:53:09 +04:00
Dawson Coleman cd176e55c5 Add ability to set TLS cipher list for listeners
Listeners will now be able to each be assigned their own OpenSSL
cipher string with a new field: tls_ciphers.  There is also a new
configuration option, default_listener_ciphers, which specifies the
cipher string to assign to new listeners when one is not explicitly
specified.

Change-Id: I77da6f14063877af0077f2c12df1aab5d5ead187
Depends-On: Id5f4c20abd40dd092558a711987953012d4ae67f
Story: 2006627
Task: 36839
2020-04-06 17:06:32 -07:00
Sam Morrison 18020e6c88 Add oslo middleware healthcheck to Octavia API
healthcheck middleware adds a /healthcheck url that allows
unauthenticated access to provide a simple check when running
octavia-api behind a load balancer

https://docs.openstack.org/oslo.middleware/latest/reference/healthcheck_plugins.html

Co-authored-by: Michael Johnson <johnsomor@gmail.com>
Change-Id: I10db6226750f7b7c703067d2ab82eea3a9875112
2020-03-19 11:24:16 -07:00
Brian Haley f6b957e8ee Remove all usage of six library
Convert all code to not require six library and instead
use python 3.x logic.

Created one helper method in common.utils for binary
representation to limit code changes.

Change-Id: I2716ce93691d11100ee951a3a3f491329a4073f0
2020-03-18 17:15:26 -04:00
Brian Haley 4af42f837f Complete dropping py27 support goal
This takes care of the last details for dropping py27
support by adding a proper min version of python in setup.cfg.

Change-Id: I693db277d802b2a54084cc1be11d8ce04ad9be2e
2020-01-16 11:32:03 -05:00
Adam Harwell d115644d3e Adjust for new octavia-lib LB signature
Depends-On: https://review.opendev.org/#/c/697359/
Change-Id: I3f0c47e72978103bb4dd83fe6f3cdd2c0e8e3210
2019-12-04 12:59:39 -08:00
Brian Haley bc94789dc9 Stop testing python 2
Since it's no longer supported past Train, lets stop
running the tests.

Change-Id: Ia30435b3bb19acd1d2f9fe217251c393d4f7480e
2019-12-02 12:08:19 -05:00
Zuul ca80bc9e03 Merge "Bump diskimage-builder minimum to 2.24.0" 2019-10-09 01:32:49 +00:00
Zuul ff483f3457 Merge "Add long-running provider agent support" 2019-09-11 16:03:56 +00:00
Michael Johnson 78b1263237 Add long-running provider agent support
This patch adds support for long-running provider driver agents to
the Octavia driver-agent.
It will fork a process for all of the enabled provider driver
agents at startup.

Change-Id: Ib7042bcc48b1dd5b37b671dd5e64728b71ab9542
Story: 2006250
Task: 35863
2019-09-10 22:25:50 +00:00
sapd 93b509cfe5 Support create amphora instance from volume based.
In some deploy production, using volume based instead of localdisk
to protect data and live migrate can perform.

This patch adds:
 - creation a cinder volume for amphora
 - boot amphora with cinder volume
 - config options for cinder client
 - unit tests for cinder functionality

Story: 2001594
Co-authored-by: Vadim Ponomarev <velizarx@gmail.com>
Co-authored-by: Margarita Shakhova <shakhova.margarita@gmail.com>
Change-Id: I8181ed696b9ab556e7741c08839d79167aff8350
2019-09-10 22:21:42 +00:00
Michael Johnson 1aeb416a7e Bump diskimage-builder minimum to 2.24.0
There was a bug[1] in diskimage-builder when using pypi mirrors
if the host doesn't have "python" available DIB will fail.
So, I am going to bump the requirements minimum in Octavia for
diskimage-builder to be the minimum version with this fix.

[1] https://bugs.launchpad.net/diskimage-builder/+bug/1577105

Change-Id: If0ff2a855ad5b9d9ef3742ad0596c97a6dbf81ed
2019-09-10 22:20:42 +00:00
Michael Johnson afa917a3de Change amphora logging to dedicated socket
In some environments running older versions of gunicorn in the
amphora image, gunicorn can fail to start do to /dev/log socket
issues (timing, configuration, etc.).

This patch sets up a dedicated rsyslog socket /run/rsyslog/octavia/log
for gunicorn and haproxy to use. This should resolve any issues with
systemd overriding the /dev/log socket.

This also bumps the gunicorn minimum verison to 19.9.0.

Change-Id: I1e1ad8fde2ad8c1ffba95b1867afb130503b0a5b
2019-08-07 09:02:22 -07:00
Carlos Goncalves 262a89c7ed Add missing lib dependencies to requirements.txt
Some Python libraries were missing in requirements.txt. This patch adds
them to the list of dependencies.

Change-Id: I6dc4a7626cd5244b445893b96a7d7351ca528dc1
2019-08-06 08:41:34 -07:00
Michael Johnson b32064cdf2 Fix a unit test for new octavia-lib
A patch[1] introduced a startup check for the socket paths that
impacted a unit test checking debtcollector is finding the new
octavia-lib location of the DriverLibrary.
This patch correctly mocks out this check for this unit test.

[1] https://review.opendev.org/#/c/665027/

Change-Id: Ife93834654385430d6a333d3d6ee67b719b6c37e
2019-08-05 08:27:43 -07:00
Michael Johnson 45cf2f328f Add project_id to all of the provider objects
This patch adds a project_id field to all of the provider driver data
model objects.

This is useful for drivers to track/associate objects on the backend.

Change-Id: I8f509677da463bc5d0a7649f2f609045cf9b2dd7
2019-07-22 20:49:19 +02:00
tonybrad de30dbb62f Update tox.ini for new upper constraints strategy
The requirements team has defined a new strategy for handling
upper constraints[1]. This patch applies those strategies to
Octavia.
This also corrects a lower constraint for oslo.messaging.

[1] http://lists.openstack.org/pipermail/openstack-discuss/ \
    2019-May/006478.html

Change-Id: I4cf0ad5d14c4378e33423a4d639ec4d0fcc2a614
2019-06-28 17:07:06 -07:00
Michael Johnson 8997def2b5 Updates Octavia to support octavia-lib
This is the base patch that updates octavia to use the new octavia-lib.
It is backwards compatible by using debtcollector moves.

It adds a new controller process called the "driver-agent".

This patch also adds unit test coverage for a few additional modules.

Depends-On: https://review.openstack.org/#/c/641180/

Change-Id: I438e1548ec0fb6111d1ab85b05015007d9d0a006
2019-03-07 14:40:22 +01:00
zhulingjie 2a057474a8 Update json module to jsonutils
json is deprecated, should use oslo_serialization.jsonutils
instead.

Change-Id: I1392004e32cc835e803c9a953b4581c75049b950
2019-02-20 16:49:43 -08:00