Pecan has made webtest an optional dependency for testing only [1].
Since it is still used for testing we need to add it to our
test-requirements.txt.
[1]: https://github.com/pecan/pecan/pull/140
Change-Id: I4a0985fc93aa071e68491fe60478bee903de39ed
This patch adds a new protocol for listeners called "PROMETHEUS" that exposes
a Prometheus endpoint. This allows detailed metrics collection from Octavia
load balancers.
Change-Id: I3e27e4e57ad955bcd7728426c91f05171a46ef7f
Since 3.7.0, oslo policy started the DeprecationWarning[1] if
deprecated_reason and deprecated_since param are not passed
in DeprecatedRule or they are passed in RuleDefault object.
These warnings are logged for every test which increase the
log size and sometime can full the log buffer and fail the
job.
[1] https://github.com/openstack/oslo.policy/blob/3.7.0/oslo_policy/policy.py#L1538
Change-Id: Ifa8c620fe76517b1c040af5905e6a6c1cdb4c922
md5 is not an approved algorithm in FIPS mode, and trying to
instantiate a hashlib.md5() will fail when the system is running in
FIPS mode.
md5 is allowed when in a non-security context. There is a plan to
add a keyword parameter (usedforsecurity) to hashlib.md5() to annotate
whether or not the instance is being used in a security context.
In the case where it is not, the instantiation of md5 will be allowed.
See https://bugs.python.org/issue9216 for more details.
Some downstream python versions already support this parameter. To
support these versions, a new encapsulation of md5() has been added to
oslo_utils. See https://review.opendev.org/#/c/750031/
In this case, md5 is used to generate etags and to check file integrity when
uploading certs. fingerprints when ssh keys are
being generated and imported. Without this patch, these operations
fail on FIPS enabled systems.
Change-Id: Ib189c6f67946851d37c31a6a8d657460c15f491e
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:
1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.
2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.
[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: I8b78c7b640ab18ddfc809cb4603decc739d494d1
The secure RBAC work requires the of oslo.policy's scope_types argument,
which was made available in 1.32.0. This commit updates to version 2.1.0
so that we're using something more relevant.
Required oslo.context bump.
Change-Id: I48e8882f629886d1b8abadff6e60aad91d1169c7
This allows us to use the Wallaby release marker in versionutils for
deprecations.
Required some other bumps to satisfy lower-constraints.
Change-Id: I3c2be8a5189dcda24c42ed7ab4d8fa33a03c5d3e
ALPN is a TLS extension for application-layer protocol negotiation
within the TLS handshake [1].
This patch extends the Pool API to include a new 'alpn_protocols'
parameter. With this parameter, users can set an ALPN preference list
(descending order of preference) to be advertised by load balancer to
members.
This patch also adds HTTP/2 over TLS support to TLS-enabled pools to the
Amphora provider driver, although default the pool ALPN protocol list
configuration setting has HTTP/2 disabled similarly to the default
listener ALPN protocol list value added in Victoria release.
[1] https://tools.ietf.org/html/rfc7301
Change-Id: I91924486bab22601c15c538c8a5282ad8bc54700
The pyroute2 module had a regression in 0.5.13, but neutron needed a fix
that was included in 0.5.13. It was easier for us to work around it than
for neutron to revert the update to 0.5.13, so we hacked a workaround[1].
A new pyroute2 release, 0.5.14, has been published, so this patch reverts
the hack and bumps our minimum version of pyroute2 to 0.5.14.
[1] https://review.opendev.org/#/c/744045
Change-Id: Ia4357a4f5db07e1a6b0beb11b58ce125215eeef1
Taskflow 4.4.0 contains essential fix for jobboard behaviour on
Storage failure [1].
Also add jobboard_redis_sentinel parameter to allow usage sentinel
for Redis jobboard. Support of this also appear in 4.4.0 version.
[1] - https://docs.openstack.org/releasenotes/taskflow/unreleased.html#bug-fixes
Change-Id: I48245b3322b0f2e5f2c11594a15632501a7e4086
The move to focal for the tox jobs has raised issues with package
versions supporting python 3.8.
This patch bumps the version for the following packages:
PyMySQL==0.8.0
pyroute2==0.5.13
SQLAlchemy==1.2.19
cffi==1.14.0
cryptography==3.0
pyOpenSSL==19.1.0
oslo.db==8.30
oslo.messaging==12.4.0
This patch makes the grenade job non-voting as there is a chain
of gate fix patches required to fully fix all of the issues.
Change-Id: I95cc182a2dbcb2ea872e822ce9372bf5d7df63bd
As Octavia allows to use RedisTaskFlowDriver or
ZookeeperTaskFlowDriver we should install python clients that
allows to work with redis and zookeeper backends.
Story: 2007892
Change-Id: I7312c8c1057618e909339aa7a4dfeb836f4b8f33
ALPN is a TLS extension for application-layer protocol negotiation
within the TLS handshake [1].
This patch extends the Listener API to include a new 'alpn_protocols'
parameter. With this parameter, users can set an ALPN preference list
(descending order of preference).
Presently, the amphora provider driver is limited to http/1.0 and
http/1.1 ALPN protocol IDs. Support for "h2" (HTTP/2 over TLS) depends
on HAProxy 2.0 or newer.
[1] https://tools.ietf.org/html/rfc7301
Change-Id: If08a8169498cdfaa75440e8971ba0caff45ac4c4
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.
Also added and enabled a hacking check that would have caught this.
Change-Id: Idb10f84fd32c50db24f844352cb85de452181439
In https://review.opendev.org/#/c/613709/ octavia was
changed to use octavia-lib for a lot of API driver-related
code and deprecation warnings put in place. Now that
we're in Victoria remove all the deprecation shims and
use octavia-lib exclusively.
Change-Id: If92988150479a7daf465af5f8df22818664a0fce
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems
Update Sphinx version as well.
Set openstackdocs_pdf_link to link to PDF file. Note that
the link to the published document only works on docs.openstack.org
where the PDF file is placed in the top-level html directory. The
site-preview places the PDF in a pdf directory.
Disable openstackdocs_auto_name to use 'project' variable as name.
Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.
openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.
See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html
Change-Id: I87889f73207ecd940963fbe601ccbb79863b96ac
Introduce TaskFlowServiceController which uses taskflow
jobboard feature and saves jobs info into persistence backend.
Jobboard could be operated via RedisTaskFlowDriver or
ZookeeperTaskFlowDriver, that could be set via the config.
RedisTaskFlowDriver is intoduced as default backend for jobboard.
Usage of jobboard allows to resume jobs in case of restart/stop
of Octavia controller services.
Persistence backend saves state of flow tasks that required in
case of resuming job. SQLAlchemy backend is used here.
Bump taskflow version to 3.7.1 and add dependency to
SQLAlchemy-Utils (required for taskflow sqlalchemy
backend support).
Story: 2005072
Task: 30806
Task: 30816
Task: 30817
Change-Id: I92ee4e879e98e4718d2e9aba56486341223a9157
Listeners will now be able to each be assigned their own OpenSSL
cipher string with a new field: tls_ciphers. There is also a new
configuration option, default_listener_ciphers, which specifies the
cipher string to assign to new listeners when one is not explicitly
specified.
Change-Id: I77da6f14063877af0077f2c12df1aab5d5ead187
Depends-On: Id5f4c20abd40dd092558a711987953012d4ae67f
Story: 2006627
Task: 36839
healthcheck middleware adds a /healthcheck url that allows
unauthenticated access to provide a simple check when running
octavia-api behind a load balancer
https://docs.openstack.org/oslo.middleware/latest/reference/healthcheck_plugins.html
Co-authored-by: Michael Johnson <johnsomor@gmail.com>
Change-Id: I10db6226750f7b7c703067d2ab82eea3a9875112
Eventually hacking will move to 2.0.0 (diskimage-builder
is holding it back), and when it does there will be a few
errors to fix. We can get ahead of it so it doesn't
break us with some small changes for these items:
F601 dictionary key $item repeated with different values
F632 use ==/!= to compare str, bytes, and int literals
E501 line too long
While doing this noticed the lower-constraints.txt for
hacking was set at 0.12.0, when test-requirements.txt
had it at 1.1.0, so fixed that as well.
Change-Id: I80d2a5f97e7a4896a8fa765c1971c8bb7e72d211
Added the bashate script style checker to the pep8
check target in tox.ini. It actually found two valid
issues - a bad function declaration and a local variable
issue, but mostly just indentation noise. Fixed all the
complaints.
Change-Id: I43b60e7dcf53acf259c8a52b248fbb8c63d3c8d4
In some deploy production, using volume based instead of localdisk
to protect data and live migrate can perform.
This patch adds:
- creation a cinder volume for amphora
- boot amphora with cinder volume
- config options for cinder client
- unit tests for cinder functionality
Story: 2001594
Co-authored-by: Vadim Ponomarev <velizarx@gmail.com>
Co-authored-by: Margarita Shakhova <shakhova.margarita@gmail.com>
Change-Id: I8181ed696b9ab556e7741c08839d79167aff8350
There was a bug[1] in diskimage-builder when using pypi mirrors
if the host doesn't have "python" available DIB will fail.
So, I am going to bump the requirements minimum in Octavia for
diskimage-builder to be the minimum version with this fix.
[1] https://bugs.launchpad.net/diskimage-builder/+bug/1577105
Change-Id: If0ff2a855ad5b9d9ef3742ad0596c97a6dbf81ed
In some environments running older versions of gunicorn in the
amphora image, gunicorn can fail to start do to /dev/log socket
issues (timing, configuration, etc.).
This patch sets up a dedicated rsyslog socket /run/rsyslog/octavia/log
for gunicorn and haproxy to use. This should resolve any issues with
systemd overriding the /dev/log socket.
This also bumps the gunicorn minimum verison to 19.9.0.
Change-Id: I1e1ad8fde2ad8c1ffba95b1867afb130503b0a5b
A patch[1] introduced a startup check for the socket paths that
impacted a unit test checking debtcollector is finding the new
octavia-lib location of the DriverLibrary.
This patch correctly mocks out this check for this unit test.
[1] https://review.opendev.org/#/c/665027/
Change-Id: Ife93834654385430d6a333d3d6ee67b719b6c37e
This patch adds a project_id field to all of the provider driver data
model objects.
This is useful for drivers to track/associate objects on the backend.
Change-Id: I8f509677da463bc5d0a7649f2f609045cf9b2dd7
The requirements team has defined a new strategy for handling
upper constraints[1]. This patch applies those strategies to
Octavia.
This also corrects a lower constraint for oslo.messaging.
[1] http://lists.openstack.org/pipermail/openstack-discuss/ \
2019-May/006478.html
Change-Id: I4cf0ad5d14c4378e33423a4d639ec4d0fcc2a614
This is the base patch that updates octavia to use the new octavia-lib.
It is backwards compatible by using debtcollector moves.
It adds a new controller process called the "driver-agent".
This patch also adds unit test coverage for a few additional modules.
Depends-On: https://review.openstack.org/#/c/641180/
Change-Id: I438e1548ec0fb6111d1ab85b05015007d9d0a006
This commit adds the functionality of octavia-status CLI for performing
upgrade checks as part of the Stein cycle upgrade-checkers goal.
It only includes a sample check which must be replaced by real checks in
future.
Change-Id: I8b6d134b0bf5b5c82a19177fed6145ef8aaf7507
Story: 2003657
Task: 26146
OpenStack requirements has added pydot to global requirements.
Even though this isn't the proper fix (still pending networkx 2.2 release)
this will resolve the dependency issue for us, so re-enabling our
flow diagrams.
Change-Id: I49856c8fbd7cb6302be4ec97a14b2c10682cf504
The builtin platform.linux_distribution[1] is deprecated and will be
removed in 3.8 and the recommended replacement is distro.
This also raises a "deprecated method" error in pylint.
This patch moves us over to the future by following the recommendation
in the python docs and switching to use the "distro" module.
[1] https://docs.python.org/2/library/platform.html \
#platform.linux_distribution
Depends-On: https://review.openstack.org/578983
Change-Id: I29e2673572eab75b553da6b01143b007701808fd
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
This patch also updates pylint to 1.5.6 which is compatible with
python3.
In updating pylint we have some issues to correct, this patch addresses
those issues so the Octavia code passes pylint 1.5.6.
Change-Id: Iec21f4c803a427059d595612336d67a35ebf9585
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.
Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.
Add openstack-tox-lower-constraints job to the zuul configuration.
See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.
Change-Id: I39bf164b6852a94d1d98d73c45b2e30c37e8971b
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Gregory Thiemonge