Commit Graph

2808 Commits

Author SHA1 Message Date
wu.chunyang 421721a3e1 Enable network_isolation by default
We have already tested this feature on our CI for an entire cycle.
It's ready to be enabled as the default value.

Change-Id: Ifbe862020f7c2b1f6684164d2a26bb5ceb37bec3
2024-04-02 15:19:29 +00:00
Zuul c8d1848c0f Merge "Bump hacking" 2024-04-02 07:33:36 +00:00
Zuul d702a30db4 Merge "Fix error can't detach the configuration group binding" 2024-04-02 04:23:19 +00:00
Zuul a2775ec2aa Merge "Get rid of nose and Proboscis" 2024-03-29 08:26:04 +00:00
Zuul e19a7d4e50 Merge "Add Datastore Version Registry Extension" 2024-03-28 17:44:36 +00:00
wu.chunyang 93f35c7f04 Get rid of nose and Proboscis
they are not maintained for a long time, and trove already switch
the tests to templest.
This commit removes the usage of them. more details see story.

Story: 2010882
Task: 48606

Depends-On: https://review.opendev.org/c/openstack/trove/+/914228

Change-Id: Ie021e0bd708cf8ed13853dea632312af38190205
2024-03-28 07:07:15 +00:00
Zuul 3863b81a18 Merge "db: add commit action after querying" 2024-03-28 07:00:14 +00:00
wu.chunyang c59d54c427 db: add commit action after querying
Change-Id: Ibe39a9b999680cc1fba4eccd9fa2ed0fa41ccb00
2024-03-28 09:06:27 +08:00
Bo Tran Van b127068c20 Add Datastore Version Registry Extension
Current, users can config default datastore registry for managers not by
verions. This make you can flexible when have some experimental
datastore.

With this patch, users with the administrator role can configure the
datastore registry external for each datastore version using a command,
without editing configuration files.

Story: #2010860
Task: #48534
Change-Id: I910a1ba4a9216ab29faeed03198113b4acb2cb81
2024-03-27 05:56:09 +00:00
Sean Mooney 3367c25e77 stop using passlib
passlib is unmaintained and has not had a release since 2020
a recent bcrypt release just broke passlib
see https://github.com/pyca/bcrypt/issues/684

trove's use of passlib is pretty tirval so this change
just removes it as a depency and delegate the random password
generation in trove.common.utils to the generate_random_key
function in the trove.common.crypto_utils module

Change-Id: I6b6c64147c627025d5f89db6032d1c54445df94f
2024-03-26 14:46:55 +00:00
Bo Tran dbb18bea10 Fix error can't detach the configuration group binding
Story: #2011063
Task: #49697
Change-Id: I275bd77736020f02678fc34332edd277f5e2a6b6
2024-03-25 11:43:04 +07:00
Takashi Kajinami bb260949d4 Bump hacking
hacking 3.1.0 is too old.

Note:
We can't directly bump hacking to 6.x.0 (which is the latest major
version) because of the existing cap by diskimage-builder. The cap is
now being updated by [1].

[1] https://review.opendev.org/c/openstack/diskimage-builder/+/909336

Change-Id: I8778a7decc6669b4d95d6886c971433e7c34c5c8
2024-03-22 00:03:05 +09:00
Andrew Bogott 86a870cc50 Establish consistency on db_info.addresses between taskmanager and instance
We had two different workflows here using the same variable but expecting
it to have a totally different format. The taskmanager dns code
was populating that variable with nova api output; the database
list code populated it manually with a custom-format structure.

This seems to have survived a long time because these are both edge
cases: the dns workflow was only traversed with dns integration
is switched on (unusual) and the database list only relied on IP
addresses when hostnames weren't present.

This patch picks a winner (the custom structure used by the database
listing code) and modifies the dns workflow to live with that same format.

This is spackle over a whole lot of bitrot, but should get
both use cases working properly.

Story: #2010077
Task: #45568
Change-Id: I5832733dd312db24d2d8047658fdd1af9f4e700a
2024-03-20 08:24:38 +00:00
wu.chunyang ab83ad7c85 db: Remove use of autocommit
Change-Id: I3f0d384f353069f1eb186bbfa2aa3d8afa969e05
2024-03-19 22:03:56 +08:00
Zuul 27e5a710ce Merge "Change postgresql socket path to a persistent path" 2024-02-23 13:23:59 +00:00
Takashi Kajinami 235bd9acd7 Remove global default_password_length
This option was deprecated multiple cycles ago[1] in favor of
the datastore specific options.

[1] f4cbaf6ccd

Change-Id: I1b40fb528d5df52f132a1d8cf027a42d8b65d544
2024-01-30 03:09:33 +00:00
wu.chunyang f151fd91d9 Change postgresql socket path to a persistent path
This patch changes the default socket patch on the host from
"/var/run/postgresql" to "/var/lib/postgresql-socket".
ensuring that the  database container starts properly after
a Nova instance is restarted.

Story: 2010599
Task: 47471

Change-Id: I54a53cc802bbd20bb9e2c6508919e4c8dda38f2a
2024-01-30 03:08:55 +00:00
Zuul c179534ba9 Merge "The backup_aes_cbc_key option should be secret" 2024-01-25 08:37:52 +00:00
Zuul 40fdb7b44f Merge "Change mysql socket path to a persistent path." 2024-01-19 04:52:06 +00:00
Takashi Kajinami 56c25337a6 The backup_aes_cbc_key option should be secret
This option takes a plain key data.

Change-Id: Ic8a438632ea93da93807c37374eee3b48d06f2d2
2024-01-13 01:04:11 +09:00
wu.chunyang 9d31e46f00 Change mysql socket path to a persistent path.
This patch changes the default socket patch on the host from
"/var/run/mysqld" to "/var/lib/mysqld". ensuring that the
database container starts properly after a Nova instance is restarted.

Story: 2010599
Task: 47471

Change-Id: Idb0366ad1d73c85126e18fccd58f1e52176b63de
2024-01-08 18:06:43 +08:00
wu.chunyang d9b4c7bd24 Fix potential network conflict
when both the management and business network have a gateway.
The guest vm may generate an unexpected default gateway
which is the businees gateway as opposed to the management one.
In this case, trove guest VM may failed to connect with the
openstack control plane such as swift service while doing the
backup restore action.
In addition, if the mgmt port and the business port have the same
IP address. This also may cause network conflict

This change disable the user defined port once the guest-agent
starts when the network isolation is enabled.

Change-Id: I7a96952f34ce5f4aead837b94daedd83c0a871d8
2023-12-28 10:03:52 +00:00
Bo Tran 8b6ff821a1 Cinder Support For Boot Volume
When use cinder to store data dir of database, also create rootdisk in cinder.

Story: #2009245
Task: #43418
Change-Id: Ia5841222c7a70cb0c88078575b4d8b4f7988d5e0
2023-12-13 09:12:18 +00:00
wu.chunyang 9f7b9a95e8 Revert "Fix docker daemon failed to set IPv6 gateway"
This reverts commit ae27a32a9f.

Reason for revert: This commit didn't fix the issue

Change-Id: Ief5a87724ccaed2f0c63cf32672a7437f384179f
2023-12-12 01:38:54 +00:00
wu.chunyang 21b4cc2cf7 Don't configure ip route for slaac ipv6 subnet
for the ipv6 subnet with ipv6_ra_mode is slaac or dhcpv6-stateless,
we don't need the configure the ip route for the nic in these
subnet. otherwise, docker daemon may returns error:
failed to set IPv6 gateway while updating gateway: file exists.
refer to: https://docs.openstack.org/neutron/latest/admin/config-ipv6.html

Change-Id: Id7f537039615479e03dcd546b7d738ca32f740db
2023-12-11 10:11:52 +08:00
wu.chunyang ae27a32a9f Fix docker daemon failed to set IPv6 gateway
This commit sets "net.ipv6.conf.eth0.accept_ra" to 0 to avoid the
file exists error.
refer to: https://github.com/moby/libnetwork/issues/2455

Change-Id: Ib0d7cec54bc79b3e29a95c298435b007e769e565
2023-12-04 15:53:44 +08:00
Gabriel Adrian Samfira e998b68866
Add region name to trove-guestagent
In a shared keystone multi-region setup, we must set the region_name
when using any of the OpenStack clients. Without the region name,
clients may attempt to use the API endpoints in the wrong region.
Backups currently fail if there are multiple regions created in the
same keystone catalog, due to the fact that the trove guest agent
attempts to connect to the S3 endpoint in another region.

Story: 2010950
Task: 48994

Change-Id: I9bd497c85c6eba67485840d07a80cca8eed27994
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-10-27 01:02:49 +03:00
wu.chunyang b5e85ae2ae follow up: 3bbeeb87e0
fix typo

Change-Id: Ib7780066cce4b3f70c40ebc1280d5e86dae340a1
2023-10-25 17:47:35 +08:00
Hirotaka Wakabayashi c167159c4a Fixes the way to check db instance status
This PR changes the way to check instance status. Since Victoria, trove
has change the db instance status name from `RUNNING` to `HEALTHY`[1].

Original problem:
Some clustering databases like Apache Cassandra fail to update cluster
status because they check db instance status using `RUNNING`.

[1]: a0a10f0b94

Story: 2010147
Task: 45791
Change-Id: Iaa032fb46ed51b6e416e7d4efdfd272924ba146b
2023-10-25 01:12:05 +00:00
Bo Tran 4c83bb8862 Separate backup docker image for each database version
Co-Authored-By: wu.chunyang <wchy1001@gmail.com>

Story: #2010770
Task: #48087
Change-Id: I08063748e15de6767b437aa443311d41e25ed578
2023-09-21 01:21:32 +00:00
Zuul 05c30cbcf6 Merge "Fix: failed to create the replicas" 2023-08-30 08:52:15 +00:00
Zuul fe13333a61 Merge "Change StrictRedis usage to Redis" 2023-08-29 12:11:51 +00:00
wu.chunyang ccaa00d452 Fix typo
trivial fix

Change-Id: I17e42b4e8ac60e75d4e2fb22618fde055ff3e358
2023-08-29 11:21:30 +08:00
wu.chunyang ae3c0a225a Fix: failed to create the replicas
guest-agent should get the user-defined port's ip for primay instance
rather than the management ip when network_isolation is set to true.

Change-Id: I6f24cf623aa2836517dffda99305d127413d9df2
2023-08-29 03:10:32 +00:00
Zuul 94ec942eaa Merge "Adds postgres guestagent test driver" 2023-08-09 10:42:35 +00:00
Hirotaka Wakabayashi d2602bbf38 Adds postgres guestagent test driver
This PR adds postgres guestagent test driver.

Story: 2010761
Task: 48059
Change-Id: Id15129cbd93502ae06cb600b31553c1050ccd9ca
2023-08-05 06:39:51 +09:00
Zuul 651cee350f Merge "Fix postgresql database creation failures from prepare func" 2023-08-04 05:28:09 +00:00
wuchunyang 2f755b64b3 Add network isolation for trove
this PR adds a network_isolation config option for trove,
with network_isolation enabled, trove guest agent will
plug the user-defined port to database container by
docker host_nic network driver which is implemented in this PR.

docker host_nic network driver is a simple driver to plug host
nic to a container. this driver supports ipv4,ipv6 and dual-stack.

for more details please see the story.

story: 2010733
task: 47957

Change-Id: I35d6f8b81a2c5e847cbed3f5bc6095dc1d387165
2023-08-03 11:35:04 +00:00
wu.chunyang 8cd982be5c Fix postgresql database creation failures from prepare func
postgresql needs to restart during the bootstrap, but the postgres
driver still reports the healthy status.

this commit resets the healthy_counts when status is not healthy,
and make state_healthy_counts configurable.

Change-Id: I746b86326790dbc667f4f0d6dabcd1a656502273
2023-08-02 02:18:08 +00:00
Zuul 5d9021592b Merge "Remove logics for old Python versions" 2023-07-21 03:44:25 +00:00
likui 773115adce Remove the idle_timeout option.
The option was replaced 4 years ago[1] by connection_recycle_time. The
option is not anymore present in sqlalchemy. It's the good time to
remove it.

[1] 6634218415
[2] https://review.opendev.org/c/openstack/oslo.db/+/778441

Change-Id: Id288e09688062b007d57f09aebcd7c447e56f04d
2023-07-20 14:26:40 +08:00
Takashi Kajinami 5535153d8e Remove logics for old Python versions
Python 2.y and Python <= 3.2 are no longer supported, so we no longer
need to maintain logics for such old versions.

Also the check_python_version method is removed by this change, because
the required python version is now enforced by setup.cfg.

Change-Id: Ifc5e618a791bdf591e0ec61075089dd992c73e0c
2023-07-20 01:57:45 +00:00
Zuul 17af8eab8c Merge "mysql: explicitly use utf8mb3" 2023-07-04 04:56:11 +00:00
Hirotaka Wakabayashi 0d4a24c71d Fixes cluster creation error
This PR adds the import statement to fix cluster creation error
and enables flake8-f821 rule checks.
F821 rule checks "undefined name"

Original problem:
we have currently got `NameError: name 'common_glance' is not defined`
error when creating a cluster because I failed to edit the patch[1].

Co-Authored-By: wu.chunyang <wchy1001@gmail.com>

[1]: 2f52b2fddb

Story: 2010151
Task: 45797
Change-Id: I846f433ff35a9638a070382bf89e02db617dcd3d
2023-06-30 14:32:32 +08:00
Alexander Epaneshnikov 3ba1f0d955 mysql: explicitly use utf8mb3
this fixes mysql v8.x.
in mysql 8.0+ utf8mb3 is deprecated and utf8 alias doesn't work.

Change-Id: I548eb43a3d5adbd84957b6b92d4b172bfdfb80a8
2023-06-29 08:21:52 +00:00
Zuul 5c12f72fc5 Merge "Fix "create database" failed for postgress" 2023-06-26 04:15:26 +00:00
wu.chunyang 0fd7bf2ad9 Enable flake8 E129 rule
E129: Visually indented line with same indent
as next logical line (E129)
refer to: https://www.flake8rules.com/rules/E129.html

Change-Id: Ib34fe5f5eb2aa5bc4980ed67beb027d2bf7186e9
2023-06-19 10:38:10 +08:00
wu.chunyang cd11d7677e Fix "create database" failed for postgress
This PR stops using the psycopg2’s connections context manager
because "CREATE DATABASE" command must be run outside any
transaction[1].
[1]: https://www.psycopg.org/docs/usage.html#transactions-control

Co-Authored-By: hungnt1 <sudo.nguyenhung@gmail.com>
Co-Authored-By: Hirotaka Wakabayashi <hiwkby@yahoo.com>

Story: 2010761
Task: 48059
Change-Id: I73ec6c659d8ad7216460055077737429c878df33
2023-06-16 13:54:08 +08:00
wu.chunyang 89514c0d5b Don't ignore E125 check
E125: https://www.flake8rules.com/rules/E125.html
Change-Id: I8156bb87ce3e4d1a19d2df9e2c3f67f789bffa46
2023-06-14 14:35:07 +08:00
wu.chunyang c1761147ba Don't ignore H306 pep8 check
H306: import in alphabeical order

Imports should be grouped in the following order:
standard library imports
related third party imports
local application/library specific imports
put a blank line between each group of imports.
https://docs.openstack.org/hacking/latest/user/hacking.html#imports

Change-Id: I2c188d27f0595ac3b2d71f4612d93829915e389c
2023-05-30 17:51:16 +08:00