they are not maintained for a long time, and trove already switch
the tests to templest.
This commit removes the usage of them. more details see story.
Story: 2010882
Task: 48606
Depends-On: https://review.opendev.org/c/openstack/trove/+/914228
Change-Id: Ie021e0bd708cf8ed13853dea632312af38190205
For the backups created prior to Victoria which may be encrypted, the restore
function in Victoria or later release should correctly decrypt the backup data.
Backup encryption from Victoria is disabled.
Bump the backup docker image tag to 1.1.0.
Change-Id: I7abf5563b22ab1914fa355c089a3793da07f2215
* MySQL 5.7 and MySQL 8.0 need different percona-xtrabackup package version.
Added Percona XtraBackup 8 support for MySQL 8.x backup and restore.
* Construct different backup container image names for MySQL 5.7 and MySQL 8.0
based on the default option value.
* Two docker images are uploaded for backup/restore:
openstacktrove/db-backup-mysql5.7:1.0.0 and
openstacktrove/db-backup-mysql8.0:1.0.0. Trove guest agent can automatically
choose the approriate one based on the datastore version.
* Added option "secure-file-priv=NULL" in MySQL config template to fix
https://github.com/docker-library/mysql/issues/541.
* Stop using IDENTIFIED BY in GRANT clause (also REVOKE). Starting with MySQL 8
creating a user implicitly using the GRANT command is not supported.
Story: #2008275
Task: #41143
Change-Id: Ibdec63324b1b39ba9b8a38dbe529da17bbb06767
Significant changes:
* Using docker image to install datastore.
* Datastore image is common to different datastores.
* Using backup docker image to do backup and restore.
* Support MariaDB replication
* Set most of the functional jobs as non-voting as nested
virtualization is not supported in CI.
Change-Id: Ia9c97a63a961eebc336b70d28dc77638144c1834
* Hard delete the datastore_configuration_parameters table record.
* Make 'datastore_version_id' nullable for 'instances' table.
* Check if the datastore version is still being used before removal.
Story: 2007563
Task: 39451
Change-Id: I84e4a31f14f9327cc01ff2d699167d91112e1565
- Execute test groups in serial to make sure no more than 2 database
instance are created at the same time.
- Remove some unneccesary tests
- Remove unneeded datastore, e.g. 'Test_Datastore_1'
- Remove unsupported trovestack subcommands
- Move unsupported DIB elements to the 'deprecated-elements' folder
- Decrease default value of 'agent_call_high_timeout' to 5min
- Add initial_deplay for pooling task
- Use socket file to connect with database instead of using localhost
IP
Change-Id: Ie5030a671fbeb453eafa6cbe04e08da7b52e33c9
- 'HEALTHY' means the db service is responsive, 'ACTIVE' means the db
service is alive.
- Remove the CI job fakemodetests, but will add similar testing task in
the future.
- Fix the periodic CI job
- Remove MongoDB and related jobs
Change-Id: I5abe9091ba203297dc87db5fba139179166321f7
The service tenant deployment model means Trove creates most of the
resources(vm, volume, security group, etc.) relating to a database
instance in the Trove service tenant rather than the end user.
With this deployment model, most of the related resources behind the
scenes are invisible to the user, which is a more secure deployment
model for either private or public cloud provider.
DevStack should follow this model as it will be recommended for Trove
deployment in production.
Changes included in this patch that are necessary in order to make that
happen:
- Add 'admin' role to Trove service user(username: trove, project:
service) in DevStack.
- Create Trove management network resources for Trove service user in
DevStack.
- Enable Trove remote client configuration by default in DevStack.
- Mainly use alt_demo user in alt_demo project for integration tests,
config trove user as the admin role user in integration tests.
- Disable the module related tests(module_groups) for now because of no
use cases but need effort to fix all the failed tests in the service
tenant model.
Story: #2005445
Task: #30489
Change-Id: I2efb69d3d50344914a875b773f62a227dba2ccaf
Functional tests are executed by trovestack script, which uses python
in default OS version which is 2.7.
This change wraps python in tox and executes trovestack int_tests in
virtual enviroment, starting them in python3.
Any future python version change could be managed by tox framework.
Change-Id: I3a849978241d3d0669ef9e1c802ff504ac3c32cb
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I9ee34642c700d1e6ba9c2f3891b7fa1f7f7e1e1d
Depends-On: I8989fd4798e80eae27408017e1543819a68b4ab1
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Setting up the connection to mysql doesn't work if the host
is a IPv6 address. Make sure we only use IPv4 addresses
at the moment.
Also removed pool_recycle and echo kwargs from EngineFacade
as they are no longer supported.
Change-Id: I6464c0031ba5b504ff103d8f62093d252deb87bb
spelling mistake in api/configurations.py at line 253.
The word "invalid" is misspelt.
Closes-Bug: #1585478
Change-Id: I84fbee9b947509108348ce312a148e0c49afbd28
Replaced basestring with six.string_types
Replaced (int, long) with six.integer_types
Replaced unicode with six.text_type
This patch is generated by the following tool using
'basestring', 'long' and 'unicode' options.
Manually replaced unicode(ex) with oslo_utils.encodeutils(ex)
https://github.com/haypo/sixer
Command:
python sixer.py -w 'basestring,long,unicode' trove/
Partially implements: blueprint trove-python3
Change-Id: I6fa42674060067663ef819247ea793ef3d8aa0da
If integration tests are executed in an environment with multiple nics,
nova complains and the instance does not start. Given that the
shared_network parameter is already available with that specific
meaning, use it on instance creation.
The default value (None) should ensure that the default behavior
does not change.
Change-Id: I8554ebf111cad829deb34d3d23d18f72d7f13554
The datastore and its version is now always passed on instance creation for
integration tests.
Previously the default datastore (and version) were used, ignoring the
settings in the test configuration.
Change-Id: Ib3e78bb68aaf992177b9d9cd1d89e84fb54830ed
Depends-On: Ifa4496eb1c9e2fe053978c708e7fa1069f12cb3f
The int-tests in Trove are very MySQL specific, which makes it difficult
to reuse code for other datastores. This changeset breaks them down
into 'groups' and 'runners.' Runners can be over-ridden to add
datastore specific handling/tests. This should allow most generic
code to be reused across datastores, while also providing for datastore
specific enhancements.
Runner implementations are stored in a new package
'trove.tests.scenario.runners'. A datastore-specific implementation can
be added to the appropriate runner module file. Its name has to match
'PrefixBaseRunnerClassName' pattern, where 'BaseRunnerClassName' is the
name of the default runner and 'Prefix' is the datastore's manager
name with the first letter capitalized.
Example:
Given the default implementation for negative cluster tests in
'trove.tests.api.runners.NegativeClusterActionsGroup'. One can
provide a custom implementation for MongoDB (with manager mongodb)
in 'trove.tests.api.runners.MongodbNegativeClusterActionsRunner'
This initial changeset adds tests for basic actions on instances
and clusters. Some basic replication tests were also migrated.
The concept of a helper class for datastore specific activies
was also created. This makes it easy to have tests use standard
methods of adding data and verifying that the datastore behaves
as it should.
Vertica was refactored to use the new infrastructure.
Running the tests can be accomplished by specifying one of the
new groups in int-tests (see int_tests.py for the complete list):
./redstack kick-start mongodb
./redstack int-tests --group=instance_actions --group=cluster
or
./redstack int-tests --group=mongodb_supported (to run all
tests supported by the MongoDB datastore)
As with the original int-tests, the datastore used is the one
referenced in test configuration file (test.conf) under the
key dbaas_datastore. This key is automatically set when
kick-start is run.
Additional Notes:
Also temporarily disabled volume size check in
instances tests.
It is supposed to assert that the used space on the
Trove volume is less that the size of the volume.
It however often fails because 'used' > 'size'.
From inspection of the instance it appears that the reported
'used' space is from the root volume instead of the
attached Trove volume. Plus it sometimes returns int instead of float.
Change-Id: I34fb974a32dc1b457026f5b9d98e20d1c7219009
Authored-By: Petr Malik <pmalik@tesora.com>
Co-Authored-By: Peter Stachowski <peter@tesora.com>
This patchset helps reducing the ignored pep8 errors.
Fixed some ignored hacking rules, as listed follows:
- E111 indentation is not a multiple of four
- E122 continuation line missing indentation or outdented
- E123 closing bracket does not match indentation of opening bracket's line
- E128 continuation line under-indented for visual indent
- E251 unexpected spaces around keyword / parameter equals
- E265 block comment should start with '# '
- E713 test for membership should be 'not in'
- H105 Don't use author tags
- H306 imports not in alphabetical order
Change-Id: Iadf2af4f6ec90420153ad63b5a41650392ef2cbd
Under a 32 bit OS, the type of max/min is long, not int.
Allow both long and int as valid types for these parameters.
Change-Id: If0ae44090229e2bf77405d81f06698e75ac54b97
Closes-Bug: #1446174
Attempt to make the integration tests run faster in the following ways:
1. Remove the duplication of restore tests: We test restore of both an
incremental and a non-incremental backup, which really tests the same
code path twice. Testing a restore of an incremental backup should be
sufficient since it's a superset of the non-incremental case. In
simpler words, there is no case when restore from a full backup will
fail and restore from an incremental backup will succeed, so it is
sufficient to test the latter scenario only.
2. Spend less time idly waiting for instances to go ACTIVE. The idea is
to be able to run other tests during the waiting periods, so that we
can parallelize tests and reduce the overall time taken to run the
suite.
Closes-bug: 1394972
Change-Id: Ia9f9c4198f2dcafdacefe46bc48ad20d4714a70e
Wikipedia's list of common misspellings [1] has a machine-readable
version. This patch fixes those misspellings mentioned in the list
which don't have multiple right variants (as e.g. "accension", which can
be both "accession" and "ascension"), such misspellings are left
untouched. The list of changes was manually re-checked for false
positives.
[1] https://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings/For_machines
Change-Id: Iccc6e4825fd31c0d72d06d5cae69aa350fbe100f
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
attrs_exist() checks the attributes in the given list against the
attributes in the expected_attrs list. The name expected_attrs is
misleading since based on the name one would expect the given list
to contain all the attributes in the expected list but thats not
what this function does. Based on this, the following changes were
made:
- renamed attr_exist() to contains_allowed_attrs() which checks
whether all the attributes in the given list is present in the
allowed_attrs list.
Change-Id: Ie169271e211aa93c1bd3f64a2723aa1cd03a56e9
Closes-Bug: #1285964
the configuration parameters payload was originally just 'max'
and 'min' for the range and was changed to 'max_value' and
'min_value' in the output. Reverting this back to the original
v1 spec of 'max' and 'min'.
Closes-Bug: #1390317
Change-Id: I9628313e4154048a6aa9a089a6e3ec18f9af1d33
- Updated _validate_configuration
- Updated from querying the database every parameter to fetching
all the db rules in one call and passing them as a parameter.
- Check for empty rules set and raise a 422
- New error response: Configuration groups are not supported for this
datastore: mysql 5.5.
- Updated invalid parameter to raise a 422 instead of the previous
404 that the load_parameters_by_name call used to raise.
- New error response message for parameter not found: "The configuration
parameter auto_increment_offset is not support for this datastore:
mysql 5.5.
- Added unit tests covering the _validation_configuration method
- Modified test_configurations_create_invalid_values integration test
to handle new error status code 422 instead of 404 for invalid
configuration values
Closes-Bug: 1367107
Change-Id: I9cc873745afe19336c1fe5ef35ffdd3efde127b3
Currently the users do not have the ability to rename an
instance. The current PUT call, does not allow the user
to modify individual attributes of the instance.
Hence this review introduces a new PATCH call for updating
instance name and/or configurations.
The long term plan is to deprecate PUT in favor of PATCH.
Implements: blueprint Allow users the ability to update the instance name
Author: Iccha Sethi <iccha.sethi@rackspace.com>
Co-Authored-By: Theron Voran <theron.voran@rackspace.com>
Co-Authored-By: Riddhi Shah <ridhi.j.shah@gmail.com>
Change-Id: I25c3b0906191afdc587f5ddd5777b3185dc080d4
with the parameter deleted we can not rerun the tests on the same
instance without clearing the datstore_configuration_paramters
table and reloading the params. This will readd the parameter
that is deleted.
Change-Id: Ic1cdfa416a14c42b6c4323612ec609915224b6ee
Closes-Bug: #1379083
Changes:
Add datastore configuration paramters to configuration model
Add ability to CRUD configuration parameters
adding a manage cmd to import the config params
partially implements blueprint configuration-parameters-in-db
Change-Id: I51d5281ba8922dc19c74e7c6e972f40e45416d50
Adds flexibility to configuration tests to make used mysql configs
configurable.
Change-Id: Id7ef621c185af6514b87ec4f79001461bc3851c3
Closes-bug: 1334888
Reasons:
- Both ConfigurationView and DetailedConfigurationView
for configuration-groups return only the Datastore
Version ID and not human the readable Datastore Name / Version
Changes:
- adding datastore version name to:
- configuration-create;
- configuration-list;
- configuration show responses.
- tests;
Change-Id: I8fbefd57d40838dafcead0077bfbc6b49c27a8eb
Closes-Bug: #1326584
This commit adds created and updated timestamps to the configuration
groups list and details calls. It also adds instance_count to the
configuration details call. Timestamps on configuration groups can
serve as a secondary indicator for making config edits, as well as a
potential time marker for correlating when detrimental config changes
were made. It will also help to see how many instances use a
particular config and users will be able to see up front how many
instances the config changes will affect.
Note that instance_count is generated on the fly and not stored in the
config db, hence the getattr() usage.
Change-Id: I0138a28e9fb4c0c045a1a26df3f8a282d80a7274
Implements: blueprint minor-config-edits
Docimpact: change to configuration payloads
The test_get_configuration_details_from_instance_validation
tests were randomly timing out during certain int-test runs.
So the timeouts for these have been upped from 10s to 20s to
allow them to succeed more consistently.
Change-Id: I38451a2172397df9703e5fc90101d81c4fa953c4
Closes-Bug: #1326591
Added support for flake8 H402: one line docstring needs punctuation.
Reasons:
- H402 is disabled in tox.ini, it needs to be enabled to make code
more flake8 compliant.
Changes:
- Updated tox.ini to enable H402 rule.
- Updated code for H402 violation.
- Updated one word docstrings to be more meaningful.
Change-Id: I5e67b641717c9704ef4b9f6267390561e4840fff
Running configuration-attach with 2 different configuration groups
against the same instance can result in unintended combinations of
dynamic parameters. In order to address this, 3 changes have been
made:
1. Return an error if a configuration-attach attempt is made to an
instance that already has an attached configuration group.
2. Only allow configuration-attach when instance is in ACTIVE state.
3. Always set RESTART_REQUIRED on configuration-detach.
Change-Id: I35770eafdc2841ac58b770851efc953e0370c2cb
Closes-Bug: #1301683
This allows users to make custom configurations groups and apply them to
multiple instances. Configuration parameters that can be set are listed
in the validation_rules.json file.
implements blueprint configuration-management
Change-Id: I99b0bfe51c131ace30774e75e30d620101ed2e0b
wu.chunyang