they are not maintained for a long time, and trove already switch
the tests to templest.
This commit removes the usage of them. more details see story.
Story: 2010882
Task: 48606
Depends-On: https://review.opendev.org/c/openstack/trove/+/914228
Change-Id: Ie021e0bd708cf8ed13853dea632312af38190205
- 'HEALTHY' means the db service is responsive, 'ACTIVE' means the db
service is alive.
- Remove the CI job fakemodetests, but will add similar testing task in
the future.
- Fix the periodic CI job
- Remove MongoDB and related jobs
Change-Id: I5abe9091ba203297dc87db5fba139179166321f7
The service tenant deployment model means Trove creates most of the
resources(vm, volume, security group, etc.) relating to a database
instance in the Trove service tenant rather than the end user.
With this deployment model, most of the related resources behind the
scenes are invisible to the user, which is a more secure deployment
model for either private or public cloud provider.
DevStack should follow this model as it will be recommended for Trove
deployment in production.
Changes included in this patch that are necessary in order to make that
happen:
- Add 'admin' role to Trove service user(username: trove, project:
service) in DevStack.
- Create Trove management network resources for Trove service user in
DevStack.
- Enable Trove remote client configuration by default in DevStack.
- Mainly use alt_demo user in alt_demo project for integration tests,
config trove user as the admin role user in integration tests.
- Disable the module related tests(module_groups) for now because of no
use cases but need effort to fix all the failed tests in the service
tenant model.
Story: #2005445
Task: #30489
Change-Id: I2efb69d3d50344914a875b773f62a227dba2ccaf
Functional tests are executed by trovestack script, which uses python
in default OS version which is 2.7.
This change wraps python in tox and executes trovestack int_tests in
virtual enviroment, starting them in python3.
Any future python version change could be managed by tox framework.
Change-Id: I3a849978241d3d0669ef9e1c802ff504ac3c32cb
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
Remove pycrypto completely from requirements and fix the
related tests.
Closes-Bug: #1749574
Depends-On: I5c0c1a238023c116af5a84d899e629f1c7c3513f
Change-Id: Ibfedd9e2ab0a5e78959108112f57103a089f02d1
Signed-off-by: Fan Zhang <zh.f@outlook.com>
Server side support for the new 'reapply' command.
This reapplies a given module to all instances that it had
previously been applied to.
Originally, a module designated live-update would automatically
be re-applied whenever it was updated. Adding a specific
command however, allows operators/users more control over
how the new payload would be distributed. Old 'modules'
could be left if desired, or updated with the new command.
Scenario tests were updated to test the new command.
DocImpact: update documentation to reflect module-reapply command
Change-Id: I4aea674ebe873a96ed22b5714263d0eea532a4ca
Depends-On: Ic4cc9e9085cb40f1afbec05caeb04886137027a4
Closes-Bug: #1554903
Fixed the module-instances command to return a paginated
list of instances. Also added a --count_only flag to the
command to return a summary of the applied instances
based on the MD5 of the module (this is most useful
for live_update modules, to see which ones haven't been
updated).
Also cleaned up the code a bit, putting some methods
into files where they made more sense (and would cause
less potential collisions during import).
Change-Id: I963e0f03875a1b93e2e1214bcb6580c507fa45fe
Closes-Bug: #1554900
As an aid to debug guest failures, a Metaclass was added
to the TestRunner class that allows failed tests to
pull back and echo the guest log from 'registered'
instances.
This uses the guest-log feature that is available for
all datastores.
IDs for instances created in the tests were registered
to report on failures.
Two guest-log tests were also commented out, as they
seem to not work properly: see
https://bugs.launchpad.net/trove/+bug/1653614
This was discovered while testing the new retrieval
code. Other tests were also modified so that
'SkipTest' exceptions would be raised properly.
Change-Id: I448bd2f0181351ef1536e20c41f9d45f95478587
Partial-Bug: 1652964
A method for specifying 'priority' modules plus a way to rank the order
in which modules are applied has been added. Two new attributes
'priority_apply' and 'apply_order' are available in the payload
on create and update. In addition, an is_admin flag was added as an
automatic attribute, to be set when someone with admin credentials
creates a module or updates an existing module with 'admin-only'
options. This allows better control on the driver plugin
side with regards to security concerns, etc. The attribute is
now passed in to the guest 'apply' interface for use by the driver.
All three of these attributes are stored in the Trove database.
An admin can create a 'non-admin' module by passing in --full_access
on the command line (or python interface). This will cause an
error if any admin-only options are selected.
Scenario tests have been added to verify that the modules are
applied in the correct order. The timestamp for the 'updated'
field on the guest was also enhanced to allow for fractional
seconds, since most applies take less than a second.
The issue where modules were allowed to be applied even if
they belonged to a different datastore has been fixed and
scenario tests added to check for this case.
Change-Id: I7fcd0cf12790564ba62e7d6451fff96f763e539d
Implements: blueprint module-management-ordering
If a node is really slow then a single test can take longer than an
hour. It will then fail as the client it has cached will now have an
expired token. To alleviate this, the caching of the clients was removed
(since the time saved on caching the clients is probably negligible
anyway).
Checking the client code was also cleaned up for a number of tests.
The cluster tests were modified, however the code was reverted as
there is no way to test it at present (they will not run as they
haven't been modified to work with the new neutron set up). This
will be fixed in a separate changereq (since they are currently not
being tested in the gate, this should not hold up development).
Change-Id: I63d5b78084486b806bff38068260e42128c0d0ed
Closes-Bug: 1647002
In the SQLAlchemy keep_alive class, MariaDB is failing
as pymysql reports an invalid packet sequence.
MariaDB seems to timeout the client in a different
way than MySQL and PXC, which manifests itself as the
aforementioned invalid sequence. It is now handled
as a special-case exception.
With this fix, the MariaDB scenario tests now pass.
The scenario tests were also tweaked a bit, which aided
in the testing of the fix. 'group=instance' was created,
plus instance_error properly interleaved with
instance_create. _has_status now calls get_instance with
the admin client so that any faults are accompanied by
a relevant stack trace. Cases where the result code
was being checked out-of-sequence were removed, and explicit
calls to check the http code for the right client were added.
The replication error messages for promote and eject were
enhanced as well to attempt to debug spurious failures.
One of those failures was 'Replication is not on after 60 seconds.'
This was fixed by setting 'MASTER_CONNECT_RETRY' in the mariadb
gtid replication strategy as was done in:
https://review.openstack.org/#/c/188933
Finally, backup_incremental was added to MariaDB supported
groups and cleaned up elsewhere.
Closes-Bug: #1621702
Change-Id: Id6bde5a34e1d79eece3084f761dcd153c38ccbad
If you apply a module, remove it, update the module with a new name,
and apply it again, module-query will still show the old name.
This also applies to datastore, datastore_version and type.
All four attributes are now updated correctly and scenario
tests added.
When writing the scenario tests it became evident that resetting
a module from a datastore to all_datastores was broken too,
so this was fixed (along with all_datastore_versions). This
fix required a change to the Trove client as well.
Change-Id: Ic2e9226bcd31c4a79d84ecb7941a47875eabd73d
Depends-On: I0f226f09db08f635c542b27d8d2f898050d327fa
Closes-Bug: #1611525
Closes-Bug: #1612430
This changeset stitches together all the different groups
in a way to have 3 instances being created at any given
time (where possible). The groupings will be:
-- module
|- configuration
-- instance_actions
-- database_actions
|- backup
-- instance_actions
-- user_actions
|- root_actions
-- guest_log
-- replication
Times (on a fast test machine) went from 2230s to 1720s, an
improvement of 23%.
The group labels were moved into the __init__.py file to facilitate
being accessed by different group implementations (to avoid a
circular import issue).
Tests were also cleaned up and homogenized wherever possible.
Change-Id: Idc3daab243261584ac5fa9350d952f5bb3984300
Closes-Bug: #1571092
The module scenario tests were broken into separate
parts to facilitate parallel runs in the gate.
Change-Id: Ie3262a9e1e5cf88a0b578f648a91e01d564b8812
Partial-Bug: #1571092
The recent addition of module support in Trove (see
https://blueprints.launchpad.net/trove/+spec/module-management ) does
not include the New Relic license module driver. This has been added.
A decorator to streamline writing drivers (by handling common errors)
was also added, and the ping driver modified to use it as well.
Since this code is dependent on having an image with New Relic
installed, no changes were made to the scenario tests with
respect to this new driver.
An addition flag was added to the 'apply' interface that passes in
whether a module was created with 'admin options.' This allows
some rudimentary access control to be implemented.
Depends-On: I6fb23b3dbbec98de9ee1e2731bcfc56ab3c0ca42
Change-Id: I282cf533c99e351d23f3b86aae727ae4bf279b64
Closes-Bug: #1571711
The scenario tests need to have the group classes broken
up to allow interleaving of the different parts of the tests
so that the gate can finish in a reasonable amount of time.
The runners, however, need to be the same for each main group,
thus allowing state to be preserved between groups of the same
feature. The creation of runners was refactored to allow for this.
The module tests were also modified to use the new paradym, which
allowed the removal of the global varaibles in the runner.
Dependencies on tests.api were also removed.
Change-Id: I66fcb2bce8c6cbc99c9ff071c242e4741500056d
Partial-Bug: #1571092
This changeset handles the details of applying,
removing, listing and retrieving 'modules'
from Trove instances.
See https://review.openstack.org/#/c/290177 for
the corresponding troveclient changes.
Scenario tests have been extended to cover the
new functionality. These tests can be run by:
./redstack int-tests --group=module
A sample module type 'driver' - ping - is included
that simply parses the module contents for a
message=Text string and returns the 'Text' as the
status message. If no 'message=' tag is found, then
the driver reports an error message.
Due to time constraints, a few unimplemented
parts/tests of the blueprint have been triaged as bugs
and are scheduled to be fixed before mitaka-rc1.
These include:
Vertica license module driver:
https://bugs.launchpad.net/trove/+bug/1554898
Incomplete module-instances command:
https://bugs.launchpad.net/trove/+bug/1554900
Incomplete 'live-update' of modules:
https://bugs.launchpad.net/trove/+bug/1554903
Co-Authored-by: Peter Stachowski <peter@tesora.com>
Co-Authored-by: Simon Chang <schang@tesora.com>
Partially Implements: blueprint module-management
Change-Id: Ia8d3ff2f4560a6d997df99d41012ea61fb0096f7
Depends-On: If62f5e51d4628cc6a8b10303d5c3893b3bd5057e
This changeset handles the details of creating,
updating, listing and deleting Trove 'modules.'
Two new tables have been added to the Trove database:
modules
instance_modules
although the instance_modules table is at present unused.
Scenario tests have been written as well, to exercise the
new functionality. These tests can be run by:
./redstack int-tests --group=module_create
In the follow-up changeset, all module tests can be run
by:
./redstack int-tests --group=module
Since module support is available for all datastores
(controlled by a CONF option) the module test has been
added to the common modules group.
Note: Trying to do admin tasks with none admin
credentials results in an Unauthorized exception
being thrown, instead of Forbidden. This
is due to the fact that Forbidden is in the
HTTPUnauthorized section of wsgi.py instead of
the HTTPForbidden section. Moving the exception
caused too many failures, so I created a 'Module'
Forbidden exception and put it in the right section.
Change-Id: I755b0431b33b870ae02d903527f071fd8e23130d
Depends-On: I54d37025275dee4731ad49ebbd21612c4464e4c4
Depends-On: I779c24472d3d96a7b2fe4ed0284fd5869cdef93b
Partially-Implements: blueprint module-maintenance
wu.chunyang