Commit Graph

57 Commits

Author SHA1 Message Date
wangzihao 91ba6d1d7c Remove six.moves
Remove six.moves Replace the following items with Python 3 style code.
- six.moves.urllib
- six.moves.http_client
- six.moves.configparser
- six.moves.cStringIO

Change-Id: I004b6b9a81079c67451395bfe31ec75d58802c16
2020-10-09 18:08:32 +08:00
likui 5bf159de36 Remove six.PY3
The Python 2.7 Support has been dropped since Ussuri.
So remove hacking rules for compatibility between python 2 and 3.

Change-Id: Id3fd78b5d3e21579975ea0d0fc65e4f66ea4582e
2020-09-03 18:30:24 +08:00
Lingxian Kong 602c4d42de Improve the function tests
- Execute test groups in serial to make sure no more than 2 database
  instance are created at the same time.
- Remove some unneccesary tests
- Remove unneeded datastore, e.g. 'Test_Datastore_1'
- Remove unsupported trovestack subcommands
- Move unsupported DIB elements to the 'deprecated-elements' folder
- Decrease default value of 'agent_call_high_timeout' to 5min
- Add initial_deplay for pooling task
- Use socket file to connect with database instead of using localhost
  IP

Change-Id: Ie5030a671fbeb453eafa6cbe04e08da7b52e33c9
2020-01-28 14:27:52 +13:00
Lingxian Kong c33fa67066 Support to create public trove instance
- The users need to specify the network to create Trove instance, but
  trove-taskmanager will create port in that network for Nova instance
  creation. Using port gives Trove more capabilities to define how the
  database service is exposed.
- Deprecate ICMP protocol for the instance.
- Restrict 'nics' parameter for creating instance.
- Add 'access' parameter for creating instance.
- Add 'public_network_id' option in order to create floating IP for the
  instance.
- Do not create records for security groups, but Trove can still delete
  existing instances for backward compatibility.
- Delete unreasonable Host, Account, Storage API.

Story: 2006500
Task: 36468
Task: 36466
Change-Id: I80827e1ad5e6b130cbf94c2bb7a909c44d5cf1e5
2019-09-11 11:28:00 +12:00
Lingxian Kong dfa5ce93d5 Improve devmode=flase when building the image
During debugging, the following changes are also included:

- Support to specify an image ID to run the integration test.
- Fix the reboot function bug.
- Remove the unsuccessful restart test.

How to run integration test with dev_mode=false:

    ADMIN_PASSWORD=password \
    SERVICE_PASSWORD=password \
    DEV_MODE=false \
    /opt/stack/trove/integration/scripts/trovestack gate-tests mysql mysql

Change-Id: I31d4ee579a554f4c98f9facb9fd4b7779665a3dd
2019-08-25 23:11:54 +12:00
Marcin Piwowarczyk ad84829882 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I9ee34642c700d1e6ba9c2f3891b7fa1f7f7e1e1d
Depends-On: I8989fd4798e80eae27408017e1543819a68b4ab1
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
2018-09-18 17:19:20 +00:00
Amrith Kumar a7115e22f7 secure oslo_messaging.rpc
This is an interim commit of the changes for secure
oslo-messaging.rpc. In this commit we introduce the code for
serializers that will encrypt all traffic being sent on
oslo_messaging.rpc.

Each guest communicates with the control plane with traffic encrypted
using a per-instance key. This includes both traffic from the
taskmanager to the guest as well as the guest and the conductor.

Per-instance keys are stored in the infrastructure database. These
keys are further encrypted in the database.

Tests that got annoyed have been placated.

Upgrade related changes have been proposed. If an instance has no key,
no encryption is performed. If the guest gets no key, it won't
encrypt, just pass through. When an instance is upgraded, keys are
added.

The output of the trove show command (and the show API) have been
augmented to show which instances are using secure RPC communication
** if the requestor is an administrator **.

A simple caching mechanism for encryption keys has been proposed; this
will avoid the frequent database access to get the encryption
keys. For Ocata, to handle the upgrade case, None as an encryption_key
is a valid one, and is therefore not cached. This is why we can't use
something like lrucache.

A brief writeup has been included in dev docs
(dev/secure_oslo_messaging.rst) which shows how the feature can be
used and would help the documentation team write up the documentation
for this capability.

Change-Id: Iad03f190c99039fd34cbfb0e6aade23de8654b28
DocImpact: see dev/secure_oslo_messaging.rst
Blueprint: secure-oslo-messaging-messages
Related: If0146f08b3c5ad49a277963fcc685f5192d92edb
Related: I04cb76793cbb8b7e404841e9bb864fda93d06504
2017-01-11 07:56:35 -05:00
Victor Stinner 41f2606435 Port run_tests.py to Python 3
* Replace urllib import with six.moves.urllib
* gettext.install() doesn't require the unicode parameter on Python 3
* tox.ini: add py34 testenv

Tests don't pass. It's the very first step to start the working on
porting Trove to Python 3.

Partially implements: blueprint trove-python3
Change-Id: Ide9ec1d2aee84905c03cacdb3ea35d0dcbf01596
2015-11-19 10:25:01 +00:00
Amrith Kumar 34ce70e299 Correct errors resulting in "No handlers ..." error message
This is a checkin for the issue of having errors that read "No
handlers could be found for logger X.Y.Z" when running tests. While
being annoying, it is also a serious issue in that some times, the
lack of logging masks errors in the tests. One such example is fixed
in this chkin (see bug 1491093).

Change-Id: I4066d226aefbd23248e88692db109d31f5c150e9
Partial-Bug: #1491075
Closes-Bug: #1491093
2015-11-10 15:42:22 -05:00
Sergey Vilgelm d31ae008f5 Remove openstack.common package
Move pastedeploy[1] and local[2] of oslo-incubator modules to
the trove.common package because they are dropped.
Use base_exception module instead of the exception of oslo-incubator module.
Use i18n module instead of gettextutils.
Remove openstack.common package and openstack-common.conf file.

[1] I741f7d6d1327c9ce23ad5203d31667aff17e387f
[2] I271debac1e05a353f9876ced77970e6030912111

Change-Id: Ie289dac7a06704d5c55a5074c0afe50184e4d2f3
2015-07-23 19:54:04 +03:00
Sergey Vilgelm 19862628e4 Switch to the oslo_log library
Remove the log module of oslo-incubator.
Move the WritableLogger wrapper to the base_wsgi module.
Add oslo.log to the requirements.

Change-Id: I724fa6090cebf40e7d7c78cc6b8458dfba9508a8
2015-07-23 19:31:58 +03:00
Sushil Kumar 843bb0cd53 Fixes hacking rules
This patchset helps reducing the ignored pep8 errors.

Fixed some ignored hacking rules, as listed follows:
- E111 indentation is not a multiple of four
- E122 continuation line missing indentation or outdented
- E123 closing bracket does not match indentation of opening bracket's line
- E128 continuation line under-indented for visual indent
- E251 unexpected spaces around keyword / parameter equals
- E265 block comment should start with '# '
- E713 test for membership should be 'not in'
- H105  Don't use author tags
- H306  imports not in alphabetical order

Change-Id: Iadf2af4f6ec90420153ad63b5a41650392ef2cbd
2015-05-29 07:27:59 +00:00
Sergey Gotliv ea148d7dfe Integration with oslo.messaging library
Port Trove to use oslo messaging library instead of obsolete messaging
code from oslo incubator.

Change-Id: Ibd886f3cb4a45250c7c434b3af711abee266671c
Implements: blueprint rpc-versioning
2015-01-06 09:16:39 +02:00
Edmond Kotowski a6358494d1 Legacy MySQL datastore is shown on datastore-list
On the 019_datastore_fix.py migration script the upgrade method was
calling the has_instances_wo_datastore_version as a function pointer
instead of passing the instance_table in and calling the function. This
meant the if block was always true because it was just checking if the
function exists and the create_legacy_version method was called everytime
causing the "Legacy MySQL" datastore to be inserted even if there was no
legacy trove instances. Now inject the instace_table into the
has_instances_wo_datastore_version and everything works as expected.

  Updated the datastore integration tests to not rely on the Legacy MySQL
datastore to exist by using the 10000000-0000-0000-0000-000000000001 GUID.
Instead the test now asserts against the data_store by the name
"Test_Datastore_1" which is the datastore the integration test
intended to validate against.

Closes-Bug: 1396427
Change-Id: Ifff33a962cc8d6ea150f6408d63b81412e196939
2014-12-02 17:47:12 -08:00
Tim Simpson 220e3e7255 Event simulator II
The previous event simulator simulated time by making all tasks that
would have been launched as threads run in the same thread. That only
worked to a point but didn't properly simulate more complex behaviors
such as clustering.

This new one handles things properly by still running tasks in
eventlet's greenthreads but forcing them to only run one at a time.

Change-Id: Ife834e8d55193da8416ba700e55e7b0c2496f532
2014-09-29 10:32:29 -05:00
Simon Chang 06e0aa25a0 Fixed database migration script issues
The db downgrade scripts are currently not dropping foreign key
constraints, causing errors when the script tries to drop related
tables on downgrade.

This commit address issues in the migration scripts, and also
introduces a new test script to test the migration scripts, so that
issues can be prevented in the future. The new test script is based
on the existing migration test script implementated in Nova.

Change-Id: I240d81afc3e43fd3711de8c156cfb43fd14850bf
Closes-Bug: #1347114
2014-09-19 16:56:31 -04:00
Tim Simpson 108fbc5ef3 Fix config parameters test for fake mode
This commit makes the config parameters tests run in fake mode.

Closes-Bug: 1370237
Change-Id: I6ce228a627b35ee418c290a9f19baa163261fb7c
2014-09-16 15:52:51 -05:00
Morgan Jones 86eacbeabc Mysql guest agent functionality for replication
Implement new strategy for replication
Implement replication strategy for mysql binlog replication.
Supporting methods in mysql service
Implement API methods in mysql manager
Define configuration settings for replication

Co-authored by: Nikhil Manchanda <SlickNik@gmail.com>
Co-authored by: Greg Lucas <glucas@tesora.com>

Partially implements: blueprint replication-v1
Change-Id: I70f0b5c37fe3c2d42426029bb627c141965eb524
2014-08-14 17:25:42 -07:00
amcrn 63d62f084e Hide Datastores In List With No Active Versions
in a datastore-list operation, for non-admin, filters out
datastores that either have no versions or have no active
versions for non-admin users. admins continue to see all
datastores, active or inactive, in a datastore-list
operation.

Change-Id: Iccaf9ae59469fd0f4cb346ee258d3132eed83660
Co-Authored-By: Andrey Shestakov <ashestakov@mirantis.com>
Closes-Bug: #1290617
2014-03-14 13:22:02 -07:00
Andrey Shestakov 2296bdfcea Datastores improvements
List of improvements:
- added default_version field in datastore show output
- admin can list deactivated versions too
- admin can see extanded info about datastore version (image_id, packages)
- add possibility to retrive version by uuid (without specified datastore)

Implements: blueprint datastore-type-version-followon

Change-Id: I9b0c3547fb6a881a01db9e6e3d8d72077bc5ee29
2014-01-21 21:50:04 +02:00
Craig Vyvial f14c87f015 move datastore manager to the datastore version
This moves the datastore manager field to the datastore version object
so that we can have managers per datastore version and they can be
shared if nessesary.

implements blueprint move-manager-to-datastore-version

Change-Id: I1c889a01005ec482bcd44e0e19475bc972e1af04
2014-01-16 09:49:42 -06:00
Nikhil Manchanda 8a1db9b689 Fixed PEP8/flake8 issues
We need to fix PEP8/flake8 issues corresponding to the new
hacking rules that got added before we can update to the
new global requirements.

Change-Id: Iff995139d258549ff9a8e3e5018e964efa2ab152
2013-12-10 15:47:09 -08:00
Sushil Kumar 5aaac4c6fa Removes unused import statements
Updates tox.ini to remove suppressed F401 standard.

Updates the code files containing unused import statements

Change-Id: Ib44bd1509fdb17f457a205f9e62caf85cebee830
Closes-Bug: #1257267
2013-12-03 19:22:40 +00:00
Andrey Shestakov 1cceaa11df Add support of datastore types
implements blueprint db-type-version

Change-Id: Ie87f72b898e993044803e7b37cad78372c2cd3f4
2013-11-19 13:28:59 +02:00
Jenkins bbca76444f Merge "Simulates events so run_tests.py executes 10x faster" 2013-10-25 22:43:42 +00:00
Robert Myers 954dd04f5d Add tests for hosts mgmt commands
* moving the tests from trove-integration for host command

Implements: Blueprint host-mgmt-tests

Change-Id: I947ae348ebd0ff83625476f0b28180db8f50afdb
2013-10-22 12:59:43 -05:00
Tim Simpson c157502a9c Simulates events so run_tests.py executes 10x faster
This restores some functionality that had been removed earlier to use
a class from Oslo. It turns out making it work with the new Oslo RPC
manager code wasn't too hard, plus the new version is hopefully easier
to follow.

Change-Id: I18e4d06359f28605cd764a10ace791997f534ed7
Implements: blueprint bring-back-event-simulator
2013-10-17 14:43:54 -05:00
Dmitriy Ukhlov 9f55a06e09 Task manager refactoring done
Manager API class removed and rpc fake impl used now for integration
testing

Change-Id: If20004f1b69e41e8243a3f58eec184f1f457affe
Implements: blueprit trove-task-manager-client-refactoring
2013-10-08 23:36:18 +03:00
Jenkins 00b03a4d7e Merge "Support Optional Super User in Instance Create" 2013-10-08 19:02:41 +00:00
Denis M 3333dd73e6 PEP8. E125, F811, F401, H703
Change-Id: I9537e364affe20b9edf6861dc9a0d6437a112d30
Implements: blueprint reduce-amount-of-pep8-ignored-rules-in-tox-tests
2013-10-07 14:00:29 +03:00
amcrn f77aab57e1 Support Optional Super User in Instance Create
The creation of initial users and/or databases is supported in an
instance-create request, but the creation of a super/root user is
not. Granted, it's possible to create or reset the root user, but it
must be done as a separate request, after the instance has completely
initialized.

Changeset: If CONF.root_on_create == True, a UUID is generated for
the root user's password and is returned in the instance-create
response as the "password" property/field.

Change-Id: I300d3f019766445b90411d5ab8651b1aa34a77e9
Implements: blueprint instance-initial-super-user
Closes-Bug: #1219624
2013-09-20 14:29:20 -07:00
Michael Basnight 7ab80b554d Rename from reddwarf to trove.
Implements Blueprint reddwarf-trove-rename

Change-Id: Ia9ee609bbc06a1d8b9d6917642529f30347541fd
2013-06-24 14:11:15 -07:00
Monty Taylor 98d42e50e3 Migrate to flake8.
Fixes bug 1172444

Change-Id: I09e1e2d03f41a175aa39760b5fc560ed0d9d30a5
2013-05-16 10:29:59 -07:00
Jenkins d6c9cf6c4d Merge "added malformed json tests" 2013-05-10 16:57:32 +00:00
justin-hopper b3c32e3f87 Controller and API changes for backups.
-API controller for backups
-adding swift url to the config
-Fixing failing tests
-Renaming 'instance' param.
 Checking for NotFound models so that the returned error is friendly
-adding feature to list all backups from a specific instance
-Adding checks for creating/deleting/restoring backups when it is not completed
-Adding unit tests for backup controller
-adding check to see if file is in swift
-Adding skeleton code for delete backup in the task manager
-Fixed backups API to pass in the backup_id during create_backup.
-adding int tests for backup controller
-Adding backup list and delete int tests
-Adding list backups for instance test
-Adding quota for create backup

BP: https://blueprints.launchpad.net/reddwarf/+spec/consistent-snapshots

Change-Id: I35c2fefcce4b3009e76ba7232c52dabf502a3ac0
2013-05-09 09:26:06 -07:00
Paul Lodronio 0ac1ba7e26 added malformed json tests
these tests are all being skipped because the code being tested is broken

bug #1177969
Change-Id: Iaf3c09bfda01b5ad883e9859906f831a3b2b5337
2013-05-08 18:06:20 -05:00
Jenkins 3c423f1885 Merge "Test no accept headers" 2013-03-11 18:48:10 +00:00
DJ Johnstone b22056082d Test no accept headers
Created a test to validate that the api still responds correctly without accept headers being passed.

Change-Id: Ia6e145d857eed7c20b1a048ceaf41e2b301c1ea1
Implements: blueprint no-accept-header-validation
2013-03-08 12:18:27 -06:00
Tim Simpson 0fc4600e01 Tests the API in XML mode.
This form of testing has been nearly available for awhile but was not
turned on because it will increase the time of a full tox test run by
eight seconds. However there has recently been new features which do
not work at all with XML, so the time loss seems like a silly reason
to not gate on this.

implements blueprint test-with-xml

Change-Id: I98d892bb5c1a0c16425f80a28a296c70120318c8
2013-03-06 18:24:17 -06:00
daniel-a-nguyen 0a71ef9e88 Rate limits implementation
added unittest for limits
    reverted changes to openstack/common
    removed commented code
    cleaned up unittest
    added int-tests
    updated reference to XMLNS
    removed 1.1 XMLMS in wsgi

Implements: blueprint rate-limits
Change-Id: I842de3a6cae1859cc246264a5836abfd97fb8074
2013-03-05 14:17:36 -08:00
Michael Basnight 4374be7a20 Fixing run_tests.py so it emits an error upon failure.
fixes bug 1144683

Change-Id: If9ef4e53d012fed1c44ea5c099f2e750c167265d
2013-03-04 10:16:10 -06:00
Ed Cranford 90f8ca81d9 Modify-user features.
Grant, revoke, list user access.
Change user password and get single user.

Partially implements blueprint modify-users

Change-Id: I0001a7a9d1c527b88a1ed965f0f077c864e602cf
2013-02-19 20:40:26 -06:00
Joe Cruz cd66821e86 Adds reset-task-status mgmt api instance action
When reset-task-status is called the instance's task status is set to NONE.

Partially implements: blueprint reset-task-status-mgmt-api

Change-Id: I3150b53c3feb586d4bcfa8a2f5807ecd53c89e26
2013-02-05 15:25:10 -06:00
Jenkins db78f392cb Merge "Add unit tests for guestagent.volume.py" 2013-01-18 22:48:59 +00:00
ruiyuan-shen 49bf821843 Add unit tests for guestagent.volume.py
- more clean up on test_manager.py
- Migrate guestagent tests to unittest folder
- modify guestagent.manager.py so it recognize '_'

Bug # 1090139

Change-Id: I718a5212cd07c8bdb4e71796ba384574ce8275e9
2013-01-17 15:11:52 -08:00
Ed Cranford ce6b98e2d1 Checks guest status during migration.
During a migration, waits for the guest and service to start up
following the status change to VERIFY_RESIZE. Confirms if everything
is all right, or reverts if not.

Implements blueprint migration-check-guest-status

Change-Id: Ia7c7ed1fd0070429fed93323ca559d1c0742bd8f
2013-01-17 16:59:16 -06:00
Jenkins 8296ea6f69 Merge "Add unit tests for guestagent.models.py Delete guestagent.utils.py since it is no longer invoked anywhere." 2013-01-12 01:21:55 +00:00
Steve Leon b1f061b2e8 Adding guest agent pkg unit tests and running them with testr
Added unit tests for reddwarf.guestagent.pkg module.
Added tests for package install, remove and version
Changing the test runner of unittests to be run by testr instead of probocis/nose
Remove probocis reference from the dbaas test
Moved dbaas.py to reddwarf/tests/unittests

tracked by bug #1090139

Change-Id: Ia25fbe996d04d39e83edef0369de0e7fccce579e
2013-01-11 15:06:25 -08:00
ruiyuan-shen 2e346c4dee Add unit tests for guestagent.models.py
Delete guestagent.utils.py since it is no longer invoked anywhere.

Bug # 1090139

Change-Id: Iad3fad6f2b5bdb813b499dc8ce7181e6f8fb0231
2013-01-09 13:27:04 -08:00
Anna Shen fc12c6f0df Add unit test for guestagent.service.py and query.py
Bug # 1090139

Change-Id: Ib6ddf1f1823ccec348cf552daaee0c127940aa29
2013-01-02 16:39:50 -08:00