Tempest is going to expose a new optional interface in the plugin
class, to automatically register service clients implemented in
a plugin. Along with this the former client manager, renamed to
ServiceClients, is going to move to a stable interface in
tempest.lib.
Co-Authored-By: Raissa Sarmento <rdearauj@redhat.com>
Change-Id: I4cc9e2b9cd5cb09dff04ee16edcf85e59aec554f
"Quota" APIs now will return two new following keys:
- 'share_groups'
- 'share_group_snapshots'
For user and project, but not share type.
Default values can be configured using following config options:
- 'quota_share_groups'
- 'quota_share_group_snapshots'
APIImpact
DocImpact
Implements BluePrint add-share-groups-quota
Change-Id: I397a8e886226cb22fa50abdf2a4a938bb04c655d
With this feature it will be possible to set quotas per share type
for all existing quota resources. It is useful for deployments with
multiple backends that are accessible via different share types.
Also, fix one of existing DB migrations that hangs on PostgreSQL.
APIImpact
DocImpact
Implements blueprint support-quotas-per-share-type
Change-Id: I8472418c2eb363cf5a76c672c7fdea72f21e4f63
For quite some time, OpenStack services have wanted to be able to send
messages to API end users (by user I do not mean the operator, but the
user that is interacting with the client).
This patch implements basic user messages with the following APIs.
GET /messages
GET /messages/<message_id>
DELETE /messages/<message_id>
Implements the basic /messages resource and tempest tests
The patch is aligned with related cinder patch where possible:
I8a635a07ed6ff93ccb71df8c404c927d1ecef005
DocImpact
APIImpact
Needed-By: I5ffb840a271c518f62ee1accfd8e20a97f45594d
Needed-By: I9ce096eebda3249687268e361b7141dea4032b57
Needed-By: Ic7d25a144905a39c56ababe8bd666b01bc0d0aef
Partially-implements: blueprint user-messages
Co-Authored-By: Jan Provaznik <jprovazn@redhat.com>
Change-Id: Ia0cc524e0bfb2ca5e495e575e17e9911c746690b
Share and share instance list API will accept new query string parameter
'export_location'. It can pass path and id of export_location to
retrieve shares filtered.
APIImpact
Partly-implement: BP support-filter-share-by-export-location
Change-Id: I5fdf6d89d0b6c7fa182ddfaac60979bc6c0fc2a6
- Added tests to validate the migration functionality
for a replicated share.
- Added tests to validate the extend and shrink
functionality on migrated share.
- Added tests to cover the preserve snapshot
functionality on migrated share.
- Current waiter logic in migration_progress
method of dummy driver causes every migration
test to take over 4 minutes to complete.
Therefore, updated the method to cut the
overall time in half and avoid the possible timeout.
Partially-Implements: bp ocata-migration-improvements
Change-Id: Ia7c7eb7b48b464aa670073e8c0795369bd972678
This new feature gives the user the ability to allow and
deny access to the snapshots, so that they could be mounted in
read-only mode to retrieve files.
APIImpact
DocImpact
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Alyson Rosa <alyson.rosa@fit-tecnologia.org.br>
Co-Authored-By: Miriam Yumi <miriam.peixoto@fit-tecnologia.org.br>
Partially-implements: blueprint manila-mountable-snapshots
Change-Id: I65f398a05f82eef31ec317d70dfa101483b44b30
Implemented several improvements to share migration
according to spec [1].
Summary of changes:
- Snapshot restriction in API has been changed to return error only
when parameter force-host-assisted-migration is True
- Added preserve_snapshot to API and migration_check_compatibility
driver interface
- Changed all driver-assisted API parameters to be mandatory
- Added validation to prevent 'force_host_assisted_migration' to be
used alongside driver-assisted parameters
- Changed "same host" validation to reject only if the combination
of "host", "new_share_network" and "new_share_type" is the same as
the source
- Updated migration driver interfaces to support snapshots
- Updated zfsonlinux driver, defaulting preserve_snapshots to False
- Updated dummy driver to support preserve_snapshots
Spec update with latest changes since [1] merged
can be found in [2].
APIImpact
DocImpact
[1] I5717e902373d79ed0d55372afdedfaa98134c24e
[2] If02180ec3b5ae05c9ff18c9f5a054c33f13edcdf
Change-Id: I764b389816319ed0ac5178cadbf809cb632035b4
Partially-implements: blueprint ocata-migration-improvements
This commit adds the ability for Manila to revert a
share to the latest available snapshot.
The feature is implemented in the LVM driver, for
testing purposes.
APIImpact
DocImpact
Co-Authored-By: Ben Swartzlander <ben@swartzlander.org>
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Implements: blueprint manila-share-revert-to-snapshot
Change-Id: Id497e13070e0003db2db951526a52de6c2182cca
Now the quota-set API only returned single attribute
'limit', this change intends to add a new API
'quota-sets/{project_id}/detail' to retrieve more info
with attributes 'in_use', 'limit', 'reserved'.
APIImpact
Implements: blueprint admin-check-tenant-quota-usage
Depends-On: Ie0eb7d32b7b032ffdb7f7dd47f68841211e7d7a6
Change-Id: I499b099a3ba7704a2108cd15f80ff507e24b7cd0
The snapshot_support extra spec has always meant two
things: a driver can take snapshots and create shares
from snapshots. As we add alternate snapshot semantics,
it is likely that some drivers will want to support
snapshots and some of the new semantics while being
unable to create new shares from snapshots.
This work adds a new extra spec,
create_share_from_snapshot_support, that removes the
overloading on snapshot_support. It also makes the
existing snapshot_support extra spec optional,
allowing admins to create types without setting
snapshot_support; shares created with such types
will not support snapshots.
APIImpact
DocImpact
Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Implements: blueprint add-create-share-from-snapshot-extra-spec
Change-Id: Ib0ad5fbfdf6297665c208149b08c8d21b3c232be
Drop the undesired executable bit for the following files:
1. manila_tempest_tests/services/share/v2/json/shares_client.py
2. manila_tempest_tests/tests/api/test_share_networks.py
Also add pep8 check for new executable files under manila_tempest_tests.
Change-Id: Iaa509fdf4e0ebeb60d71b053f62459992c938468
This patch adds a 'new_share_type_id' parameter to Share Migration,
where the destination share can be provisioned under a different
share type of choice.
Host-assisted migration handles it by creating a totally new share,
as before.
Driver-assisted migration handles by creating the destination
instance model with the new share type, the driver is responsible
for making the necessary changes to satisfy the provided model.
In order to accomplish this, a database change was required,
transferring the 'share_type_id' field from the 'shares' table
to the 'share_instances' table.
APIImpact
Partially implements: blueprint newton-migration-improvements
Change-Id: I3200eaaa5b66d9b8ce1cbd16c1658db8516c70fb
At Austin 2016 summit there were several improvements to
Share migration feature discussed. This patch implements
these changes.
Changes are:
- Added 'Writable' API parameter: user chooses whether share must
remain writable during migration.
- Added 'Preserve Metadata' API parameter: user chooses whether
share must preserve all file metadata on migration.
- Added 'Non-disruptive' API parameter: user chooses whether
migration of share must be performed non-disruptively.
- Removed existing 'Notify', thus removing 1-phase migration
possibility.
- Renamed existing 'Force Host Copy' parameter to 'Force
Host-assisted Migration'.
- Renamed all 'migration_info' and 'migration_get_info' entries to
'connection_info' and 'connection_get_info'.
- Updated driver interfaces with the new API parameters, drivers
must respect them.
- Changed share/api => scheduler RPCAPI back to asynchronous.
- Added optional SHA-256 validation to perform additional check if
bytes were corrupted during copying.
- Added mount options configuration to Data Service so CIFS shares
can be mounted.
- Driver may override _get_access_mapping if supports a different
access_type/protocol combination than what is defined by default.
- Added CIFS share protocol support and 'user' access type
support to Data Service.
- Reset Task State API now allows task_state to be unset using
'None' value.
- Added possibility to change share-network when migrating a share.
- Bumped microversion to 2.22.
- Removed support of all previous versions of Share Migration APIs.
APIImpact
DocImpact
Implements: blueprint newton-migration-improvements
Change-Id: Ief49a46c86ed3c22d3b31021aff86a9ce0ecbe3b
Tempest tests were not appropriate for driver-assisted migration,
so this was fixed.
Also, improved docstrings and fixed workflow for drivers when
implementing 2-phase migration to be accurate with tempest and
handle AZs, which were previously locked to the source share's
AZ.
Driver-assisted migration now creates an additional
share instance to better handle and support driver methods.
Updated allow_access and deny_access APIs to allow users to mount
migrating shares before issuing 'migration-complete'.
APIImpact
Closes-bug: #1594922
Change-Id: If4bfaf7e9d963b83c13a6fea241c2eda14f7f409
Get gateway information from network plugin and put it into network_info.
It is required by EMC Unity storage to create a interface.
APIImpact
DocImpact
Change-Id: I8614b8686af7fa5764b49e8e3cb4a4855dc3a5f4
Implements: blueprint add-gateway-info
All tempest tests create shares with fixed size, 1GB. This patch add
a config option in order to allow configurable share sizes as cinder
does (Cinder change-ID reference:
I2897f6d6bd970f73867f56d9d23a768cafcbfd80).
TrivialFix
Change-Id: Ifd09519b63c6c1a0ec8449e5dae1ffe0caa1b879
By default, these tests are disabled; 3rd party CI systems
should enable these tests through the
RUN_MANILA_REPLICATION_TESTS environment variable
if their driver supports share replication.
Partially implements: blueprint manila-share-replication
Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Change-Id: Idaf3c61600337cc30568b982dfb23948209c1d6e
Removed functionality of Share Migration relying on Manila Share
Service node, moved code to Data Service node for copy phase.
Added parameter 'notify' and share/api methods for future
implementation (see dependent patches).
Added new copy operation statuses, in order to implement future
API calls to obtain progress and cancel migration.
Added possibility of 2-phase migration for driver migration and
generic (fallback) migration.
Added admin export location support and removed approach of
replacing IP with config parameter.
Added Admin-only API entry points to:
- Migration Cancel (only during copying)
- Reset Task State field
- Migration Get Progress (only during copying)
- Migration Complete (2nd phase migration)
- Notify parameter on Migrate Share
APIImpact
DocImpact
Implements: blueprint data-service-migration
Change-Id: I1d65aac2f36942cd70eb214be561d59a15a4ba26
tempest_lib was renamed to tempest.lib which causes problems
when running tests because tempest imports plugins from other
projects and crashes.
Closes bug: 1552520
Change-Id: Id7d43dee256856062bd0dadad9c469c1500ee36e
Add APIs to support manage and unmanage share snapshots.
Also add support in the Generic driver.
This only supports for DHSS=False driver mode.
Add provider_location column to the share_snapshots table
to save data used to identify the snapshot on the backend.
Also need to bump microversion.
APIImpact
DocImpact
Change-Id: I87a066173c85d969607d132accd9f0e9bd49c235
Implements: blueprint manage-unmanage-snapshot
In case the CG name is None, use the ID instead so we have
context on the failure.
Also, add the current CG status to the timeout message.
Change-Id: Ifb579d319f4230a662a218f479070a420d7ba38e
Closes-Bug: #1547056
Currently, the scheduler will ignore any properties
that are specific to the share instance. This is because
they are not passed into the scheduler filters as part of
the resource properties.
Change-Id: I8c3274e4d2779562dd8ee529473dd7cad521dbee
Closes-Bug: #1515624
- Add update_access() method to driver interface
- Move all code related to access operations to ShareInstanceAccess
class
- Statuses from individual access rules are now mapped to
share_instance's access_rules_status
- Add 'access_rules_status' field to share instance, which indicates
current status of applying access rules
APIImpact
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Tiago Pasqualini da Silva <tiago.pasqualini@gmail.com>
Implements: bp new-share-access-driver-interface
Change-Id: Iff1ec2e3176a46e9f6bd383b38ffc5d838aa8bb8
_get_access_action_name() purpose is to get proper action name
based on microversion, currently works only for allow_access,
but is used by list_access_rules() and delete_access_rule() as well.
This change fixes the method to work for all access rules methods.
This change also improves tempest tests to use both V1 and V2 client
in access rules tests.
Closes-Bug: #1528908
Change-Id: I3f1b109621f3ef72b1b9415e7da9d2cefb7c9000
Some upcoming features require more than one export location and
possibility to mark them with specific labels like fast/slow or
rw/ro.
So, implement 'export locations metadata' feature:
- It allows to set any key-value pairs for each export location.
- These key-value pairs can be set only by share manager using
response from various share driver methods.
- Example of update is implemented using Generic driver
"create_instance" method.
- Metadata can be updated for any export location in any place
of share manager where db function "share_export_locations_update"
is called.
- Keys from export location metadata table will be added to 'share' and
'share instances' views as export location attributes.
Also:
- Add new attr 'is_admin_only' for existing export locations model.
If set to True, then only admins will be able to see them. Unless
policy is changed.
- Add APIs for reading export locations by share and share instance IDs.
- Remove 'export_location' and 'export_locations' attrs
from 'share' and 'share instance' views.
- Bump microversion as new APIs implemented.
APIImpact
Implements bp export-location-metadata
Change-Id: I36d1aa8d9302e097ffb08d239cf7a81101d2c1cb
Manila microversions have following template:
x.y
where 'x' and 'y' both digits.
And now tempest transforms string 'x.y' to float but it is incorrect
thing to do because float assumes that each left value is bigger than
right one. And it is not suitable for microversion comparisons.
Examples:
Microversions true conditions:
2.9 < 2.10
2.9 < 2.81
Float true conditions:
2.9 > 2.10
2.9 > 2.81
So, create new file 'manila_tempest_tests/utils.py' and place there
old and new functions that serve all microversion actions. In addition,
port another existing utility function called 'rand_ip'.
Change-Id: I88bf2cb51fd8de1bc89bf169bda7a05ca5a0b8ab
Closes-Bug: #1518996
This fix allows manila manage command to explicitly
set visibility (key '--public') similar to using API
"create".
Change-Id: I8725719b8c7ff5557fa20ebbb5314f3e770ffcc0
Closes-Bug: #1436865
After port of extensions to core API we need to rename their URLs.
So, rename URLs and bump microversion for it.
Make new URLs work only with new microversion(s) 2.7+
and old with old microversions 1.0-2.6.
Add separate API router for v2 API as now we should split v1 and v2
logic.
Also, move updated APIs under v2 directory that will be used by both
API routers - v1 and v2.
List of updated collections is following:
- os-availability-zone -> availability-zones
- os-services -> services
- os-quota-sets -> quota-sets
- os-quota-class-sets -> quota-class-sets
- os-share-manage -> shares/manage
- os-share-unmanage -> shares/%s/action
List of updated member actions is following:
- os-share-unmanage/%(share_id)s/unmanage -> shares/%(share_id)s/action
- types/%(id)s/os-share-type-access -> types/%(id)s/share_type_access
List of updated action names is following:
- os-access_allow -> access_allow
- os-access_deny -> access_deny
- os-access_list -> access_list
- os-reset_status -> reset_status
- os-force_delete -> force_delete
- os-migrate_share -> migrate_share
- os-extend -> extend
- os-shrink -> shrink
List of updated attribute names is following:
- os-share-type-access -> share-type-access
Partially implements bp ext-to-core
Change-Id: I82f00114db985b4b3bf4db0a64191559508ac600
Changes:
- Register admin actions API as core API.
- Remove extension code for admin actions.
- Leave rename of admin actions API 'post' data for future update which
will be done with bump of microversion
after port of all extensions to core API.
Partially implements bp ext-to-core
Closes-Bug: #1510620
Change-Id: Ic8804cff5fa46003d2991372c6eb2a854b886dec
A previous patch added experimental=True to hide task_state field
for any API call that would return share fields, such as "create"
and "get" calls if different than microversion "2.5" with
experimental=True. Turns out the field task_state should be
returned always if microversion is "2.5" or above because it
relates to a change in DB, not only to the usage of Share Migration
feature. This patch fixes it by removing experimental=True
dependency to show task_state field.
Closes-bug: #1497352
Change-Id: Ic07d63ebcfe25c1cfa3fac1e75e2e186fbcec25b
After Share Migration was included, the task_state field was
added to be displayed on GET requests, but API versions prior to
2.5 should not see this field. This patch fixes that by using
microversions.
Closes-bug: #1494746
Change-Id: Ie755ae53fe8efdf1702a0ecbabf022a5fe4beb93
To prevent a microversioned client from managing a non-microversioned
Manila server, Manila must update its REST endpoints by adding /v2 for
all microversioned APIs.
This commit does the following:
* Add /v2 to the URL map, connected to all the same /v1 API methods
* Renumber the microversion sequence starting from 2.0
* Update the versions API to reflect v2
* Publish the new endpoint to Keystone in the DevStack plug-in
* Update relevant documentation
* Update Tempest tests for microversions
APIImpact
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Closes-Bug: 1488624
Change-Id: I56a516b5f81914557dd2465746629431cfd6deac
Raissa Sarmento