Added support for display count info
in share list&detail APIs:
1. /v2/{project_id}/shares?with_count=True
2. /v2/{project_id}/shares/detail?with_count=True
Partially-Implements bp add-amount-info-in-list-api
Change-Id: I12c41a46140b04f26565d8934e0326480477c612
"Quota" APIs now will return two new following keys:
- 'share_groups'
- 'share_group_snapshots'
For user and project, but not share type.
Default values can be configured using following config options:
- 'quota_share_groups'
- 'quota_share_group_snapshots'
APIImpact
DocImpact
Implements BluePrint add-share-groups-quota
Change-Id: I397a8e886226cb22fa50abdf2a4a938bb04c655d
With this feature it will be possible to set quotas per share type
for all existing quota resources. It is useful for deployments with
multiple backends that are accessible via different share types.
Also, fix one of existing DB migrations that hangs on PostgreSQL.
APIImpact
DocImpact
Implements blueprint support-quotas-per-share-type
Change-Id: I8472418c2eb363cf5a76c672c7fdea72f21e4f63
Please read spec for design detail [1].
Add support to validate IPv6 based addresses
in allow access API when access type is IP.
[1] f7202a6cfe32a057f752a4e393f848f8a0211c36
APIImpact
DocImpact
Change-Id: Ica242007e77a7e21b2151e5fc32401b501d961b2
Partial-Implements: blueprint support-ipv6-access
For quite some time, OpenStack services have wanted to be able to send
messages to API end users (by user I do not mean the operator, but the
user that is interacting with the client).
This patch implements basic user messages with the following APIs.
GET /messages
GET /messages/<message_id>
DELETE /messages/<message_id>
Implements the basic /messages resource and tempest tests
The patch is aligned with related cinder patch where possible:
I8a635a07ed6ff93ccb71df8c404c927d1ecef005
DocImpact
APIImpact
Needed-By: I5ffb840a271c518f62ee1accfd8e20a97f45594d
Needed-By: I9ce096eebda3249687268e361b7141dea4032b57
Needed-By: Ic7d25a144905a39c56ababe8bd666b01bc0d0aef
Partially-implements: blueprint user-messages
Co-Authored-By: Jan Provaznik <jprovazn@redhat.com>
Change-Id: Ia0cc524e0bfb2ca5e495e575e17e9911c746690b
Add like filter support in ``shares``, ``snapshots``,
``share-networks``, ``share-groups`` list APIs.
APIImpact
Implements BP like-filter
Change-Id: I5fdf6d89d0b6c7fa182ddfaac60979bc6c0fc2a5
Share and share instance list API will accept new query string parameter
'export_location'. It can pass path and id of export_location to
retrieve shares filtered.
APIImpact
Partly-implement: BP support-filter-share-by-export-location
Change-Id: I5fdf6d89d0b6c7fa182ddfaac60979bc6c0fc2a6
Previous change [1] added logic for handling of availability zones with
share groups and [2] for scheduling with share group capability
to create consistent snapshots. Those two added two new DB fields
for 'share_groups' DB model, but not to API response that requires
microversion bump. So, add these fields to API and bump microversion.
Following two new fields will be available with new microversion:
- 'availability_zone_id'
- 'consistent_snapshot_support'
Also, add tempest tests to these API changes.
[1] I000adeb53fe8435465cbedc3c539e6aaae6503c5
[2] I05553c308ae40c4ddc2c6469ff1c1a3da36a87da
Partially-Implements BP manila-share-groups
Change-Id: I343d0c6f3a5c7b58d88e95dba4af984fae738954
That will be used for scheduling share groups based on their possibility
to create consistent snapshots.
Also apply following tempest plugin changes:
- Add new 'capability_sg_consistent_snapshot_support' tempest config
option, that will be used for creation of new share group types and used
to prove that scheduling works as expected.
- Fix some share group test attributes from 'only API involved' to
'API and Backend are involved', because it is so indeed.
Change-Id: I05553c308ae40c4ddc2c6469ff1c1a3da36a87da
Partially-Implements BP manila-share-groups
When we do manila access-list, the 'created_at' and 'updated_at' time
should be shown to the user. And then the user could determine which
is the recent access rule. As the design, the recent access-allow
rule caused the error access-status of share instance.
APIImpact
Closes-Bug: #1682795
Change-Id: Iad6070d60ec77b7de9cc9679cfa7478876084da1
This option is used to control how revert-to-snapshot
extra-spec is used when creating a share type for several tests,
but for the same tests, it is more appropriate to either not
create a share-type (thus use the default or "don't care"
behavior) or do so based on the value of
run_revert_to_snapshot_tests.
TrivialFix
Change-Id: Ie8a0030a896634eb8deaff17ea72371419eecdea
- Added tests to validate the migration functionality
for a replicated share.
- Added tests to validate the extend and shrink
functionality on migrated share.
- Added tests to cover the preserve snapshot
functionality on migrated share.
- Current waiter logic in migration_progress
method of dummy driver causes every migration
test to take over 4 minutes to complete.
Therefore, updated the method to cut the
overall time in half and avoid the possible timeout.
Partially-Implements: bp ocata-migration-improvements
Change-Id: Ia7c7eb7b48b464aa670073e8c0795369bd972678
This new feature gives the user the ability to allow and
deny access to the snapshots, so that they could be mounted in
read-only mode to retrieve files.
APIImpact
DocImpact
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Alyson Rosa <alyson.rosa@fit-tecnologia.org.br>
Co-Authored-By: Miriam Yumi <miriam.peixoto@fit-tecnologia.org.br>
Partially-implements: blueprint manila-mountable-snapshots
Change-Id: I65f398a05f82eef31ec317d70dfa101483b44b30
Remove the experimental consistency group APIs and
replace them with the experimental Share Group APIs.
DocImpact
APIImpact
Partially-implements-blueprint: manila-share-groups
Change-Id: I79a80a62ae4e0015d6161edc2b93fd1f9ba69537
- Add Database migration to introduce the column on the
share instances model.
- Set the field to True if creating read-only secondary
replicas, unset while promoting them.
- Set the field to True if drivers don't support writable access
to migrating shares, or if using host assisted migration.
Unset if migration fails, or is canceled.
- Expose the field via share-instances and share-replicas
APIs to administrators.
Supporting read only-access rules is part of the minimum
driver requirements in manila.
APIImpact
DocImpact
Implements: bp fix-and-improve-access-rules
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Change-Id: Ie8425f36f02cbcede0aaa9f3fe1f5f3cf23df8b8
Implemented several improvements to share migration
according to spec [1].
Summary of changes:
- Snapshot restriction in API has been changed to return error only
when parameter force-host-assisted-migration is True
- Added preserve_snapshot to API and migration_check_compatibility
driver interface
- Changed all driver-assisted API parameters to be mandatory
- Added validation to prevent 'force_host_assisted_migration' to be
used alongside driver-assisted parameters
- Changed "same host" validation to reject only if the combination
of "host", "new_share_network" and "new_share_type" is the same as
the source
- Updated migration driver interfaces to support snapshots
- Updated zfsonlinux driver, defaulting preserve_snapshots to False
- Updated dummy driver to support preserve_snapshots
Spec update with latest changes since [1] merged
can be found in [2].
APIImpact
DocImpact
[1] I5717e902373d79ed0d55372afdedfaa98134c24e
[2] If02180ec3b5ae05c9ff18c9f5a054c33f13edcdf
Change-Id: I764b389816319ed0ac5178cadbf809cb632035b4
Partially-implements: blueprint ocata-migration-improvements
- Pull up policy check to beginning of the APIs.
- Avoid making access rules changes when one or
more instances of the share are in an invalid state.
- Add back the per rule share instance access status.
This restoration provides better visibility for which
rules were applied successfully.
- Remove 'updating' and 'updating_multiple' as valid
states for the share instance access rules status.
- Deprecate the access rule state 'new' in favor of
'queued_to_apply' and the share instance access rules
status 'out_of_sync' in favor of 'syncing'.
In a new API micro-version:
- Allow access rule changes irrespective of the share's
access_rules_status.
- Expose new access rule states and share's
access_rules_status values.
Access rules for each share instance now transition
from 'queued_to_apply' to 'applying' to 'active' or 'error';
and from 'active', 'queued_to_apply', 'applying' or 'error'
to 'queued_to_deny' to 'denying' to 'deleted'.
APIImpact
DocImpact
Partially-implements: bp fix-and-improve-access-rules
Co-Authored-By: Mike Rooney <rooneym@netapp.com>
Change-Id: Ic25e63215b5ba723cbc8cab7c51789c698e76f28
This commit adds the ability for Manila to revert a
share to the latest available snapshot.
The feature is implemented in the LVM driver, for
testing purposes.
APIImpact
DocImpact
Co-Authored-By: Ben Swartzlander <ben@swartzlander.org>
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Implements: blueprint manila-share-revert-to-snapshot
Change-Id: Id497e13070e0003db2db951526a52de6c2182cca
Nova network was deprecated in Newton and is no longer supported for
regular deployments in Ocata [1].
Complete the removal of nova network support from manila begun in [2]
by removing support for ''nova_net_id`` in the share networks API,
by removing the corresponding column from the share networks DB model,
and by removing the nova network plugins themselves. Unit tests for
share network api views were refactored to remove redundancy while
extending coverage to the new microversion introduced with this patch
and maintaining full coverage of earlier microversions.
APIImpact
DocImpact
UpgradeImpact
Partially-implements: bp remove-nova-net-plugin
[1] http://docs.openstack.org/releasenotes/nova/unreleased.html
[2] I846b760fa7c01f7f86768399a2bfad9ced7e57cd
Change-Id: I8b9a559fbea61979f01737ed1dc272276c4f1269
Now the quota-set API only returned single attribute
'limit', this change intends to add a new API
'quota-sets/{project_id}/detail' to retrieve more info
with attributes 'in_use', 'limit', 'reserved'.
APIImpact
Implements: blueprint admin-check-tenant-quota-usage
Depends-On: Ie0eb7d32b7b032ffdb7f7dd47f68841211e7d7a6
Change-Id: I499b099a3ba7704a2108cd15f80ff507e24b7cd0
The snapshot_support extra spec has always meant two
things: a driver can take snapshots and create shares
from snapshots. As we add alternate snapshot semantics,
it is likely that some drivers will want to support
snapshots and some of the new semantics while being
unable to create new shares from snapshots.
This work adds a new extra spec,
create_share_from_snapshot_support, that removes the
overloading on snapshot_support. It also makes the
existing snapshot_support extra spec optional,
allowing admins to create types without setting
snapshot_support; shares created with such types
will not support snapshots.
APIImpact
DocImpact
Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Implements: blueprint add-create-share-from-snapshot-extra-spec
Change-Id: Ib0ad5fbfdf6297665c208149b08c8d21b3c232be
In addition to functional tests.
Also, make some corrections to scenario tests code to make it work
with shares located on localhost.
Co-Authored-By: Alexey Ovchinnikov <aovchinnikov@mirantis.com>
Co-Authored-By: Valeriy Ponomaryov <vponomaryov@mirantis.com>
Change-Id: Ice61a7593f5e4e443cffbe78ecd71d683a9c274e
Administrators intend to get the pool's information filtered
by share type(actually filtered by share_type's *extra_spec*)
more directly.
The blueprint is to add a filter key 'share_type' to cover
this situation.
APIImpact
Implements: blueprint pool-list-by-share-type
Change-Id: Ifd64bb84d03a02aa0a118cc42e1d1b373c439884
Service available group already exists.Therefore we don't need to
register this group here again.
Change-Id: I9680abe0a177a80860ec6ba1dbacc04fac41758d
Closes-Bug: #1621036
At Austin 2016 summit there were several improvements to
Share migration feature discussed. This patch implements
these changes.
Changes are:
- Added 'Writable' API parameter: user chooses whether share must
remain writable during migration.
- Added 'Preserve Metadata' API parameter: user chooses whether
share must preserve all file metadata on migration.
- Added 'Non-disruptive' API parameter: user chooses whether
migration of share must be performed non-disruptively.
- Removed existing 'Notify', thus removing 1-phase migration
possibility.
- Renamed existing 'Force Host Copy' parameter to 'Force
Host-assisted Migration'.
- Renamed all 'migration_info' and 'migration_get_info' entries to
'connection_info' and 'connection_get_info'.
- Updated driver interfaces with the new API parameters, drivers
must respect them.
- Changed share/api => scheduler RPCAPI back to asynchronous.
- Added optional SHA-256 validation to perform additional check if
bytes were corrupted during copying.
- Added mount options configuration to Data Service so CIFS shares
can be mounted.
- Driver may override _get_access_mapping if supports a different
access_type/protocol combination than what is defined by default.
- Added CIFS share protocol support and 'user' access type
support to Data Service.
- Reset Task State API now allows task_state to be unset using
'None' value.
- Added possibility to change share-network when migrating a share.
- Bumped microversion to 2.22.
- Removed support of all previous versions of Share Migration APIs.
APIImpact
DocImpact
Implements: blueprint newton-migration-improvements
Change-Id: Ief49a46c86ed3c22d3b31021aff86a9ce0ecbe3b
For backends with internal authentication system,
e.g. Ceph, that return ``access_key`` (credential) for
client identities that are granted share access:
* Retrieve ``access_key`` as return value of driver's
update_access()
* Store ``access_key`` in ShareAccessMapping model
* Expose it in access_list API
APIImpact
DocImpact
Partially implements bp auth-access-keys
Co-Authored-By: John Spray <jspray@redhat.com>
Change-Id: I486064f117cf3001dba7735ca92a7d89aee3ce5b
MTU value can be different for each neutron network.
E.g. for high-performance use-cases it's very important to also
support MTU's with jumbo frames.
This patch exposes this information to the drivers. Each driver
should setup its resources accordingly.
The tempest test actually just covers the api change. Better coverage will
be added when the container driver lands.
APIImpact
DocImpact
Change-Id: I9b4efae620ec9f6790547c8fffc58872d43277f5
Implements: bp add-network-mtu
Related-Bug: #1612528
Added dummy driver for ease of testing and development of REST APIs
and internal manila interfaces.
It can be enabled as backend in the same way as all other share drivers.
But, it does not provide any resources.
Also, update tempest to be able to work with share-networks when
networks are defined via config and not API.
CI hooks are configured to run tempest twice using both available
driver modes. Where DHSS=True mode uses StandAloneNetworkPlugin as
network backend.
Change-Id: I9053dddfc643cb5a6602f15235b91bbaea727dbd
Implements BluePrint dummy-driver
Get gateway information from network plugin and put it into network_info.
It is required by EMC Unity storage to create a interface.
APIImpact
DocImpact
Change-Id: I8614b8686af7fa5764b49e8e3cb4a4855dc3a5f4
Implements: blueprint add-gateway-info
All tempest tests create shares with fixed size, 1GB. This patch add
a config option in order to allow configurable share sizes as cinder
does (Cinder change-ID reference:
I2897f6d6bd970f73867f56d9d23a768cafcbfd80).
TrivialFix
Change-Id: Ifd09519b63c6c1a0ec8449e5dae1ffe0caa1b879
In a cloud with multi-tenancy, administrators
would like to see the project_id of the tenant
while listing snapshots of all tenants.
Likewise, we also should expose the user_id field
as it would help in cases where tenants have multiple
users.
DocImpact
User and administrator references need to capture
this information.
APIImpact
API changes are micro-versioned.
Change-Id: I3ad655dc9ab7440f205b0e153fccaa99abb79bbd
Closes-Bug: #1587161
Filter noqa is used to avoid expected pep8 failures.
But now, tempest has lots of these without any reason.
Therefore, remove this filter and unused modules that were
discovered as unused without it.
Change-Id: I96f8a4a7e7a302f08581eed9e037f388e4c86802
Add "user_id" detail when we run command
"manila show/create/manage ...". Make the operator know
which user created this share.
APIImpact
Closes-Bug: #1562846
Change-Id: I2858c7f63182288f354b96448f0970d3642d4bf7
By default, these tests are disabled; 3rd party CI systems
should enable these tests through the
RUN_MANILA_REPLICATION_TESTS environment variable
if their driver supports share replication.
Partially implements: blueprint manila-share-replication
Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Change-Id: Idaf3c61600337cc30568b982dfb23948209c1d6e
Removed functionality of Share Migration relying on Manila Share
Service node, moved code to Data Service node for copy phase.
Added parameter 'notify' and share/api methods for future
implementation (see dependent patches).
Added new copy operation statuses, in order to implement future
API calls to obtain progress and cancel migration.
Added possibility of 2-phase migration for driver migration and
generic (fallback) migration.
Added admin export location support and removed approach of
replacing IP with config parameter.
Added Admin-only API entry points to:
- Migration Cancel (only during copying)
- Reset Task State field
- Migration Get Progress (only during copying)
- Migration Complete (2nd phase migration)
- Notify parameter on Migrate Share
APIImpact
DocImpact
Implements: blueprint data-service-migration
Change-Id: I1d65aac2f36942cd70eb214be561d59a15a4ba26
Update the export location retrieval APIs for shares and share
instances to do the following:
1. Restore the API-to-view-builder calling convention of
index-->summary and show-->detail. In so doing, modify which
values are returned by the list commands (currently, all but
the timestamps). The admin context from the request determines
whether the admin-only values are returned.
2. Report the UUID field from the export location table as
'id' to be consistent will all other objects returned via the
Manila REST API.
3. Add the preferred flag to the output of the API. Drivers
can report preferred:True or preferred:False in their export
location metadata, and this standard flag will be returned
via the REST interface, like this:
+-------------------+--------------------------------------+
| Property | Value |
+-------------------+--------------------------------------+
| is_admin_only | False |
| uuid | df828d44-0b04-47fa-8ee5-516ffc199ca7 |
| share_instance_id | 1b40e873-331e-4e1c-ab53-38ec95b3bfcc |
| path | 10.0.0.100:/share_1b40e873 |
| created_at | 2016-02-18T21:12:51.000000 |
| updated_at | 2016-02-18T21:12:51.000000 |
| preferred | True |
+-------------------+--------------------------------------+
APIImpact
Implements: blueprint update-export-location-retrieval-apis
Change-Id: Ia63477d4f3e28ab4f53d7b9d51756cc798c977b9
This driver enables use of the Ceph filesystem for
Manila shares. Clients require a native CephFS
client to access the share. The interface
to Ceph is CephFSVolumeClient, included in
the 'Jewel' Ceph release and later.
APIImpact
The API microversion is bumped to 2.13 to add a
new access type, cephx, supported by the driver.
Co-Authored-By: Ramana Raja <rraja@redhat.com>
Implements: blueprint cephfs-driver
Change-Id: I33801215f64eacb9dade4d92f11f659173abb7f5
Add APIs to support manage and unmanage share snapshots.
Also add support in the Generic driver.
This only supports for DHSS=False driver mode.
Add provider_location column to the share_snapshots table
to save data used to identify the snapshot on the backend.
Also need to bump microversion.
APIImpact
DocImpact
Change-Id: I87a066173c85d969607d132accd9f0e9bd49c235
Implements: blueprint manage-unmanage-snapshot
We have possibility to disable some sets of Manila functional tests
using config options and cannot do it for quota tests.
So, add new 'run_quota_tests' option to be able to do so.
Change-Id: I0ce3170a2568f24a86573e4f45661a4e862af6ac
This patch provides the scheduler support to filter share
backends matching replication capabilities reported by the
hosts and the replication_type extra_spec provided via the
share_type during share creation.
It also adds wsgi routes, API endpoints and driver entry
routines to support the actions: list, show, create, delete
and promote share replicas. It augments the ShareInstance
DB model with a 'replica_state' attribute and the Share DB
Model with 'replication_type' attribute to support these
workflows.
Replica states are periodically updated from the respective
backends that the replicas are created on.
APIImpact
Impact on existing APIs:
In Microversion 2.11, the /shares APIs return 2 additional
fields during index and show calls for each share: 'has_replicas'
and 'replication_type'. Similarly, the field 'replica_state' is
added to the API response for /share-instances.
Also, deletion of a share that has replicas is forbidden,
returning error code 403.
DocImpact
Co-Authored-By: Alex Meade <mr.alex.meade@gmail.com>
Implements: blueprint manila-share-replication
Change-Id: I10515d55b1291c34777a31d8c6a3a1954f551235
Raissa Sarmento