Replaces Huawei driver's use of Python's
native XML library methods with defusedxml's
methods instead. The native library is vulnerable
to XML attacks while defusedxml isn't. This also
makes Bandit 3 issues happier.
Change-Id: I27d1204ec7dafd3b578d1261c3fd2e371ae405fb
Export locations are usually too difficult to memo
rize.Currently, there is no way to determine the
export location before the share is created, so
users wait until the share creation request gets
completed, and then they check the export
locations to mount the share. The generated
export locations are often not human readable
and it is hard to memorize and control them.
Implements: bp/human-readable-export-locations
Change-Id: I72ac7e24ddd4330d76cafd5e7f78bac2b0174883
'reserved_share_extend_percentage' backend config option allows Manila
to consider different reservation percentage for share extend
operation. With this option, under existing limit of
'reserved_share_percentage', we do not want user to create new share if
limit is hit, but allow user to extend existing share.
DocImpact
Closes-Bug: #1961087
Change-Id: I000a7f530569ff80495b1df62a91981dc5865023
Manila can now configure a share network with multiple subnets
in an availability zone. Also, it can add a new subnet for an
availability that has share servers, which will triger an update
share server allocations.
Changes:
- API:
- Bump version to 2.70.
- setup share network with multiple subents per az.
- Block manage server with multiple subnets.
- Allow add subnet for in-use share servers.
- `share_network_subnet_id` is dropped from ShareServer view
- `share_network_subnet_ids` is added in ShareServer view
- `network_allocation_update_support` is added to ShareServer and
ShareNetwork views.
- Add a check operation for share network subnet create.
- DB:
- Remove `share_network_subnet_id` from share_servers.
- Create mapping table `share_server_share_network_subnet_mappings`.
- Fix queries with new db design.
- Add migration downgrade and upgrade alembic.
- Add `share_network_subnet_id` to the NetworkAllocations.
- Scheduler:
- Change `AvailabilityZoneFilter` to take in account if the
host supports the allocation required by the setup request.
- Manager:
- Bump RPC API version.
- `_setup_server` allocating multiple subnets.
- Modify signature of driver `_setup_server` interface, passing a
list of `network_info` for each subnet.
- Share server DB creation to inform a list of subnets and
create with `network_allocation_update_support`.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations`.
- Drivers:
- For legacy compatibility, all drivers implementing `_setup_server`
consume the first element of the `network_ino`.
- Dummy Driver:
- Implement `_setup_server` with new signature as multiple subnet.
- Modify the `backend_details` to save allocations for all subnets.
- Report update allocation and share server multiple subnet support.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations` interfaces.
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Fábio Oliveira <fabioaurelio1269@gmail.com>
Co-Authored-By: Nahim Alves de Souza <nahimsouza@outlook.com>
Co-Authored-By: Caique Mello <caique_mellosbo@hotmail.com>
DocImpact
APIImpact
Partially-Implements: blueprint multiple-share-network-subnets
Change-Id: I7de9de4ae509182e9494bba604979cce03acceec
Python2 is no longer supported, so in this patch
set we remove the usage of the six (py2 and py3
compatibility library) in favor of py3 syntax.
Change-Id: I3ddfad568a1b578bee23a6d1a96de9551e336bb4
This config option allows different value for reservation percentage,
mostly useful on the platforms, where shares can only be created from
the snapshot on the host where snapshot was taken. The lower value of
this config option against existing (reserved_share_percentage) allows
to create shares from the snapshot on the same host up to a higher
threshold even though non-snapshot/regular share create fails.
In case this config option is not set, the shares created from snapshot
will use reservation percentage value set in 'reserved_share_percentage'.
This will be useful for users who want to keep same reservation percentage
for both non-snapshot/regular and snapshot shares.
DocImpact
Closes-Bug: #1938060
Change-Id: I390da933fe92875e3c7ee40709eacacc030278dc
This patch implements the update of security service's association
with in-use share networks. The following changes were added:
- New share network APIs: `share_network_security_service_update`
and `share_network_reset_state`.
- A new `status` attribute was added to share network model to
identify when it's in a modification state, called 'network_change'.
Other supported status that were added: 'active' and 'error'.
- New 'security_service_update_support' property was added to both
share server and share network models, to identify when this resources
are able to process security service update for in-use share networks.
- New driver interface was added to support update of security service's
configuration of a given share server.
DocImpact
APIImpact
Partially Implements: bp add-security-service-in-use-share-networks
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Douglas Viroel <viroel@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Change-Id: I129a794dfd2d179fa2b9a2fed050459d6f00b0de
Fix:
E241 multiple spaces after ':' (or ',')
Fix also a few other problems found by hacking in files changed.
Change-Id: I4c00551f29ad65df2631c8ae55fe2f022e872544
Fix pylint E103 warning raised by usage of
self.__class__ to refer to the derived class
in super() methods.
self.__class__ is a reasonable first argument
to super() in any method of a class, as long
as the method is not going to be invoked in
derived classes.
Python3 removes this ambiguity by not requiring
arguments for the super() method.
[1] https://docs.pylint.org/en/1.6.0/features.html#id33
Change-Id: I6071b6cfd8cff2be3853d739f71b94da990cda97
While Huawei driver mounting CIFS filesystem, the username parameter
key passed to the mount command is by the form of "user=***", which's
compatible as "username=***" in Ubuntu. However, for some other
systems, such as Redhat, it is not valid.
This patch changes it to the more compatible form "username=***".
Change-Id: Ifa2ca7bcdceef2d47e2841a27ddf51d86f1ef63c
Closes-Bug: #1740816
Implement the revert_to_snapshot interface in Huawei driver to
support this capability.
Implements: blueprint huawei-driver-support-snapshot-revert
Change-Id: Ib92afcce59bd5ee018b3a99cff47d892c88145e4
For the latest Python 2.7 release, urllib uses the SSL certification
while launching URL connection by default, which causes Huawei driver
failed to connect backend storage because it doesn't support SSL
certification.
This patch fixes this issue by updating Huawei driver logic to use
requests lib for Huawei storage connection, and specifying no use of
SSL certification.
Change-Id: Ia761819a352163176d353f61682cf47a1f429966
Closes-Bug: #1733451
Please read spec for design detail [1].
Support IPv6 in IP drivers, networks and share
type extra specs.
Co-Authored-By: TommyLikeHu(tommylikehu@gmail.com)
Co-Authored-By: Ben Swartzlander <ben@swartzlander.org>
[1] f7202a6cfe32a057f752a4e393f848f8a0211c36
DocImpact
Partial-Implements: blueprint support-ipv6-access
Change-Id: I96d3389262e9829b8b4344870cdf5c76abd22828
That will be used for scheduling share groups based on their possibility
to create consistent snapshots.
Also apply following tempest plugin changes:
- Add new 'capability_sg_consistent_snapshot_support' tempest config
option, that will be used for creation of new share group types and used
to prove that scheduling works as expected.
- Fix some share group test attributes from 'only API involved' to
'API and Backend are involved', because it is so indeed.
Change-Id: I05553c308ae40c4ddc2c6469ff1c1a3da36a87da
Partially-Implements BP manila-share-groups
This new feature gives the user the ability to allow and
deny access to the snapshots, so that they could be mounted in
read-only mode to retrieve files.
APIImpact
DocImpact
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Alyson Rosa <alyson.rosa@fit-tecnologia.org.br>
Co-Authored-By: Miriam Yumi <miriam.peixoto@fit-tecnologia.org.br>
Partially-implements: blueprint manila-mountable-snapshots
Change-Id: I65f398a05f82eef31ec317d70dfa101483b44b30
Remove the experimental consistency group APIs and
replace them with the experimental Share Group APIs.
DocImpact
APIImpact
Partially-implements-blueprint: manila-share-groups
Change-Id: I79a80a62ae4e0015d6161edc2b93fd1f9ba69537
- Pull up policy check to beginning of the APIs.
- Avoid making access rules changes when one or
more instances of the share are in an invalid state.
- Add back the per rule share instance access status.
This restoration provides better visibility for which
rules were applied successfully.
- Remove 'updating' and 'updating_multiple' as valid
states for the share instance access rules status.
- Deprecate the access rule state 'new' in favor of
'queued_to_apply' and the share instance access rules
status 'out_of_sync' in favor of 'syncing'.
In a new API micro-version:
- Allow access rule changes irrespective of the share's
access_rules_status.
- Expose new access rule states and share's
access_rules_status values.
Access rules for each share instance now transition
from 'queued_to_apply' to 'applying' to 'active' or 'error';
and from 'active', 'queued_to_apply', 'applying' or 'error'
to 'queued_to_deny' to 'denying' to 'deleted'.
APIImpact
DocImpact
Partially-implements: bp fix-and-improve-access-rules
Co-Authored-By: Mike Rooney <rooneym@netapp.com>
Change-Id: Ic25e63215b5ba723cbc8cab7c51789c698e76f28
This commit adds the ability for Manila to revert a
share to the latest available snapshot.
The feature is implemented in the LVM driver, for
testing purposes.
APIImpact
DocImpact
Co-Authored-By: Ben Swartzlander <ben@swartzlander.org>
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Implements: blueprint manila-share-revert-to-snapshot
Change-Id: Id497e13070e0003db2db951526a52de6c2182cca
Nova network was deprecated in Newton and is no longer supported for
regular deployments in Ocata [1].
Complete the removal of nova network support from manila begun in [2]
by removing support for ''nova_net_id`` in the share networks API,
by removing the corresponding column from the share networks DB model,
and by removing the nova network plugins themselves. Unit tests for
share network api views were refactored to remove redundancy while
extending coverage to the new microversion introduced with this patch
and maintaining full coverage of earlier microversions.
APIImpact
DocImpact
UpgradeImpact
Partially-implements: bp remove-nova-net-plugin
[1] http://docs.openstack.org/releasenotes/nova/unreleased.html
[2] I846b760fa7c01f7f86768399a2bfad9ced7e57cd
Change-Id: I8b9a559fbea61979f01737ed1dc272276c4f1269
Array's API supports filtering of filesystem id by
share name, current method gets all filesystem ids,
then filter results in the driver code.
Use NAME filter of array's API instead of the
old way.
Change-Id: I0600f7a519ef6f4ba1c43e88498d6a0dc4859e00
The snapshot_support extra spec has always meant two
things: a driver can take snapshots and create shares
from snapshots. As we add alternate snapshot semantics,
it is likely that some drivers will want to support
snapshots and some of the new semantics while being
unable to create new shares from snapshots.
This work adds a new extra spec,
create_share_from_snapshot_support, that removes the
overloading on snapshot_support. It also makes the
existing snapshot_support extra spec optional,
allowing admins to create types without setting
snapshot_support; shares created with such types
will not support snapshots.
APIImpact
DocImpact
Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Implements: blueprint add-create-share-from-snapshot-extra-spec
Change-Id: Ib0ad5fbfdf6297665c208149b08c8d21b3c232be
huawei driver uses a tag '!$$$' to indicate if username/password
is encoded or not in huawei configuration file.
Currently 'find' method is used to check if this tag is included
in the configuration string, but it'll misread if the tag is
included not in front of the string.
So change the comparing statement to a more accurate way, compare
the front slice of the string directly to the tag.
Change-Id: Ic343970578477362460340b01e5766e03c7d63a4
Closes-Bug: 1635073
While huawei driver reads username/password configuration from
configuration file, if the value is plain text, huawei driver'll
encode this text and write back to configuration file.
huawei driver leverages base64.b64encode to implement the encoding.
The encoded result is a byte string, and then is converted to
unicode string.
In py2, this works fine. However, in py3, the unicode string
converted from byte string is like "b'***'", *** is the orginal
string supposed to be.
So the encoded string is broken, and exception occurs while decoding
from it.
This patch fixes the incorrect unicode string converting and make
the encoded result string right.
Change-Id: I2483ae4999484b3a67c7d443f9b3cc214a3473b2
Closes-Bug: #1613242
Currently, there are 3 statuses for qos:
- 'active', 'inactivated' and 'idle'
If qos status is 'idle', the qos deletion will fail because
huawei driver doesn't deactivate it before deleting.
This change fixes this bug by deactivating qos first if qos
status is not 'inactivated'.
Change-Id: I40f937c22d77d8e07fba176bdc09ca461f01733a
Closes-Bug: #1625424
Currently, huawei driver will use the "AllocType" configuration
or default "thick" type to create share, if "thin-provisioning"
not specified in share_type, but scheduler default uses thin
type in this situation, so huawei driver isn't consistent with
scheduler.
This patch removes the "AllocType" configuration from huawei
driver configuration file and default use "thin" type if
"thin-provisioning" not given.
Depend on change I238a7962425ea35c356c5ed2e31b8f68462b3769.
DocImpact
Change-Id: I5b7128657e089113da44bc862e9c864de2ec59b2
Closes-Bug: #1609718
The cause of failing unit tests is a bug in huawei driver.
When huawei driver logins to backend, it reads username/password from
huawei configuration file. If the username/password are configured as
plain text, huawei driver'll encode them and overwrite the encoded text
to the configuration file.
The encoding logic code is like this:
six.text_type(base64.b64encode("***"))
For py27, this code works fine. However, for py34, this text_type
converting adds some extra words to result unicode string, like
"b'***'"('***' is the actual encoded result string).
Once driver reads the username/password again, it'll get the incorrect
text and then fail while jsonutils.dumps.
All the failed unit tests tried to call login twice, and triggered the
error condition said above.
This patch is just a workaround, because those failed tests actually
no need to login twice, the redundant login calls are removed.
Another patch will be commited to fix the problem thoroughly in huawei
driver. Bug https://launchpad.net/bugs/1612149 has been raised for
that work, and will also analyze what changed to trigger this issue
only recently.
Change-Id: Ia6bf7bbb9ffb9644085bbdf4f5576f09215a877f
Closes-Bug: #1612149
Adding support for reporting disk type of pool in huawei manila
driver. When creating share we can specify the disk type of the
pool we want to use ssd/sas/nl_sas/mix, manila driver will report
the disk type of the pools.
Implements: blueprint huawei-pool-disktype-support
Change-Id: I1f694e8c93f79851f838d226830d9ab5344cf35a
This patch ports cinder's DriverFilter and GoodnessWeigher to manila.
These can use two new properties provided by backends,
'filter_function' and 'goodness_function', which can be used to filter
and weigh qualified backends, respectively.
Reference for cinder spec: I59b607a88953a346aa35e67e785a0417a7ce8cc9
Reference for cinder commit: I38408ab49b6ed869c1faae746ee64a3bae86be58
DocImpact
Change-Id: I873f4152e16efdeb30ceae26335a7974dc9b4b69
Implements: blueprint driver-filter-goodness-weigher
Data in the file system consists of fixed-length disk blocks.
The size of the blocks (also known as file system sector size)
affects disk space usage and performance.
The value of blocks can be 4 KB, 8 KB, 16 KB, 32 KB, or 64 KB.
Change-Id: I95a9aa7e071c470c21dc0fa009ba96a7c7439ce6
Implements: blueprint huawei-driver-block-size-config
In access-allow and access-deny, change 0.0.0.0/0 to asterisk(*)
to represent IP addresses of all clients.
Change-Id: I7b79cf5fde88ba8eb273d71d6eb40bea5b744673
Closes-Bug: #1576557
Manage share snapshot on the array,
before managing the snapshot, make sure that the
source share is managed by OpenStack.
Implements: blueprint huawei-driver-manage-snapshot
Change-Id: I085106dccec5d371771fa112898e980bae182d11
When share is deleted in the backend, user is unable to delete
share because of error thrown in update_access(). Make backend raise
ShareResourceNotFound exception in case share not found during
"update_access" operation, because share manager expects it for
proper handling of share deletion, thus allowing share to be
deleted in this case, without having to use force-delete.
After add ShareResourceNotFound exception, it will need a parameter
named share id(share['id']), so add share id to share info in
create_share_from_snapshot function.
Change-Id: I9756ff882e6960b07f5f0abac94057c687830ad0
Closes-Bug: #1585035
Removed functionality of Share Migration relying on Manila Share
Service node, moved code to Data Service node for copy phase.
Added parameter 'notify' and share/api methods for future
implementation (see dependent patches).
Added new copy operation statuses, in order to implement future
API calls to obtain progress and cancel migration.
Added possibility of 2-phase migration for driver migration and
generic (fallback) migration.
Added admin export location support and removed approach of
replacing IP with config parameter.
Added Admin-only API entry points to:
- Migration Cancel (only during copying)
- Reset Task State field
- Migration Get Progress (only during copying)
- Migration Complete (2nd phase migration)
- Notify parameter on Migrate Share
APIImpact
DocImpact
Implements: blueprint data-service-migration
Change-Id: I1d65aac2f36942cd70eb214be561d59a15a4ba26
Add 'None' as a legal value of 'network_type'
Add choose the port which has least number of logic port
to setup server in flat network.
Change-Id: Id6bd63fce62004f8ff54e6959931cb7a175bbf93
Closes-Bug: #1549183
Add update_access() method to huawei driver.
Do not remove allow_access() and deny_access().
Change-Id: Ic96ee3166b1b9f6a604688c0eea8f1c8131be919
Implements: bp huawei-driver-implement-update-access
Ubuntu