Export locations are usually too difficult to memo
rize.Currently, there is no way to determine the
export location before the share is created, so
users wait until the share creation request gets
completed, and then they check the export
locations to mount the share. The generated
export locations are often not human readable
and it is hard to memorize and control them.
Implements: bp/human-readable-export-locations
Change-Id: I72ac7e24ddd4330d76cafd5e7f78bac2b0174883
Rename a number of APIs to use singular, rather than plural, like every
other API uses:
- share_instances_status_update ->
share_instance_status_update
- share_instances_get_all ->
share_instance_get_all
- share_instances_get_all_by_host ->
share_instance_get_all_by_host
- share_instances_get_all_by_share_network ->
share_instance_get_all_by_share_network
- share_instances_get_all_by_share_server ->
share_instance_get_all_by_share_server
- share_instances_get_all_by_share ->
share_instance_get_all_by_share
- share_instances_get_all_by_share_group_id ->
share_instance_get_all_by_share_group_id
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Ic48fe0d1631a6e1a8ee9a50741cc1b31c9187c37
This reverts commit e885852648.
Reason for revert: This change is breaking drivers that are able to thin provisioning shares. The shares join in the sqlalchemy/api is not working properly, as the share doesn't have an attribute called shares. The join should be performed differently.
Change-Id: Ia5b4e0d57294c3972f55f4fa9a2750ab0251f7bf
The scheduler need to estimate the allocated capacity for thin
provisioning hosts. To do this more efficiently, use a sum fucntion in
query, instead of loading all share instances for a specific host.
Closes-Bug: #2020187
Change-Id: I1f6883c50c74c0822e787de28ba5118a8292e348
'reserved_share_extend_percentage' backend config option allows Manila
to consider different reservation percentage for share extend
operation. With this option, under existing limit of
'reserved_share_percentage', we do not want user to create new share if
limit is hit, but allow user to extend existing share.
DocImpact
Closes-Bug: #1961087
Change-Id: I000a7f530569ff80495b1df62a91981dc5865023
Manila can now configure a share network with multiple subnets
in an availability zone. Also, it can add a new subnet for an
availability that has share servers, which will triger an update
share server allocations.
Changes:
- API:
- Bump version to 2.70.
- setup share network with multiple subents per az.
- Block manage server with multiple subnets.
- Allow add subnet for in-use share servers.
- `share_network_subnet_id` is dropped from ShareServer view
- `share_network_subnet_ids` is added in ShareServer view
- `network_allocation_update_support` is added to ShareServer and
ShareNetwork views.
- Add a check operation for share network subnet create.
- DB:
- Remove `share_network_subnet_id` from share_servers.
- Create mapping table `share_server_share_network_subnet_mappings`.
- Fix queries with new db design.
- Add migration downgrade and upgrade alembic.
- Add `share_network_subnet_id` to the NetworkAllocations.
- Scheduler:
- Change `AvailabilityZoneFilter` to take in account if the
host supports the allocation required by the setup request.
- Manager:
- Bump RPC API version.
- `_setup_server` allocating multiple subnets.
- Modify signature of driver `_setup_server` interface, passing a
list of `network_info` for each subnet.
- Share server DB creation to inform a list of subnets and
create with `network_allocation_update_support`.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations`.
- Drivers:
- For legacy compatibility, all drivers implementing `_setup_server`
consume the first element of the `network_ino`.
- Dummy Driver:
- Implement `_setup_server` with new signature as multiple subnet.
- Modify the `backend_details` to save allocations for all subnets.
- Report update allocation and share server multiple subnet support.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations` interfaces.
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Fábio Oliveira <fabioaurelio1269@gmail.com>
Co-Authored-By: Nahim Alves de Souza <nahimsouza@outlook.com>
Co-Authored-By: Caique Mello <caique_mellosbo@hotmail.com>
DocImpact
APIImpact
Partially-Implements: blueprint multiple-share-network-subnets
Change-Id: I7de9de4ae509182e9494bba604979cce03acceec
This config option allows different value for reservation percentage,
mostly useful on the platforms, where shares can only be created from
the snapshot on the host where snapshot was taken. The lower value of
this config option against existing (reserved_share_percentage) allows
to create shares from the snapshot on the same host up to a higher
threshold even though non-snapshot/regular share create fails.
In case this config option is not set, the shares created from snapshot
will use reservation percentage value set in 'reserved_share_percentage'.
This will be useful for users who want to keep same reservation percentage
for both non-snapshot/regular and snapshot shares.
DocImpact
Closes-Bug: #1938060
Change-Id: I390da933fe92875e3c7ee40709eacacc030278dc
if rabbitmq is too much pressure or blockage.scheduler
will not received service capabilities, but once the
message queue(rabbitmq) returns to normal, scheduler
will received many service capabilities, these service
capabilities are acquired by manila share at different
times, so the timestamp of service capabilities shoud
added at share manage layer(before rpc), but not
scheduler layer(after rpc), once scheduler get an newer
service capabilities, there is no need to update an
earlier service capabilities.
Closes-Bug: #1908963
Change-Id: I6ce99ed4451c5d02cb4446861fa59e55a94951a5
This patch implements the update of security service's association
with in-use share networks. The following changes were added:
- New share network APIs: `share_network_security_service_update`
and `share_network_reset_state`.
- A new `status` attribute was added to share network model to
identify when it's in a modification state, called 'network_change'.
Other supported status that were added: 'active' and 'error'.
- New 'security_service_update_support' property was added to both
share server and share network models, to identify when this resources
are able to process security service update for in-use share networks.
- New driver interface was added to support update of security service's
configuration of a given share server.
DocImpact
APIImpact
Partially Implements: bp add-security-service-in-use-share-networks
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Douglas Viroel <viroel@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Change-Id: I129a794dfd2d179fa2b9a2fed050459d6f00b0de
On those backends, there is no need to calculate provisioned_capacity_gb
as it is not used during the scheduling. This calculation was not
scaling properly on big environments as it implies many database
queries.
Closes-Bug: #1869712
Change-Id: Icb8947991723a2a1cc334a72276a35cf50fc1b7d
This fix doesn't fully address the issue given that share backends
are allowed to use a list of capabilities and reporting thin_provisioning as
[False] will still allow the calculation of provisioned_capacity_gb.
Bug #1869712 will be fixed in a follow up patch.
This reverts commit cd9292b931.
Change-Id: I922f161980e8a1ef7cf111b1e30c02991073d6e5
On those backends, there is no need to calculate provisioned_capacity_gb
as it is not used during the scheduling. This calculation was not
scaling properly on big environments as it implies many database
queries.
Change-Id: If1ea4ceedc495dc6c6e247feccfbdc4899ad725c
Closes-Bug: #1869712
Python 3 is more strict in ordering container elements and raises errors
when attempted to order unorderable types like dicts.
So, fix such places and replace them with proper things.
Ref: https://review.openstack.org/#/c/210252/
Change-Id: I47f2e61d0721ec2759427dd0be0e22c15b21e1a6
Partially-Implements: bp py3-compatibility
Please read spec for design detail [1].
Support IPv6 in IP drivers, networks and share
type extra specs.
Co-Authored-By: TommyLikeHu(tommylikehu@gmail.com)
Co-Authored-By: Ben Swartzlander <ben@swartzlander.org>
[1] f7202a6cfe32a057f752a4e393f848f8a0211c36
DocImpact
Partial-Implements: blueprint support-ipv6-access
Change-Id: I96d3389262e9829b8b4344870cdf5c76abd22828
Filtering issues are most common when creating a share. User messages
now keep information which was the last executed filter if hosts
filtering failed.
DocImpact
Partially-implements: blueprint user-messages
Change-Id: I9ce096eebda3249687268e361b7141dea4032b57
That will be used for scheduling share groups based on their possibility
to create consistent snapshots.
Also apply following tempest plugin changes:
- Add new 'capability_sg_consistent_snapshot_support' tempest config
option, that will be used for creation of new share group types and used
to prove that scheduling works as expected.
- Fix some share group test attributes from 'only API involved' to
'API and Backend are involved', because it is so indeed.
Change-Id: I05553c308ae40c4ddc2c6469ff1c1a3da36a87da
Partially-Implements BP manila-share-groups
This new feature gives the user the ability to allow and
deny access to the snapshots, so that they could be mounted in
read-only mode to retrieve files.
APIImpact
DocImpact
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Alyson Rosa <alyson.rosa@fit-tecnologia.org.br>
Co-Authored-By: Miriam Yumi <miriam.peixoto@fit-tecnologia.org.br>
Partially-implements: blueprint manila-mountable-snapshots
Change-Id: I65f398a05f82eef31ec317d70dfa101483b44b30
Remove the experimental consistency group APIs and
replace them with the experimental Share Group APIs.
DocImpact
APIImpact
Partially-implements-blueprint: manila-share-groups
Change-Id: I79a80a62ae4e0015d6161edc2b93fd1f9ba69537
This commit adds the ability for Manila to revert a
share to the latest available snapshot.
The feature is implemented in the LVM driver, for
testing purposes.
APIImpact
DocImpact
Co-Authored-By: Ben Swartzlander <ben@swartzlander.org>
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Implements: blueprint manila-share-revert-to-snapshot
Change-Id: Id497e13070e0003db2db951526a52de6c2182cca
The snapshot_support extra spec has always meant two
things: a driver can take snapshots and create shares
from snapshots. As we add alternate snapshot semantics,
it is likely that some drivers will want to support
snapshots and some of the new semantics while being
unable to create new shares from snapshots.
This work adds a new extra spec,
create_share_from_snapshot_support, that removes the
overloading on snapshot_support. It also makes the
existing snapshot_support extra spec optional,
allowing admins to create types without setting
snapshot_support; shares created with such types
will not support snapshots.
APIImpact
DocImpact
Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Implements: blueprint add-create-share-from-snapshot-extra-spec
Change-Id: Ib0ad5fbfdf6297665c208149b08c8d21b3c232be
Administrators intend to get the pool's information filtered
by share type(actually filtered by share_type's *extra_spec*)
more directly.
The blueprint is to add a filter key 'share_type' to cover
this situation.
APIImpact
Implements: blueprint pool-list-by-share-type
Change-Id: Ifd64bb84d03a02aa0a118cc42e1d1b373c439884
Currently 'provisioned_capacity_gb' is expected from
the drivers as part of capability reporting.
If driver does not provide it, it defaults to 'allocated_capacity_gb'.
But if driver does not provide both 'allocated_capacity_gb' and
'provisioned_capacity_gb' then 'provisioned_capacity_gb' defaults to 0.
Which affects later calculation of 'provisioned_ratio'.
Hence fixing it by summing up sizes of all the shares of
that host and taking that as 'provisioned_capacity_gb',
in such case.
Change-Id: I844d176eb6f0f5e7b0eb3cbd66c4b413b6757f51
Closes-Bug: #1606691
Scheduler picks up a host even when the manila-share
service status is disabled, Because the
db.service_get_all_by_topic can not get the
manila-share(status=disabled), so if the manila-share
(status=disabled) already exists in self.host_state_map,
it can not be removed from self.host_state_map.
This change add code to get active hosts and then remove
non-active host from the cache every time when scheduler
handles a new request.
Change-Id: I580e018470e55ef064b54e7c6bcb6f35b32f92f9
Closes-bug: #1557854
Backends supporting replication of shares could be
limited by vendor or storage model or software. Cloud
administrators may also need to configure backends
outside of Manila to support replication between these
backends.
Hence, Manila should allow administrators to specify
'replication_domain's. This configuration is reported to the
scheduler as a backend capability as part of the
_update_share_stats() driver call. The scheduler
should acknowledge this 'capability' when scheduling
shares supporting replication and replicas themselves.
Introduce a new scheduler filter that uses a backend
reported "capability" called 'replication_domain'.
Implements bp: manila-dr-scheduler-enhancements
DocImpact
Change-Id: I194054a9a1071c2d577a588e3299ae81490cd0f8
This patch provides the scheduler support to filter share
backends matching replication capabilities reported by the
hosts and the replication_type extra_spec provided via the
share_type during share creation.
It also adds wsgi routes, API endpoints and driver entry
routines to support the actions: list, show, create, delete
and promote share replicas. It augments the ShareInstance
DB model with a 'replica_state' attribute and the Share DB
Model with 'replication_type' attribute to support these
workflows.
Replica states are periodically updated from the respective
backends that the replicas are created on.
APIImpact
Impact on existing APIs:
In Microversion 2.11, the /shares APIs return 2 additional
fields during index and show calls for each share: 'has_replicas'
and 'replication_type'. Similarly, the field 'replica_state' is
added to the API response for /share-instances.
Also, deletion of a share that has replicas is forbidden,
returning error code 403.
DocImpact
Co-Authored-By: Alex Meade <mr.alex.meade@gmail.com>
Implements: blueprint manila-share-replication
Change-Id: I10515d55b1291c34777a31d8c6a3a1954f551235
QoS is a common attribute, so add the QoS
capability in common capabilities doc.
This is a simple first step towards what the
blueprints proposes.
Partially implements: bp manila-support-qos
Change-Id: I377bf0abcc62239c9a1a5ee5c28c336b2b6c410a
Some of tests use different method of assertTrue(isinstance(A, B)) or
assertEqual(type(A), B). The correct way is to use assertIsInstance(A,
B) provided by testtools.
Change-Id: I9053c9deaa71ee12e10db73b19250b3a32f3d93c
Closes-bug: #1268480
The unit test test_get_all_host_states_share(),
testing host_manager's get_all_host_states_share()
function makes an unnecessary call to the manila/utils.py
function service_is_up(). This test fails sporadically
due to initialization problems with the fake data provided.
This patch mocks out the service availability check.
The service availability check already had its own
unit tests.
TrivialFix
Change-Id: I78090e86f5558d6b2192b0559445b7530380bcac
There are a few places where duplicate keys are defined in the
dictionary. Removed all such occurrences throughout the code.
TrivialFix
Change-Id: I3f22a2ce3bc4d67bfaed2dee7918610b3225e26e
With oslo-incubator going away, we need to pull those classes into
the Manila code base, along with their unit tests. This presents a
good opportunity to do some long-needed housecleaning. This commit
does the following:
1. Moves the scheduler classes from openstack.common to manila.
2. Adds the unit tests from olso-incubator into Manila.
3. Removes duplication among the combined scheduler modules.
4. Moves scheduler drivers into a sub-module.
5. Normalizes class and module naming throughout the scheduler.
6. Splits some unit test files so they match the names of the
modules that they test.
7. Converts usage of mox & oslotest to mock & unittest.
8. Adds a few unit tests to boost coverage levels.
Implements: blueprint reorganize-manila-scheduler
Change-Id: I7aa237e17787e89a95bb198093ea9bc9498279cd
Drivers that report no value for the "compression" common capability
should be treated in the scheduler as if compression=False.
Change-Id: I60ebfd24f623ec3da081e48afa39cc5c2fee52b0
Closes-Bug: #1504746
Fix incorrect order assertEqual(observed, expected) as below.
assertEqual(observed, expected) => assertEqual(expected, observed)
Target of this patch:
manila/tests/scheduler/*
manila/tests/volume/*
and all others
Change-Id: I063d35c01835ed6a5c62f016274b02e29333aade
Closes-Bug: #1280522
For a driver that doesn't support snapshots, a share type with
snapshot_support = '<is> False' will prevent share creation.
Root cause is code in HostManager that improperly checks whether
snapshot_support (and other boolean properties) is set on each
pool. This commit fixes the check for the boolean pool capabilities.
Change-Id: I6a488788926cca119498b3de992beec3b0545259
Closes-Bug: #1498646
Drivers that report no value for the "dedupe" common capability
should be treated in the scheduler as if dedupe=False.
Change-Id: I5dd49b5a28563106d35739c64f450118c4583f78
Partial-Bug: #1487467
This patch implements the scheduler changes for
scheduling the creation of consistency groups,
shares within consistency groups, and cgsnapshots.
The consistency group scheduling filter was added
to allow the filtering of hosts that are not supported
by a specified consistency group. If no CG is specified
on share create then this filter is a no-op. CGs will be
scheduled only to a backend that supports all of its
specified share types.
Partially implements bp manila-consistency-groups
Change-Id: Ia03191085cefb47a17ce99ad3f30ba70412f5802
As was discussed on Liberty Mid-Cycle meetup
instances of "infinite" capacity should be replaced
with "unknown" and this capacity should be sorted
to the bottom in the capacity weigher.
Change-Id: I9b37a5ff9cceb35a964b2a0d243688eb73e5cadc
Closes-Bug: #1487478
According to new common capabilities documentation
Manila has common capability thin_provisioning with
boolean value. That's why instances of
thin_provisioning_support and thick_provisioning_support
capabilities should be replaced with
thin_provisioning=True/False.
Depends-On: I2c5bdf5ef9bfd6c8e7e06dac1fde08ad56cc6af1
Change-Id: If366fb242e4903f0b2693132d312b9318f4d89a4
Partial-Bug: #1487467
On last midcycle meetup was decided to make snapshots optional feature.
Features:
- Add new boolean capability 'snapshot_support' to base share driver
so every existing share driver will inherit it. Make value of it be
calculated from the fact of redefinition of three main driver
methods for snapshots 'create_snapshot', 'delete_snapshot' and
'create_share_from_snapshot'.
- Set extra spec 'snapshot_support' with share type creation by default
to 'True'
- Restrict deletion of extra spec 'snapshot_support' that is expected
to exist
- Allow to redefine new extra spec 'snapshot_support'
- Restrict API 'snapshot create' for share created with
share type that has extra spec 'snapshot_support' equal to 'False'.
- Add migration where new extra spec 'snapshot_support' is added
to all share types that do not have it yet.
Partially implements bp snapshots-optional
Change-Id: I069d9e911c7d7a708fa518b38ed10572a45e5f42
Python 3 is more strict in ordering container elements and raises errors
when attempted to order unorderable types like dicts.
So, fix such places and replace them with proper things.
Partially-Implements: bp py3-compatibility
Change-Id: Ife20b4b112aae9ff4a784c54e2ad7b2f518ab4a9
Func 'xrange' is absent in py3. So, use six.moves.range instead for py34
compatibility.
Change-Id: I76217598ae08849c2a93746392fc552dc0a51de1
Partially-Implements: bp py3-compatibility
jayaanand.borra@netapp.com