Export locations are usually too difficult to memo
rize.Currently, there is no way to determine the
export location before the share is created, so
users wait until the share creation request gets
completed, and then they check the export
locations to mount the share. The generated
export locations are often not human readable
and it is hard to memorize and control them.
Implements: bp/human-readable-export-locations
Change-Id: I72ac7e24ddd4330d76cafd5e7f78bac2b0174883
services are down
enhance the user message when all of the share
manager services are down or are still
initializing
removed a duplicate test: test_create_share_non_admin
Closes-Bug: #1886690
Change-Id: I168564a5b054d17762ad668ebbe4f5e7b562197b
Rename a number of APIs to use singular, rather than plural, like every
other API uses:
- share_instances_status_update ->
share_instance_status_update
- share_instances_get_all ->
share_instance_get_all
- share_instances_get_all_by_host ->
share_instance_get_all_by_host
- share_instances_get_all_by_share_network ->
share_instance_get_all_by_share_network
- share_instances_get_all_by_share_server ->
share_instance_get_all_by_share_server
- share_instances_get_all_by_share ->
share_instance_get_all_by_share
- share_instances_get_all_by_share_group_id ->
share_instance_get_all_by_share_group_id
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Ic48fe0d1631a6e1a8ee9a50741cc1b31c9187c37
Add the new scheduler weigher NetAppAIQWeigher that relies on
the NetApp Acitve IQ software to weigh the hosts. It only
works with NetApp only hosts.
It is also adding a new NetApp specific pool information
called ``netapp_cluster_name`` that contains the name
of the cluster where the pool is located.
Implements: netapp-active-iq-scheduler-weigher
Signed-off-by Felipe Rodrigues <felipefuty01@gmail.com>
Change-Id: I36b08066545afdaa37e053eee319bc9cd489efdc
1. Change context as first argument to function.
2. Fix spelling mistake in version history
3. Add new host_admin RBAC policy which is applied in onlyHostFilter
since non-admin user as well needs to create share on specific host.
Change-Id: Id2c09ebab874ec983da7f26370932d46a0447801
When manila services are stopped or restarted via stop(), the DB
entries are not deleted, they are destroyed only in kill() method. In
cluster deployments, where multiple instances of manila services are
deployed via PODs, unique hostname is derived from node name. However
if pods are deployed again and launched on new hosts/nodes, the old
entries of manila service remains as it is.
Fix it by adding 'state' column in 'services' table and introducing
per service cleanup function. On service stop, state is changed to
'stopped' and cleanup function will delete 'stopped' services unless
they are 'up' again before cleanup periodic interval.
Closes-bug: #1990839
Change-Id: I8b71c4c27ff8fcb25616a95a5ed8362a7f4ffc61
Instead of using all default filters of share create operation, use only
necessary list of filters for share extend operation. A new config
option `scheduler_default_extend_filters` is added to provide such list
of filters e.g. CapacityFilter.
Closes-Bug: #1973621
Change-Id: I162325f3096227b74a58b9a55de4660668454d4b
When trying to compare two values that are non-numeric using the driver
filter, the filter function will give an error. This is not desirable as
it might be interesting to support comparatives with non-numeric values
provided by the filter objects (share, host, etc). For example, the
following formula failed before the fix:
filter_function = '(share.project_id == "bb212f09317a4f4a8952ef3f729c2551")'
Copied from cinder 87a7e80a2c
Closes-Bug: #1975715
Change-Id: Icbfabb3bc0f608ebdd0784337db0921cc7763c53
'reserved_share_extend_percentage' backend config option allows Manila
to consider different reservation percentage for share extend
operation. With this option, under existing limit of
'reserved_share_percentage', we do not want user to create new share if
limit is hit, but allow user to extend existing share.
DocImpact
Closes-Bug: #1961087
Change-Id: I000a7f530569ff80495b1df62a91981dc5865023
The default scheduler_hints value is None, and if we don't pass
the scheduler_hints , It will report TypeError when creating share
with JsonFilter with Rest API. The TypeError exception is added to
solve this problem.
Change-Id: Iad89491cbbaccac8df161f8f1157c7ebf3291458
Closes-Bug: #1959472
Fixed the issue in the share migration where the replica quotas
were not being accounted. Manila now tries to allocate share
replica quotas for the migrated share if the destination share type
has a `replication_type`.
If the migration fails or get cancelled, the quotas will be
reverted.
Change-Id: Ie0a08a61c79dc8ea2a7a240363e94b26a80e64a4
Closes-Bug: 1910752
Manila can now configure a share network with multiple subnets
in an availability zone. Also, it can add a new subnet for an
availability that has share servers, which will triger an update
share server allocations.
Changes:
- API:
- Bump version to 2.70.
- setup share network with multiple subents per az.
- Block manage server with multiple subnets.
- Allow add subnet for in-use share servers.
- `share_network_subnet_id` is dropped from ShareServer view
- `share_network_subnet_ids` is added in ShareServer view
- `network_allocation_update_support` is added to ShareServer and
ShareNetwork views.
- Add a check operation for share network subnet create.
- DB:
- Remove `share_network_subnet_id` from share_servers.
- Create mapping table `share_server_share_network_subnet_mappings`.
- Fix queries with new db design.
- Add migration downgrade and upgrade alembic.
- Add `share_network_subnet_id` to the NetworkAllocations.
- Scheduler:
- Change `AvailabilityZoneFilter` to take in account if the
host supports the allocation required by the setup request.
- Manager:
- Bump RPC API version.
- `_setup_server` allocating multiple subnets.
- Modify signature of driver `_setup_server` interface, passing a
list of `network_info` for each subnet.
- Share server DB creation to inform a list of subnets and
create with `network_allocation_update_support`.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations`.
- Drivers:
- For legacy compatibility, all drivers implementing `_setup_server`
consume the first element of the `network_ino`.
- Dummy Driver:
- Implement `_setup_server` with new signature as multiple subnet.
- Modify the `backend_details` to save allocations for all subnets.
- Report update allocation and share server multiple subnet support.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations` interfaces.
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Fábio Oliveira <fabioaurelio1269@gmail.com>
Co-Authored-By: Nahim Alves de Souza <nahimsouza@outlook.com>
Co-Authored-By: Caique Mello <caique_mellosbo@hotmail.com>
DocImpact
APIImpact
Partially-Implements: blueprint multiple-share-network-subnets
Change-Id: I7de9de4ae509182e9494bba604979cce03acceec
e.g. manila create NFS 1 --name Share1 --share-network net1 \
--scheduler_hint="only_host=host1@generic1#GENERIC1"
Since there is no way to create share server in manila, we can use a
workaround of creating first share on specific host
(e.g. host@backend#pool). This will then create the share server
automatically on that host and admin can use idle host when other
hosts are overloaded.
New microversion 2.67 introduced.
DocImpact
Closes-Bug: #1946462
Change-Id: I603434cac246e2c0946672d3f0fe469ed5423fa4
pyparsing renamed a method used to define the operator
precedence hierarchy [1]. The method's behavior was preserved
although opt-in override enhancements to the paranthesis were
introduced.
Needed-by: https://review.opendev.org/818614/
[1] ab2f220dd2
Change-Id: I77ea03284c1db6b4eb171ef6e2e88ac506304502
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
This patch implements hard affinity and anti-affinity filter for
manila scheduler. Users can specify affinity/anti-affinity share
ids to the field "share.scheduler_hints.same_host" or
"share.scheduler_hints.different_host" in the request payload
when creating a manila share. The scheduler_hints are stored as
share metadata. The filter properties are populated from this
metadata during share migration and so filters will be applied
for share migration as well.
Both fields can be a single share UUID or multiple uuids
separated by comma. For example,
`{
"share": {
"scheduler_hints": {
"same_host": "share_uuid_1,share_uuid_2",
"different_host": "share_uuid_3"
}
}
}`
Implements: bp/affinity-antiaffinity-filter
Change-Id: Ic42d8a0c1d22e77ae64e0ca014607b28fd336467
Co-authored-by: Maurice Escher <maurice.escher@sap.com>
This config option allows different value for reservation percentage,
mostly useful on the platforms, where shares can only be created from
the snapshot on the host where snapshot was taken. The lower value of
this config option against existing (reserved_share_percentage) allows
to create shares from the snapshot on the same host up to a higher
threshold even though non-snapshot/regular share create fails.
In case this config option is not set, the shares created from snapshot
will use reservation percentage value set in 'reserved_share_percentage'.
This will be useful for users who want to keep same reservation percentage
for both non-snapshot/regular and snapshot shares.
DocImpact
Closes-Bug: #1938060
Change-Id: I390da933fe92875e3c7ee40709eacacc030278dc
Add an force parameter to the API layer that lets the user choose
whether to go through the scheduler or not, which is boolean.and
default is False,set True means extend share directly, set False
means extend share will go through scheduler.
Add an new min_version 2.64 to extend share api. force parameter
only support min_version >= 2.64.
Closes-Bug:#1855391
Change-Id: I6da36a687a37c78a7fb7d3f252318d03d4a05133
Attempting to create a share without identifying a share type results in an
error.This adds a user message informing the user that no default share type
was identified while attempting to create a share.
Closes-Bug: #1870280
Change-Id: Ib417499d4394b939eb289460ac8dbb6b22d789f4
This change adds a new hook to Manager class called init_host_with_rpc()
to allow services like scheduler to do something once RPC is ready.
Copied from cinder 65fa80c361
and 60c563f72d
Change-Id: Iac6507a6e395c55f0fec453650009f08c2bb6563
Closes-Bug: #1271568
https://review.opendev.org/768137 fixed
bug #1908963 by adding a timestamp field to
the "update_service_capabilities" rpc API
call to the scheduler service.
It's possible that during an upgrade, the
share service is upgraded before a given
scheduler service, and an updated client
could start sending messages with this
extra data that wasn't expected. Lets
bump the version so that the client
messages are just waiting for an updated
scheduler to show up and consume the
messages rather than raise syntax
errors.
Partial-Bug: #1908963
Change-Id: I0d9fc311ffd296bd6153b7190f8f5c42f494a39d
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
if we failed to manage a share, we don't need to commit the quota usages. so
we should skip quota usages cuts when delete or unmange the share with
status of "error_manage". and the size of error_manage share should be
zero.
Closes-Bug:#1883506
Change-Id: I5c81dd6780890c55c8c6a92491c3f4f507531fdb
The share protocol requested was being
ignored by the scheduler and this would
cause shares to get scheduled to hosts
that don't support the specified protocol.
Change-Id: I2e87264865b645781c481383c039fecbfd7c6eb1
Closes-Bug: #1783736
if rabbitmq is too much pressure or blockage.scheduler
will not received service capabilities, but once the
message queue(rabbitmq) returns to normal, scheduler
will received many service capabilities, these service
capabilities are acquired by manila share at different
times, so the timestamp of service capabilities shoud
added at share manage layer(before rpc), but not
scheduler layer(after rpc), once scheduler get an newer
service capabilities, there is no need to update an
earlier service capabilities.
Closes-Bug: #1908963
Change-Id: I6ce99ed4451c5d02cb4446861fa59e55a94951a5
This patch implements the update of security service's association
with in-use share networks. The following changes were added:
- New share network APIs: `share_network_security_service_update`
and `share_network_reset_state`.
- A new `status` attribute was added to share network model to
identify when it's in a modification state, called 'network_change'.
Other supported status that were added: 'active' and 'error'.
- New 'security_service_update_support' property was added to both
share server and share network models, to identify when this resources
are able to process security service update for in-use share networks.
- New driver interface was added to support update of security service's
configuration of a given share server.
DocImpact
APIImpact
Partially Implements: bp add-security-service-in-use-share-networks
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Douglas Viroel <viroel@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Change-Id: I129a794dfd2d179fa2b9a2fed050459d6f00b0de
remove usage of six library from the following directory:
1:common
2:data
3:db
4:message
5:network
6:scheduler
Change-Id: I9db0abf2b0847157074ca6ba84b5451bfe3f20d0
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
This patch enables the creation of a share from snapshot
specifying another pool or backend. In the scheduler, a
new filter and weigher were implemented in order to consider
this operation if the backend supports it. Also, a new
field called 'progress' was added in the share and share
instance. The 'progress' field indicates the status
of the operation create share from snapshot (in percentage).
Finally, a new periodic task was added in order to constantly
check the share status.
Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend
DOCImpact
Change-Id: Iab13a0961eb4a387a502246e5d4b79bc9046e04b
Co-authored-by: carloss <ces.eduardo98@gmail.com>
Co-authored-by: dviroel <viroel@gmail.com>
On those backends, there is no need to calculate provisioned_capacity_gb
as it is not used during the scheduling. This calculation was not
scaling properly on big environments as it implies many database
queries.
Closes-Bug: #1869712
Change-Id: Icb8947991723a2a1cc334a72276a35cf50fc1b7d
This fix doesn't fully address the issue given that share backends
are allowed to use a list of capabilities and reporting thin_provisioning as
[False] will still allow the calculation of provisioned_capacity_gb.
Bug #1869712 will be fixed in a follow up patch.
This reverts commit cd9292b931.
Change-Id: I922f161980e8a1ef7cf111b1e30c02991073d6e5
Fix:
E305 expected 2 blank lines after class or function definition, found 1
Fix also other problems found by hacking in these files.
Change-Id: I880afc40adf974cbb0e60f8dc5931f77d51f451b
On those backends, there is no need to calculate provisioned_capacity_gb
as it is not used during the scheduling. This calculation was not
scaling properly on big environments as it implies many database
queries.
Change-Id: If1ea4ceedc495dc6c6e247feccfbdc4899ad725c
Closes-Bug: #1869712
Currently the value of extra_specs is not case-insenstive. If some
share_type has extra_specs that is `storage_protocol=NFS`, It will
not be filtered by `storage_protocol=nfs`. Make extra_specs value
as case-insensitive.
Change-Id: I51376d1b13a5300cfdbe19f6eadc341db45efcd6
Closes-Bug: #1850029
If a share type includes the share_backend_name spec,
the scheduler may fail to schedule share replicas unless
all backends in a replication domain share the same
value for the capability "share_backend_name". Having
the same "share_backend_name" isn't desirable all the
time.
Ignore the share_backend_name spec from share type
while creating replica, so the scheduler will filter
a available backend according to the selection policy.
Change-Id: Ic8f7e6230298c222cc6cb5e4e4e8189524aaa81f
Closes-Bug: #1634734
Fixed the bug stating: Driver filter disallows using
queries with share_backend_name. The driver filter was
checking for hard equality between the share_backend_name
specified in the share type and the name reported by the
host [1]. This defeats the purpose of the capabilities
filter giving the ability to use "<in>" (selection)
operator in the extra-spec. Thus this commit fixes it by
updating the required files according to
openstack/cinder/commit/b32011 as now the driver does not
check for shared_backend_name.
Closes-Bug: #1815700
Change-Id: If384392d0f140922bb85a1dc50b0c4494c231e2f
Administrators configure share types and make them
available to projects within an OpenStack cloud.
These share types will define capabilities to match
back-end storage pools that manila provisions shares
within. Administrators may want to limit share types
to specific Availability zones, given they may have
different classes of storage in different availability
zones in the cloud. A major use case of this is edge
computing, where, provisioning can be driven to specific
edge locations with the help of share types.
This commit will:
- Introduce 'availability_zones' as a new common share type
extra spec that is user visible when configured.
- In and beyond microversion 2.48, validate that the AZ
chosen in the POST /shares API is supported by the configured
availability zones for the share type being used.
- Share types can be filtered by AZs through extra-specs:
$ manila type-list --extra-specs availability_zone=nova
now gives you all types that explicitly (and implicitly)
are supported within the AZ 'nova'.
- Improve experimental APIs:
- Add validation of AZ to POST /share-replicas
- Add validation of AZ to POST /share-groups
- Add validation of AZ to
POST /shares/id {'action': 'migration_start'}
- Also fix old unit tests by using a helper method to
generate appropriate mock values.
DocImpact
Change-Id: Idf274cd73e3b1b33f49668fca768ae676ca30164
Implements: bp share-type-supported-azs
jayaanand.borra@netapp.com