When a user has access to the APIs to reset status,
task state or replica state but doesn't have access to
the share, they must be prevented from performing
those actions. This enforcement allows granular control
of these actions and the resources themselves.
Change-Id: Ic3be777b238a467d1b7bd1daa6aa088dedb095b0
Closes-Bug: #1955627
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
RBAC enforcement in manila happens in stages:
1) Does user have access to the API
2) Does user have access to the resource
3) Is user permitted to perform the API action on the resource
If (1) fails, user gets a HTTP 403, if (2) fails,
they get a HTTP 404; if (3) fails, they get a HTTP 403.
More often than not, (2) prevents "existence" detection
of resources that don't belong to the user; except in
case of "public" resources (e.g.: shares can be "public").
In some share API methods, policy checks for (1) are
happening after a bunch of processing. This leads to
some inconsistency.
Fix these occurrences to ensure a consistent user
experience.
Change-Id: I5b1f1ce517efed000f17b1e0901e183a1913ba9f
Related-Bug: #2004230
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
During replication, at least one share backed is in working and serving
the share. So update the check of instance state from (non-transitional
+ non-error) states to only non-transitional states.
Closes-bug: #1965561
Change-Id: Id23d04f8d18a0e30b511c4d501a1910f2f2b4ca6
Manila can now configure a share network with multiple subnets
in an availability zone. Also, it can add a new subnet for an
availability that has share servers, which will triger an update
share server allocations.
Changes:
- API:
- Bump version to 2.70.
- setup share network with multiple subents per az.
- Block manage server with multiple subnets.
- Allow add subnet for in-use share servers.
- `share_network_subnet_id` is dropped from ShareServer view
- `share_network_subnet_ids` is added in ShareServer view
- `network_allocation_update_support` is added to ShareServer and
ShareNetwork views.
- Add a check operation for share network subnet create.
- DB:
- Remove `share_network_subnet_id` from share_servers.
- Create mapping table `share_server_share_network_subnet_mappings`.
- Fix queries with new db design.
- Add migration downgrade and upgrade alembic.
- Add `share_network_subnet_id` to the NetworkAllocations.
- Scheduler:
- Change `AvailabilityZoneFilter` to take in account if the
host supports the allocation required by the setup request.
- Manager:
- Bump RPC API version.
- `_setup_server` allocating multiple subnets.
- Modify signature of driver `_setup_server` interface, passing a
list of `network_info` for each subnet.
- Share server DB creation to inform a list of subnets and
create with `network_allocation_update_support`.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations`.
- Drivers:
- For legacy compatibility, all drivers implementing `_setup_server`
consume the first element of the `network_ino`.
- Dummy Driver:
- Implement `_setup_server` with new signature as multiple subnet.
- Modify the `backend_details` to save allocations for all subnets.
- Report update allocation and share server multiple subnet support.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations` interfaces.
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Fábio Oliveira <fabioaurelio1269@gmail.com>
Co-Authored-By: Nahim Alves de Souza <nahimsouza@outlook.com>
Co-Authored-By: Caique Mello <caique_mellosbo@hotmail.com>
DocImpact
APIImpact
Partially-Implements: blueprint multiple-share-network-subnets
Change-Id: I7de9de4ae509182e9494bba604979cce03acceec
Add support share Recycle Bin, the end user can soft delete
share to Recycle Bin, and can restore the share within 7 days,
otherwise the share will be deleted automatically.
DocImpact
APIImpact
Partially-Implements: blueprint manila-share-support-recycle-bin
Change-Id: Ic838eec5fea890be6513514053329b1d2d86b3ba
This patch implements hard affinity and anti-affinity filter for
manila scheduler. Users can specify affinity/anti-affinity share
ids to the field "share.scheduler_hints.same_host" or
"share.scheduler_hints.different_host" in the request payload
when creating a manila share. The scheduler_hints are stored as
share metadata. The filter properties are populated from this
metadata during share migration and so filters will be applied
for share migration as well.
Both fields can be a single share UUID or multiple uuids
separated by comma. For example,
`{
"share": {
"scheduler_hints": {
"same_host": "share_uuid_1,share_uuid_2",
"different_host": "share_uuid_3"
}
}
}`
Implements: bp/affinity-antiaffinity-filter
Change-Id: Ic42d8a0c1d22e77ae64e0ca014607b28fd336467
Co-authored-by: Maurice Escher <maurice.escher@sap.com>
Add an force parameter to the API layer that lets the user choose
whether to go through the scheduler or not, which is boolean.and
default is False,set True means extend share directly, set False
means extend share will go through scheduler.
Add an new min_version 2.64 to extend share api. force parameter
only support min_version >= 2.64.
Closes-Bug:#1855391
Change-Id: I6da36a687a37c78a7fb7d3f252318d03d4a05133
Fixes bug #1915706 that was discovered in Manila UI.
Issue is invalid input error being raised, even when
availability_zone provided is the same, as parent share's.
The reason is availability_zone is name and source_share_az
is id in the filter [1]. Now, regardless of availability_zone
provided is id or name, it will be always converted to a name.
And if availability_zone is the same as parent share's
operation will complete successfully.
[1] 2e27c71877/manila/share/api.py (L200)
Closes-Bug: #1915706
Change-Id: I5039b7adf563d926053b423a4cc6e175d3f1e5cf
This bugfix [1] modified the totalcount returned by pagination
query when the argument 'limit' was specified. It caused
manila to do not return precise count of shares in a query that
satisfied the conditions.
This bug has been fixed and now manila is returning the precise
values of shares matched in a given query. Also, manila is now
performing filtering actions in the database to have more
performatic results.
[1] https://review.opendev.org/#/c/688542/
Closes-Bug: #1860061
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Change-Id: I6ddd919bbd5180593cc52bf986912f65a2dab3a7
This patch implements the update of security service's association
with in-use share networks. The following changes were added:
- New share network APIs: `share_network_security_service_update`
and `share_network_reset_state`.
- A new `status` attribute was added to share network model to
identify when it's in a modification state, called 'network_change'.
Other supported status that were added: 'active' and 'error'.
- New 'security_service_update_support' property was added to both
share server and share network models, to identify when this resources
are able to process security service update for in-use share networks.
- New driver interface was added to support update of security service's
configuration of a given share server.
DocImpact
APIImpact
Partially Implements: bp add-security-service-in-use-share-networks
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Douglas Viroel <viroel@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Change-Id: I129a794dfd2d179fa2b9a2fed050459d6f00b0de
Manila APIs have had the requirement to include
project_id in the URLs since the very beginning.
This comes from an old assumption that our APIs
would be differentiated per-tenant on the cloud,
and we would allow different kinds of API endpoints
(public, admin, internal, etc). While it is possible
to set up different endpoints against the API
service, the same and complete API is exposed at
each of these endpoints.
We don't _need_ the project_id information that
we receive in the URL for any of our APIs to
function. We rather authorize tenants by gathering
information from the Identity service (Keystone)
and wrapping that into a RequestContext object
that we then rely on to ensure namespace isolation.
Removing the requirement for "project_id" simplifies
our API endpoint structure in the service catalog
as well as provides a way for system scoped users
to interact with manila without having to declare
their project.
In order to make project_id optional in urls, the
possible values of project_id have to be constrained.
This change introduces a new configuration option
so deployers may control that. This configuration
option defaults to accepting UUIDs with and without
dashes.
Since manila can be used in standalone deployments
without the need for Keystone, this change introduces
a noauth middleware that can work without project_id
in the URL paths.
The API version has been incremented to signal this
change to end users. When 2.60 is available, deployments
may drop "project_id" in the service catalog endpoint
for Manila and end users applications can stop needing
it as well (if they don't already rely on the service
catalog for this data).
APIImpact
Implements: bp remove-project-id-from-urls
Change-Id: I5127e150e8a71e621890f30dba6720b3932cf583
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
The share "manage" API checks whether
an existing/known share is being imported by
matching the export path provided to existing
shares.
This lookup does not consider the
fact that shares may have multiple export
locations, because it relies on an old/deprecated
"export_location" property on shares which
was added to provide backwards compatibility
to the API that presented only one export
location per share.
Further, it's possible to get a
"ERROR: Invalid share: Share already exists"
exception even when no such share exists in the
database.
Fix the lookup by using the "export_location_path"
based lookup which is faster, since it performs
a meaningful join on the export locations table;
and remove the parameters "protocol"
and "share_type_id" - these things make no
difference when there's a duplicated export
location. We'll consider "host" as a lookup
parameter since we can't be sure that export
locations are unique in a deployment - but they
ought to be unique for a given host.
Closes-Bug: #1848608
Closes-Bug: #1893718
Change-Id: I1d1aef0c2b48764789b43b91b258def6464b389f
Co-Authored-By: Goutham Pacha Ravi <gouthampravi@gmail.com>
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
This patch modifies the database api layer to fix the pagination
slowness bug, which causes a delay while the administrator tries
to list shares using the `--limit` option.
This change was tested in a very busy environment with 800 shares.
Before the change, the operation took about 25 seconds to be
finished. Now, the operation takes about 3 seconds in the same
environment.
Change-Id: I89659452b0e033631f1318a2eabb7e120c9e5743
Closes-bug: #1795463
This patch adds the possibility to create share networks with
multiple subnets in Manila. It also updates the share server api
to receive "share_network_subnet_id" instead of "share_network_id".
Each share network subnet must be associated with only one
availability zone. Each share network must have a single default
share network subnet.
DocImpact
APIImpact
Depends-On: I13bb48e7c03e16c26946ccf9d48e80592391a3d1
Partially-implements: bp share-network-multiple-subnets
Change-Id: Id8814a8b26c9b9dcb1fe71d0d7e9b79e8b8a9210
Closes-Bug: #1588144
Co-Authored-By: lseki <luciomitsuru.seki@fit-tecnologia.org.br>
Co-Authored-By: dviroel <viroel@gmail.com>
When doing `manila list --count True` without shares, the user gets
an error. This happens because _list_view do not handle count
properly for 0 shares. This patch-set adds a fix for this case.
Change-Id: Ic6b45260ae39da9ec2c29d05c76d85be1e20635d
Closes-bug: #1822815
When manila API is served behind a proxy, the
"script_name" in the request can have the proxy
component in it. So, this patch fixes the version
selection logic by looking for the version in the
script name string instead of equivalence.
In addition, this patch adds some missing unit
tests and fixes tests that invoke a mocked
wsgi app for testing request context.
Change-Id: I0363d7174f3d7ddefa8ced59b182faed665e9c36
Partial-Bug: #1815038
Closes-Bug: #1818081
This patch adds Manage/Unmanage of share servers in
Manila. It also updates the Manage Share API to accept
a "share_server_id" parameter, and updates Unmanage
of Share and Snapshots API to allow unmanaging of
shares and snapshots in DHSS=True.
Managed share servers are not deleted automatically
by manila, and if a single share is unmanaged in
DHSS=True, the respective share server will not be
deleted automatically as well.
Managing share servers require that the driver
implements 2 functions:
- get_share_server_network_info: obtain IPs from
share server.
- manage_server: perform required operations to
manage and return dict of backend_details.
Unmanaging share servers require that the driver
overrides unmanage_server function.
The IPs obtained from the backend are validated
by the Network plugin, so ports with the exact
IPs must exist in the subnet and admin subnet
associated with the share network specified
when managing the share server (unless
StandaloneNetworkPlugin is used).
It is recommended to rename the backend resource
if possible when managing the share server, to
prevent issues with re-managing a share server
that has already been managed.
This patch bumps the API microversion to 2.49.
APIImpact
DocImpact
Depends-On: I17c74b2aa242918188eeff368232c762a4b31093
Partially-implements: bp manage-unmanage-with-share-servers
Change-Id: I108961e7436ba13550ef2b8f02079c6e599a6166
Public shares are visible across all keystone
projects and are "accessible" by all projects
within a multi-tenant cloud. More often than not,
only privileged users can create and manage shares
that EVERYONE on the cloud can really mount and use.
Manila deployers want a way to disable
unprivileged users from create public shares. Feedback
from deployers has been that public shares created on one
private tenant network (DHSS=True scenario) cannot
be used within another private tenant network belonging
to a different project. So, if users unintentionally create
public shares, they end up pollute the dashboards of users
in other projects. So, we need to make public share
creation a "privileged" feature. This can be achieved by
introducing a manila API policy that defaults to
only allowing the "admin" role. So, this commit will:
- Introduce two new policies:
- share:create_public_share and
- share:set_public_share
- Set the default check string for these policies
to rule:admin_api. They will accept the default
rule temporarily, and log a deprecation warning
message.
- Remove some redundant policy checks in code
and move the policy check up in the API so we
can fail unauthorized requests faster.
When making an API change that potentially changes
the return code from being "successful" (HTTP 2xx)
to returning failure (in this case: HTTP 403,
NotAuthorized), we typically maintain API backwards
compatibility by failing the requests ONLY with newer
API microversions. Following this pattern for API
policy changes will introduce a security hole, i.e.,
users can always create public shares with previous
versions even when an administrator explicitly
sets up policy to disable the action. This is why
this change will not maintain API backwards
compatibility.
APIImpact
Closes-Bug: #1801763
Change-Id: Ib4fc9a82b6a3e5f8e50f0bc8a89d0445eecab028
Administrators configure share types and make them
available to projects within an OpenStack cloud.
These share types will define capabilities to match
back-end storage pools that manila provisions shares
within. Administrators may want to limit share types
to specific Availability zones, given they may have
different classes of storage in different availability
zones in the cloud. A major use case of this is edge
computing, where, provisioning can be driven to specific
edge locations with the help of share types.
This commit will:
- Introduce 'availability_zones' as a new common share type
extra spec that is user visible when configured.
- In and beyond microversion 2.48, validate that the AZ
chosen in the POST /shares API is supported by the configured
availability zones for the share type being used.
- Share types can be filtered by AZs through extra-specs:
$ manila type-list --extra-specs availability_zone=nova
now gives you all types that explicitly (and implicitly)
are supported within the AZ 'nova'.
- Improve experimental APIs:
- Add validation of AZ to POST /share-replicas
- Add validation of AZ to POST /share-groups
- Add validation of AZ to
POST /shares/id {'action': 'migration_start'}
- Also fix old unit tests by using a helper method to
generate appropriate mock values.
DocImpact
Change-Id: Idf274cd73e3b1b33f49668fca768ae676ca30164
Implements: bp share-type-supported-azs
rpc_backend was an old oslo_messaging option
that was deprecated in stable/newton and removed
in master/stein [2], so stop using it in our
unit tests.
The coverage job in manila was always non-voting,
however, a773e31420
added a common coverage job-template from the
openstack-zuul-jobs repository. This change
inadvertently made the non-voting cover job a
voting job.
Our coverage script isn't perfect, it is known
to fail erroneously on non-code changes, and
some genuine failures are just an indication
to developers and reviewers to make the best
effort to adhere to the script's strict
checks [3].
These changes are unrelated, but must be fixed
together to pass the gate, since we have a
chicken-and-egg problem with both failures
preventing separate fixes from merging
separately.
[1] https://review.openstack.org/#/c/317285/
[2] https://review.openstack.org/#/c/580910/
[3] http://git.openstack.org/cgit/openstack/manila/tree/tools/cover.sh
Closes-Bug: #1796759
Related-Bug: #1797512
Change-Id: Ie349c3866d51ea4e706369ad67bc1155f62f2651
Fix pylint E103 warning raised by usage of
self.__class__ to refer to the derived class
in super() methods.
self.__class__ is a reasonable first argument
to super() in any method of a class, as long
as the method is not going to be invoked in
derived classes.
Python3 removes this ambiguity by not requiring
arguments for the super() method.
[1] https://docs.pylint.org/en/1.6.0/features.html#id33
Change-Id: I6071b6cfd8cff2be3853d739f71b94da990cda97
When request default share type and show share type detailed,
the share type results is not correct, because the values of
attribute required_extra_specs is always empty.
Change-Id: I68fc4e2ba30bbd87b5417fe48688edd96cbcee5d
Closes-Bug: #1733742
Added support for display count info
in share list&detail APIs:
1. /v2/{project_id}/shares?with_count=True
2. /v2/{project_id}/shares/detail?with_count=True
Partially-Implements bp add-amount-info-in-list-api
Change-Id: I12c41a46140b04f26565d8934e0326480477c612
Please read spec for design detail [1].
Add support to validate IPv6 based addresses
in allow access API when access type is IP.
[1] f7202a6cfe32a057f752a4e393f848f8a0211c36
APIImpact
DocImpact
Change-Id: Ica242007e77a7e21b2151e5fc32401b501d961b2
Partial-Implements: blueprint support-ipv6-access
Add like filter support in ``shares``, ``snapshots``,
``share-networks``, ``share-groups`` list APIs.
APIImpact
Implements BP like-filter
Change-Id: I5fdf6d89d0b6c7fa182ddfaac60979bc6c0fc2a5
Share and share instance list API will accept new query string parameter
'export_location'. It can pass path and id of export_location to
retrieve shares filtered.
APIImpact
Partly-implement: BP support-filter-share-by-export-location
Change-Id: I5fdf6d89d0b6c7fa182ddfaac60979bc6c0fc2a6
'Share groups' feature was not finished completely in Ocata timeframe.
There was absent possibility to specify 'availability zone' for
creation of a share group. So, implement it.
The fact of its implementation fixes mentioned bug below.
Closes-Bug: #1662944
Partially-Implements BP manila-share-groups
Change-Id: I000adeb53fe8435465cbedc3c539e6aaae6503c5
The shares view uses the 'Share' model to access certain properties such
as the share type or the share type id. These properties are, however,
only proxied to the 'ShareInstance' model. This patch proposes to access
these properties through the 'ShareInstance' model directly in order to
avoid warning messages for proxied properties which pollute the m-api logs.
Closes-Bug: #1692058
Change-Id: I28fa366e106df51c7406673874bd993a44c500bf
When we do manila access-list, the 'created_at' and 'updated_at' time
should be shown to the user. And then the user could determine which
is the recent access rule. As the design, the recent access-allow
rule caused the error access-status of share instance.
APIImpact
Closes-Bug: #1682795
Change-Id: Iad6070d60ec77b7de9cc9679cfa7478876084da1
Currently, The user access name is limited to 32 characters in manila API service,
but actually the user access name is longer than 32 characters. so we need
to change user access name limit from 32 to 255 characters
APIImpact
Closes-bug: 1674908
Change-Id: I68d8afabcd3fef57e472b4067ea8949e0aa8f53a
Also, allow filtering by host based on policy 'list_by_host' that
defaults to 'admin only'.
Do not bump API, because it is not considered as expected behavior,
hence should not be kept for old microversions.
Co-Authored-By: Valeriy Ponomaryov <vponomaryov@mirantis.com>
APIImpact
Change-Id: I799bb7378927b6c3ee0f9fe88fd9876a03dd85b5
Closes bug: 1664370
In order to revert to a snapshot in the LVM driver (and
very likely other drivers) the list of access rules is
needed, so this change modifies the driver interface to
provide this extra information.
This change requires preventing a revert to snapshot
operation while access rules on the affected share are
out of sync.
Closes bug: 1658133
Change-Id: Ia6678bb0e484f9c8f8b05d90e514801ae9baa94b
Remove the experimental consistency group APIs and
replace them with the experimental Share Group APIs.
DocImpact
APIImpact
Partially-implements-blueprint: manila-share-groups
Change-Id: I79a80a62ae4e0015d6161edc2b93fd1f9ba69537
Implemented several improvements to share migration
according to spec [1].
Summary of changes:
- Snapshot restriction in API has been changed to return error only
when parameter force-host-assisted-migration is True
- Added preserve_snapshot to API and migration_check_compatibility
driver interface
- Changed all driver-assisted API parameters to be mandatory
- Added validation to prevent 'force_host_assisted_migration' to be
used alongside driver-assisted parameters
- Changed "same host" validation to reject only if the combination
of "host", "new_share_network" and "new_share_type" is the same as
the source
- Updated migration driver interfaces to support snapshots
- Updated zfsonlinux driver, defaulting preserve_snapshots to False
- Updated dummy driver to support preserve_snapshots
Spec update with latest changes since [1] merged
can be found in [2].
APIImpact
DocImpact
[1] I5717e902373d79ed0d55372afdedfaa98134c24e
[2] If02180ec3b5ae05c9ff18c9f5a054c33f13edcdf
Change-Id: I764b389816319ed0ac5178cadbf809cb632035b4
Partially-implements: blueprint ocata-migration-improvements
Goutham Pacha Ravi