With oslo-incubator going away, we need to pull those classes into
the Manila code base, along with their unit tests. This presents a
good opportunity to do some long-needed housecleaning. This commit
does the following:
1. Moves the scheduler classes from openstack.common to manila.
2. Adds the unit tests from olso-incubator into Manila.
3. Removes duplication among the combined scheduler modules.
4. Moves scheduler drivers into a sub-module.
5. Normalizes class and module naming throughout the scheduler.
6. Splits some unit test files so they match the names of the
modules that they test.
7. Converts usage of mox & oslotest to mock & unittest.
8. Adds a few unit tests to boost coverage levels.
Implements: blueprint reorganize-manila-scheduler
Change-Id: I7aa237e17787e89a95bb198093ea9bc9498279cd
Share Migration allows a share to be migrated from
one host#pool to another host#pool through the
"manila migrate <share> <host#pool>" command. It first
calls the driver to perform it in an optimized way if
possible. If the driver returns that it did not migrate,
it performs a generic migration.
A new field has been added to "shares" table: task_state,
which tracks migration status.
For driver migration, the method migrate_share in driver
base class should be overridden.
For generic migration, drivers may use new config options
to achieve the necessary configuration:
- migration_mounting_backend_ip: If backend has additional
exports IP for admin network, specify it here.
- migration_data_copy_node_ip: IP of entity performing
migration between backends, such as manila node or
data copy service node. This may not apply for
DHSS = true drivers.
- migration_protocol_mount_command: specify mount command
with protocol and additional parameters. Advisable to restrict
protocols per backend. Defaults to "mount -t <share_proto>".
If additional customization is needed, drivers may override
certain methods:
- _mount_share: return the mount command.
- _umount_share: return the umount command.
- _get_access_rule_for_data_copy: return an access rule with
the IP address which will allow the manila node or data copy node
to mount the share after added permission through
allow-access API command.
Change-Id: I8dde892cb7c0180b2b56d8c7d680dfe2320c2ec7
Implements: blueprint share-migration
This patch implements the scheduler changes for
scheduling the creation of consistency groups,
shares within consistency groups, and cgsnapshots.
The consistency group scheduling filter was added
to allow the filtering of hosts that are not supported
by a specified consistency group. If no CG is specified
on share create then this filter is a no-op. CGs will be
scheduled only to a backend that supports all of its
specified share types.
Partially implements bp manila-consistency-groups
Change-Id: Ia03191085cefb47a17ce99ad3f30ba70412f5802
On last midcycle meetup was decided to make snapshots optional feature.
Features:
- Add new boolean capability 'snapshot_support' to base share driver
so every existing share driver will inherit it. Make value of it be
calculated from the fact of redefinition of three main driver
methods for snapshots 'create_snapshot', 'delete_snapshot' and
'create_share_from_snapshot'.
- Set extra spec 'snapshot_support' with share type creation by default
to 'True'
- Restrict deletion of extra spec 'snapshot_support' that is expected
to exist
- Allow to redefine new extra spec 'snapshot_support'
- Restrict API 'snapshot create' for share created with
share type that has extra spec 'snapshot_support' equal to 'False'.
- Add migration where new extra spec 'snapshot_support' is added
to all share types that do not have it yet.
Partially implements bp snapshots-optional
Change-Id: I069d9e911c7d7a708fa518b38ed10572a45e5f42
Rework availability zones support which was inherited from Cinder:
- Add public API extension
- Preserve AZ if creating a share from a snapshot
- Always set AZ in Share API or Share Manager
- Update db schema and create db migration
- Update appropriate unit tests
APIImpact
Partially-Implements: blueprint availability-zones
Change-Id: Iea9fbc3fea5c0128772115c028989121f397e0c5
This commit adds a possibility to create/delete
share instances and allow/disable access to individual
share instances through Share API. It's required minimum
in Manila core to implement share migration functionality.
Also this commit updates methods in Share RPC API,
Share manager, Scheduler RPC API and Share manager
to maintain consistency.
Partially implements bp share-instances
Change-Id: I3af93071b5851944b59a3c5b1a0c2296e1267bfe
A share_type is set to None, when creating a new share
without specifying a share type and a default_share_type
option is set to None.
Make scheduler check presense of a share type and
raise error if it is absent.
The raised error is logged finally.
Closes-Bug: #1477789
Change-Id: I6cdf3ce3940af4095b8e6afca4861ad1c2fc4795
The scheduler API 'get_pools' was copied from Cinder and has several
problems. The REST endpoint, /scheduler-stats/get_pools, is not a
REST-appropriate name. The API allows filtering but filters are not
currently supported by the Manila scheduler. The API was added as an
extension, not part of the v1 API as it should be. After restarting
the scheduler service, the API doesn't return any data until a share
has been created (Cinder was used as a model for fixing this one).
This patch fixes all of these issues.
Closes-Bug: #1422042
Change-Id: I7e32efb5390fd4b0a62f644ca5a12dcad14d8a9b
Currently the driver_handles_share_servers is an optional extra_spec
on each share_type that allows the scheduler to filter. We want to
make this a required extra spec, so that every share type has this
value set to true or false. It's not be possible to remove that extra
spec. Modifying the extra spec is allowed as long as the value
remains either true or false.
- Add validation checks to types_manage and types_extra_specs API
controllers
- Add required_extra_specs to data returned by API methods
types_show() and types_index()
- Add db migration, which adds required extra_spec to all existing
share_types
- Add get_required_extra_specs(), validate_required_extra_spec(),
get_valid_required_extra_specs()
methods to share_types module
- Add appropriate unit tests
Partially implements bp share-type-require-driver-mode
Change-Id: I60a877b05a642ea2945b15248e885cf6f7f89aa0
Addon of required extra spec 'driver_handles_share_servers' to share types
requires simultanious change of client and server side. So, add
temporary workaround for scheduler to ignore mentioned spec and unblock gates,
until server and client side have all changes done.
This commit should be reverted right after sync is completed.
Change-Id: I72228c0f610b43573e0aea661d80e5ae4a289e46
This change adds support for share_type, share_types, and share_type_id
along side the corresponding volume_type, volume_types and
volume_type_id fields in the API. This change was made as share_type and
its references are more intuitive in comparison to volume_type.
Both entries are returned to ensure API compatibility but preference is
given to share_type[s,_id].
The database migration introduced in this change should work on MySQL
and Postgres, but will not work on SQLite. Because there is no point
in supporting SQLite, tests for it are removed.
Implements blueprint: rename-volume-type-to-share-type
Change-Id: Ifa23ee6520b5238c7de3d5bce7b7bbde02a4a61d
Module 'log' from oslo-incubator was removed after release of oslo_log library.
So, start using oslo_log, but keep oslo-incubator code yet other common modules
within Manila codebase use it.
Implements bp use-oslo-log-lib
Change-Id: I88224f7c2bd99adb78140dfc3fa73cea437f29cd
This change introduces pool-aware scheduler to address the need for
supporting multiple pools from one storage controller.
Derived from the Cinder Implementation of Pool-aware Scheduler -
https://review.openstack.org/#/c/98715/ and
https://review.openstack.org/#/c/119938
Implements blueprint: dynamic-storage-pools
Change-Id: I3aee5ed1f96f972f7d40fbd981393559587c1a23
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
oslo.messaging is the only exception because this package doesn't
currently support non-namespaced imports.
Change-Id: I3987e651bc880c8ffa7c0105df0298679dcd3a43
LOG.warn etc. should be translated separately and thus messages need to
be marked with _LW for LOG.warn, _LI for LOG.info and _LE for LOG.errors
and LOG.exception.
Mark all LOG invocations with proper translation marker.
Use ',' instead of '%' when adding variables to log messages to allow
lazy evaluation.
Add new hacking checks for these.
Change-Id: I31d3ee50f30c63d7d647b1c2b1eae50bf96f0c74
oslo.i18n provides the i18n function that were provided by
oslo-incubator's gettextutils module
Import _ where needed, oslo.i18n deprecated the builtin method.
Closes-Bug: #1382187
Change-Id: I12aa1c725aa4bb52a9aa46e9c3d2b303839de48b
LOG.debug("Hello %s" % xyz) should be LOG.debug("Hello %s", xyz).
This allows the logging package to skip creating the
formatted log message if the message is not going to be emitted because
of the current log level.
Change-Id: Ie0c91cd940017fd80d1d84b0e56780a1da980858
Fix H405:
H405 multi line docstring summary not separated with an empty line
With this patch, flake8 passes all checks (incl. hacking checks)
except the on-purpose disabled check H904.
Closes-Bug: #1333290
Change-Id: If41259aefc4d6d588da5036f1f25df17b88db04f
Fix the following issues in some files:
F401 'XXX' imported but unused
E123 closing bracket does not match indentation of opening bracket's
line
E126 continuation line over-indented for hanging indent
E128 continuation line under-indented for visual indent
E129 visually indented line with same indent as next logical line
H904 Wrap long lines in parentheses instead of a backslash
Partial-Bug: #1333290
Change-Id: Ic9b3a1e47fd24d2835f35f2ef71c4519908ff049
According to the OpenStack translation policy available at
https://wiki.openstack.org/wiki/LoggingStandards debug messages
should not be translated. Like mentioned in several changes in
Nova by garyk this is to help prioritize log translation.
Change-Id: Idd917acd97481d6b644e307adb002354b37f34a1
Modify FilterScheduler._schedule_share to call new new method
consume_from_share (it was calling consume_from_volume).
Add HostState.consume_from_share to host_manager.
Change-Id: I124674afe739ccf9648f91fcfbc63fd3be7caecc
Closes-Bug: #1253860
Moving file flags.py to manila/common/config.py,
replacing FLAGS by CONF. Rename modules fake_flags to conf_fixture,
test_flags to test_conf, declare_flags to declare_conf,
runtime_flags to runtime_conf like it was done in cinder, nova, glance etc.
Implement bp: use-oslo-conf
Change-Id: I38d869123e5e706d3b06f1844b97ead05e22668f
Clinton Knight