Add share backup feature in Data Copy Service and
Share Service. It will allow the user to create, restore
and delete backups as well as listing backups and showing
the details of a specific backup.
APIImpact
DOCImpact
Change-Id: I7d10cf47864cd21932315375d84dc728ff738f23
Implement: blueprint share-backup
Currently we register all parameters about the quota feature to
[DEFAULT] section. However it is difficult for operators to identity
that some of these parameters like reservation_expire are used in the
quota feature, based of names and descriptions of the parameters.
This change migrates all quota options to the new [quota] section,
so that operators can easily understand which parameters are used
in the quota feature.
Closes-Bug: #1934025
Change-Id: I83b5750da7083718d39efb56262a49dae0a05f48
This feature allows admin to set share size limit for a project.
The defaults will either come from the default values
set in the quota configuration option or via manila.conf
if the user has configured default values for quotas there.
The quota_per_share_gigabytes defaults to -1["No Limit"] always
unless changed in manila.conf by admin.
Closes-Bug: #1811943
Change-Id: Ida126c8c419b8bf4d2a194f061a0809d52b47ab8
Fixes the OverQuota issue while managing shares. Now, when a share
manage request is received and there is no available quota to this
resource, manila will allow the quotas to be exceeded and the
administrator will need to adjust it.
Change-Id: If6edfa79965f1a0e6959b436c53a714217d23f7d
Closes-Bug: #1863298
This patch adds new quotas for share replicas and replica sizes.
This quotas can be related to either tenants and users or tenants
and share types. Now, when creating a share replica, manila will
check if there are resources available for that specific request.
Partially-Implements: bp limit-share-replicas-per-share
Change-Id: I8ba7bc6f167c28d6c169b2187d0e1bda7cad3f69
Upon share type deletion, we were leaving behind
share type quotas, usages and reservations. These
couldn't be cleaned up without resorting to manual
intervention.
Cleanup quotas when a share type is being destroyed,
since the resources that they pertained to are gone
too.
Closes-Bug: #1811680
Change-Id: I04b1fe88808596aa8c05429b964190d654587f9a
Fix pylint E103 warning raised by usage of
self.__class__ to refer to the derived class
in super() methods.
self.__class__ is a reasonable first argument
to super() in any method of a class, as long
as the method is not going to be invoked in
derived classes.
Python3 removes this ambiguity by not requiring
arguments for the super() method.
[1] https://docs.pylint.org/en/1.6.0/features.html#id33
Change-Id: I6071b6cfd8cff2be3853d739f71b94da990cda97
"Quota" APIs now will return two new following keys:
- 'share_groups'
- 'share_group_snapshots'
For user and project, but not share type.
Default values can be configured using following config options:
- 'quota_share_groups'
- 'quota_share_group_snapshots'
APIImpact
DocImpact
Implements BluePrint add-share-groups-quota
Change-Id: I397a8e886226cb22fa50abdf2a4a938bb04c655d
With this feature it will be possible to set quotas per share type
for all existing quota resources. It is useful for deployments with
multiple backends that are accessible via different share types.
Also, fix one of existing DB migrations that hangs on PostgreSQL.
APIImpact
DocImpact
Implements blueprint support-quotas-per-share-type
Change-Id: I8472418c2eb363cf5a76c672c7fdea72f21e4f63
Manila's sqlalchemy API has decorators to require
a context or admin context argument to its DB
methods.
Add missing context-check decorators where context
argument is required in accord with the following
principles:
1. Private methods should begin with underscore and
and public methods should not.
2. All public methods should have appropriate context
requirement decorators.
3. No private methods have context requirement
decorators since these are redundant if
principle #2 is enforced.
Correct unit tests that inappropriately called these
methods without context as well.
Closes-Bug: #1580690
Change-Id: Ic448d40ef83a02837dd9bc2c6465080387305ca1
Some of tests use different method of assertTrue(isinstance(A, B)) or
assertEqual(type(A), B). The correct way is to use assertIsInstance(A,
B) provided by testtools.
Change-Id: I9053c9deaa71ee12e10db73b19250b3a32f3d93c
Closes-bug: #1268480
PEP-0274 introduced dict comprehensions to replace dict constructor
with a sequence of length-2 sequences, these are benefits copied
from [1]:
The dictionary constructor approach has two distinct disadvantages
from the proposed syntax though. First, it isn't as legible as a
dict comprehension. Second, it forces the programmer to create an
in-core list object first, which could be expensive.
Manila does not support python 2.6, we can leverage this.
There is deep dive about PEP-0274[2] and basic tests about
performance[3].
Note: This commit doesn't handle dict constructor with kwagrs.
This commit also adds a hacking rule.
[1]http://legacy.python.org/dev/peps/pep-0274/
[2]http://doughellmann.com/2012/11/12/the-performance-impact-of-using
-dict-instead-of-in-cpython-2-7-2.html
[3]http://paste.openstack.org/show/480757/
Change-Id: I87d26a46ef0d494f92afb1d3bebda2797a12413c
Closes-Bug: #1524771
Replace assertEqual(None, *) with assertIsNone in tests to have
more clear messages in case of failure.
Closes-Bug: #1280522
Change-Id: I2d5285add72678b5a61b4f66879c4d066afc34c7
- Move all DB API unit tests to
manila/tests/db/sqlalchemy/test_api.py
- Move DB API helper functions used in unit tests
to manila/tests/db_utils.py
- Split DB API unit tests in test_api.py module to
appropriate test case classes
- Reduce code duplication in test_api module
Partially implements bp cleanup-db-api-code
Change-Id: Ia6ce9acbb4640e83b5d892dcd1cff886edc8513b
We have several entities in Manila that do have statuses:
- Shares
- Snapshots
- Share Servers
- Share Access Rules
But some share and all share server statuses use uppercase for it.
Make all of them consistent and transform them to lowercase.
Add migrations for it.
Also, remove two unused statuses and reuse all statuses from common place.
Change-Id: I53e6a768c98977d3d94e761349b7446a9dfb4936
Closes-Bug: #1459598
Manila has used single gigabytes quota for shares and snapshots.
Config opt 'no_snapshot_gb_quota' is set to False by default, that has been
used for enabling/disabling of snapshot gigabytes quota considerations.
Add separate snapshot gigabyte quota and remove opt 'no_snapshot_gb_quota' that
is not needed anymore. To be able to set infinite quota for snapshot gigabytes,
just set value for new config option 'quota_snapshot_gigabytes' to '-1'.
Change-Id: I43cb95ff3d0d3e6191f520b52edb67a0a44e9c2c
Implements BP add-snapshot-gb-quota
This change introduces pool-aware scheduler to address the need for
supporting multiple pools from one storage controller.
Derived from the Cinder Implementation of Pool-aware Scheduler -
https://review.openstack.org/#/c/98715/ and
https://review.openstack.org/#/c/119938
Implements blueprint: dynamic-storage-pools
Change-Id: I3aee5ed1f96f972f7d40fbd981393559587c1a23
The Manila test module uses a class called StubOutForTesting, which
is an awkward holdover from when the unit tests were using Mox.
Alex wrote a more elegant and capable mocker method,
manila.test.TestCase.mock_object, in the Cinder project which we
have now added to Manila. So to remove the duplicate functionality
and legacy code, all references to self.stubs.Set() in the Manila
tests should be replaced with self.mock_object() and any related
dead code should be removed.
Resolves-bug: #1415602
Change-Id: Ie5f203c0b2cb57ad00d9ec8db0529075b6fef261
get_project_quotas support a "remain" parameter to add the remaining
space into the returned data structure. This will be tested by the
introduced test.
Change-Id: I612d58365d2b584b0a93ffd65d91cd26839420f6
Partial-Bug: #1348635
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
oslo.messaging is the only exception because this package doesn't
currently support non-namespaced imports.
Change-Id: I3987e651bc880c8ffa7c0105df0298679dcd3a43
Change usage of modules that are deprecated in oslo-incubator.
Use the corresponding module from oslo.utils.
A followup patch will sync with oslo-incubator and remove the now
obsolete modules.
Change-Id: I4f949de57e333832dcc7c1e256ce82e2db0144cb
Partial-Bug: #1382189
Now Manila has too small quota for share networks, that affects gate tests with
exception "ShareNetworksLimitExceeded".
Changes:
- increased quota from 5 to 10;
- updated description of quota with proper info.
Change-Id: I1e3127780785ec689d02a4a0bcce42d903df7d72
Closes-Bug: #1368460
Manila uses oslo-incubator/rpc as an RPC library.
During Havana, oslo/rpc was cleaned up and moved into
oslo.messaging with a more stable and well-defined API.
oslo-incubator/rpc considered as deprecated and should be
replaced with oslo.messaging in Manila.
Sum changes:
* New dependency oslo.messaging>=1.3.0 is required
* The new rpc module has init() and cleanup() methods which manage the
global oslo.messaging transport state. The TRANSPORT and NOTIFIER
globals are conceptually similar to the current RPCIMPL global,
except we're free to create and use alternate Transport objects.
* The rpc.get_{client,server,notifier}() methods are just helpers
which wrap the global messaging state, specifiy serializers and
specify the use of the eventlet executor.
* In oslo.messaging, a request context is expected to be a dict, so
RequestContextSerializer was added which can serialize to and from
dicts using RequestContext.{to,from}_dict()
* The allowed_rpc_exception_modules configuration option is replaced
by an allowed_remote_exmods get_transport() parameter. This is not
something that users ever need to configure, but it is something
each project using oslo.messaging needs to be able to customize.
* We maintain a global NOTIFIER object and create specializations of
it with specific publisher IDs in order to avoid notification driver
loading overhead.
* rpc.py contains transport aliases for backwards compatibility
purposes. setup.cfg also contains notification driver aliases for
backwards compat.
* messaging.ConfFixture is used in tests to override oslo.messaging
config options, rather than making assumptions about the options
registered by the library.
Partially-implements bp oslo-messaging
Change-Id: I42cd582f3e1ff96c8f6e8957122b8e9176b1771d
It allows us to use a bunch of skip
decorators, addCleanup method with
2.6 and 2.7 python versions.
Provides more strict usage of setUp
and tearDown methods.
Change-Id: I62144ba43d62e3becb90427d5d6600f97db458df
The set_time_override function in timeutils was written as a
helper function to mock utcnow for unittests before 'mock' was
generally used. Now that we have mock and fixture, we no longer
need to use it.
Change-Id: Ida9b5b0130a38f45e1f51bf4fbdb1d1a5851cf82
Partial-Bug: #1266962
model_query() function was changed recently
to use 'model' object as a second positional
argument, so all callers need to adhere to
this requirement.
Fix 'bad' model_query().
Remove 'skips' from tests that were failing
due to bad model_query() calls.
Closes-Bug: #1275026
Change-Id: Ibf3d45c6a37493cc0ced7cc6e6c552f27b6e4338
Because Manila DB uses 'soft delete', we can't use
unique constaints properly. Solution is to change
'deleted' column from Boolean to the type of 'id'
column and write value of 'id' to 'delete'
on delete operation. To apply unique constr only
to not deleted records we will be adding
'deleted' column to every uc from now on.
Closes-Bug: #1272303
Change-Id: Ia673b91c7d7f700a25824766486a82f07203f3b6
Added per user-tenant quota support.
Added user-quotas extension, that turns on user quota support if it is
loaded.
Added force parameter, that lets to ignore check if admin want to
force update when run 'manila quota-update'
Added 'extended-quotas' extension that has provides ability for admins
to be able to delete a non-default quota (absolute limit) for a
tenant, so that tenant's quota will revert back to the configured default,
and makes the force parameter always be passed if the client wants
to set the new quota lower than what is already used and reserved.
Added user quota support to db.api, sqlalchemy.api, sqlalchemy.models.
Added migrations for user quota support.
Implement bp: user-quota-support
Change-Id: Ifb8f8a041c2fa54e2ed3a8219e87607b161438ca
Moving file flags.py to manila/common/config.py,
replacing FLAGS by CONF. Rename modules fake_flags to conf_fixture,
test_flags to test_conf, declare_flags to declare_conf,
runtime_flags to runtime_conf like it was done in cinder, nova, glance etc.
Implement bp: use-oslo-conf
Change-Id: I38d869123e5e706d3b06f1844b97ead05e22668f
zhongjun