From eb3ba53a44b3af01232a4e3af8dd17dfa936c5f3 Mon Sep 17 00:00:00 2001 From: zhongjun Date: Tue, 28 Mar 2017 16:55:03 +0800 Subject: [PATCH] Add spec for share backup Currently, manila doesn't support backup and restore share features in manila itself. This spec intends to make this possible. Implement: blueprint share-backup Change-Id: I3b345f396e581575a403c728136eb644565c5928 --- doc/source/index.rst | 10 + specs/antelope/backup_share.rst | 494 ++++++++++++++++++++++++++++++++ 2 files changed, 504 insertions(+) create mode 100644 specs/antelope/backup_share.rst diff --git a/doc/source/index.rst b/doc/source/index.rst index 91b5be0..259a1b4 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -15,6 +15,16 @@ These specifications can be implemented over multiple releases. specs/release_independent/* +Antelope approved specs +======================= + +.. toctree:: + :glob: + :maxdepth: 1 + + specs/antelope/* + + Zed approved specs ================== diff --git a/specs/antelope/backup_share.rst b/specs/antelope/backup_share.rst new file mode 100644 index 0000000..5911196 --- /dev/null +++ b/specs/antelope/backup_share.rst @@ -0,0 +1,494 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +============================= +Manila backup & restore share +============================= + +Blueprint: https://blueprints.launchpad.net/manila/+spec/share-backup + +Backup and restore share is a valuable feature for most of storage +users, especially for NAS user, but currently, manila itself doesn't +support backup and restore share features, this spec proposes a backup +and restore solution based on the existing one from cinder. + +Problem description +=================== + +Today we can't use manila commands to backup shares. These shares reside on +the storage backend itself. Providing a way to backup shares directly will +allow the user to protect the shares on a backup device, separately from the +same storage backend. + +Use Cases +========= + +There are users who have many shares and would like to protect these shares. +This proposal of share backup provides a manila level of data protection. + +There are other projects in OpenStack focusing on data protection such as +Freezer, Karbor, Raksha, etc. They are all in different stages of design, +development, or adoption. The backup API in manila is not a replacement of +those projects. Instead, manila APIs can be consumed by those higher level +projects for data protection and can also be used directly by users who do +not need those higher level projects. + +When user backing up files from a share in a naive way, we lose metadata. +By having backup API in Manila, we can retain metadata (when backend is +identical) + +Also when talking about share backup, most of the users would like to +enforce backup policies on them. Such as incremental backup every two days, +create backup local or remote, only keep latest 5 copies, etc. The policies +are not part of this spec and will be considered for future. + +Proposed change +=============== + +* New API collection for backups + + In order to support backup, We will introduce the basic operations create + /update/delete/list/show/restore for backups. We also allow to reset the + backup state in some scenarios. + +* New database resource backup + + When creating a backup, user can specify share_id and availability_zone. + Information related to backup i.e. backup_id, share_id, status, AZ and + host are stored in database. + +* New backup driver + + A new type of driver called backup driver will be introduced to take + responsibility for share related operations. The backup driver will + configured at the manila data service node, and provide the basic abilities + + 1. Create a backup. + 2. Delete a backup. + 3. Restore a backup in specified share. + + Correspondingly, we will implement these with a new and simple backup driver + called 'NFSBackupDriver'. This driver is copied from cinder `[1]`_ and aims + to provide the basic backup ability. The vendor that supports nfs, must + provide space for nfs to interconnect with nfs backup drivers. The + implementation of nfs backup driver will be generic though. The backup + process for this driver would be: + + 1. Make sure share is in available state and not busy. + 2. Allow read access to share and write access to backup share. + 3. Mount the share and backend driver's share(i.e. backup share) to the + data service node. + 4. Copy data from share to backup share. + 5. Unmount the share and backup share. + 6. Deny access to share and backup share. + + Also, several new configuration options will be added to support backup + driver, at present, only one backup backend is allowed for + simplicity (we don't have to report the backup driver's capabilities + and filtering the backends). By default no backup driver will + be enabled:: + + # OPTION1: enabled backup backend + enabled_backup_backend = backup1 + [backup1] + # OPTION2: which backup driver will be used for this backend + backup_driver = manila.data.drivers.test.TestBackupDriver + +* New status for backup and share: + + 1. backup( + creating, available, + deleting, deleted, error_deleting + backup_restoring, error) + 2. share( + backing_creating, backup_restoring, + backup_restoring_error) + + During backup, share will be marked as busy and other operations on share + such as delete, soft_delete, migration, extend, shrink, ummanage, + revert_to_snapshot, crate_snapshot, create_replica etc can not be performed + unless share become available. Finally, whether or not the share is + successfully backed up, the state of the share is rolled back to the + available state. In case backup fails, share task_state will contain the + failure information. Also, share message will be recorded. + +* New clean up actions + + The backup and restore actions could break when service is down, so new + clean up action will be added to reset the status and clean temporary + files (if involved). + +* New quotas for backup + + 1. quota_backups, indicate the share backups allowed per project. + 2. quota_backup_gigabytes, indicate the total amount of storage, in + gigabytes, allowed for backups per project. + + Correspondingly, we will add new configuration options. + + +Alternatives +------------ + +We could use the third-party projects to backup the file shares. + +Data model impact +----------------- + +* Add table for backup + + +-----------------------+--------------+------+-----+---------+-------+ + | Field | Type | Null | Key | Default | Extra | + +=======================+==============+======+=====+=========+=======+ + | created_at | datetime | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | updated_at | datetime | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | deleted_at | datetime | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | deleted | tinyint(1) | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | id | varchar(36) | NO | PRI | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | share_id | varchar(36) | NO | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | user_id | varchar(255) | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | project_id | varchar(255) | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | host | varchar(255) | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | availability_zone | varchar(255) | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | display_name | varchar(255) | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | display_description | varchar(255) | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | status | varchar(255) | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + | size | int(11) | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + + +* New field in shares table + + +-----------------------+--------------+------+-----+---------+-------+ + | Field | Type | Null | Key | Default | Extra | + +=======================+==============+======+=====+=========+=======+ + | source_backup_id | varchar(255) | YES | | NULL | | + +-----------------------+--------------+------+-----+---------+-------+ + + +CLI API impact +-------------- + +Add new commands to openstackclient(OSC): + +openstack share backup create [--name ] + [--description ] + [--availabity-zone ] + + +* name: Name of backup. Default=None +* description: Backup description. Default=None. +* availability-zone: Availability-zone of backup. +* share: Name or ID of share to backup. + + +openstack share backup restore + +* backup: ID of backup to restore. + + +openstack share backup list [--share ] + +* share: Filter backups by a share name or ID. + + +openstack share backup show + +* backup: ID or name of backup + + +openstack share backup set + [--name ] + [--description ] + +* backup: ID or name of backup +* name: Name of backup. Default=None +* description: Backup description. Default=None. + + +openstack share backup delete + [--force ] + +* backup: ID or name of a backup. +* force: Allows deleting backup of a share when its status is other than + "available" or "error". Default=False. + + +REST API impact +--------------- +APIs will be experimental, until some cycles of testing, and the eventual +graduation of them. + +**Creating a share backup**:: + + POST /v2/share-backups + +Request:: + + { + "share_backup": { + "share": "77eb3421-4549-4789-ac39-0d5185d68c28", + "name": "backup_share", + "description": "This is my backup", + } + } + +Backup details ``name`` and ``description`` are optional. + +If the share is not known to manila, the API will respond with +``404 Not Found``. +If the share is not in available state or share has snapshots, +the API will respond with ``400 Invalid Share``. +If the project's share backup quota has exceeded, the API will +respond with ``413 QuotaError``. + +Response(202 Accepted):: + + { + "share_backup": { + "share_id": "77eb3421-4549-4789-ac39-0d5185d68c28", + "created_at": "2016-06-01T21:12:12.617687", + "updated_at": "2016-06-01T21:12:12.617687", + "id": "77eb3421-4549-4789-ac39-0d5185d68c29", + "project_id": "e10a683c20da41248cfd5e1ab3d88c62", + "display_name": "backup_share", + "display_description": "This is my backup", + "status": "creating" + } + } + +**Delete a backup**:: + + DELETE /v2/share-backups/{backup_id} + +If the share backup is not known to manila, the API will respond with +``404 Not Found``. +If the share backup state not in ``available`` or ``error``, the API +will respond with ``400 Invalid State``. +API will respond ``202`` if request is accepted. + +**Detailed listing of share backups**:: + + GET /v2/share-backups/detail + +Response(200 OK):: + + { + "share_backups": [ + { + "availability_zone": "az1", + "created_at": "2016-04-02T10:35:27.000000", + "id": "2ef47aee-8844-490c-804d-2a8efe561c65", + "display_name": "my_backup", + "display_description": "this is description", + "size": 1, + "status": "available", + "share_id": "e5185058-943a-4cb4-96d9-72c184c337d6", + }, + { + "availability_zone": "az2", + "created_at": "2016-05-02T10:35:27.000000", + "id": "2ef47aee-8844-423c-804d-2a8efe561623", + "display_name": "my_backup_1", + "display_description": "this is description", + "size": 2, + "status": "available", + "share_id": "e5185058-943a-4cb4-96d9-72c184c33dsd", + } + ] + } + +**Restore a share backup**:: + + POST /v2/share-backups/{backup_id}/action + +Request:: + + { + "restore": null + } + +The backup will be restored in source share (i.e. share from which backup +was created) will be used for restore. +In case, source share or share backup is not known to manila, API will +repsond with ``404 Not Found``. +In case, source share size is different than size of backup, API will +respond with ``400 Invalid Share``. +However, API will respond ``202`` if request is accepted. + + +**Update a share backup information**:: + + PUT /v2/share-backups/{backup_id} + +Request:: + + { + "share_backup": { + "display_name": "test share backup", + } + } + +If the share backup is not know to manila, the API will respond with +``404 Not Found``. +API will respond ``200`` if request is accepted. + + +Driver impact +------------- +The backup driver needs to implement these functions:: + + def backup(self, backup, share): + """Create a backup of a specified share. + + The driver should return the backup model with new created + backup content if creation is successful, manila + will update the model after this. For example:: + ``` + {"backup": + { + "name": "backup_one", + "id": "e5185058-943a-4cb4-96d9-72c184c33d12", + "created_at": "2016-04-02T10:35:27.000000", + } + } + ``` + + :param backup: backup object. + :param share: share object. + :returns: backup: backup object with backup object updated. + """ + return + + + def restore(self, backup, share): + """Restore a shared backup. + + Driver will restore the specified backup. + + :param backup: backup object. + :param share: share object. + """ + return + + def delete(self, backup): + """Delete a saved backup. + + Driver will delete the share backup. + + :param backup: backup object. + """ + return + + +Security impact +--------------- + +During backup process the data node would have access to read the share +data. If the deny access phase fails, the node will continue forever with +access to the user's data. + +Notifications impact +-------------------- + +None + +Other end user impact +--------------------- + +End user will be unavailable or restricted to other share operations +while it is backing up. Such as: extend/shrink share, replication share, +share-group operation, migration share. + +Performance Impact +------------------ + +None + +Other deployer impact +--------------------- + +The deployer will be able to backup a share. + +Developer impact +---------------- + + + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + * kpdev(kinpaa@gmail.com) + +Work Items +---------- + +* Implement core feature. +* Implement backup in NFSBackupDriver. +* Implement backup command in python-manilaclient. +* Implement tempest support. +* Implement manila-ui support. +* Support manila backup in devstack plugin. + +Future Work Items +----------------- + +* New table 'backup_metadata' where the data service or the user can store some + useful metadata such as the location of the backup. The metadata can possibly + used to restore the backup, even in the event of a catastrophic database + failure. +* Add support to create share from backup, where 'openstack share create' API + will accept --backup option and create share of the size as that + of backup. In addition, backup data will be copied to share. +* Add support to handle backup failover in case manila service is interrupted, + i.e. when service is back online, monitor the backups, check on their + completion and recover if posssible. +* Restore operation can be enhanced to consider restore to share of different + size than backup. If restore share is smaller in size, it will be expanded + before restore and if its larger in size, it will be shrinked before restore. + +Dependencies +============ + +None + +Testing +======= + +* Unit tests +* Tempest tests + +Documentation Impact +==================== + +- Docstrings +- Devref +- User guide +- Admin guide +- Release notes + +References +========== + +_`[1]`: https://specs.openstack.org/openstack/cinder-specs/specs/kilo/nfs-backup.html +