diff --git a/specs/ocata/approved/cinder_os_brick.rst b/specs/ocata/approved/cinder_os_brick.rst new file mode 100644 index 0000000..cacb5ba --- /dev/null +++ b/specs/ocata/approved/cinder_os_brick.rst @@ -0,0 +1,284 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +============================================= +Freezer Cinder Volumes backup using OS Bricks +============================================= + +* https://blueprints.launchpad.net/freezer/+spec/cinder-osbrick + +Provide efficient way to backup Cinder Volumes leveraging os_bricks + +Problem description +=================== + +Currently Freezer provides basic features to execute Cinder volumes backup. +The current approach present significant challenges, +due mainly to the difficulty of downloading Cinder Volumes without passing +through Glance. This can be an issue for time and scalability reasons, +(i.e. volumes of few hundreds GB size, potential error probability increase, +as more services are part of the process, unailability of cinder-backup) + +Use Cases +--------- + +* Users that want to backup cinder volumes. + +* Store backed up volumes in a different storage media than Swift. + This is important for disaster recovery purpose, as it should be + possible to restore the volume even if the swift or other services are + down in the original OpenStack deployment. + +* OpenStack distributions deployed without cinder-backup module. + +* Provide a more efficient way of executing incrementals backup. + +* Avoid uploading volumes image to Glance to be processed. + +* Volumes can be backed up while attached or detached (hot and cold). + +- Hot backup will be provide a crach consistent backup and the data present + in the volumes can be accesses at all times during backup +- If Cold backup is executed, the Volume is detached first, + then the backup is executed. + + +Proposed change +=============== + +Implement in the freezer-agent a new engine called cinder-osbrick. + +The new engine cinder-osbrick execute backup and restore related +operations direclty on the Volumes, without passing through Glance API. + +The freezer-agent needs to back up a single volume, all the volumes +owned by the tenant or all volumes from all tenants (admin). +Volumes backup and restore can happen in parallel (i.e. 10 Volumes +simultaneously can be backup or restore) + + +Technical details +----------------- + +Openstack provide the os_brick library to attach volumes: + +* https://github.com/openstack/os-brick + +It mainly provides the following features: + +* Volumes discovery +* Volumes attach +* Volumes removal + +Related docs: + +* http://docs.openstack.org/developer/os-brick/api/index.html +* http://docs.openstack.org/developer/os-brick/tutorial.html + +The python client module that could be used is brick-cinderclient-ex: + +* https://github.com/openstack/python-brick-cinderclient-ext + +It is preferrable to implement the Volumes related operations from cinder +in python, rather wrapping around any possible related os-brick command. + +* The freezer-scheduler and the freezer-agent needs to support, + in the json and ini config file respectively, engine specific settings. + + +Backup workflow with osbrick: +----------------------------- + +* freezer-agent workflow: + +(Common steps) + +single-vol-backup: + - Backup any available metadata of the Volume + - A Snapshot is execute on the volume (--force if volume is attached) + - Snapshot is converted to Volume, in oder to be mounted using osbrick + - The new Volume is attached using os_brick. The Volume can be attached + using iSCSI, Local or FC according the information provided by + os_brick about the volume. + - The new Volume is mounted on the node where the freezer-agent is executing + - The freezer-agent will execute a backup of the volume content, starting + from the volume mount (i.e. volume root /) + - Every single file in the volume is backed up. + - If the execution is part of an incremental backup, each file/block is + compared against the previous execution. + - Data can be storage on each supported freezer storage backend + - When finished, the new Volume is detached + - Once detached, the new volume is removed + +Backup of a Single Volume: + 1) The freezer-agent take the volume id as input param (either from ini + file or json file provided to the scheduler): + 2) single-vol-backup from Common steps + +Backup of all Volumes owned by a tenant: + 1) freezer-agent discover all the volumes owned by the tenant from Cinder API + 2) Iterate over each Volume + 3) single-vol-backup from Common steps + +Backup of all Volumes (admin): + 1) freezer-agent get the list of all Volumes available from Cinder API + 2) Iterate over each volume + 3) single-vol-backup from Common steps + +Backup of all volumes part of a Consistency Group + 1) get list of all volumes from the Consistency Group. It can be provided + as a single element id or a list of elements comma separated: + 2) freezer-agent get the list of all Volumes available from Cinder API + 3) Iterate over each volume + 4) single-vol-backup from Common steps + + +Restore workflow with osbrick +----------------------------- + +* freezer-agent workflow: + +(Common steps) + +single-vol-restore: + - Get the original Volume metadata + - Check if the volume id exists + - If the same volume id exists + + + snapshot the volume + + convert from snap to volume + + attach the volume + + mount the volume + + restore the backup data in the volume filesystem + + if meta-override option is provided, the volume metadata from backup + is applied to the current Volume meta + - If the volume id does not exist + + Create a new Volume with the same metadata from backup + + attach the volume with os-brick + + mount the volume + + restore the backup data in the volume filesystem + - unmount + - deattach the volum + - if remove_old_vol is provided, any existing volume not matching with the + new ones will be removed (Dangerous Option) + +Restote of a single volume: + 1) The freezer-agent take the volume id as input param (either from ini + file or json file provided to the scheduler): + 2) single-vol-restore from Common steps + +Restore of all volumes owned by a tenant: + 1) freezer-agent discover all the volumes owned by the tenant from Cinder API + 2) Iterate over each volume + 3) single-vol-restore from Common steps + +Restore of all volumes from all tenants (admin): + 1) freezer-agent get the list of all Volumes available from Cinder API + 2) Iterate over each volume + 3) single-vol-restore from Common steps + +Restore of all volumes part of a Consistency Group + 1) get list of all volumes from the Consistency Group. It can be provided as + a single element id or a list of elements comma separated: + 2) Iterate over each volume + 3) single-vol-restore from Common steps + +Data model impact +----------------- +* new engine in the db +* DB model for single, all tenant, tenant owned volumes and consistency groups + +New Options to be added: +------------------------ +* engine-os-brick +* recreate-vol-on-error +* meta-override +* consistency groups [id] +* all_tenants +* all_tenant_volumes +* single_volume_id +* remove_old_vol + +Alternatives +------------ + + +Impacts +------- +* freezer-agent +* freezer-api +* freezer-web-ui + +REST API impact +----------------------- + +* API needs to support this new engine + +Security impact +--------------------- + +None + +Notifications impact +--------------------------- + +TBD. + +Other end user impact +------------------------------ + +None. TBD. + +Performance Impact +------------------ + +None. + +Other deployer impact +------------------------------ + + +Developer impact +------------------------ + + +Implementation +============== + +Assignee(s) +----------------- + +Primary assignee: + +Other contributors: + daemontool + +Work Items +---------- + + +Dependencies +============ + + +Testing +======= + +TBD. + +Documentation Impact +==================== + + +* Freezer API installation doc +* Freezer agent docu +* Freezer web ui doc + +References +========== + +.. https://etherpad.openstack.org/p/freezer_cinder-os-brick +