summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJenkins <jenkins@review.openstack.org>2017-03-16 14:14:47 +0000
committerGerrit Code Review <review@openstack.org>2017-03-16 14:14:47 +0000
commit710167e17c02e00bc402d3d356c8f5dcdd643e1e (patch)
treefd3a570ac1ff9893af544cad8591e1de6c06296b
parentdc0ffee9f78111a27ce6358eef85d633dca9dced (diff)
parent810206fc84656628c26e8195632d8d2229058438 (diff)
Merge "Cinder volumes backup using os-brick from Freezer"
-rw-r--r--specs/ocata/approved/cinder_os_brick.rst284
1 files changed, 284 insertions, 0 deletions
diff --git a/specs/ocata/approved/cinder_os_brick.rst b/specs/ocata/approved/cinder_os_brick.rst
new file mode 100644
index 0000000..cacb5ba
--- /dev/null
+++ b/specs/ocata/approved/cinder_os_brick.rst
@@ -0,0 +1,284 @@
1..
2 This work is licensed under a Creative Commons Attribution 3.0 Unported
3 License.
4
5 http://creativecommons.org/licenses/by/3.0/legalcode
6
7=============================================
8Freezer Cinder Volumes backup using OS Bricks
9=============================================
10
11* https://blueprints.launchpad.net/freezer/+spec/cinder-osbrick
12
13Provide efficient way to backup Cinder Volumes leveraging os_bricks
14
15Problem description
16===================
17
18Currently Freezer provides basic features to execute Cinder volumes backup.
19The current approach present significant challenges,
20due mainly to the difficulty of downloading Cinder Volumes without passing
21through Glance. This can be an issue for time and scalability reasons,
22(i.e. volumes of few hundreds GB size, potential error probability increase,
23as more services are part of the process, unailability of cinder-backup)
24
25Use Cases
26---------
27
28* Users that want to backup cinder volumes.
29
30* Store backed up volumes in a different storage media than Swift.
31 This is important for disaster recovery purpose, as it should be
32 possible to restore the volume even if the swift or other services are
33 down in the original OpenStack deployment.
34
35* OpenStack distributions deployed without cinder-backup module.
36
37* Provide a more efficient way of executing incrementals backup.
38
39* Avoid uploading volumes image to Glance to be processed.
40
41* Volumes can be backed up while attached or detached (hot and cold).
42
43- Hot backup will be provide a crach consistent backup and the data present
44 in the volumes can be accesses at all times during backup
45- If Cold backup is executed, the Volume is detached first,
46 then the backup is executed.
47
48
49Proposed change
50===============
51
52Implement in the freezer-agent a new engine called cinder-osbrick.
53
54The new engine cinder-osbrick execute backup and restore related
55operations direclty on the Volumes, without passing through Glance API.
56
57The freezer-agent needs to back up a single volume, all the volumes
58owned by the tenant or all volumes from all tenants (admin).
59Volumes backup and restore can happen in parallel (i.e. 10 Volumes
60simultaneously can be backup or restore)
61
62
63Technical details
64-----------------
65
66Openstack provide the os_brick library to attach volumes:
67
68* https://github.com/openstack/os-brick
69
70It mainly provides the following features:
71
72* Volumes discovery
73* Volumes attach
74* Volumes removal
75
76Related docs:
77
78* http://docs.openstack.org/developer/os-brick/api/index.html
79* http://docs.openstack.org/developer/os-brick/tutorial.html
80
81The python client module that could be used is brick-cinderclient-ex:
82
83* https://github.com/openstack/python-brick-cinderclient-ext
84
85It is preferrable to implement the Volumes related operations from cinder
86in python, rather wrapping around any possible related os-brick command.
87
88* The freezer-scheduler and the freezer-agent needs to support,
89 in the json and ini config file respectively, engine specific settings.
90
91
92Backup workflow with osbrick:
93-----------------------------
94
95* freezer-agent workflow:
96
97(Common steps)
98
99single-vol-backup:
100 - Backup any available metadata of the Volume
101 - A Snapshot is execute on the volume (--force if volume is attached)
102 - Snapshot is converted to Volume, in oder to be mounted using osbrick
103 - The new Volume is attached using os_brick. The Volume can be attached
104 using iSCSI, Local or FC according the information provided by
105 os_brick about the volume.
106 - The new Volume is mounted on the node where the freezer-agent is executing
107 - The freezer-agent will execute a backup of the volume content, starting
108 from the volume mount (i.e. volume root /)
109 - Every single file in the volume is backed up.
110 - If the execution is part of an incremental backup, each file/block is
111 compared against the previous execution.
112 - Data can be storage on each supported freezer storage backend
113 - When finished, the new Volume is detached
114 - Once detached, the new volume is removed
115
116Backup of a Single Volume:
117 1) The freezer-agent take the volume id as input param (either from ini
118 file or json file provided to the scheduler):
119 2) single-vol-backup from Common steps
120
121Backup of all Volumes owned by a tenant:
122 1) freezer-agent discover all the volumes owned by the tenant from Cinder API
123 2) Iterate over each Volume
124 3) single-vol-backup from Common steps
125
126Backup of all Volumes (admin):
127 1) freezer-agent get the list of all Volumes available from Cinder API
128 2) Iterate over each volume
129 3) single-vol-backup from Common steps
130
131Backup of all volumes part of a Consistency Group
132 1) get list of all volumes from the Consistency Group. It can be provided
133 as a single element id or a list of elements comma separated:
134 2) freezer-agent get the list of all Volumes available from Cinder API
135 3) Iterate over each volume
136 4) single-vol-backup from Common steps
137
138
139Restore workflow with osbrick
140-----------------------------
141
142* freezer-agent workflow:
143
144(Common steps)
145
146single-vol-restore:
147 - Get the original Volume metadata
148 - Check if the volume id exists
149 - If the same volume id exists
150
151 + snapshot the volume
152 + convert from snap to volume
153 + attach the volume
154 + mount the volume
155 + restore the backup data in the volume filesystem
156 + if meta-override option is provided, the volume metadata from backup
157 is applied to the current Volume meta
158 - If the volume id does not exist
159 + Create a new Volume with the same metadata from backup
160 + attach the volume with os-brick
161 + mount the volume
162 + restore the backup data in the volume filesystem
163 - unmount
164 - deattach the volum
165 - if remove_old_vol is provided, any existing volume not matching with the
166 new ones will be removed (Dangerous Option)
167
168Restote of a single volume:
169 1) The freezer-agent take the volume id as input param (either from ini
170 file or json file provided to the scheduler):
171 2) single-vol-restore from Common steps
172
173Restore of all volumes owned by a tenant:
174 1) freezer-agent discover all the volumes owned by the tenant from Cinder API
175 2) Iterate over each volume
176 3) single-vol-restore from Common steps
177
178Restore of all volumes from all tenants (admin):
179 1) freezer-agent get the list of all Volumes available from Cinder API
180 2) Iterate over each volume
181 3) single-vol-restore from Common steps
182
183Restore of all volumes part of a Consistency Group
184 1) get list of all volumes from the Consistency Group. It can be provided as
185 a single element id or a list of elements comma separated:
186 2) Iterate over each volume
187 3) single-vol-restore from Common steps
188
189Data model impact
190-----------------
191* new engine in the db
192* DB model for single, all tenant, tenant owned volumes and consistency groups
193
194New Options to be added:
195------------------------
196* engine-os-brick
197* recreate-vol-on-error
198* meta-override
199* consistency groups [id]
200* all_tenants
201* all_tenant_volumes
202* single_volume_id
203* remove_old_vol
204
205Alternatives
206------------
207
208
209Impacts
210-------
211* freezer-agent
212* freezer-api
213* freezer-web-ui
214
215REST API impact
216-----------------------
217
218* API needs to support this new engine
219
220Security impact
221---------------------
222
223None
224
225Notifications impact
226---------------------------
227
228TBD.
229
230Other end user impact
231------------------------------
232
233None. TBD.
234
235Performance Impact
236------------------
237
238None.
239
240Other deployer impact
241------------------------------
242
243
244Developer impact
245------------------------
246
247
248Implementation
249==============
250
251Assignee(s)
252-----------------
253
254Primary assignee:
255
256Other contributors:
257 daemontool
258
259Work Items
260----------
261
262
263Dependencies
264============
265
266
267Testing
268=======
269
270TBD.
271
272Documentation Impact
273====================
274
275
276* Freezer API installation doc
277* Freezer agent docu
278* Freezer web ui doc
279
280References
281==========
282
283.. https://etherpad.openstack.org/p/freezer_cinder-os-brick
284