Merge "Remove bogus responses for dynamic large objects"
This commit is contained in:
commit
60e524d82e
|
@ -43,11 +43,10 @@ format="SVG" scale="60"/>
|
||||||
interface. There are also language-specific APIs that utilize
|
interface. There are also language-specific APIs that utilize
|
||||||
the RESTful API but make it much easier for developers to
|
the RESTful API but make it much easier for developers to
|
||||||
integrate into their applications.</para>
|
integrate into their applications.</para>
|
||||||
<para>For more details on the OpenStack Object Storage service,
|
<para>For more details about the OpenStack Object Storage service,
|
||||||
please refer to <link
|
see <link
|
||||||
xlink:href="http://docs.openstack.org/developer/swift/"
|
xlink:href="http://docs.openstack.org/developer/swift/"
|
||||||
>http://docs.openstack.org/developer/swift/</link>
|
>http://docs.openstack.org/developer/swift/</link>.</para>
|
||||||
</para>
|
|
||||||
<para>We welcome feedback, comments, and bug reports at <link
|
<para>We welcome feedback, comments, and bug reports at <link
|
||||||
xlink:href="http://bugs.launchpad.net/swift"
|
xlink:href="http://bugs.launchpad.net/swift"
|
||||||
>http://bugs.launchpad.net/swift</link>.</para>
|
>http://bugs.launchpad.net/swift</link>.</para>
|
||||||
|
|
|
@ -1,4 +1,4 @@
|
||||||
PUT /<api version>/<account>/<container>/<object> HTTP/1.1
|
PUT /{api_version}/{account}/{container}/{object} HTTP/1.1
|
||||||
Host: storage.clouddrive.com
|
Host: storage.clouddrive.com
|
||||||
X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
|
X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
|
||||||
ETag: 8a964ee2a5e88be344f36c22562a6486
|
ETag: 8a964ee2a5e88be344f36c22562a6486
|
||||||
|
|
|
@ -1,4 +1,4 @@
|
||||||
PUT /<api version>/<account>/<container>/<object> HTTP/1.1
|
PUT /{api_version}/{account}/{container}/{object} HTTP/1.1
|
||||||
Host: storage.clouddrive.com
|
Host: storage.clouddrive.com
|
||||||
X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
|
X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
|
||||||
ETag: 8a964ee2a5e88be344f36c22562a6486
|
ETag: 8a964ee2a5e88be344f36c22562a6486
|
||||||
|
|
|
@ -1,17 +1,17 @@
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
"path": "mycontainer/objseg1",
|
"path":"mycontainer/objseg1",
|
||||||
"etag": "0228c7926b8b642dfb29554cd1f00963",
|
"etag":"0228c7926b8b642dfb29554cd1f00963",
|
||||||
"size_bytes": 1468006
|
"size_bytes":1468006
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"path": "mycontainer/pseudodir/seg-obj2",
|
"path":"mycontainer/pseudodir/seg-obj2",
|
||||||
"etag": "5bfc9ea51a00b790717eeb934fb77b9b",
|
"etag":"5bfc9ea51a00b790717eeb934fb77b9b",
|
||||||
"size_bytes": 1572864
|
"size_bytes":1572864
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"path": "other-container/seg-final",
|
"path":"other-container/seg-final",
|
||||||
"etag": "b9c3da507d2557c1ddc51f27c54bae51",
|
"etag":"b9c3da507d2557c1ddc51f27c54bae51",
|
||||||
"size_bytes": 256
|
"size_bytes":256
|
||||||
}
|
}
|
||||||
]
|
]
|
|
@ -1,6 +1,6 @@
|
||||||
PUT /<api version>/<account>/<container>/<object> HTTP/1.1
|
PUT /{api_version}/{account}/{container}/{object} HTTP/1.1
|
||||||
Host: storage.clouddrive.com
|
Host: storage.clouddrive.com
|
||||||
X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
|
X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
|
||||||
Content-Length: 0
|
Content-Length: 0
|
||||||
X-Object-Meta-PIN: 1234
|
X-Object-Meta-PIN: 1234
|
||||||
X-Object-Manifest: container/object/segments
|
X-Object-Manifest: {container}/{prefix}
|
|
@ -26,8 +26,7 @@
|
||||||
</listitem>
|
</listitem>
|
||||||
<listitem>
|
<listitem>
|
||||||
<para><literal>X-Container-Meta-Quota-Count</literal>. The
|
<para><literal>X-Container-Meta-Quota-Count</literal>. The
|
||||||
number of objects that can be stored in a container.
|
number of objects that can be stored in a container.</para>
|
||||||
</para>
|
|
||||||
</listitem>
|
</listitem>
|
||||||
</itemizedlist>
|
</itemizedlist>
|
||||||
<para>When you exceed a container quota, subsequent requests to
|
<para>When you exceed a container quota, subsequent requests to
|
||||||
|
|
|
@ -40,8 +40,7 @@
|
||||||
</listitem>
|
</listitem>
|
||||||
<listitem>
|
<listitem>
|
||||||
<para>The destination container can be a source container
|
<para>The destination container can be a source container
|
||||||
for synchronization for another destination container.
|
for synchronization for another destination container.</para>
|
||||||
</para>
|
|
||||||
</listitem>
|
</listitem>
|
||||||
<listitem>
|
<listitem>
|
||||||
<para>The destination container can be the original source
|
<para>The destination container can be the original source
|
||||||
|
|
|
@ -18,116 +18,274 @@
|
||||||
xml:id="large-object-creation">
|
xml:id="large-object-creation">
|
||||||
<title>Large objects</title>
|
<title>Large objects</title>
|
||||||
<para>By default, the content of an object cannot be greater than
|
<para>By default, the content of an object cannot be greater than
|
||||||
5 GB. However, you can use segment objects and manifest
|
5 GB. However, you can use a number of smaller objects to
|
||||||
objects to store more content.</para>
|
construct a large object. The large object is comprised of two
|
||||||
<section xml:id="segment-objects">
|
types of objects:</para>
|
||||||
<title>Segment objects</title>
|
<itemizedlist>
|
||||||
<para>You can divide your content into segments, and upload
|
<listitem>
|
||||||
each segment into its own segment object. Segment objects
|
<para><emphasis role="bold">Segment objects</emphasis>
|
||||||
do not have any special features. You create, update,
|
store the object content. You can divide your content
|
||||||
download, and delete segment objects just as you would
|
into segments, and upload each segment into its own
|
||||||
normal objects.</para>
|
segment object. Segment objects do not have any
|
||||||
</section>
|
special features. You create, update, download, and
|
||||||
<section xml:id="manifest-objects">
|
delete segment objects just as you would normal
|
||||||
<title>Manifest objects</title>
|
objects.</para>
|
||||||
<para>A manifest object points to segment objects. When you
|
</listitem>
|
||||||
download a manifest object, Object Storage concatenates
|
<listitem>
|
||||||
the contents of the segment objects and returns this in
|
<para>A <emphasis role="bold">manifest object</emphasis>
|
||||||
the response body of the request.</para>
|
links the segment objects into one logical large
|
||||||
<para>This behavior extends to the response headers returned
|
object. When you download a manifest object, Object
|
||||||
by &GET; and &HEAD; requests. The
|
Storage concatenates and returns the contents of the
|
||||||
<literal>Content-Length</literal> response header
|
segment objects in the response body of the request.
|
||||||
value is the total size of all segment objects. Object
|
This behavior extends to the response headers returned
|
||||||
Storage calculates the <literal>ETag</literal> response
|
by &GET; and &HEAD; requests. The
|
||||||
header value by taking the <literal>ETag</literal> value
|
<literal>Content-Length</literal> response header
|
||||||
of each segment, concatenating them together, and
|
value is the total size of all segment objects. Object
|
||||||
returning the MD5 checksum of the result.</para>
|
Storage calculates the <literal>ETag</literal>
|
||||||
<note>
|
response header value by taking the
|
||||||
<para>If you make a © request by using a manifest
|
<literal>ETag</literal> value of each segment,
|
||||||
object as the source, the new object is a normal, and
|
concatenating them together, and returning the MD5
|
||||||
not a segment, object. If the total size of the source
|
checksum of the result. The manifest object types
|
||||||
segment objects exceeds 5 GB, the © request
|
are:</para>
|
||||||
fails. However, you can make a duplicate of the
|
<variablelist wordsize="10">
|
||||||
manifest object and this new object can be larger than
|
<varlistentry>
|
||||||
5 GB.</para>
|
<term><emphasis role="bold">Static large
|
||||||
</note>
|
objects</emphasis></term>
|
||||||
<para>The manifest object types are:</para>
|
<listitem>
|
||||||
|
<para>The manifest object content is an
|
||||||
|
ordered list of the names of the segment
|
||||||
|
objects in JSON format. See <xref
|
||||||
|
linkend="static-large-objects"
|
||||||
|
/>.</para>
|
||||||
|
</listitem>
|
||||||
|
</varlistentry>
|
||||||
|
<varlistentry>
|
||||||
|
<term><emphasis role="bold">Dynamic large
|
||||||
|
objects</emphasis></term>
|
||||||
|
<listitem>
|
||||||
|
<para>The manifest object has no content but
|
||||||
|
it has a
|
||||||
|
<literal>X-Object-Manifest</literal>
|
||||||
|
metadata header. The value of this header
|
||||||
|
is
|
||||||
|
<literal>{container}/{prefix}</literal>,
|
||||||
|
where <literal>{container}</literal> is
|
||||||
|
the name of the container where the
|
||||||
|
segment objects are stored, and
|
||||||
|
<literal>{prefix}</literal> is a
|
||||||
|
string that all segment objects have in
|
||||||
|
common. See <xref
|
||||||
|
linkend="dynamic-large-object-creation"
|
||||||
|
/>.</para>
|
||||||
|
</listitem>
|
||||||
|
</varlistentry>
|
||||||
|
</variablelist>
|
||||||
|
</listitem>
|
||||||
|
</itemizedlist>
|
||||||
|
<note>
|
||||||
|
<para>If you make a © request by using a manifest object
|
||||||
|
as the source, the new object is a normal, and not a
|
||||||
|
segment, object. If the total size of the source segment
|
||||||
|
objects exceeds 5 GB, the © request fails.
|
||||||
|
However, you can make a duplicate of the manifest object
|
||||||
|
and this new object can be larger than 5 GB.</para>
|
||||||
|
</note>
|
||||||
|
<section xml:id="static-large-objects">
|
||||||
|
<title>Static large objects</title>
|
||||||
|
<para>To create a static large object, divide your content
|
||||||
|
into pieces and create (upload) a segment object to
|
||||||
|
contain each piece.</para>
|
||||||
|
<para>You must record the <literal>ETag</literal> response
|
||||||
|
header that the &PUT; operation returns. Alternatively,
|
||||||
|
you can calculate the MD5 checksum of the segment prior to
|
||||||
|
uploading and include this in the <literal>ETag</literal>
|
||||||
|
request header. This ensures that the upload cannot
|
||||||
|
corrupt your data.</para>
|
||||||
|
<para>List the name of each segment object along with its size
|
||||||
|
and MD5 checksum in order.</para>
|
||||||
|
<para>Create a manifest object. Include the
|
||||||
|
<parameter>?multipart-manifest=put</parameter> query
|
||||||
|
string at the end of the manifest object name to indicate
|
||||||
|
that this is a manifest object.</para>
|
||||||
|
<para>The body of the &PUT; request on the manifest object
|
||||||
|
comprises a json list, where each element contains the
|
||||||
|
following attributes:</para>
|
||||||
<itemizedlist>
|
<itemizedlist>
|
||||||
<listitem>
|
<listitem>
|
||||||
<para><emphasis role="bold">Static large
|
<para><code>path</code>. The container and object name
|
||||||
objects</emphasis>. The manifest object
|
in the format:
|
||||||
content is an ordered list of the names of the
|
<code>{container-name}/{object-name}</code></para>
|
||||||
segment objects in JSON format.</para>
|
|
||||||
</listitem>
|
</listitem>
|
||||||
<listitem>
|
<listitem>
|
||||||
<para><emphasis role="bold">Dynamic large
|
<para><code>etag</code>. The MD5 checksum of the
|
||||||
objects</emphasis>. The manifest object has no
|
content of the segment object. This value must
|
||||||
content.</para>
|
match the <literal>ETag</literal> of that
|
||||||
<para>However, it has
|
object.</para>
|
||||||
<literal>X-Object-Manifest</literal> metadata
|
</listitem>
|
||||||
header. The value of this header is
|
<listitem>
|
||||||
<literal><container>/<prefix></literal>,
|
<para><code>size_bytes</code>. The size of the segment
|
||||||
where <literal><container></literal> is the
|
object. This value must match the
|
||||||
name of the container where the segment objects
|
<literal>Content-Length</literal> of that
|
||||||
are stored, and <literal><prefix></literal> is
|
object.</para>
|
||||||
a string that all segment objects have in
|
|
||||||
common.</para>
|
|
||||||
</listitem>
|
</listitem>
|
||||||
</itemizedlist>
|
</itemizedlist>
|
||||||
<para>While both types of manifest objects have similar
|
<example>
|
||||||
behavior, the following table describes their
|
<title>Static large object manifest list</title>
|
||||||
differences:</para>
|
<para>This example shows three segment objects. You can
|
||||||
|
use several containers and the object names do not
|
||||||
|
have to conform to a specific pattern, in contrast to
|
||||||
|
dynamic large objects.</para>
|
||||||
|
<literallayout class="monospaced"><xi:include href="samples/slo-manifest-example.txt" parse="text"/></literallayout>
|
||||||
|
</example>
|
||||||
|
<para>The <literal>Content-Length</literal> request header
|
||||||
|
must contain the length of the json content—not the
|
||||||
|
length of the segment objects. However, after the &PUT;
|
||||||
|
operation completes, the <literal>Content-Length</literal>
|
||||||
|
metadata is set to the total length of all the object
|
||||||
|
segments. A similar situation applies to the
|
||||||
|
<literal>ETag</literal>. If used in the &PUT;
|
||||||
|
operation, it must contain the MD5 checksum of the json
|
||||||
|
content. The <literal>ETag</literal> metadata value is
|
||||||
|
then set to be the MD5 checksum of the concatenated
|
||||||
|
<literal>ETag</literal> values of the object segments.
|
||||||
|
You can also set the <literal>Content-Type</literal>
|
||||||
|
request header and custom object metadata.</para>
|
||||||
|
<para>When the &PUT; operation sees the
|
||||||
|
<parameter>?multipart-manifest=put</parameter> query
|
||||||
|
parameter, it reads the request body and verifies that
|
||||||
|
each segment object exists and that the sizes and ETags
|
||||||
|
match. If there is a mismatch, the &PUT;operation
|
||||||
|
fails.</para>
|
||||||
|
<para>If everything matches, the manifest object is created.
|
||||||
|
The <literal>X-Static-Large-Object</literal> metadata is
|
||||||
|
set to <literal>true</literal> indicating that this is a
|
||||||
|
static object manifest.</para>
|
||||||
|
<para>Normally when you perform a &GET; operation on the
|
||||||
|
manifest object, the response body contains the
|
||||||
|
concatenated content of the segment objects. To download
|
||||||
|
the manifest list, use the
|
||||||
|
<parameter>?multipart-manifest=get</parameter> query
|
||||||
|
parameter. The resulting list is not formatted the same as
|
||||||
|
the manifest you originally used in the &PUT;
|
||||||
|
operation.</para>
|
||||||
|
<para>If you use the &DELETE; operation on a manifest object,
|
||||||
|
the manifest object is deleted. The segment objects are
|
||||||
|
not affected. However, if you add the
|
||||||
|
<parameter>?multipart-manifest=delete</parameter>
|
||||||
|
query parameter, the segment objects are deleted and if
|
||||||
|
all are successfully deleted, the manifest object is also
|
||||||
|
deleted.</para>
|
||||||
|
<para>To change the manifest, use a &PUT; operation with the
|
||||||
|
<parameter>?multipart-manifest=put</parameter> query
|
||||||
|
parameter. This request creates a manifest object. You can
|
||||||
|
also update the object metadata in the usual way.</para>
|
||||||
|
</section>
|
||||||
|
<section xml:id="dynamic-large-object-creation">
|
||||||
|
<title>Dynamic large objects</title>
|
||||||
|
<para>You must segment objects that are larger than 5 GB
|
||||||
|
before you can upload them. You then upload the segment
|
||||||
|
objects like you would any other object and create a
|
||||||
|
dynamic large manifest object. The manifest object tells
|
||||||
|
Object Storage how to find the segment objects that
|
||||||
|
comprise the large object. The segments remain
|
||||||
|
individually addressable, but retrieving the manifest
|
||||||
|
object streams all the segments concatenated. There is no
|
||||||
|
limit to the number of segments that can be a part of a
|
||||||
|
single large object.</para>
|
||||||
|
<para>To ensure the download works correctly, you must upload
|
||||||
|
all the object segments to the same container and ensure
|
||||||
|
that each object name is prefixed in such a way that it
|
||||||
|
sorts in the order in which it should be concatenated. You
|
||||||
|
also create and upload a manifest file. The manifest file
|
||||||
|
is a zero-byte file with the extra
|
||||||
|
<literal>X-Object-Manifest</literal>
|
||||||
|
<code>{container}/{prefix}</code> header, where
|
||||||
|
<code>{container}</code> is the container the object
|
||||||
|
segments are in and <code>{prefix}</code> is the common
|
||||||
|
prefix for all the segments. You must UTF-8-encode and
|
||||||
|
then URL-encode the container and common prefix in the
|
||||||
|
<literal>X-Object-Manifest</literal> header.</para>
|
||||||
|
<para>It is best to upload all the segments first and then
|
||||||
|
create or update the manifest. With this method, the full
|
||||||
|
object is not available for downloading until the upload
|
||||||
|
is complete. Also, you can upload a new set of segments to
|
||||||
|
a second location and update the manifest to point to this
|
||||||
|
new location. During the upload of the new segments, the
|
||||||
|
original manifest is still available to download the first
|
||||||
|
set of segments.</para>
|
||||||
|
<example>
|
||||||
|
<title>Upload segment of large object request:
|
||||||
|
HTTP</title>
|
||||||
|
<literallayout class="monospaced"><xi:include href="samples/large-object-upload-segment-req.txt" parse="text"/></literallayout>
|
||||||
|
</example>
|
||||||
|
<para>No response body is returned. A status code of
|
||||||
|
<returnvalue>2<replaceable>nn</replaceable></returnvalue>
|
||||||
|
(between 200 and 299, inclusive) indicates a successful
|
||||||
|
write; status <errorcode>411</errorcode>
|
||||||
|
<errortext>Length Required</errortext> denotes a missing
|
||||||
|
<literal>Content-Length</literal> or
|
||||||
|
<literal>Content-Type</literal> header in the request.
|
||||||
|
If the MD5 checksum of the data written to the storage
|
||||||
|
system does NOT match the (optionally) supplied ETag
|
||||||
|
value, a <errorcode>422</errorcode>
|
||||||
|
<errortext>Unprocessable Entity</errortext> response is
|
||||||
|
returned.</para>
|
||||||
|
<para>You can continue uploading segments like this example
|
||||||
|
shows, prior to uploading the manifest.</para>
|
||||||
|
<example>
|
||||||
|
<title>Upload next segment of large object request:
|
||||||
|
HTTP</title>
|
||||||
|
<literallayout class="monospaced"><xi:include href="samples/large-object-upload-next-segment-req.txt" parse="text"/></literallayout>
|
||||||
|
</example>
|
||||||
|
<para>Next, upload the manifest you created that indicates the
|
||||||
|
container the object segments reside within. Note that
|
||||||
|
uploading additional segments after the manifest is
|
||||||
|
created causes the concatenated object to be that much
|
||||||
|
larger but you do not need to recreate the manifest file
|
||||||
|
for subsequent additional segments.</para>
|
||||||
|
<example>
|
||||||
|
<title>Upload manifest request: HTTP</title>
|
||||||
|
<literallayout class="monospaced"><xi:include href="samples/upload-manifest-req.txt" parse="text"/></literallayout>
|
||||||
|
</example>
|
||||||
|
<example>
|
||||||
|
<title>Upload manifest response: HTTP</title>
|
||||||
|
<literallayout class="monospaced"><xi:include href="samples/upload-manifest-resp.txt" parse="text"/></literallayout>
|
||||||
|
</example>
|
||||||
|
<para>The <literal>Content-Type</literal> in the response for
|
||||||
|
a &GET; or &HEAD; on the manifest is the same as the
|
||||||
|
<literal>Content-Type</literal> set during the &PUT;
|
||||||
|
request that created the manifest. You can easily change
|
||||||
|
the <literal>Content-Type</literal> by reissuing the &PUT;
|
||||||
|
request.</para>
|
||||||
|
</section>
|
||||||
|
<section xml:id="comparison_dynamic_static_objects">
|
||||||
|
<title>Comparison of static and dynamic large objects</title>
|
||||||
|
<para>While static and dynamic objects have similar behavior,
|
||||||
|
this table describes their differences:</para>
|
||||||
<table rules="all">
|
<table rules="all">
|
||||||
<caption>Static and dynamic large objects</caption>
|
<caption>Static and dynamic large objects</caption>
|
||||||
|
<col width="20%"/>
|
||||||
|
<col width="40%"/>
|
||||||
|
<col width="40%"/>
|
||||||
<thead>
|
<thead>
|
||||||
<tr>
|
<tr>
|
||||||
<th>Object type</th>
|
<td/>
|
||||||
<th>End-to-end integrity</th>
|
<th>Static large object</th>
|
||||||
<th>Upload order</th>
|
<th>Dynamic large object</th>
|
||||||
<th>Removal or addition of segment objects</th>
|
|
||||||
<th>Segment object size and number</th>
|
|
||||||
<th>Segment object container name</th>
|
|
||||||
<th>Manifest Object Metadata</th>
|
|
||||||
<th>Making a copy of the manifest object</th>
|
|
||||||
</tr>
|
</tr>
|
||||||
</thead>
|
</thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
<tr>
|
<tr>
|
||||||
<td><para>Static large object</para></td>
|
<th>End-to-end integrity</th>
|
||||||
<td><para>Assured. The list of segments includes
|
<td><para>Assured. The list of segments includes
|
||||||
the MD5 checksum (<literal>ETag</literal>)
|
the MD5 checksum (<literal>ETag</literal>)
|
||||||
of each segment. You cannot upload the
|
of each segment. You cannot upload the
|
||||||
manifest object if the
|
manifest object if the
|
||||||
<literal>ETag</literal> in the list
|
<literal>ETag</literal> in the list
|
||||||
differs from the segment object already
|
differs from the uploaded segment object.
|
||||||
uploaded. If a segment is somehow lost, an
|
If a segment is somehow lost, an attempt
|
||||||
attempt to download the manifest object
|
to download the manifest object results in
|
||||||
results in an error.</para></td>
|
an error.</para></td>
|
||||||
<td><para>The segment objects must be uploaded
|
|
||||||
before the manifest object.</para></td>
|
|
||||||
<td><para>You cannot add or remove segment objects
|
|
||||||
from the manifest. However, you can create
|
|
||||||
a completely new manifest object of the
|
|
||||||
same name with a different manifest
|
|
||||||
list.</para></td>
|
|
||||||
<td><para>Segment objects must be at least 1 MB in
|
|
||||||
size (by default). The final segment
|
|
||||||
object can be any size. At most 1000
|
|
||||||
segments are supported (by
|
|
||||||
default).</para></td>
|
|
||||||
<td><para>The manifest list includes the container
|
|
||||||
name of each object. Segment objects can
|
|
||||||
be in different containers.</para></td>
|
|
||||||
<td><para>The object has
|
|
||||||
<literal>X-Static-Large-Object</literal>
|
|
||||||
set to <literal>true</literal>. You do not
|
|
||||||
set this metadata directly. Instead the
|
|
||||||
system sets it when you &PUT; a static
|
|
||||||
manifest object.</para></td>
|
|
||||||
<td><para/></td>
|
|
||||||
</tr>
|
|
||||||
<tr>
|
|
||||||
<td><para>Dynamic large object</para></td>
|
|
||||||
<td><para>Not guaranteed. The eventual consistency
|
<td><para>Not guaranteed. The eventual consistency
|
||||||
model means that although you have
|
model means that although you have
|
||||||
uploaded a segment object, it might not
|
uploaded a segment object, it might not
|
||||||
|
@ -136,28 +294,77 @@
|
||||||
it appears in the container, it does not
|
it appears in the container, it does not
|
||||||
form part of the content returned in
|
form part of the content returned in
|
||||||
response to a &GET; request.</para></td>
|
response to a &GET; request.</para></td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<th>Upload order</th>
|
||||||
|
<td><para>You must upload the segment objects
|
||||||
|
before you upload the manifest
|
||||||
|
object.</para></td>
|
||||||
<td><para>You can upload manifest and segment
|
<td><para>You can upload manifest and segment
|
||||||
objects in any order. You are recommended
|
objects in any order. You are recommended
|
||||||
to upload the manifest object after the
|
to upload the manifest object after the
|
||||||
segments in case a premature download of
|
segments in case a premature download of
|
||||||
the manifest occurs. However, this is not
|
the manifest occurs. However, this is not
|
||||||
enforced.</para></td>
|
enforced.</para></td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<th>Removal or addition of segment objects</th>
|
||||||
|
<td><para>You cannot add or remove segment objects
|
||||||
|
from the manifest. However, you can create
|
||||||
|
a completely new manifest object of the
|
||||||
|
same name with a different manifest
|
||||||
|
list.</para></td>
|
||||||
<td><para>You can upload new segment objects or
|
<td><para>You can upload new segment objects or
|
||||||
remove existing segments. The names must
|
remove existing segments. The names must
|
||||||
simply match the
|
simply match the
|
||||||
<literal><prefix></literal>
|
<literal>{prefix}</literal> supplied
|
||||||
supplied in
|
in
|
||||||
<literal>X-Object-Manifest</literal>.</para></td>
|
<literal>X-Object-Manifest</literal>.</para></td>
|
||||||
<td><para>Segment objects can be of any
|
</tr>
|
||||||
size.</para></td>
|
<tr>
|
||||||
|
<th>Segment object size and number</th>
|
||||||
|
<td><para>Segment objects must be at least
|
||||||
|
1 MB in size (by default). The final
|
||||||
|
segment object can be any size. At most,
|
||||||
|
1000 segments are supported (by
|
||||||
|
default).</para></td>
|
||||||
|
<td><para>Segment objects can be any
|
||||||
|
size.</para></td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<th>Segment object container name</th>
|
||||||
|
<td><para>The manifest list includes the container
|
||||||
|
name of each object. Segment objects can
|
||||||
|
be in different containers.</para></td>
|
||||||
<td><para>All segment objects must be in the same
|
<td><para>All segment objects must be in the same
|
||||||
container</para></td>
|
container.</para></td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<th>Manifest object metadata</th>
|
||||||
|
<td><para>The object has
|
||||||
|
<literal>X-Static-Large-Object</literal>
|
||||||
|
set to <literal>true</literal>. You do not
|
||||||
|
set this metadata directly. Instead the
|
||||||
|
system sets it when you &PUT; a static
|
||||||
|
manifest object.</para></td>
|
||||||
<td><para>The <literal>X-Object-Manifest</literal>
|
<td><para>The <literal>X-Object-Manifest</literal>
|
||||||
value is the
|
value is the
|
||||||
<literal><container>/<prefix></literal>
|
<literal>{container}/{prefix}</literal>,
|
||||||
indicating where the segment objects are
|
which indicates where the segment objects
|
||||||
located. You supply this request header in
|
are located. You supply this request
|
||||||
the &PUT; operation</para></td>
|
header in the &PUT; operation.</para></td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<th>Copying the manifest object</th>
|
||||||
|
<td><para>Include the
|
||||||
|
<parameter>?multipart-manifest=get</parameter>
|
||||||
|
query string in the © request. The
|
||||||
|
new object contains the same manifest as
|
||||||
|
the original. The segment objects are not
|
||||||
|
copied. Instead, both the original and new
|
||||||
|
manifest objects share the same set of
|
||||||
|
segment objects.</para>
|
||||||
|
</td>
|
||||||
<td><para>The © operation does not create a
|
<td><para>The © operation does not create a
|
||||||
manifest object. To duplicate a manifest
|
manifest object. To duplicate a manifest
|
||||||
object, use the &GET; operation to read
|
object, use the &GET; operation to read
|
||||||
|
@ -172,198 +379,5 @@
|
||||||
</tr>
|
</tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table>
|
</table>
|
||||||
<section xml:id="dynamic-large-object-creation">
|
|
||||||
<title>Dynamic large objects</title>
|
|
||||||
<para>You must segment objects that are larger than 5 GB
|
|
||||||
before you can upload them. You then upload the
|
|
||||||
segment objects like you would any other object and
|
|
||||||
create a dynamic large manifest object. The manifest
|
|
||||||
object tells Object Storage how to find the segment
|
|
||||||
objects that comprise the large object. The segments
|
|
||||||
remain individually addressable, but retrieving the
|
|
||||||
manifest object streams all the segments concatenated.
|
|
||||||
There is no limit to the number of segments that can
|
|
||||||
be a part of a single large object.</para>
|
|
||||||
<para>To ensure the download works correctly, you must
|
|
||||||
upload all the object segments to the same container
|
|
||||||
and ensure that each object name is prefixed in such a
|
|
||||||
way that it sorts in the order in which it should be
|
|
||||||
concatenated. You also create and upload a manifest
|
|
||||||
file. The manifest file is a zero-byte file with the
|
|
||||||
extra <literal>X-Object-Manifest</literal>
|
|
||||||
<code><container>/<prefix></code> header,
|
|
||||||
where <code><container></code> is the container
|
|
||||||
the object segments are in and
|
|
||||||
<code><prefix></code> is the common prefix
|
|
||||||
for all the segments. You must UTF-8-encode and then
|
|
||||||
URL-encode the container and common prefix in the
|
|
||||||
<literal>X-Object-Manifest</literal>
|
|
||||||
header.</para>
|
|
||||||
<para>It is best to upload all the segments first and then
|
|
||||||
create or update the manifest. With this method, the
|
|
||||||
full object is not available for downloading until the
|
|
||||||
upload is complete. Also, you can upload a new set of
|
|
||||||
segments to a second location and then update the
|
|
||||||
manifest to point to this new location. During the
|
|
||||||
upload of the new segments, the original manifest is
|
|
||||||
still available to download the first set of
|
|
||||||
segments.</para>
|
|
||||||
<example>
|
|
||||||
<title>Upload segment of large object request:
|
|
||||||
HTTP</title>
|
|
||||||
<literallayout class="monospaced"><xi:include href="samples/large-object-upload-segment-req.txt" parse="text"/></literallayout>
|
|
||||||
</example>
|
|
||||||
<example>
|
|
||||||
<title>Upload segment of large object response:
|
|
||||||
HTTP</title>
|
|
||||||
<literallayout class="monospaced">s</literallayout>
|
|
||||||
</example>
|
|
||||||
<para>No response body is returned. A status code of
|
|
||||||
<returnvalue>2<replaceable>nn</replaceable></returnvalue>
|
|
||||||
(between 200 and 299, inclusive) indicates a
|
|
||||||
successful write; status <errorcode>411</errorcode>
|
|
||||||
<errortext>Length Required</errortext> denotes a
|
|
||||||
missing <literal>Content-Length</literal> or
|
|
||||||
<literal>Content-Type</literal> header in the request.
|
|
||||||
If the MD5 checksum of the data written to the storage
|
|
||||||
system does NOT match the (optionally) supplied ETag
|
|
||||||
value, a <errorcode>422</errorcode>
|
|
||||||
<errortext>Unprocessable Entity</errortext> response
|
|
||||||
is returned.</para>
|
|
||||||
<para>You can continue uploading segments like this
|
|
||||||
example shows, prior to uploading the manifest.</para>
|
|
||||||
<example>
|
|
||||||
<title>Upload next segment of large object request:
|
|
||||||
HTTP</title>
|
|
||||||
<literallayout class="monospaced"><xi:include href="samples/large-object-upload-next-segment-req.txt" parse="text"/></literallayout>
|
|
||||||
</example>
|
|
||||||
<example>
|
|
||||||
<title>Upload next segment of large object response:
|
|
||||||
HTTP</title>
|
|
||||||
<literallayout class="monospaced">w</literallayout>
|
|
||||||
</example>
|
|
||||||
<para>Next, upload the manifest you created that indicates
|
|
||||||
the container the object segments reside within. Note
|
|
||||||
that uploading additional segments after the manifest
|
|
||||||
is created causes the concatenated object to be that
|
|
||||||
much larger but you do not need to recreate the
|
|
||||||
manifest file for subsequent additional
|
|
||||||
segments.</para>
|
|
||||||
<example>
|
|
||||||
<title>Upload manifest request: HTTP</title>
|
|
||||||
<literallayout class="monospaced"><xi:include href="samples/upload-manifest-req.txt" parse="text"/></literallayout>
|
|
||||||
</example>
|
|
||||||
<example>
|
|
||||||
<title>Upload manifest response: HTTP</title>
|
|
||||||
<literallayout class="monospaced"><xi:include href="samples/upload-manifest-resp.txt" parse="text"/></literallayout>
|
|
||||||
</example>
|
|
||||||
<para>The response's <literal>Content-Type</literal> for a
|
|
||||||
&GET; or &HEAD; on the manifest is the same as the
|
|
||||||
<literal>Content-Type</literal> set during the
|
|
||||||
&PUT; request that created the manifest. You can
|
|
||||||
easily change the <literal>Content-Type</literal> by
|
|
||||||
reissuing the &PUT; request.</para>
|
|
||||||
</section>
|
|
||||||
<section xml:id="static-large-objects">
|
|
||||||
<title>Static large objects</title>
|
|
||||||
<procedure>
|
|
||||||
<title>To create a static large object</title>
|
|
||||||
<step>
|
|
||||||
<para>Divide your content into pieces and create
|
|
||||||
(upload) a segment object to contain each
|
|
||||||
piece. You must record the <literal>ETag</literal>
|
|
||||||
response header returned by the&PUT;
|
|
||||||
operation. Alternatively, you can calculate
|
|
||||||
the MD5 checksum of the segment prior to
|
|
||||||
uploading and include this in the
|
|
||||||
<literal>ETag</literal> request header. This
|
|
||||||
ensures that the upload cannot corrupt your
|
|
||||||
data.</para>
|
|
||||||
</step>
|
|
||||||
<step>
|
|
||||||
<para>List the name of each segment object along
|
|
||||||
with its size and MD5 checksum in order.
|
|
||||||
Create a manifest object. You indicate that
|
|
||||||
this is a manifest object by including the
|
|
||||||
<parameter>?multipart-manifest=put</parameter> query
|
|
||||||
string at the end of the manifest object
|
|
||||||
name.</para>
|
|
||||||
</step>
|
|
||||||
</procedure>
|
|
||||||
<para>The body of the &PUT; request on the manifest object
|
|
||||||
comprises a json list, where each element contains the
|
|
||||||
following attributes:</para>
|
|
||||||
<itemizedlist>
|
|
||||||
<listitem>
|
|
||||||
<para><code>path</code>. The container and object
|
|
||||||
name in this format:
|
|
||||||
<code><container-name>/<object-name></code></para>
|
|
||||||
</listitem>
|
|
||||||
<listitem>
|
|
||||||
<para><code>etag</code>. The MD5 checksum of the
|
|
||||||
content of the segment object. This value must
|
|
||||||
match the <literal>ETag</literal> of that
|
|
||||||
object.</para>
|
|
||||||
</listitem>
|
|
||||||
<listitem>
|
|
||||||
<para><code>size_bytes</code>. The size of the
|
|
||||||
segment object. This value must match the
|
|
||||||
<literal>Content-Length</literal> of that
|
|
||||||
object.</para>
|
|
||||||
</listitem>
|
|
||||||
</itemizedlist>
|
|
||||||
<example>
|
|
||||||
<title>Static large object manifest list</title>
|
|
||||||
<para>This example shows three segment objects. You
|
|
||||||
can use several containers and the object names do
|
|
||||||
not have to conform to a specific pattern, in
|
|
||||||
contrast to dynamic large objects.</para>
|
|
||||||
<literallayout class="monospaced"><xi:include href="samples/slo-manifest-example.txt" parse="text"/></literallayout>
|
|
||||||
</example>
|
|
||||||
<para>The <literal>Content-Length</literal> request header must
|
|
||||||
contain the length of the json content. Not the length
|
|
||||||
of the segment objects. However, after the &PUT;
|
|
||||||
operation completes, the <literal>Content-Length</literal>
|
|
||||||
metadata is set to the total length of all the object
|
|
||||||
segments. A similar situation applies to the
|
|
||||||
<literal>ETag</literal>. If used in the &PUT; operation,
|
|
||||||
it must contain the MD5 checksum of the json content.
|
|
||||||
The <literal>ETag</literal> metadata value is then set to be
|
|
||||||
the MD5 checksum of the concatenated <literal>ETag</literal>
|
|
||||||
values of the object segments. You can also set the
|
|
||||||
<literal>Content-Type</literal> request header and
|
|
||||||
custom object metadata.</para>
|
|
||||||
<para>When the &PUT; operation sees the
|
|
||||||
<parameter>?multipart-manifest=put</parameter>
|
|
||||||
query parameter, it reads the request body and verifies
|
|
||||||
that each segment object exists and that the sizes and
|
|
||||||
ETags match. If there is a mismatch, the
|
|
||||||
&PUT;operation fails.</para>
|
|
||||||
<para>If everything matches, the manifest object is
|
|
||||||
created. The <literal>X-Static-Large-Object</literal>
|
|
||||||
metadata is set to <literal>true</literal> indicating
|
|
||||||
that this is a static object manifest.</para>
|
|
||||||
<para>Normally when you perform a &GET; operation on the
|
|
||||||
manifest object, the response body contains the
|
|
||||||
concatenated content of the segment objects. To
|
|
||||||
download the manifest list, use the
|
|
||||||
<parameter>?multipart-manifest=get</parameter> query parameter.
|
|
||||||
The resulting list is not formatted the same as the
|
|
||||||
manifest you originally used in the &PUT;
|
|
||||||
operation.</para>
|
|
||||||
<para>If you use the &DELETE; operation on a manifest
|
|
||||||
object, the manifest object is deleted. The segment
|
|
||||||
objects are not affected. However, if you add the
|
|
||||||
<parameter>?multipart-manifest=delete</parameter> query parameter,
|
|
||||||
the segment objects are deleted and if all are
|
|
||||||
successfully deleted, the manifest object is also
|
|
||||||
deleted.</para>
|
|
||||||
<para>To change the manifest, use a &PUT; operation with
|
|
||||||
the <parameter>?multipart-manifest=put</parameter>
|
|
||||||
query parameter. This request creates a
|
|
||||||
new manifest object. You can
|
|
||||||
also update the object metadata in the usual
|
|
||||||
way.</para>
|
|
||||||
</section>
|
|
||||||
</section>
|
</section>
|
||||||
</section>
|
</section>
|
||||||
|
|
|
@ -4,9 +4,9 @@
|
||||||
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0"
|
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0"
|
||||||
xml:id="static-website">
|
xml:id="static-website">
|
||||||
<title>Create static website</title>
|
<title>Create static website</title>
|
||||||
<para>To discover whether your Object Storage system supports
|
<para>To discover whether your Object Storage system supports this
|
||||||
this feature, see <xref linkend="discoverability"
|
feature, see <xref linkend="discoverability"/>. Alternatively,
|
||||||
/>. Alternatively, check with your service provider.</para>
|
check with your service provider.</para>
|
||||||
<para>You can use your Object Storage account to create a static
|
<para>You can use your Object Storage account to create a static
|
||||||
website. This mode is normally only active for anonymous
|
website. This mode is normally only active for anonymous
|
||||||
requests, which provide no authentication token. To use it
|
requests, which provide no authentication token. To use it
|
||||||
|
@ -21,7 +21,7 @@
|
||||||
<literal>X-Container-Meta-Web-Error</literal>. (The latter
|
<literal>X-Container-Meta-Web-Error</literal>. (The latter
|
||||||
header is discussed below, under <link
|
header is discussed below, under <link
|
||||||
linkend="Set_Error_Pages_for_Static_Website-dle4005">Set
|
linkend="Set_Error_Pages_for_Static_Website-dle4005">Set
|
||||||
Error Pages for Static Website</link>.) With
|
error pages for static website</link>.) With
|
||||||
<literal>X-Container-Meta-Web-Index</literal>, you
|
<literal>X-Container-Meta-Web-Index</literal>, you
|
||||||
determine the index file (or default page served, such as
|
determine the index file (or default page served, such as
|
||||||
<literal>index.html</literal>) displays your website. When
|
<literal>index.html</literal>) displays your website. When
|
||||||
|
@ -41,9 +41,9 @@
|
||||||
style sheet (for example,
|
style sheet (for example,
|
||||||
<literal>lists.css</literal>).</para>
|
<literal>lists.css</literal>).</para>
|
||||||
<section xml:id="Examples_for_static_web-dle4025">
|
<section xml:id="Examples_for_static_web-dle4025">
|
||||||
<title>Static Web Middleware through swift</title>
|
<title>Static web middleware through swift</title>
|
||||||
<example>
|
<example>
|
||||||
<title>Make Container Publicly Readable</title>
|
<title>Make container publicly readable</title>
|
||||||
<para>Make the container publicly readable. Once the
|
<para>Make the container publicly readable. Once the
|
||||||
container is publicly readable, you may access your
|
container is publicly readable, you may access your
|
||||||
objects directly, but you must set the index file to
|
objects directly, but you must set the index file to
|
||||||
|
@ -52,27 +52,27 @@
|
||||||
<screen><prompt>$</prompt> <userinput>swift post -r '.r:*' container</userinput></screen>
|
<screen><prompt>$</prompt> <userinput>swift post -r '.r:*' container</userinput></screen>
|
||||||
</example>
|
</example>
|
||||||
<example>
|
<example>
|
||||||
<title>Set Site Index File</title>
|
<title>Set site index file</title>
|
||||||
<para>Set the index file. In this case,
|
<para>Set the index file. In this case,
|
||||||
<literal>index.html</literal> is the default file
|
<literal>index.html</literal> is the default file
|
||||||
displayed when the site displays.</para>
|
displayed when the site displays.</para>
|
||||||
<screen><prompt>$</prompt> <userinput>swift post -m 'web-index:index.html' container</userinput></screen>
|
<screen><prompt>$</prompt> <userinput>swift post -m 'web-index:index.html' container</userinput></screen>
|
||||||
</example>
|
</example>
|
||||||
<example>
|
<example>
|
||||||
<title>Enable File Listing</title>
|
<title>Enable file listing</title>
|
||||||
<para>Turn on file listing. If you do not set the index
|
<para>Turn on file listing. If you do not set the index
|
||||||
file, list the objects in the container. Instructions
|
file, list the objects in the container. Instructions
|
||||||
on styling the list with the CSS follow.</para>
|
on styling the list with the CSS follow.</para>
|
||||||
<screen><prompt>$</prompt> <userinput>swift post -m 'web-listings: true' container</userinput></screen>
|
<screen><prompt>$</prompt> <userinput>swift post -m 'web-listings: true' container</userinput></screen>
|
||||||
</example>
|
</example>
|
||||||
<example>
|
<example>
|
||||||
<title>Enable CSS for File Listing</title>
|
<title>Enable CSS for file listing</title>
|
||||||
<para>Style the file listing.</para>
|
<para>Style the file listing.</para>
|
||||||
<programlisting language="ini"><xi:include href="samples/file-listings-css-set-req.txt" parse="text"/></programlisting>
|
<programlisting language="ini"><xi:include href="samples/file-listings-css-set-req.txt" parse="text"/></programlisting>
|
||||||
</example>
|
</example>
|
||||||
</section>
|
</section>
|
||||||
<section xml:id="Set_Error_Pages_for_Static_Website-dle4005">
|
<section xml:id="Set_Error_Pages_for_Static_Website-dle4005">
|
||||||
<title>Set Error Pages for Static Website</title>
|
<title>Set error pages for static website</title>
|
||||||
<para>You can create and set custom error pages for visitors
|
<para>You can create and set custom error pages for visitors
|
||||||
to your website; currently, only 401 (Unauthorized) and
|
to your website; currently, only 401 (Unauthorized) and
|
||||||
404 (Not Found) errors are supported. To do this, set the
|
404 (Not Found) errors are supported. To do this, set the
|
||||||
|
@ -93,10 +93,10 @@
|
||||||
<para>Set the <literal>X-Container-Meta-Web-Error</literal>
|
<para>Set the <literal>X-Container-Meta-Web-Error</literal>
|
||||||
metadata once for your entire static website.</para>
|
metadata once for your entire static website.</para>
|
||||||
<example>
|
<example>
|
||||||
<title>Set Error Pages for Static Website Request</title>
|
<title>Set error pages for static website request</title>
|
||||||
<programlisting language="ini"><xi:include href="samples/error-page-set-req.txt" parse="text"/></programlisting>
|
<programlisting language="ini"><xi:include href="samples/error-page-set-req.txt" parse="text"/></programlisting>
|
||||||
</example>
|
</example>
|
||||||
<para>Any 2<varname>nn</varname> response indicates success.
|
<para>Any 2<varname>nn</varname> response indicates
|
||||||
</para>
|
success.</para>
|
||||||
</section>
|
</section>
|
||||||
</section>
|
</section>
|
||||||
|
|
Loading…
Reference in New Issue