openstack
/
liberasurecode
Code Issues Proposed changes

v1.0-rc2

This commit is contained in:
Tushar Gohad 2015-03-04 22:34:58 -07:00
parent 40f4337407
commit a99f69513c
1 changed files with 441 additions and 441 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

882
README.md
View File

@ -1,344 +1,344 @@
liberasurecode liberasurecode
============== ==============
liberasurecode is an Erasure Code API library written in C with pluggable Erasure Code backends. liberasurecode is an Erasure Code API library written in C with pluggable Erasure Code backends.
---- ----
Highlights Highlights
========== ==========
* Unified Erasure Coding interface for common storage workloads. * Unified Erasure Coding interface for common storage workloads.
* Pluggable Erasure Code backends - As of v1.0-rc1, liberasurecode supports the following backends: * Pluggable Erasure Code backends - As of v1.0-rc2, liberasurecode supports the following backends:
- 'Jerasure' - Erasure Coding library that supports Reed-Solomon, Cauchy backends [1] - 'Jerasure' - Erasure Coding library that supports Reed-Solomon, Cauchy backends [1]
- 'ISA-L' - Intel Storage Acceleration Library - SIMD accelerated Erasure Coding backends [2] - 'ISA-L' - Intel Storage Acceleration Library - SIMD accelerated Erasure Coding backends [2]
- 'SHSS' - NTT Lab Japan's hybrid Erasure Coding backend [4] - 'SHSS' - NTT Lab Japan's hybrid Erasure Coding backend [4]
- 'Flat XOR HD' - built-in to liberasurecode, based on [3] - 'Flat XOR HD' - built-in to liberasurecode, based on [3]
- 'NULL' template backend implemented to help future backend writers - 'NULL' template backend implemented to help future backend writers
* True 'plugin' architecture - liberasurecode uses Dynamically Loaded (DL) libraries to realize a true 'plugin' architecture. This also allows one to build liberasurecode indepdendent of the Erasure Code backend libraries. * True 'plugin' architecture - liberasurecode uses Dynamically Loaded (DL) libraries to realize a true 'plugin' architecture. This also allows one to build liberasurecode indepdendent of the Erasure Code backend libraries.
* Cross-platform - liberasurecode is known to work on Linux (Fedora/Debian flavors), Solaris, BSD and Darwin/Mac OS X. * Cross-platform - liberasurecode is known to work on Linux (Fedora/Debian flavors), Solaris, BSD and Darwin/Mac OS X.
* Community support - Developed alongside Erasure Code authority Kevin Greenan, liberasurecode is an actively maintained open-source project with growing community involvement (Openstack Swift, Ceph, PyECLib). * Community support - Developed alongside Erasure Code authority Kevin Greenan, liberasurecode is an actively maintained open-source project with growing community involvement (Openstack Swift, Ceph, PyECLib).
---- ----
License License
========== ==========
liberasurecode is distributed under the terms of the **BSD** license. liberasurecode is distributed under the terms of the **BSD** license.
---- ----
Active Users Active Users
==================== ====================
* PyECLib - Python EC library: https://pypi.python.org/pypi/PyECLib * PyECLib - Python EC library: https://pypi.python.org/pypi/PyECLib
* Openstack Swift Object Store - https://wiki.openstack.org/wiki/Swift * Openstack Swift Object Store - https://wiki.openstack.org/wiki/Swift
---- ----
liberasurecode API Definition liberasurecode API Definition
============================= =============================
---- ----
Backend Support Backend Support
--------------- ---------------
``` c ``` c
typedef enum { typedef enum {
EC_BACKEND_NULL = 0, /* "null" */ EC_BACKEND_NULL = 0, /* "null" */
EC_BACKEND_JERASURE_RS_VAND = 1, /* "jerasure_rs_vand" */ EC_BACKEND_JERASURE_RS_VAND = 1, /* "jerasure_rs_vand" */
EC_BACKEND_JERASURE_RS_CAUCHY = 2, /* "jerasure_rs_cauchy" */ EC_BACKEND_JERASURE_RS_CAUCHY = 2, /* "jerasure_rs_cauchy" */
EC_BACKEND_FLAT_XOR_HD = 3, /* "flat_xor_hd" */ EC_BACKEND_FLAT_XOR_HD = 3, /* "flat_xor_hd" */
EC_BACKEND_ISA_L_RS_VAND = 4, /* "isa_l_rs_vand" */ EC_BACKEND_ISA_L_RS_VAND = 4, /* "isa_l_rs_vand" */
EC_BACKEND_SHSS = 5, /* "shss" */ EC_BACKEND_SHSS = 5, /* "shss" */
EC_BACKENDS_MAX, EC_BACKENDS_MAX,
} ec_backend_id_t; } ec_backend_id_t;
``` ```
---- ----
User Arguments User Arguments
-------------- --------------
``` c ``` c
/** /**
* Common and backend-specific args * Common and backend-specific args
* to be passed to liberasurecode_instance_create() * to be passed to liberasurecode_instance_create()
*/ */
struct ec_args { struct ec_args {
int k; /* number of data fragments */ int k; /* number of data fragments */
int m; /* number of parity fragments */ int m; /* number of parity fragments */
int w; /* word size, in bits (optional) */ int w; /* word size, in bits (optional) */
int hd; /* hamming distance (=m for Reed-Solomon) */ int hd; /* hamming distance (=m for Reed-Solomon) */
union { union {
struct { struct {
uint64_t arg1; /* sample arg */ uint64_t arg1; /* sample arg */
} null_args; /* args specific to the null codes */ } null_args; /* args specific to the null codes */
struct { struct {
uint64_t x, y; /* reserved for future expansion */ uint64_t x, y; /* reserved for future expansion */
uint64_t z, a; /* reserved for future expansion */ uint64_t z, a; /* reserved for future expansion */
} reserved; } reserved;
} priv_args1; } priv_args1;
void *priv_args2; /** flexible placeholder for void *priv_args2; /** flexible placeholder for
* future backend args */ * future backend args */
ec_checksum_type_t ct; /* fragment checksum type */ ec_checksum_type_t ct; /* fragment checksum type */
}; };
``` ```
--- ---
User-facing API Functions User-facing API Functions
------------------------- -------------------------
``` c ``` c
/* liberasurecode frontend API functions */ /* liberasurecode frontend API functions */
/** /**
* Create a liberasurecode instance and return a descriptor * Create a liberasurecode instance and return a descriptor
* for use with EC operations (encode, decode, reconstruct) * for use with EC operations (encode, decode, reconstruct)
* *
* @param id - one of the supported backends as * @param id - one of the supported backends as
* defined by ec_backend_id_t * defined by ec_backend_id_t
* @param ec_args - arguments to the EC backend * @param ec_args - arguments to the EC backend
* arguments common to all backends * arguments common to all backends
* k - number of data fragments * k - number of data fragments
* m - number of parity fragments * m - number of parity fragments
* w - word size, in bits * w - word size, in bits
* hd - hamming distance (=m for Reed-Solomon) * hd - hamming distance (=m for Reed-Solomon)
* ct - fragment checksum type (stored with the fragment metadata) * ct - fragment checksum type (stored with the fragment metadata)
* backend-specific arguments * backend-specific arguments
* null_args - arguments for the null backend * null_args - arguments for the null backend
* flat_xor_hd, jerasure do not require any special args * flat_xor_hd, jerasure do not require any special args
* *
* @return liberasurecode instance descriptor (int > 0) * @return liberasurecode instance descriptor (int > 0)
*/ */
int liberasurecode_instance_create(const ec_backend_id_t id, int liberasurecode_instance_create(const ec_backend_id_t id,
struct ec_args *args); struct ec_args *args);
/** /**
* Close a liberasurecode instance * Close a liberasurecode instance
* *
* @param desc - liberasurecode descriptor to close * @param desc - liberasurecode descriptor to close
* *
* @return 0 on success, otherwise non-zero error code * @return 0 on success, otherwise non-zero error code
*/ */
int liberasurecode_instance_destroy(int desc); int liberasurecode_instance_destroy(int desc);
/** /**
* Erasure encode a data buffer * Erasure encode a data buffer
* *
* @param desc - liberasurecode descriptor/handle * @param desc - liberasurecode descriptor/handle
* from liberasurecode_instance_create() * from liberasurecode_instance_create()
* @param orig_data - data to encode * @param orig_data - data to encode
* @param orig_data_size - length of data to encode * @param orig_data_size - length of data to encode
* @param encoded_data - pointer to _output_ array (char **) of k data * @param encoded_data - pointer to _output_ array (char **) of k data
* fragments (char *), allocated by the callee * fragments (char *), allocated by the callee
* @param encoded_parity - pointer to _output_ array (char **) of m parity * @param encoded_parity - pointer to _output_ array (char **) of m parity
* fragments (char *), allocated by the callee * fragments (char *), allocated by the callee
* @param fragment_len - pointer to _output_ length of each fragment, assuming * @param fragment_len - pointer to _output_ length of each fragment, assuming
* all fragments are the same length * all fragments are the same length
* *
* @return 0 on success, -error code otherwise * @return 0 on success, -error code otherwise
*/ */
int liberasurecode_encode(int desc, int liberasurecode_encode(int desc,
const char *orig_data, uint64_t orig_data_size, /* input */ const char *orig_data, uint64_t orig_data_size, /* input */
char ***encoded_data, char ***encoded_parity, /* output */ char ***encoded_data, char ***encoded_parity, /* output */
uint64_t *fragment_len); /* output */ uint64_t *fragment_len); /* output */
/** /**
* Cleanup structures allocated by librasurecode_encode * Cleanup structures allocated by librasurecode_encode
* *
* The caller has no context, so cannot safely free memory * The caller has no context, so cannot safely free memory
* allocated by liberasurecode, so it must pass the * allocated by liberasurecode, so it must pass the
* deallocation responsibility back to liberasurecode. * deallocation responsibility back to liberasurecode.
* *
* @param desc - liberasurecode descriptor/handle * @param desc - liberasurecode descriptor/handle
* from liberasurecode_instance_create() * from liberasurecode_instance_create()
* @param encoded_data - (char **) array of k data * @param encoded_data - (char **) array of k data
* fragments (char *), allocated by liberasurecode_encode * fragments (char *), allocated by liberasurecode_encode
* @param encoded_parity - (char **) array of m parity * @param encoded_parity - (char **) array of m parity
* fragments (char *), allocated by liberasurecode_encode * fragments (char *), allocated by liberasurecode_encode
* *
* @return 0 in success; -error otherwise * @return 0 in success; -error otherwise
*/ */
int liberasurecode_encode_cleanup(int desc, char **encoded_data, int liberasurecode_encode_cleanup(int desc, char **encoded_data,
char **encoded_parity); char **encoded_parity);
/** /**
* Reconstruct original data from a set of k encoded fragments * Reconstruct original data from a set of k encoded fragments
* *
* @param desc - liberasurecode descriptor/handle * @param desc - liberasurecode descriptor/handle
* from liberasurecode_instance_create() * from liberasurecode_instance_create()
* @param fragments - erasure encoded fragments (> = k) * @param fragments - erasure encoded fragments (> = k)
* @param num_fragments - number of fragments being passed in * @param num_fragments - number of fragments being passed in
* @param fragment_len - length of each fragment (assume they are the same) * @param fragment_len - length of each fragment (assume they are the same)
* @param force_metadata_checks - force fragment metadata checks (default: 0) * @param force_metadata_checks - force fragment metadata checks (default: 0)
* @param out_data - _output_ pointer to decoded data * @param out_data - _output_ pointer to decoded data
* @param out_data_len - _output_ length of decoded output * @param out_data_len - _output_ length of decoded output
* *
* @return 0 on success, -error code otherwise * @return 0 on success, -error code otherwise
*/ */
int liberasurecode_decode(int desc, int liberasurecode_decode(int desc,
char **available_fragments, /* input */ char **available_fragments, /* input */
int num_fragments, uint64_t fragment_len, /* input */ int num_fragments, uint64_t fragment_len, /* input */
int force_metadata_checks, /* input */ int force_metadata_checks, /* input */
char **out_data, uint64_t *out_data_len); /* output */ char **out_data, uint64_t *out_data_len); /* output */
/** /**
* Cleanup structures allocated by librasurecode_decode * Cleanup structures allocated by librasurecode_decode
* *
* The caller has no context, so cannot safely free memory * The caller has no context, so cannot safely free memory
* allocated by liberasurecode, so it must pass the * allocated by liberasurecode, so it must pass the
* deallocation responsibility back to liberasurecode. * deallocation responsibility back to liberasurecode.
* *
* @param desc - liberasurecode descriptor/handle * @param desc - liberasurecode descriptor/handle
* from liberasurecode_instance_create() * from liberasurecode_instance_create()
* @param data - (char *) buffer of data decoded by librasurecode_decode * @param data - (char *) buffer of data decoded by librasurecode_decode
* *
* @return 0 on success; -error otherwise * @return 0 on success; -error otherwise
*/ */
int liberasurecode_decode_cleanup(int desc, char *data); int liberasurecode_decode_cleanup(int desc, char *data);
/** /**
* Reconstruct a missing fragment from a subset of available fragments * Reconstruct a missing fragment from a subset of available fragments
* *
* @param desc - liberasurecode descriptor/handle * @param desc - liberasurecode descriptor/handle
* from liberasurecode_instance_create() * from liberasurecode_instance_create()
* @param available_fragments - erasure encoded fragments * @param available_fragments - erasure encoded fragments
* @param num_fragments - number of fragments being passed in * @param num_fragments - number of fragments being passed in
* @param fragment_len - size in bytes of the fragments * @param fragment_len - size in bytes of the fragments
* @param destination_idx - missing idx to reconstruct * @param destination_idx - missing idx to reconstruct
* @param out_fragment - output of reconstruct * @param out_fragment - output of reconstruct
* *
* @return 0 on success, -error code otherwise * @return 0 on success, -error code otherwise
*/ */
int liberasurecode_reconstruct_fragment(int desc, int liberasurecode_reconstruct_fragment(int desc,
char **available_fragments, /* input */ char **available_fragments, /* input */
int num_fragments, uint64_t fragment_len, /* input */ int num_fragments, uint64_t fragment_len, /* input */
int destination_idx, /* input */ int destination_idx, /* input */
char* out_fragment); /* output */ char* out_fragment); /* output */
/** /**
* Return a list of lists with valid rebuild indexes given * Return a list of lists with valid rebuild indexes given
* a list of missing indexes. * a list of missing indexes.
* *
* @desc: liberasurecode instance descriptor (obtained with * @desc: liberasurecode instance descriptor (obtained with
* liberasurecode_instance_create) * liberasurecode_instance_create)
* @fragments_to_reconstruct list of indexes to reconstruct * @fragments_to_reconstruct list of indexes to reconstruct
* @fragments_to_exclude list of indexes to exclude from * @fragments_to_exclude list of indexes to exclude from
* reconstruction equation * reconstruction equation
* @fragments_needed list of fragments needed to reconstruct * @fragments_needed list of fragments needed to reconstruct
* fragments in fragments_to_reconstruct * fragments in fragments_to_reconstruct
* *
* @return 0 on success, non-zero on error * @return 0 on success, non-zero on error
*/ */
int liberasurecode_fragments_needed(int desc, int liberasurecode_fragments_needed(int desc,
int *fragments_to_reconstruct, int *fragments_to_reconstruct,
int *fragments_to_exclude, int *fragments_to_exclude,
int *fragments_needed); int *fragments_needed);
``` ```
Erasure Code Fragment Checksum Types Supported Erasure Code Fragment Checksum Types Supported
---------------------------------------------- ----------------------------------------------
``` c ``` c
/* Checksum types supported for fragment metadata stored in each fragment */ /* Checksum types supported for fragment metadata stored in each fragment */
typedef enum { typedef enum {
CHKSUM_NONE = 0, /* "none" (default) */ CHKSUM_NONE = 0, /* "none" (default) */
CHKSUM_CRC32 = 1, /* "crc32" */ CHKSUM_CRC32 = 1, /* "crc32" */
CHKSUM_TYPES_MAX, CHKSUM_TYPES_MAX,
} ec_checksum_type_t; } ec_checksum_type_t;
``` ```
Erasure Code Fragment Checksum API Erasure Code Fragment Checksum API
---------------------------------- ----------------------------------
``` c ``` c
struct struct
fragment_metadata fragment_metadata
{ {
uint32_t idx; /* 4 */ uint32_t idx; /* 4 */
uint32_t size; /* 4 (raw data bytes size of each fragment used for encode/fragment_to_string) */ uint32_t size; /* 4 (raw data bytes size of each fragment used for encode/fragment_to_string) */
uint32_t frag_backend_metadata_size; /* 4 (extra metadata bytes size of backend specification) */ uint32_t frag_backend_metadata_size; /* 4 (extra metadata bytes size of backend specification) */
uint64_t orig_data_size; /* 8 */ uint64_t orig_data_size; /* 8 */
uint8_t chksum_type; /* 1 */ uint8_t chksum_type; /* 1 */
uint32_t chksum[LIBERASURECODE_MAX_CHECKSUM_LEN]; /* 16 */ uint32_t chksum[LIBERASURECODE_MAX_CHECKSUM_LEN]; /* 16 */
uint8_t chksum_mismatch; /* 1 */ uint8_t chksum_mismatch; /* 1 */
} fragment_metadata_t; } fragment_metadata_t;
/** /**
* Get opaque metadata for a fragment. The metadata is opaque to the * Get opaque metadata for a fragment. The metadata is opaque to the
* client, but meaningful to the underlying library. It is used to verify * client, but meaningful to the underlying library. It is used to verify
* stripes in verify_stripe_metadata(). * stripes in verify_stripe_metadata().
* *
* @param desc - liberasurecode descriptor/handle * @param desc - liberasurecode descriptor/handle
* from liberasurecode_instance_create() * from liberasurecode_instance_create()
* @param fragment - fragment data pointer * @param fragment - fragment data pointer
* @param fragment_metadata - pointer to allocated buffer of size at least * @param fragment_metadata - pointer to allocated buffer of size at least
* sizeof(struct fragment_metadata) to hold fragment metadata struct * sizeof(struct fragment_metadata) to hold fragment metadata struct
* *
* @return 0 on success, non-zero on error * @return 0 on success, non-zero on error
*/ */
int liberasurecode_get_fragment_metadata(int desc, int liberasurecode_get_fragment_metadata(int desc,
char *fragment, fragment_metadata_t *fragment_metadata); char *fragment, fragment_metadata_t *fragment_metadata);
/** /**
* Verify a subset of fragments generated by encode() * Verify a subset of fragments generated by encode()
* *
* @param desc - liberasurecode descriptor/handle * @param desc - liberasurecode descriptor/handle
* from liberasurecode_instance_create() * from liberasurecode_instance_create()
* @param fragments - fragments part of the EC stripe to verify * @param fragments - fragments part of the EC stripe to verify
* @param num_fragments - number of fragments part of the EC stripe * @param num_fragments - number of fragments part of the EC stripe
* *
* @return 1 if stripe checksum verification is successful, 0 otherwise * @return 1 if stripe checksum verification is successful, 0 otherwise
*/ */
int liberasurecode_verify_stripe_metadata(int desc, int liberasurecode_verify_stripe_metadata(int desc,
char **fragments, int num_fragments); char **fragments, int num_fragments);
/** /**
* This computes the aligned size of a buffer passed into * This computes the aligned size of a buffer passed into
* the encode function. The encode function must pad fragments * the encode function. The encode function must pad fragments
* to be algined with the word size (w) and the last fragment also * to be algined with the word size (w) and the last fragment also
* needs to be aligned. This computes the sum of the algined fragment * needs to be aligned. This computes the sum of the algined fragment
* sizes for a given buffer to encode. * sizes for a given buffer to encode.
* *
* @param desc - liberasurecode descriptor/handle * @param desc - liberasurecode descriptor/handle
* from liberasurecode_instance_create() * from liberasurecode_instance_create()
* @param data_len - original data length in bytes * @param data_len - original data length in bytes
* *
* @return aligned length, or -error code on error * @return aligned length, or -error code on error
*/ */
int liberasurecode_get_aligned_data_size(int desc, uint64_t data_len); int liberasurecode_get_aligned_data_size(int desc, uint64_t data_len);
/** /**
* This will return the minimum encode size, which is the minimum * This will return the minimum encode size, which is the minimum
* buffer size that can be encoded. * buffer size that can be encoded.
* *
* @param desc - liberasurecode descriptor/handle * @param desc - liberasurecode descriptor/handle
* from liberasurecode_instance_create() * from liberasurecode_instance_create()
* *
* @return minimum data length length, or -error code on error * @return minimum data length length, or -error code on error
*/ */
int liberasurecode_get_minimum_encode_size(int desc); int liberasurecode_get_minimum_encode_size(int desc);
/** /**
* This will return the fragment size, which is each fragment data * This will return the fragment size, which is each fragment data
@ -352,107 +352,107 @@ int liberasurecode_get_minimum_encode_size(int desc);
* + frag_backend_metadata_size * + frag_backend_metadata_size
*/ */
int liberasurecode_get_fragment_size(int desc, int data_len); int liberasurecode_get_fragment_size(int desc, int data_len);
``` ```
---- ----
Error Codes Error Codes
----------- -----------
``` c ``` c
/* Error codes */ /* Error codes */
typedef enum { typedef enum {
EBACKENDNOTSUPP = 200, /* EC Backend not supported by the library */ EBACKENDNOTSUPP = 200, /* EC Backend not supported by the library */
EECMETHODNOTIMPL = 201, /* EC Backend method not implemented */ EECMETHODNOTIMPL = 201, /* EC Backend method not implemented */
EBACKENDINITERR = 202, /* EC Backend could not be initialized */ EBACKENDINITERR = 202, /* EC Backend could not be initialized */
EBACKENDINUSE = 203, /* EC Backend in use (locked) */ EBACKENDINUSE = 203, /* EC Backend in use (locked) */
} LIBERASURECODE_ERROR_CODES; } LIBERASURECODE_ERROR_CODES;
``` ```
--- ---
Build and Install Build and Install
================= =================
To build the liberasurecode repository, perform the following from the To build the liberasurecode repository, perform the following from the
top-level directory: top-level directory:
``` sh ``` sh
$ ./autogen.sh $ ./autogen.sh
$ ./configure $ ./configure
$ make $ make
$ make test $ make test
$ sudo make install $ sudo make install
``` ```
---- ----
Code organization Code organization
================= =================
``` ```
|-- include |-- include
| +-- erasurecode | +-- erasurecode
| | +-- erasurecode.h --> liberasurecode frontend API header | | +-- erasurecode.h --> liberasurecode frontend API header
| | +-- erasurecode_backend.h --> liberasurecode backend API header | | +-- erasurecode_backend.h --> liberasurecode backend API header
| +-- xor_codes --> headers for the built-in XOR codes | +-- xor_codes --> headers for the built-in XOR codes
| |
|-- src |-- src
| |-- erasurecode.c --> liberasurecode API implementation | |-- erasurecode.c --> liberasurecode API implementation
| | (frontend + backend) | | (frontend + backend)
| |-- backends | |-- backends
| | +-- null | | +-- null
| | +--- null.c --> 'null' erasure code backend (template backend) | | +--- null.c --> 'null' erasure code backend (template backend)
| | +-- xor | | +-- xor
| | +--- flat_xor_hd.c --> 'flat_xor_hd' erasure code backend (built-in) | | +--- flat_xor_hd.c --> 'flat_xor_hd' erasure code backend (built-in)
| | +-- jerasure | | +-- jerasure
| | +-- jerasure_rs_cauchy.c --> 'jerasure_rs_vand' erasure code backend (jerasure.org) | | +-- jerasure_rs_cauchy.c --> 'jerasure_rs_vand' erasure code backend (jerasure.org)
| | +-- jerasure_rs_vand.c --> 'jerasure_rs_cauchy' erasure code backend (jerasure.org) | | +-- jerasure_rs_vand.c --> 'jerasure_rs_cauchy' erasure code backend (jerasure.org)
| | +-- isa-l | | +-- isa-l
| | +-- isa_l_rs_vand.c --> 'isa_l_rs_vand' erasure code backend (Intel) | | +-- isa_l_rs_vand.c --> 'isa_l_rs_vand' erasure code backend (Intel)
| | +-- shss | | +-- shss
| | +-- shss.c --> 'shss' erasure code backend (NTT Labs) | | +-- shss.c --> 'shss' erasure code backend (NTT Labs)
| | | |
| |-- builtin | |-- builtin
| | +-- xor_codes --> XOR HD code backend, built-in erasure | | +-- xor_codes --> XOR HD code backend, built-in erasure
| | | code implementation (shared library) | | | code implementation (shared library)
| | +-- xor_code.c | | +-- xor_code.c
| | +-- xor_hd_code.c | | +-- xor_hd_code.c
| | | |
| +-- utils | +-- utils
| +-- chksum --> fragment checksum utils for erasure | +-- chksum --> fragment checksum utils for erasure
| +-- alg_sig.c coded fragments | +-- alg_sig.c coded fragments
| +-- crc32.c | +-- crc32.c
| |
|-- doc --> API Documentation |-- doc --> API Documentation
| +-- Doxyfile | +-- Doxyfile
| +-- html | +-- html
| |
|--- test --> Test routines |--- test --> Test routines
| +-- builtin | +-- builtin
| | +-- xor_codes | | +-- xor_codes
| +-- liberasurecode_test.c | +-- liberasurecode_test.c
| +-- utils | +-- utils
| |
|-- autogen.sh |-- autogen.sh
|-- configure.ac |-- configure.ac
|-- Makefile.am |-- Makefile.am
|-- README |-- README
|-- NEWS |-- NEWS
|-- COPYING |-- COPYING
|-- AUTHORS |-- AUTHORS
|-- INSTALL |-- INSTALL
+-- ChangeLog +-- ChangeLog
``` ```
--- ---
References References
========== ==========
[1] Jerasure, C library that supports erasure coding in storage applications, http://jerasure.org [1] Jerasure, C library that supports erasure coding in storage applications, http://jerasure.org
[2] Intel(R) Storage Acceleration Library (Open Source Version), https://01.org/intel%C2%AE-storage-acceleration-library-open-source-version [2] Intel(R) Storage Acceleration Library (Open Source Version), https://01.org/intel%C2%AE-storage-acceleration-library-open-source-version
[3] Greenan, Kevin M et al, "Flat XOR-based erasure codes in storage systems", http://www.kaymgee.com/Kevin_Greenan/Publications_files/greenan-msst10.pdf [3] Greenan, Kevin M et al, "Flat XOR-based erasure codes in storage systems", http://www.kaymgee.com/Kevin_Greenan/Publications_files/greenan-msst10.pdf
[4] Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>, Ryuta Kon <kon.ryuta@po.ntts.co.jp>, "NTT SHSS Erasure Coding backend" [4] Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>, Ryuta Kon <kon.ryuta@po.ntts.co.jp>, "NTT SHSS Erasure Coding backend"