5626cd5791
...if users *really* want to. They opt-in at run time by setting
LIBERASURECODE_WRITE_LEGACY_CRC=1
in the environment; leaving it unset, set to an empty string, or set to
the string "0" continues to write zlib crcs.
UpgradeImpact
=============
This option is intended to allow a smooth upgrade from liberasurecode
1.5.0 and earlier in a system with multiple readers and writers:
* Before upgrade, ensure the environment variable is set on all nodes.
This will be ignored by earlier versions.
* Upgrade liberasurecode on each node in the system, restarting any
services that use it. Every node continues writing CRCs that are
still usable by nodes that have not yet upgraded.
* Now that every node is capable of reading zlib CRCs, remove the
environment variable from each node to start writing new CRCs.
If you are already using 1.6.0 or later, just upgrade normally.
Closes-Bug: #1886088
Closes-Bug: #1867937
Related-Bug: #1666320
Needed-By: https://review.opendev.org/#/c/739164/
Change-Id: I9adfbe631a2dddc592fd08f8a325f3e8331b92f1
|
||
|---|---|---|
| doc | ||
| include | ||
| playbooks/unittests | ||
| roles | ||
| src | ||
| test | ||
| .gitignore | ||
| .gitreview | ||
| .mailmap | ||
| .zuul.yaml | ||
| AUTHORS | ||
| COPYING | ||
| ChangeLog | ||
| Makefile.am | ||
| NEWS | ||
| README | ||
| README.md | ||
| Xorcode.pc.in | ||
| autogen.sh | ||
| bindep.txt | ||
| configure.ac | ||
| erasurecode.pc.in | ||
| erasurecode_rs_vand.pc.in | ||
| get_flags_from_cpuid.c | ||
README.md
liberasurecode
liberasurecode is an Erasure Code API library written in C with pluggable Erasure Code backends.
Highlights
-
Unified Erasure Coding interface for common storage workloads.
-
Pluggable Erasure Code backends - liberasurecode supports the following backends:
- 'liberasurecode_rs_vand' - Native, software-only Erasure Coding implementation that supports a Reed-Solomon backend
- 'Jerasure' - Erasure Coding library that supports Reed-Solomon, Cauchy backends [1]
- 'ISA-L' - Intel Storage Acceleration Library - SIMD accelerated Erasure Coding backends [2]
- 'SHSS' - NTT Lab Japan's hybrid Erasure Coding backend [4]
- 'Flat XOR HD' - built-in to liberasurecode, based on [3]
- 'libphazr' - Phazr.IO's erasure code backend with built-in privacy [5]
- 'NULL' template backend implemented to help future backend writers
-
True 'plugin' architecture - liberasurecode uses Dynamically Loaded (DL) libraries to realize a true 'plugin' architecture. This also allows one to build liberasurecode indepdendent of the Erasure Code backend libraries.
-
Cross-platform - liberasurecode is known to work on Linux (Fedora/Debian flavors), Solaris, BSD and Darwin/Mac OS X.
-
Community support - Developed alongside Erasure Code authority Kevin Greenan, liberasurecode is an actively maintained open-source project with growing community involvement (Openstack Swift, Ceph, PyECLib, NTT Labs).
Active Users
- PyECLib - Python EC library: https://github.com/openstack/pyeclib
- Openstack Swift Object Store - https://wiki.openstack.org/wiki/Swift
Build and Install
Install dependencies -
Debian/Ubuntu hosts:
$ sudo apt-get install build-essential autoconf automake libtool
Fedora/RedHat/CentOS hosts:
$ sudo yum install -y gcc make autoconf automake libtool
To build the liberasurecode repository, perform the following from the top-level directory:
$ ./autogen.sh
$ ./configure
$ make
$ make test
$ sudo make install
References
[1] Jerasure, C library that supports erasure coding in storage applications, http://jerasure.org
[2] Intel(R) Storage Acceleration Library (Open Source Version), https://01.org/intel%C2%AE-storage-acceleration-library-open-source-version
[3] Greenan, Kevin M et al, "Flat XOR-based erasure codes in storage systems", http://www.kaymgee.com/Kevin_Greenan/Publications_files/greenan-msst10.pdf
[4] Kota Tsuyuzaki tsuyuzaki.kota@lab.ntt.co.jp, "NTT SHSS Erasure Coding backend"
[5] Jim Cheung support@phazr.io, "Phazr.IO libphazr erasure code backend with built-in privacy"