Commit Graph

3 Commits

Author SHA1 Message Date
Paul Bourke 13b9dbfcdd Update .gitignore
Source
https://raw.githubusercontent.com/github/gitignore/master/Python.gitignore

with some modifications.

Change-Id: I9eb17bba855381d475a104d222d1c8a803671fb5
2016-01-19 17:03:10 +00:00
Michal Jastrzebski a9d13fc32f Create manifest and implement sqlite driver
Use class inheritance for driver in case we need something other than
sqlite as time goes on.

SQLite is only 10% larger than the custom format used with osdk. When
compressed the SQLite manifest only fractionally larger than osdk. The
time it takes to generate and process is about 2x longer. However, the
benefit we get from using common contructs (sqlite) and tracking and
changing the manifest schema is enough for me to think this is right
way to proceed.

To generate a manifest with 2^24 objects (64TB disk) run:
tools/generate_manifest.py --backupsize 64000 --manifest /pathtosave/manifest

This will generate a 809MB file

Co-Authored-By: Sam Yaple <sam@yaple.net>
Change-Id: Ic431bfa52b6fcaeb1c6a64cf270cbb36c496335e
2016-01-11 18:15:17 +00:00
SamYaple fdfb83197c Initial Commit
Change-Id: Iaa4726694388e24e944a37da52c851bba1ead1ce
2016-01-04 16:28:29 +00:00