aad02ad30d
Suffix hash invalidations in hashes.invalid can be lost when two concurrent calls to get_hashes consolidate the hashes of a new partition with no hashes.pkl: - suffix S has been invalidated and is listed in hashes.invalid - process X calls get_hashes when there is no existing hashes.pkl - process X removes hashes.invalids file in consolidate_hashes - process X calculates the hash of suffix S (note, process X has not yet written hashes.pkl) - process Y invalidates suffix S, appends S to hashes.invalid, so the hash of suffix S *should* be recalculated at some point - process Z calls get_hashes->consolidate_hashes, deletes hashes.invalid because there is still no hashes.pkl - process Z fails - process X writes hashes.pkl with stale hash value for suffix S - the invalidation of S that was made by process Y is lost The solution is to never remove hashes.invalid during consolidate_hashes without first recording any invalid suffixes in hashes and writing hashes to disk in hashes.pkl. This is already the behaviour when hashes.pkl exists. The cost of an additional write to hashes.pkl, introduced by this patch, is only incurred once, when get_hashes first encounters a partition with no hashes.pkl. Related-Change: Ia43ec2cf7ab715ec37f0044625a10aeb6420f6e3 Change-Id: I08c8cf09282f737103e580c1f57923b399abe58c |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| auditor.py | ||
| diskfile.py | ||
| expirer.py | ||
| mem_diskfile.py | ||
| mem_server.py | ||
| reconstructor.py | ||
| replicator.py | ||
| server.py | ||
| ssync_receiver.py | ||
| ssync_sender.py | ||
| updater.py | ||