Difference between revisions of "User:StefanoZacchiroli/Content deduplication"
Jump to navigation
Jump to search
(→test 1) |
|||
| Line 14: | Line 14: | ||
* avg_block_size: 8 KB | * avg_block_size: 8 KB | ||
* max_block_size: 64 KB | * max_block_size: 64 KB | ||
| + | |||
| + | Results: | ||
| + | * original size (uncompressed): 55.89 GB | ||
| + | * dedup chunk size (uncompressed): 19.87 GB (35.55%) | ||
Revision as of 12:42, 7 January 2018
Some experiments on deduplicating contents at sub-file granularity using Rabin fingerprints.
Linux kernel, Git repo
- origin: git.kernel.org, on 2018-01-06
- 1.653.941 content blobs, for a total of 19 GB (compressed)
test 1
Rabin fingerprint parameters:
- prime: 3
- window_size: 48 KB
- min_block_size: 2 KB
- avg_block_size: 8 KB
- max_block_size: 64 KB
Results:
- original size (uncompressed): 55.89 GB
- dedup chunk size (uncompressed): 19.87 GB (35.55%)