On Sun, 6 Nov 2016 14:30:52 +0100, James Pharaoh <james@xxxxxxxxxxxxxxxxxxxxxxx> wrote : > Hi all, > > I'm pleased to announce my btrfs deduplication utility, written in > Rust. This operates on whole files, is fast, and I believe > complements the existing utilities (duperemove, bedup), which exist > currently. > > Please visit the homepage for more information: > > http://btrfs-dedupe.com > Thanks for having shared your work. Please be aware of these other similar softwares: - jdupes: https://github.com/jbruchon/jdupes - rmlint: https://github.com/sahib/rmlint And of course fdupes. Some intesting points I have seen in them: - use xxhash to identify potential duplicates (huge speedup) - ability to deduplicate read-only snapshots - identify potential reflinked files (see also my email here: https://www.spinics.net/lists/linux-btrfs/msg60081.html) - ability to filter out hardlinks - triangle problem: see jdupes readme - jdupes has started the process to be included in Debian I hope that will help and that you can share some codes with them ! -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
