Am Wed, 1 Nov 2017 02:51:58 -0400 schrieb Dave <davestechshop@xxxxxxxxx>: > > > >> To reconcile those conflicting goals, the only idea I have come up > >> with so far is to use btrfs send-receive to perform incremental > >> backups > > > > As already said by Romain Mamedov, rsync is viable alternative to > > send-receive with much less hassle. According to some reports it > > can even be faster. > > Thanks for confirming. I must have missed those reports. I had never > considered this idea until now -- but I like it. > > Are there any blogs or wikis where people have done something similar > to what we are discussing here? I used rsync before, backup source and destination both were btrfs. I was experiencing the same btrfs bug from time to time on both devices, luckily not at the same time. I instead switched to using borgbackup, and xfs as the destination (to not fall the same-bug-in-two-devices pitfall). Borgbackup achieves a much higher deduplication density and compression, and as such also is able to store much more backup history in the same storage space. The first run is much slower than rsync (due to enabled compression) but successive runs are much faster (like 20 minutes per backup run instead of 4-5 hours). I'm currently storing 107 TB of backup history in just 2.2 TB backup space, which counts a little more than one year of history now, containing 56 snapshots. This is my retention policy: * 5 yearly snapshots * 12 monthly snapshots * 14 weekly snapshots (worth around 3 months) * 30 daily snapshots Restore is fast enough, and a snapshot can even be fuse-mounted (tho, in that case mounted access can be very slow navigating directories). With latest borgbackup version, the backup time increased to around 1 hour from 15-20 minutes in the previous version. That is due to switching the file cache strategy from mtime to ctime. This can be tuned to get back to old performance, but it may miss some files during backup if you're doing awkward things to file timestamps. I'm also backing up some servers with it now, then use rsync to sync the borg repository to an offsite location. Combined with same-fs local btrfs snapshots with short retention times, this could be a viable solution for you. -- Regards, Kai Replies to list-only preferred. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
