Re: Extremely slow metadata performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Marc MERLIN <marc@xxxxxxxxxxx> schrieb:

> I'm one of those people who uses cp -al and rsync to do backups. Indeed
> I should likely rework the flow to use subvolumes and snapshots.
> You also mentioned reflinks, and it sounds like I can use
> cp -a --reflink instead of cp -al.
> 
> Also, would the dedupe code in btrfs effectively allow for the same
> thing after the fact if you use cp without --reflink? Is it stable
> enough nowadays?

You may want to try my backup script as a starting point:

https://gist.github.com/kakra/5520370

It uses a scratch area to create a mirror of your system, then - if it was 
successful - take a snapshot of it. Next time the backup runs, the scratch 
area is updated with rsync using its features for in-file delta updates, so 
you get optimal small deltas between each snapshot. Normally, rsync would 
create a new copy of each file in local mode, so using these features is 
important.

My script is meant to be run from systemd with auto-mount backup disk. But 
it should be easy to adapt to cron if you need to. All the work is done 
within the bash script.

You may wonder about the scratch area. I took this route to ensure that 
snapshots always contain clean and consistent backups. If there was a 
problem during rsync, there will be no snapshot. It's that easy. Usually I 
see other solutions taking a snapshot first, then modify this snapshot. So 
you never know if snapshots are clean and consistent. With my approach you 
get clean snapshots and even if the last backup broke and you are thus 
missing a snapshot, you still have data in the scratch area to recover from.

In my scenario my backup script is able to hold several weeks of daily 
backups in the backlog. The destination is mounted with compress-force=zlib 
so I get good compression, too. However, I still had no time for 
implementing automatic cleanup of old snapshots. Feel free to add it.

HTH
Kai

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Filesystem Development]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux