Re: Data Deduplication with the help of an online filesystem check

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, Apr 28, 2009 at 2:02 PM, Chris Mason <chris.mason@xxxxxxxxxx> wrote:
> On Tue, 2009-04-28 at 07:22 +0200, Thomas Glanzmann wrote:
>> Hello Chris,
>>
>> > There is a btrfs ioctl to clone individual files, and this could be used
>> > to implement an online dedup.  But, since it is happening from userland,
>> > you can't lock out all of the other users of a given file.
>>
>> > So, the dedup application would be responsible for making sure a given
>> > file was not being changed while the dedup scan was running.
>>
>> I see, does that mean that I can not do ,,dedup'' for files that are
>> currently opened by a userland program?
>
> No, but it does mean the dedup done from userland is racey.  Picture

Race disappears if (background) dedupe is run against snapshot(s).

Regards,
Andrey

> this:
>
> process A:
>    create some_file # some_file matches the contents of another file
>
> dedup proc:
>    check some_file
>    decide to start block dedup
>
> process A:
>     modify some_file
>
> dedup proc:
>     progress through block dedup
>
> So, this will happily replace blocks in some_file with the dedup blocks.
> But there's no way to atomically swap them.
>
> We could create new ioctls for this, basically a variant of the clone
> file ioctl that makes sure a given set of pages has a given sum (or
> strict memory contents) before doing the swap.
>
> But they don't exist yet.
>
> -chris
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Filesystem Development]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux