Re: announcement of deduplication in Sun ZFS

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, Nov 2, 2009 at 3:04 PM, Chris Samuel <chris@xxxxxxxxxxx> wrote:
> On Tue, 3 Nov 2009 07:51:52 am Alex Dedul wrote:
>
> > In any way thats a nice and interesting feature, thanks! :)
>
> My concern would be that it increases the impact of a corruption of the block
> that has been de-dup'd - in other words if the block that now represents the
> same data in lots of files gets trashed then all those files have corrupt
> data. This implies that you would want to keep around at least one other copy
> of that data to be resilient in the face of corruption (checksums will let you
> detect it, but not necessarily recover from it without duplicate copies).
>
> Given that ZFS prides itself on detecting errors it would be strange if they
> hadn't considered this in the implementation but I couldn't see any mention of
> it.

In the PSARC description and mailing list thread, there's mention of a
setting for how many references before another copy of the block is
kept.  The default was going to be 100.  So after 100 references to
the same block, a second copy of the block would be stored on disk.
After 200 references, a third copy of the block would be stored on
disk.  And so on.

There was some confusion on whether or not the 100 would be the
minimum or the maximum, though.

Either way, the admin sets the policy for how many extra copies to
keep around.  And it would play nicely with the "copies=X" setting as
well (ie if copies=2, then you'd start with 2 copies of each deduped
block, and after 100 references, you'd store 2 more copies on disk,
and so on).

--
Freddie Cash
fjwcash@xxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Filesystem Development]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux