On 2014-01-20 10:36, Bob Marley wrote: > On 20/01/2014 15:57, Ian Hinder wrote: >> i.e. that there is parity information stored with every piece of data, >> and ZFS will "correct" errors automatically from the parity information. > > So this is not just parity data to check correctness but there are many > more additional bits to actually correct these errors, based on an > algorithm like reed-solomon ? > > Where can I find information on how much "parity" is stored in ZFS ? > >> I start to suspect that there is confusion here between checksumming >> for data integrity and parity information. If this is really how ZFS >> works, then if memory corruption interferes with this process, then I >> can see how a scrub could be devastating. > > I don't . If you have additional bits to correct errors (other than > detect errors), this will never be worse than having less of them. > All algorithms I know of, don't behave any worse if the erroneous bits > are in the checksum part, or if the algorithm is correct+detect instead > of just detect. > If the algorithm stores X+2Y extra bits (supposed ZFS case) in order to > detect&correct Y erroneous bits and detect additional X erroneous bits, > this will not be worse than having just X checksum bits (btrfs case). > > So does ZFS really uses detect&correct parity? I'd expect this to be > quite a lot computationally expensive > >> I don't know if ZFS really works like this. It sounds very odd to do >> this without an additional checksum check. This sounds very different >> to what you say below that btrfs does, which is only to check against >> redundantly-stored copies, which I agree sounds much safer. The second >> link above from the ZFS FAQ just says that if you place a very high >> value on data integrity, you should be using ECC memory anyway, which >> I'm sure we can all agree with. >> hxxp://zfsonlinux.org/faq.html#DoIHaveToUseECCMemory: >>> 1.16 Do I have to use ECC memory for ZFS? >>> Using ECC memory for ZFS is strongly recommended for enterprise >>> environments where the strongest data integrity guarantees are >>> required. Without ECC memory rare random bit flips caused by cosmic >>> rays or by faulty memory can go undetected. If this were to occur ZFS >>> (or any other filesystem) will write the damaged data to disk and be >>> unable to automatically detect the corruption. > > The above sentence imho means that the data can get corrupted just prior > to its first write. > This is obviously applicable to every filesystem on earth, without ECC, > especially if it happens prior to the computation of the parity. > > BM I apparently misunderstood what I had read about ZFS. As for the parity though, it's equivalent to RAID5, RAID6, or distributed striped triple-parity RAID. Looking further into ZFS itself, I'm starting to wonder why ECC would be recommended for ZFS in cases other than using it on a single disk, it should be able to handle SEU's as long as they don't hit the executable code itself (it uses SHA256, which makes the chances of a single-bit error going undetected astronomical in scale). -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
