Re: Btrfs duperemove corrupt data while dedup

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Thx Filipe,
this is full fix my issue

2015-09-29 15:49 GMT+03:00 Filipe Manana <fdmanana@xxxxxxxxx>:
> On Tue, Sep 29, 2015 at 1:38 PM, Timofey Titovets <nefelim4ag@xxxxxxxxx> wrote:
>> FYI:
>> Looks like patch:
>> Btrfs: fix read corruption of compressed and shared extents
>
> Try the second part (patch on top of that one) that fixes an
> additional corner case that I missed earlier:
>
> https://patchwork.kernel.org/patch/7275851/
>
> thanks
>
>>
>> Partial fixed my issue
>>
>> 2015-08-26 22:33 GMT+03:00 Timofey Titovets <nefelim4ag@xxxxxxxxx>:
>>> Hello guys,
>>> i like btrfs, and i want put it in production soon,
>>> one of the feature that i want use, is a deduplication.
>>>
>>> i frequently testing duperemove on btrfs and already see this problem before.
>>> i know what btrfs before, change mtime while deduping, but after dedup
>>> fixes from Mark (https://github.com/markfasheh), i've try to get
>>> checksums.
>>>
>>> As i know duperemove use kernel ioctl for deduping, i.e. it's not a
>>> duperemove issue, kernel must keep data consistent.
>>>
>>> File system is fresh and btrfs check not show any metadata corruption.
>>>
>>> Github issue:
>>> https://github.com/markfasheh/duperemove/issues/91
>>>
>>> System info:
>>> $ uname -a
>>> Linux titovetst-beplan 4.2.0-rc8-next-20150825-0959-ARCH #1 SMP Wed
>>> Aug 26 10:27:18 MSK 2015 x86_64 GNU/Linux
>>>
>>> Mount options:
>>> rw,relatime,compress=lzo,space_cache,subvolid=257,subvol=/@home
>>>
>>> Okay, how i find it:
>>>
>>> md5sum_recursive(){
>>>         find $@ -type f -exec md5sum {} \;
>>> }
>>>
>>> cp -av --reflink=always ~/<src> ~/<dest>
>>> md5sum_recursive ~/<dest> > ~/dedup.before
>>> duperemove -vhrdb 8k ~/<dest>
>>> md5sum_recursive ~/<dest> > ~/dedup.after
>>> diff -up ~/dedup.before ~/dedup.after
>>>
>>> what i've got (full diff in attach):
>>> --- /home/nefelim4ag/dedup.after        2015-08-26 21:36:55.773452558 +0300
>>> +++ /home/nefelim4ag/dedup.before       2015-08-26 21:21:01.203600761 +0300
>>> @@ -25139,9 +25139,9 @@ caf9d41036e46b85d90a9541e8bc9ce1  /home/
>>> ....
>>> -0ccbc9c81a51f59dcf2ac0d102de37cb
>>> /home/nefelim4ag/L4D2/left4dead2/pak01_003.vpk
>>> +e665b502ee977dc1c619ecbd415c91b8
>>> /home/nefelim4ag/L4D2/left4dead2/pak01_000.vpk
>>> ....
>>>
>>> Files sizes not changed and it's > 1MB.
>>>
>>> Every time i've get a random data corruption.
>>> Only dependencies what i've find it is what smallest block -> more
>>> corruptions and vise versa, i.e. more data deduped -> more corrupted.
>>>
>>> Smart of the disk, it's not looks, like damaged. (attach)
>>>
>>> What i can provide to help fix this issue?
>>> If it's needed, i can recompile kernel with some parameters if it can
>>> help, of course.
>>>
>>> Thanks.
>>>
>>> --
>>> Have a nice day,
>>> Timofey.
>>
>> --
>> Have a nice day,
>> Timofey.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
>  Unreasonable men adapt the world to themselves.
>  That's why all progress depends on unreasonable men."



-- 
Have a nice day,
Timofey.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Filesystem Development]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux