On Fri, Nov 13, 2009 at 12:44:47AM +0300, Alexander Beregalov wrote: > 2009/11/12 Chris Mason <chris.mason@xxxxxxxxxx>: > > On Thu, Nov 12, 2009 at 08:41:44PM +0300, Alexander Beregalov wrote: > >> 2009/11/12 Chris Mason <chris.mason@xxxxxxxxxx>: > >> > On Thu, Nov 12, 2009 at 06:43:00PM +0300, Alexander Beregalov wrote: > >> >> 2009/11/12 Chris Mason <chris.mason@xxxxxxxxxx>: > >> >> > On Thu, Nov 12, 2009 at 05:16:34PM +0300, Alexander Beregalov wrote: > >> >> >> 2009/11/12 Chris Mason <chris.mason@xxxxxxxxxx>: > >> >> >> > On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote: > >> >> >> >> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify > >> >> >> >> failed";echo; done > >> >> >> >> checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 > >> >> >> >> checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 > >> >> >> > > >> >> >> > Was your filesystem mounted at the time? It's strange that the checksum > >> >> >> > keeps changing, that points to the data in the block changing. > >> >> >> > >> >> >> It was not. > >> >> >> Any tips how I can try to find a root cause? > >> >> >> It is raid0 of two disks. > >> >> > > >> >> > Well, I'd start by looking for anything else that could be touching the > >> >> > FS or the devices. You shouldn't see more than two different csums in this > >> >> > configuration (one from each disk). > >> >> I always see only these two blocks in output. > >> > > >> > The wanted is always the same but the found is always different? > >> > >> Yes, exactly. The bits marked as X look as random. > > > > Ok, I've added a new tool to the btrfs-unstable-progs repo. It is > > called btrfs-map-logical and it will print the logical->physical mapping > > for any metadata or data block in the FS. The FS should not be mounted > > when you run this program...it'll mostly work mounted but the results > > won't be 100% accurate. > > > > For your case: > > > > btrfs-map-logical -l 31945617408 -c 1 -o mirror1 /dev/sde > > btrfs-map-logical -l 31945617408 -c 2 -o mirror2 /dev/sde > > > > This will print to the screen the mappings for that block and also save > > the contents of the first mirror of the block to a file called mirror1. > > The second command will save the contents of the second mirror. > > > > If the FS isn't mounted and isn't being changed by someone else, you > > should get exactly the same mappings every time and exactly the same > > contents in the files mirror1 and mirror2 every time. > > > > Just let me know if you have any problems. The most recent commit in > > the btrfs-progs repo is: > > > > commit ab8fb4c99516c186641bda1dbc0e788f68b4dc77 > > Author: Chris Mason <chris.mason@xxxxxxxxxx> > > Date: Thu Nov 12 14:34:09 2009 -0500 > > > > Add btrfs-map-logical program to map and read logical block numbers > > > > If you check kernel.org and this commit isn't there, just try again ;) > > The mirror takes a few minutes to update. I'm pretty sure it goes > > faster if you hit reload over and over again, but that's just a guess. > > > > -chris > > > > sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 1 -o /tmp/mirror1 > /dev/sde; md5sum /tmp/mirror1 > mirror 1 logical 31945617408 physical 16913231872 device /dev/sde > mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf > bcce5ab8cc65c948650376e936952547 /tmp/mirror1 > > sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 2 -o /tmp/mirror2 > /dev/sde; md5sum /tmp/mirror2 > mirror 1 logical 31945617408 physical 16913231872 device /dev/sde > mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf > 347034331d8879096e262a736fd02ba0 /tmp/mirror2 > > sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 1 -o /tmp/mirror1 > /dev/sde; md5sum /tmp/mirror1 > mirror 1 logical 31945617408 physical 16913231872 device /dev/sde > mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf > bcce5ab8cc65c948650376e936952547 /tmp/mirror1 > > sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 2 -o /tmp/mirror2 > /dev/sde; md5sum /tmp/mirror2 > mirror 1 logical 31945617408 physical 16913231872 device /dev/sde > mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf > 347034331d8879096e262a736fd02ba0 /tmp/mirror2 You're getting different answers for the two blocks, which is ok (one could be corrupt). What doesn't make sense is that btrfsck isn't dealing with this correctly. When you read both copies again, you get the same answers as the first time, so this looks like a btrfsck bug to me. I'll try to reproduce. If you're in a hurry, you could do something like this (with the FS unmounted). Keep a good copy of both mirror1 and mirror2 dd if=/tmp/mirror1 of=/dev/sde bs=1 seek=16913231872 count=4096 dd if=/tmp/mirror1 of=/dev/sdf bs=1 seek=16893308928 count=4096 sync btrfsck. If that doesn't work: dd if=/tmp/mirror2 of=/dev/sde bs=1 seek=16913231872 count=4096 dd if=/tmp/mirror2 of=/dev/sdf bs=1 seek=16893308928 count=4096 btrfsck again. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
