2009/11/12 Chris Mason <chris.mason@xxxxxxxxxx>: > On Thu, Nov 12, 2009 at 08:41:44PM +0300, Alexander Beregalov wrote: >> 2009/11/12 Chris Mason <chris.mason@xxxxxxxxxx>: >> > On Thu, Nov 12, 2009 at 06:43:00PM +0300, Alexander Beregalov wrote: >> >> 2009/11/12 Chris Mason <chris.mason@xxxxxxxxxx>: >> >> > On Thu, Nov 12, 2009 at 05:16:34PM +0300, Alexander Beregalov wrote: >> >> >> 2009/11/12 Chris Mason <chris.mason@xxxxxxxxxx>: >> >> >> > On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote: >> >> >> >> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify >> >> >> >> failed";echo; done >> >> >> >> checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 >> >> >> >> checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 >> >> >> > >> >> >> > Was your filesystem mounted at the time? It's strange that the checksum >> >> >> > keeps changing, that points to the data in the block changing. >> >> >> >> >> >> It was not. >> >> >> Any tips how I can try to find a root cause? >> >> >> It is raid0 of two disks. >> >> > >> >> > Well, I'd start by looking for anything else that could be touching the >> >> > FS or the devices. You shouldn't see more than two different csums in this >> >> > configuration (one from each disk). >> >> I always see only these two blocks in output. >> > >> > The wanted is always the same but the found is always different? >> >> Yes, exactly. The bits marked as X look as random. > > Ok, I've added a new tool to the btrfs-unstable-progs repo. It is > called btrfs-map-logical and it will print the logical->physical mapping > for any metadata or data block in the FS. The FS should not be mounted > when you run this program...it'll mostly work mounted but the results > won't be 100% accurate. > > For your case: > > btrfs-map-logical -l 31945617408 -c 1 -o mirror1 /dev/sde > btrfs-map-logical -l 31945617408 -c 2 -o mirror2 /dev/sde > > This will print to the screen the mappings for that block and also save > the contents of the first mirror of the block to a file called mirror1. > The second command will save the contents of the second mirror. > > If the FS isn't mounted and isn't being changed by someone else, you > should get exactly the same mappings every time and exactly the same > contents in the files mirror1 and mirror2 every time. > > Just let me know if you have any problems. The most recent commit in > the btrfs-progs repo is: > > commit ab8fb4c99516c186641bda1dbc0e788f68b4dc77 > Author: Chris Mason <chris.mason@xxxxxxxxxx> > Date: Thu Nov 12 14:34:09 2009 -0500 > > Add btrfs-map-logical program to map and read logical block numbers > > If you check kernel.org and this commit isn't there, just try again ;) > The mirror takes a few minutes to update. I'm pretty sure it goes > faster if you hit reload over and over again, but that's just a guess. > > -chris > sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 1 -o /tmp/mirror1 /dev/sde; md5sum /tmp/mirror1 mirror 1 logical 31945617408 physical 16913231872 device /dev/sde mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf bcce5ab8cc65c948650376e936952547 /tmp/mirror1 sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 2 -o /tmp/mirror2 /dev/sde; md5sum /tmp/mirror2 mirror 1 logical 31945617408 physical 16913231872 device /dev/sde mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf 347034331d8879096e262a736fd02ba0 /tmp/mirror2 sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 1 -o /tmp/mirror1 /dev/sde; md5sum /tmp/mirror1 mirror 1 logical 31945617408 physical 16913231872 device /dev/sde mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf bcce5ab8cc65c948650376e936952547 /tmp/mirror1 sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 2 -o /tmp/mirror2 /dev/sde; md5sum /tmp/mirror2 mirror 1 logical 31945617408 physical 16913231872 device /dev/sde mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf 347034331d8879096e262a736fd02ba0 /tmp/mirror2 Does that mean that the problem is not in FS and I need to check discs, IO controller, memory. The same result with mount FS, but when I unmounted it there was a bug: BUG: unable to handle kernel paging request at 6b6b6c9f IP: [<f9a3c696>] caching_kthread+0x286/0x410 [btrfs] *pde = 00000000 Oops: 0002 [#1] last sysfs file: /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed Modules linked in: hwmon_vid btrfs zlib_deflate zlib_inflate sata_sil i2c_nforce2 Pid: 15258, comm: btrfs-cache-311 Not tainted (2.6.32-rc6-00346-gaa021ba #4) EIP: 0060:[<f9a3c696>] EFLAGS: 00010286 CPU: 0 EIP is at caching_kthread+0x286/0x410 [btrfs] EAX: 6b6b6b6b EBX: f6f9e098 ECX: 0000000e EDX: effc5f00 ESI: f4d95ab0 EDI: f26f4000 EBP: e84e7f98 ESP: e84e7f20 DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 Process btrfs-cache-311 (pid: 15258, ti=e84e6000 task=f6450000 task.ti=e84e6000) Stack: 00000000 ffffffff 81c00000 00000007 4324c000 effc5508 effc54b8 f26f5bd0 <0> f62c20c0 000b6000 00000000 effc54b0 ffffffff ffffffff 00000032 e7b14678 <0> f26f4000 f4d95ab0 c0007f98 00074324 00a80000 00000010 00000000 074324d0 Call Trace: [<f9a3c410>] ? caching_kthread+0x0/0x410 [btrfs] [<c103ce4c>] ? kthread+0x6c/0x80 [<c103cde0>] ? kthread+0x0/0x80 [<c1003513>] ? kernel_thread_helper+0x7/0x14 Code: c7 8b 45 9c b9 01 00 00 00 ba 03 00 00 00 c7 04 24 00 00 00 00 e8 fb 4d 5e c7 8b 45 b4 e8 53 d5 ff ff 8b 75 cc 8b 86 a0 00 00 00 <ff> 88 34 01 00 00 31 c0 83 c4 6c 5b 5e 5f c9 c3 66 90 77 09 3b EIP: [<f9a3c696>] caching_kthread+0x286/0x410 [btrfs] SS:ESP 0068:e84e7f20 CR2: 000000006b6b6c9f ---[ end trace 72d6516d4c697ae6 ]--- device: 'btrfs-4': device_unregister device: 'btrfs-4': device_create_release -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
