In summary: Looks like minimal damage remains and yet I'm still suffering "Input/output error" from btrfs and btrfsck appears to have looped... A diff check suggests the damage to be in one (heavily linked to) tree of a few MBytes. Would a scrub clear out the damaged trees? Worth debugging? Thanks, Martin Further detail: On 07/10/13 20:03, Chris Murphy wrote: > > On Oct 7, 2013, at 8:56 AM, Martin <m_btrfs@xxxxxxxxx> wrote: > >> >> Or try "mount -o recovery,noatime" again? > > Because of this: free space inode generation (0) did not match free > space cache generation (1607) > > Try mount option clear_cache. You could then use iotop to make sure > the btrfs-freespace process becomes inactive before unmounting the > file system; I don't think you need to wait in order to use the file > system, nor do you need to unmount then remount without the option. > But if it works, it should only be needed once, not as a persistent > mount option. Thanks for that. So, trying: mount -v -t btrfs -o recovery,noatime,clear_cache /dev/sdc gave: kernel: device label bu_A devid 1 transid 17448 /dev/sdc kernel: btrfs: enabling inode map caching kernel: btrfs: enabling auto recovery kernel: btrfs: force clearing of disk cache kernel: btrfs: disk space caching is enabled kernel: btrfs: bdev /dev/sdc errs: wr 0, rd 27, flush 0, corrupt 0, gen 0 btrfs-freespace appeared occasionally briefly in atop but there's no noticeable disk activity. All very rapidly done? Running a diff check to see if all ok and what might be missing gave the syslog output: kernel: verify_parent_transid: 165 callbacks suppressed kernel: parent transid verify failed on 915444506624 wanted 16974 found 13021 kernel: parent transid verify failed on 915444506624 wanted 16974 found 13021 kernel: parent transid verify failed on 915444506624 wanted 16974 found 13021 kernel: parent transid verify failed on 915444506624 wanted 16974 found 13021 kernel: parent transid verify failed on 915444506624 wanted 16974 found 13021 kernel: parent transid verify failed on 915444506624 wanted 16974 found 13021 The diff eventually failed with "Input/output error". 'mv' to move this failed directory tree out of the way worked. Attempting to use 'ln -s' gave the attached syslog output and the filesystem was made "Read-only". Remounting: mount -v -o remount,recovery,noatime,clear_cache,rw /dev/sdc and the mv looks fine. Trying the 'ln -s' again gives: ln: creating symbolic link `./portage': Read-only file system unmounting gave the syslog message: kernel: btrfs: commit super ret -30 Mounting again: mount -v -t btrfs -o recovery,noatime,clear_cache /dev/sdc showed that the symbolic link was put in place ok. Rerunning the diff check eventually found another "Input/output error". So unmounted and tried again: btrfsck --repair --init-extent-tree /dev/sdc Failed with: btrfs unable to find ref byte nr 911367733248 parent 0 root 1 owner 2 offset 0 btrfs unable to find ref byte nr 911367737344 parent 0 root 1 owner 1 offset 1 btrfs unable to find ref byte nr 911367741440 parent 0 root 1 owner 0 offset 1 leaf free space ret -297791851, leaf data size 3995, used 297795846 nritems 2 checking extents btrfsck: extent_io.c:606: free_extent_buffer: Assertion `!(eb->refs < 0)' failed. enabling repair mode Checking filesystem on /dev/sdc UUID: 38a60270-f9c6-4ed4-8421-4bf1253ae0b3 Creating a new extent tree Failed to find [911367733248, 168, 4096] Failed to find [911367737344, 168, 4096] Failed to find [911367741440, 168, 4096] Rerunning again and this time btrfsck is sat there at 100% CPU for the last 24 hours. Full output so far is: parent transid verify failed on 911904604160 wanted 17448 found 17449 parent transid verify failed on 911904604160 wanted 17448 found 17449 parent transid verify failed on 911904604160 wanted 17448 found 17449 parent transid verify failed on 911904604160 wanted 17448 found 17449 Ignoring transid failure Nothing syslog and no disk activity. Looped?... >> Or is it dead? >> >> (The 1.5TB of backup data is replicated elsewhere but it would be >> good to rescue this version rather than completely redo from >> scratch. Especially so for the sake of just a few MBytes of one >> corrupt directory tree.) > > Right. If you snapshot the subvolume containing the corrupt portion > of the file system, the snapshot probably inherits that corruption. > But if you write to only one of them, if those writes make the > problem worse, should be isolated only to the one you write to. I > might avoid writing to it, honestly. To save time, get increasingly > aggressive to get data out of this directory and once you succeed, > blow away the file system and start from scratch. > > You could also then try kernel 3.12 rc4, as there are some btrfs bug > fixes I'm seeing in there also, but I don't know if any of them will > help your case. If you try it, mount normally, then try to get your > data. If that doesn't work, try the recovery option. Maybe you'll get > different results. As suspected, thanks. Would a scrub clear out the damaged trees? Anything useful to try? Any debug value in looking at the fail cases? Is there a btrfsck mode of making good everything that is certain and dumping any remaining fragments into "lost + found"? (Or is that way down the developments yet?) Aside: btrfs looks to be usable enough, especially so with the disk format now stable, to at least offer the well established features as 'stable'...? (This is the first fail I've had, and considering the sata failed, is no surprise... Too severe a test! But can the limited damage be recovered...?) Thanks, Martin -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
