Am 15.01.19 um 23:24 schrieb Oliver Freyermuth: > Am 14.01.19 um 01:48 schrieb Oliver Freyermuth: >> Am 13.01.19 um 22:51 schrieb Oliver Freyermuth: >>> I just upgraded to 4.20.1 from 4.19 (not sure if related) and my btrfs backup volume entered read-only mode when running btrfs-cleaner, >>> i.e. when purging old subvolumes. >>> >>> I have attached the kernel log from when this happens. >>> >>> What is the best way to proceed from here? Running "btrfs check repair" on the device? >>> Worst case it's not a huge issue to lose the data stored there, it's my backup volume after all. >>> But it would be good to understand the cause and know if there is a better fix than starting from scratch. >> attached is the output of "btrfs check -p /dev/sdc2". >> I can't guarantee the volume has never been cleanly unmounted. >> >> I found several past occasions of this here: >> https://www.spinics.net/lists/linux-btrfs/msg69040.html >> and here: >> https://unix.stackexchange.com/questions/369133/dealing-with-btrfs-ref-backpointer-mismatches-backref-missing >> but without conclusive result. >> >> Please let me know what's the best way to proceed. From these links, it seems >> btrfs check --repair >> _should_ help, but I would prefer to get some advice first whether this is really the best approach. >> > I have now salvaged all my backup subvolumes with btrfs send (using btrbk archive) to a new btrfs partition. > Interestingly, when the old partition was mounted r/w initially and remounted r/o after the described issue was triggered by btrfs-cleaner: > > [34758.491644] BTRFS: error (device sdc2) in __btrfs_free_extent:6828: errno=-2 No such entry > [34758.491647] BTRFS info (device sdc2): forced readonly > [34758.491652] BTRFS: error (device sdc2) in btrfs_run_delayed_refs:2978: errno=-2 No such entry > > btrfs send appeared to fail on some subvolumes with: > > [41822.676040] BTRFS error (device sdc2): parent transid verify failed on 52633681920 wanted 88063 found 87999 > [41822.676260] BTRFS error (device sdc2): parent transid verify failed on 52633681920 wanted 88063 found 87999 > [41822.676266] BTRFS info (device sdc2): no csum found for inode 22175978 start 0 > [41822.683112] BTRFS warning (device sdc2): csum failed root 25758 ino 22175978 off 4427459514368 csum 0x5d3b8d26 expected csum 0x00000000 mirror 1 > > Unmounting and remounting the broken file system r/o, all visible subvolumes could be transferred without that issue. > I presume that there's also a bug when the automatic remount as r/o happens since csum 0x00000000 does not look correct. > > Since there's now nothing to lose and I received no other advice up to now, I'm running "btrfs check --repair" now just for the sake of learning > whether this appears to fix it. I'll shortly report back when that's done. > > If anybody can suggest a better solution in case this happens again (the issue appears to be wide-spread) I would be happy to learn. btrfs check --repair started to do it's thing - and died. Below is the log and trace in the hope that it may help to fix the BUG_ON. That's with btrfs-progs 4.19.1 on Kernel 4.20.1. I'll run repair again, but I guess the volume is hosed, broken somewhere in subvolume deletion. Still seems fine when mounting r/o, though. -------------------------------------------------------------------------------------- $ btrfs check -p --repair /dev/sdc2 enabling repair mode Opening filesystem to check... Checking filesystem on /dev/sdc2 UUID: 3ded2960-989e-4890-9756-d6e60433e42f [1/7] checking root items (0:06:27 elapsed, 12335857 items checked) Fixed 0 roots. ref mismatch on [711065600 16384] extent item 0, found 1elapsed, 1184265 items checked) tree backref 711065600 parent 18178 root 18178 not found in extent tree backpointer mismatch on [711065600 16384] adding new tree backref on start 711065600 len 16384 parent 0 root 18178 Repaired extent references for 711065600 ref mismatch on [928907264 16384] extent item 0, found 1 tree backref 928907264 parent 25744 root 25744 not found in extent tree backpointer mismatch on [928907264 16384] owner ref check failed [928907264 16384] repair deleting extent record: key [928907264,169,1] adding new tree backref on start 928907264 len 16384 parent 0 root 25744 Repaired extent references for 928907264 ref mismatch on [28052652032 16384] extent item 0, found 1 tree backref 28052652032 parent 18178 root 18178 not found in extent tree backpointer mismatch on [28052652032 16384] owner ref check failed [28052652032 16384] repair deleting extent record: key [28052652032,169,1] adding new tree backref on start 28052652032 len 16384 parent 0 root 18178 Repaired extent references for 28052652032 ref mismatch on [28088516608 16384] extent item 0, found 1 tree backref 28088516608 parent 18178 root 18178 not found in extent tree backpointer mismatch on [28088516608 16384] owner ref check failed [28088516608 16384] repair deleting extent record: key [28088516608,169,1] adding new tree backref on start 28088516608 len 16384 parent 0 root 18178 Repaired extent references for 28088516608 ref mismatch on [52375928832 16384] extent item 0, found 1 tree backref 52375928832 parent 18178 root 18178 not found in extent tree backpointer mismatch on [52375928832 16384] adding new tree backref on start 52375928832 len 16384 parent 0 root 18178 Repaired extent references for 52375928832 ref mismatch on [185114099712 16384] extent item 0, found 1 tree backref 185114099712 parent 18178 root 18178 not found in extent tree backpointer mismatch on [185114099712 16384] adding new tree backref on start 185114099712 len 16384 parent 0 root 18178 Repaired extent references for 185114099712 ref mismatch on [283321597952 16384] extent item 0, found 1 tree backref 283321597952 parent 18178 root 18178 not found in extent tree backpointer mismatch on [283321597952 16384] owner ref check failed [283321597952 16384] repair deleting extent record: key [283321597952,169,1] adding new tree backref on start 283321597952 len 16384 parent 0 root 18178 Repaired extent references for 283321597952 ref mismatch on [419430154240 16384] extent item 0, found 1 tree backref 419430154240 parent 18178 root 18178 not found in extent tree backpointer mismatch on [419430154240 16384] owner ref check failed [419430154240 16384] repair deleting extent record: key [419430154240,169,1] adding new tree backref on start 419430154240 len 16384 parent 0 root 18178 Repaired extent references for 419430154240 ref mismatch on [419638804480 16384] extent item 0, found 1 tree backref 419638804480 parent 18178 root 18178 not found in extent tree backpointer mismatch on [419638804480 16384] owner ref check failed [419638804480 16384] repair deleting extent record: key [419638804480,169,1] adding new tree backref on start 419638804480 len 16384 parent 0 root 18178 Failed to find [52107100160, 168, 16384] btrfs unable to find ref byte nr 52107116544 parent 0 root 2 owner 1 offset 0 transaction.c:168: btrfs_commit_transaction: BUG_ON `ret` triggered, value -5 btrfs(+0x507f9)[0x55ec25cf97f9] btrfs(btrfs_commit_transaction+0x193)[0x55ec25cf9dd3] btrfs(+0x1d74e)[0x55ec25cc674e] btrfs(cmd_check+0x1104)[0x55ec25d0f2e4] btrfs(main+0x82)[0x55ec25cc70f2] /lib64/libc.so.6(__libc_start_main+0xe7)[0x7f41b421dae7] btrfs(_start+0x2a)[0x55ec25cc72ca] --------------------------------------------------------------------------------------
