On 2018年06月14日 22:23, Daniel Underwood wrote: >>> Your very first task right now is to mount ro, and update your >>> backups. Don't do anything else until you've done that. It's a >>> testimony to Btrfs that this file system mounts at all, even ro, so >>> take advantage of this fact before you can't mount it anymore. > > Backups are in place for the important parts, though I'd prefer not to > use them if possible. > > For btrfs-progs, I am using 4.16.1 installed from > https://github.com/kdave/btrfs-progs. > > Regarding em5, it had errors when I initially added it to the array > that were due to a faulty SAS card. The card was replaced and I > haven't seen errors since until this popped up. Regarding what you > said about my dmesg, the numbers are actually still the same (bdev > /dev/mapper/em5 errs: wr 164286, rd 3444262, flush 2110, corrupt 3, > gen 181) after doing a backup and reboot. I would think they would > have changed unless btrfs is just ignoring that disk now. Just to make > sure, I've checked to make sure there weren't any loose connections. > The logs for check (with and without lowmem mode), `btrfs fi us`, and > `smartctl -x` are attached. The checks are only for em5, but I can do > other disks if necessary (it's a 6-disk raid10-style setup). Note that > there are a lot of errors on the SMART info, but they seem to be back > from when I was having hardware issues. From the output, specially the lowmem mode output since original mode handles extent tree corruption poorly and aborted, it's your extent tree corrupted and causing the bug. Thus, you should be able to mount the fs RO and copy all the data back without much hassle. Just need to pay attention for csum error. And considering how many extent tree corruption, I don't think it's a good idea to manually fix the fs. The last chance is, to try --repair --init-extent-tree as your last chance, if you still want to salvage the filesystem. The lowmem mode shows no extra bug, thus it's possible for --init-extent-tree to re-init extent tree and save the day. But personally speaking I'm not fully confident of the operation, thus it may fail and you may need to use the backup. BTW, even --init-extent-tree succeeded, you may still need to run btrfs check again to check if all the bugs are fixed. But at least from the lowmem output, the remaining errors are all fixable. Thanks, Qu > > --- > > On Thu, Jun 7, 2018 at 4:50 PM, Chris Murphy <lists@xxxxxxxxxxxxxxxxx> wrote: >> On Thu, Jun 7, 2018 at 2:38 PM, Chris Murphy <lists@xxxxxxxxxxxxxxxxx> wrote: >> >> >>> Your very first task right now is to mount ro, and update your >>> backups. Don't do anything else until you've done that. It's a >>> testimony to Btrfs that this file system mounts at all, even ro, so >>> take advantage of this fact before you can't mount it anymore. >> >> After you've done the backup, you need to find out why one of these >> devices is being so unreliable. That has to be fixed first. You can >> recreate a new Btrfs or some other file system, and you'll just run >> into the exact same problem down the road. Next, it might be useful to >> see the output from btrfs-progs 4.16.1 'btrfs check' and 'btrfs check >> --mode=lowmem' both of which are slow, the second one is really slow >> but is a different implementation so it's helpful to see both outputs. >> That's safe as long as you do not use --repair. >> >> Also we need to see the output from 'btrfs fi us <mountpoint>' with >> the volume mounted (ro). Off hand I think the most likely outcome is >> that you get a backup from the ro mounted file system, and you'll have >> to recreate it from scratch and restore from backups. In other words, >> no matter what you need a current backup. >> >> -- >> Chris Murphy > > >
Attachment:
signature.asc
Description: OpenPGP digital signature
