Re: Chicken-egg: uncorrectable checksum error prevents RAID1 rebalancing

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Chris Murphy posted on Sun, 24 Jan 2016 15:45:49 -0700 as excerpted:

> Maybe it's an orphaned item. I don't know much about those, whether
> btrfs check finds or removes them. But you can safely use
> btrfs-debug-tree while the fs is mounted.
> 
> btrfs-debug-tree <dev> | grep -A3 -B3 ORPHAN
> 
> That'd find object type ORPHAN and item type ORPHAN_ITEM. Again, no idea
> what to do about it if either are found, and matches inode 515.
> You could also do a search using
> 
> btrfs-debug-tree <dev> | grep -A3 -B3 "(515 "

Orphans, in POSIX filesystem context, are files that are open, but 
deleted.  Think shared object libraries (*.so*) that are loaded by a 
running program at the time they're updated, in which context it's a 
security issue as well since some of those updates might have been 
security related and existing processes continue to use the old, 
vulnerable versions until they are restarted.  These files can be 
logically deleted so nothing else can access them, but the filesystem 
can't remove all references to them until all running programs holding 
them open have closed them.  That's what makes them orphan -- nothing new 
can access them, but existing references mean they can't yet be fully 
removed from the filesystem, either.

Typically such orphans block unmounting or remounting the filesystem 
readonly (unless it's forced read-only by filesystem error or kernel 
emergency-SRQ sequence, etc, which override things and locks orphans in 
their existing state on the filesystem, just as a crash would, except it 
doesn't necessarily immediately crash, tho a livelock often follows 
pretty quickly).

There are various tools available that can find these files and tell you 
which processes are holding them open and/or kill or restart those 
processes in ordered to release the locks and close the files, allowing 
them to be fully deleted.  I routinely use one after updates here called 
lib_users, which simply tells me what processes are holding open these 
deleted files, so I can decide whether I want to restart them, kill them, 
or simply ignore the problem until later.  

In a system reboot (or simple descent to single user or systemd emergency 
mode to stop most services, systemd itself can then be reexecuted using 
systemctl daemon-reexec, if it's holding open deleted files as well, 
before returning to normal mode), normal processes and system services 
are stopped, releasing these files, so the filesystem can clean them up 
before unmount or remount-readonly.

(Here, my root, including /usr, remains mounted read-only by default, 
only being mounted writable for updates.  Thus my routine use of 
lib_users to find services I need to restart, or if there's too many, 
decide I want to temporarily go to systemd emergency mode, before 
returning to normal mode, releasing all these deleted files in the 
process, so I can cleanly remount / read-only once again.  That of course 
is the reason I know a bit more about this than many, since it's part of 
my update routine now.)

Of course in an unclean shutdown situation, the filesystem will not have 
had the chance to clean up the orphans before the umount or remount-
readonly, so these orphans remain around at reboot and must be cleaned up 
then.  Btrfs at least, does this automatically, so btrfs check need not 
be run to do it, as the check has to be done on an unmounted filesystem, 
and at mount, btrfs will handle it, itself.  However, btrfs check may 
still clean them up as well (I'm not sure), simply to avoid other issues 
the orphan files might have that are easiest eliminated by simply 
eliminating the files, since that's what would happen on mount anyway.


If it's orphan files, just quitting X (or logging out and back in if you 
use a *DM graphical login, I don't, preferring to login at the CLI and 
run startx) tends to release a lot of them, no reboot needed.  But 
sometimes quitting X and restarting other services holding open deleted 
files may be necessary as well, and if there's enough such services, 
descending to emergency/single-user mode, and possibly restarting init/
systemd itself, if it is holding open any such files, may be necessary.  
But a full reboot isn't normally needed.

Tho of course restarting X, and descending to emergency/single-user-mode 
even more so, can in practice be as bad as having to reboot.  Going to 
emergency/single-user especially, pretty much all you save in practice is 
your uptime statistics and caches that would be lost on a full reboot.  
But if you're running over a year uptime (yeah, right, on a still not 
fully stable btrfs where the recommendation is to keep to reasonably 
current kernels, hardly the stuff of year-plus uptimes!), or have 
spinning rust and lots of RAM and thus cache that would have to be 
reloaded from the slow spinning rust, going single-user to avoid dropping 
them can be well worth the hassle over simply rebooting.


Meanwhile, if it /is/ an orphan file, it's an interesting situation, 
since that means the checksum was fine and btrfs was able to load the 
file when whatever is holding it open/orphan was started, but that isn't 
the case now.  Which means we have evidence of near-real-time (well, 
depending on how long the file has been open) block decay and checksum 
invalidation, which has serious implications in terms of storage device 
health.  Be sure to run smartctl -AH, or similar, to see what's going on, 
and track it closely for awhile, as you may well have a failing device!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Filesystem Development]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux