Re: [REGRESSION] Hang during backup with rsync

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 2015-04-30 21:48, Duncan wrote:
Martin Steigerwald posted on Thu, 30 Apr 2015 19:29:57 +0200 as excerpted:

The hang was: Mouse pointer in KDE not movable anymore, Ctrl-Alt-F1 had
no effect. I waited for a minute at least. Maybe it would have reacted
after a longer time, but I wanted my machine back. Disks where idle, if
I remember correctly. After reboot both filesystems mount okay.

This response is in regard to what to do at an apparent hang, and has
nothing directly to do with btrfs...

Two comments:

1) Depending on your graphics hardware and driver config, a modern
"KMS" (kernel modesetting) setup is more likely to "soft" hang in X mode
and not switch back to text mode, even when the system is otherwise not
hung and a VT switch would have worked fine pre-KMS-era.

While I'm no kernel or graphics expert, the problem from here /seems/ to
be that a modern KMS kernel generally uses high-res framebuffer mode at
the CLI as well, and because the basic kernel handling is unified
framebuffer and kernel-mode-switching for both X and CLI modes, switching
from X to CLI doesn't involve switching to the entirely separate VGA mode
driver and with it the forced hardware reset that it used to.  Without
that driver switch and forced reset, even if the switch actually occurs
successfully in terms of what you might type, what is actually displayed
may remain frozen, such that if you only have a local session, you
generally have to reboot anyway, but if you already have a CLI login
going in the VT you tried to switch to or can login blind, sometimes you
can at least manage a controlled reboot, by doing an init 6 or systemctl
reboot or whatever, even if the display is frozen and shows nothing.  Of
course it doesn't always work, but given the chance to avoid an unclean
shutdown, try it and see.

So no response at an attempted VT switch (your ctrl-alt-F1) doesn't mean
what it used to...

Something else to try in this case is Ctrl-Alt-Backspace, (Most linux distros have that configured to outright kill X running on the current VT) followed by Ctrl-Alt-Delete, which defaults on all modern distros to be equivalent to running 'shutdown -r now' from a root shell. Also, you may try Alt-SysRq-V, which is supposed to 'restore the framebuffer console' (except on ARM systems, where it is used to dump the contents of hardware tracing modules).
2) Along the same lines, there's the kernel's magic-sysrequest (sysrq/srq)
functionality.  Assuming you have it enabled in your kernel, you can try
a series of alt-sysrq-key sequences and very possibly use that to avoid
an entirely uncontrolled shutdown, even when major functionality upto and
including all of userspace is non-functional.

There's enough explanations written and googlable on the subject that
I'll avoid a full explanation here, but the main point I have to make is
that in addition to often allowing a semi-controlled shutdown/reboot, by
using the keys in the prescribed sequence and noting at which point (if
any) you actually get a response, you get at least some indication of how
badly your system was actually locked up.
So, this is great advice in theory, except that a large majority of distributions targeted at enterprise level usage (Fedora, Ubuntu, RHEL, SuSE, CentOS, etc) have this functionality disabled at runtime by default 'for security' (which is BS, because if an attacker has the kind of access required to use SysRq, then he has sufficient access to be able to bring the system to it's knees through other methods as well).
What I'd try first, right after the VT switch didn't work, is alt-srq-k.
Called the secure-term sequence as it can be used to help avoid suspected
keyloggers of certain (but not all) types, this tells the kernel to force-
kill anything running on your current VT and reset it.  This can be used
to kill an unresponsive X, for instance, and normally you'll get
automatically switched to a CLI login, either due to automatic switching
back to a previous VT (in the case of X on its own VT), or to automatic
respawning of the login after the kernel kills it along with whatever
else you were doing if you were already at the CLI.

This alt-srq-k sequence is thus a good first fallback if ctrl-alt-Fx
appears to do nothing, since it apparently forces the VT reset that
switching to a VGAmode CLI used to, that switching to a KMS mode CLI
doesn't.

If that doesn't work, it's time for the usual REISUB sequence,

* alt-srq-r (unraw the input, take out of X mode)

* alt-srq-e (tErminate, aka SIGTERM, all of userspace, allowing anything
still alive to terminate gracefully if it can)

* alt-srq-i (kIll, aka SIGKILL, all userspace, forcefully killing
anything that ignored the SIGTERM but still allowing the kernel to do
normal cleanup if it can)

(Tho from my own experience, if the K and R sequences don't help, then
the E and I sequences aren't likely to do much either, as they're
probably locked up bad enough that nothing will be gained, but OTOH,
nothing is lost by trying them, either.)

I've found that sometimes Alt-SysRq-J is needed at this point in the sequence to get things to correctly write data out (it resumes I/O to filesystems that have been frozen with the fsfreeze command or ioctl), and it has no negative impact if not needed, so it's generally a good idea to just use it anyway.
* alt-srq-s (Sync, force an emergency sync to storage of anything still
write-cached)

alt-srq-s can be used at any time, without disrupting normal operation
except for any I/O triggered by the forced sync.  I've come to use it
regularly immediately before I do anything that I think /might/ trigger
system instability, so everything's synced before I try it, just in
case.  Think of this as a forced version of the sync command.

* alt-srq-u (remoUnt read-only, forcing all still functional filesystems
read-only)

The S and U steps are critical to a semi-controlled shutdown, and where
they work, can often mean the difference between a filesystem with no
errors on reboot as the kernel saved and cleanly mounted read-only to the
extent it could, and various filesystem corruptions, if these steps
weren't done or if the kernel was badly enough corrupted it was afraid to
write anything lest it make the problem worse.

* alt-srq-b (reBoot, force a reboot without any further cleanup).


Secondarily, for the sake of completeness, you can also use Alt-SysRq-o in place of Alt-SysRq-b to (try) to get the system to power off instead of reboot. Also, it's significant to note that the exact keys used vary depending on the keymap loaded in the kernel, on a Dvorak keyboard for example, the sequence is instead P.COGX (which IIRC is the same sequence of scancodes as on QWERTY keyboards). Furthermore, if you were running under X, you may need to add the Ctrl key to the combination to get it properly acknowledged.

It's worth mentioning also that many laptop keyboards (and some other modern keyboards) need you to use Ctrl-Alt-PrintScreen (or even some other odd key combination, for example the Dell laptop that I'm typing this on needs Fn-Alt-Home) instead of Alt-SysRq


Attachment: smime.p7s
Description: S/MIME Cryptographic Signature


[Index of Archives]     [Linux Filesystem Development]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux