On 2017-08-14 11:13, Graham Cobb wrote:
On 14/08/17 15:23, Austin S. Hemmelgarn wrote:
Assume you have higher level verification.
But almost no applications do. In real life, the decision
making/correction process will be manual and labour-intensive (for
example, running fsck on a virtual disk or restoring a file from backup).
Quite a few applications actually _do_ have some degree of secondary
verification or protection from a crash. Go look at almost any database
software. It usually will not have checksumming, but it will almost
always have support for a journal, which is enough to cover the
particular data loss scenario we're talking about (unexpected unclean
shutdown).
Would you rather not be able
to read the data regardless of if it's correct or not, or be able to
read it and determine yourself if it's correct or not?
It must be controllable on a per-file basis, of course. For the tiny
number of files where the app can both spot the problem and correct it
(for example if it has a journal) the current behaviour could be used.
In my own experience, the things that use nodatacow fall into one of 4
categories:
1. Cases where the data is non-critical, and data loss will be
inconvenient but not fatal. Systemd journal files are a good example of
this, as are web browser profiles when you're using profile sync.
2. Cases where the upper level can reasonably be expected to have some
degree of handling, even if it's not correction. VM disk images and
most database applications fall into this category.
3. Cases where data corruption will take out the application anyway.
Poorly written database software is the primary example of this.
4. Things that shouldn't be using nodatacow because data safety is the
most important aspect of the system.
The first two cases work perfectly fine with the current behavior and
are arguably no better off either way. The third is functionally fine
with the current behavior provided that the crash doesn't change state
(which isn't a guarantee), but could theoretically benefit from the
determinism of knowing the app will die if the data is bad. The fourth
is what most people seem to want this for, and don't realize that even
if this is implemented, they will be no better off on average.
But, on MY system, I absolutely would **always** select the first option
(-EIO). I need to know that a potential problem may have occurred and
will take manual action to decide what to do. Of course, this also needs
a special utility (as Christoph proposed) to be able to force the read
(to allow me to examine the data) and to be able to reset the checksum
(although that is presumably as simple as rewriting the data).
And I and most other sysadmins I know would prefer the opposite with the
addition of a secondary notification method. You can still hook the
notification to stop the application, but you don't have to if you don't
want to (and in cases 1 and 2 I listed above, you probably don't want to).
This is what happens normally with any filesystem when a disk block goes
bad, but with the additional benefit of being able to examine a
"possibly valid" version of the data block before overwriting it.
Looking at this from a different angle: Without background, what would
you assume the behavior to be for this? For most people, the assumption
would be that this provides the same degree of data safety that the
checksums do when the data is CoW.
Exactly. The naive expectation is that turning off datacow does not
prevent the bitrot checking from working. Also, the naive expectation
(for any filesystem operation) is that if there is any doubt about the
reliability of the data, the error is reported for the user to deal with.
The problem is that the naive expectation about data safety appears to
be that adding checksumming support for nodatacow will improve safety,
which it WILL NOT do. All it will do is add some reporting that will
have a 50%+ rate of false positives (there is the very real possibility
that the unexpected power loss will corrupt the checksum or the data if
you're on anything but a traditional hard drive). If you have something
that you need data safety for and can't be arsed to pay attention to
whether or not your system had an unclean shutdwon, then you have two
practical options:
1. Don't use nodatacow.
2. Do some form of higher level verification.
Nothing about that is going to magically change because you suddenly
have checksums telling you the data might be bad.
Now, you _might_ be better off in a situation where the data got
corrupted for some other reason (say, a media error for example), but
even then you should have higher level verification, and it won't
provide much benefit unless you're using replication or parity in BTRFS.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
