Re: Filesystem unable to recover from ENOSPC

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Apr 10, 2014 at 1:34 PM, Hugo Mills <hugo@xxxxxxxxxxxxx> wrote:
> On Thu, Apr 10, 2014 at 01:00:35PM -0700, Chip Turner wrote:
>> btrfs show:
>> Label: none  uuid: 04283a32-b388-480b-9949-686675fad7df
>> Total devices 1 FS bytes used 135.58GiB
>> devid    1 size 238.22GiB used 238.22GiB path /dev/sdb2
>>
>> btrfs fi df:
>> Data, single: total=234.21GiB, used=131.82GiB
>> System, single: total=4.00MiB, used=48.00KiB
>> Metadata, single: total=4.01GiB, used=3.76GiB
>
>    I'm surprised it's managed to use that much of the metadata
> allocation. The FS usually hits this problem with a much smaller
> used-to-total ratio.

It perhaps was related to how I build it -- basically I dumped many
files (around 8.5 million, 95% of which are hardlinks rather than
actual files) onto it at once, created a snapshot, and then began
deleting files.  The btrfs filesystem they were coming from was fine
with the files and its own original load, but something about the
snapshotting and deleting seemed to push it over the edge.

The issue happened over a couple of days so I don't recall exactly,
but I may also have run out of space and had to delete a lot of files
(basically, I left off the '-H' to rsync, so, when I copied to the
system, it made copies rather than hard links).  This occurred before
the original full copy and hence before the snapshot and deletion.

>    One thing you could do is btrfs dev add a small new device to the
> filesystem (say, a USB stick, or a 4 GiB loopback file mounted over
> NBD or something). Then run the filtered balance. Then btrfs dev del
> the spare device.

Ah, this worked great.  It fixed it in about ten seconds.

I'm curious about the space report; why doesn't Data+System+Metadata
add up to the total space used on the device?  Was the fs stuck in a
state where it couldn't clean up because it couldn't write more
metadata (and hence adding a few gb allowed it to balance)?  After the
balance, the used space dropped to around 150GB, roughly what I'd
expect.

>    The fact that this FS has ended up in this state should be
> considered a bug. It used to be quite common, then josef fixed a load
> of problems, and it's been rare for about a year. Only recently we've
> been seeing more of this kind of problem, and I think there's been a
> bit of a regression somewhere.

Is there any other data I can provide to help fix this?  I can see if
it's reproducible (I'm migrating disks so I can just keep doing it and
see if it happens again).

Chip

-- 
Chip Turner - cturner@xxxxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Filesystem Development]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux