On Mon, Jan 11, 2016 at 12:47 AM, cheater00 . <cheater00@xxxxxxxxx> wrote: > On Sun, Jan 10, 2016 at 3:14 PM, Henk Slager <eye1tm@xxxxxxxxx> wrote: >> On Sat, Jan 9, 2016 at 9:26 PM, Hugo Mills <hugo@xxxxxxxxxxxxx> wrote: >>> On Sat, Jan 09, 2016 at 09:00:47PM +0100, cheater00 . wrote: >>>> Hello, >>>> I can repeatedly trigger this bug by making the "data" portion fill >>>> up. If you remember the partition is 6 TB but in btrfs filesystem df >>>> Data is shown as only 2TB when in fact it should be nearly 6TB. So >>>> this has nothing to do with kernel bugs. The filesystem on disk is >>>> structured incorrectly. How do i fix this? How do I make "Data" >>>> bigger? What is it exactly? >>> >>> This is *exactly* the behaviour of the known kernel bug. The bug is >>> that the FS *should* be extending the data allocation when it gets >>> near to full, and it's not. There is no way of manually allocating >>> more (because the FS should be doing it automatically). There is no >>> known way of persuading the FS to it when it isn't. >> >> Probably this is 'the' bug we talk about: >> https://bugzilla.kernel.org/show_bug.cgi?id=74101 > > Yes, would seem like it. > >> Size of the fs is much smaller, but also problem occurs when fill-level is <50% >> >> btrfs fs resize did nothing you mention, but AFAIK you should see >> something in dmesg when you do that. > > > I remounted (to make the filesystem rw again) and got the attached > log. There are some errors in there. But bear in mind that this disk > has been hanging on in ro mode for a few days now. > > I did this: > # btrfs fi resize -1T Media > Resize 'Media' of '-1T' > # btrfs filesystem resize -1G Media > Resize 'Media' of '-1G' > # btrfs filesystem resize -1T Media > Resize 'Media' of '-1T' > # btrfs filesystem resize max Media > Resize 'Media' of 'max' > > and it resulted in the following lines in dmesg: > [189115.919160] BTRFS: new size for /dev/sdc1 is 4901661835264 > [189177.306291] BTRFS: new size for /dev/sdc1 is 4900588093440 > [189181.950289] BTRFS: new size for /dev/sdc1 is 3801076465664 > [189232.064357] BTRFS: new size for /dev/sdc1 is 6001173463040 > > (note the device changed from sdd to sdc when I rebooted last) > >> And what is the output of gdisk -l /dev/sdd , just to check? > > (the device changed to sdc since I've rebooted) > > # gdisk -l /dev/sdc > GPT fdisk (gdisk) version 0.8.8 > > Partition table scan: > MBR: protective > BSD: not present > APM: not present > GPT: present > > Found valid GPT with protective MBR; using GPT. > Disk /dev/sdc: 11721045168 sectors, 5.5 TiB > Logical sector size: 512 bytes > Disk identifier (GUID): 0DEF5509-8730-4AB4-A846-79DA3C376F66 > Partition table holds up to 128 entries > First usable sector is 34, last usable sector is 11721045134 > Partitions will be aligned on 2048-sector boundaries > Total free space is 3181 sectors (1.6 MiB) > > Number Start (sector) End (sector) Size Code Name > 1 2048 11721043967 5.5 TiB 8300 > > >> Have you had the fs already filled up to e.g. 95% before or has is >> always been not more than 2TiB? > > It has never been more than 2TB, I've had it for quite some time now > but it's always hovered around 1TB. > >> Why are the single (empty )profiles for metadata and system still >> there? They should have been removed already by the various balancing >> operations that are advised in the btrfs-wiki. > > Not sure. Would this happen automatically? Or is this something I > should have done? > I have another fs on the same model/size disk, which isn't exhibiting > this bug, and it has those profiles as well. There is nothing really wrong (just by quickly browsing the numbers) as far as I can see. So it seems the 'btrfs space allocation mechanism' somehow gives up under certain circumstances. The fs was created by bit older tools/kernel, that is why these dummy single chunks are there. They should not harm, but at least a full balance will remove them. Also a parttial balance -musage=0 with latest tools should also remove them, i don't remember exactly. If they are not needed, keep the fs as simple as possible and remove them I would say. I was just wondering what balancing options you have tried. > Here's what both look like: > > buggy fs ("Media"): > Data, single: total=1.98TiB, used=1.98TiB > System, DUP: total=8.00MiB, used=240.00KiB > System, single: total=4.00MiB, used=0.00B > Metadata, DUP: total=5.50GiB, used=3.49GiB > Metadata, single: total=8.00MiB, used=0.00B > GlobalReserve, single: total=512.00MiB, used=0.00B > > non-buggy fs: > Data, single: total=5.43TiB, used=5.40TiB > System, DUP: total=8.00MiB, used=608.00KiB > System, single: total=4.00MiB, used=0.00B > Metadata, DUP: total=12.50GiB, used=11.21GiB > Metadata, single: total=8.00MiB, used=0.00B > GlobalReserve, single: total=512.00MiB, used=0.00B > > >> What is the output of btrfs check /dev/sdd ? The usb resets >> mentioned might have introduced some errors to the fs (that is what I >> have experienced at least, but it depends on timing etc) > > I'll run that overnight and report tomorrow. > >> What you could try is to create an image+'copy' of the fs with >> btrfs-image just after you get ENOSPC abd then do various tests with >> that (make sure unmount or even better unplug the physical hdd!). Like >> mounting and then try to add a file, convert all metadata + system >> from dup to single and then try to add a file. It all doesn't give >> real space, but it might give hints to what could be wrong. > > I can't do that because I would have to buy an extra disk which is 300 euro. btrfs-image only images/dumps the metadata. In your case it is ~6GiB. # umount /dev/sdc1 # cd /sparse-file-featured-fs-mount/ # btrfs-image /dev/sdc1 Mediafs.metadump # btrfs-image -r Mediafs.metadump Mediafs.img # losetup -f /Mediafs.img # mount /dev/loopX /mnt # dd if=/dev/zero of=/mnt/testfile bs=1M count=5000 The file Mediafs.img is initially using ~6GiB space, its reported size is same as real partition on the harddisk. The dd command increases it with 5GB and simulates a large file copy. So you need upto 10's of GBs freespace to do this tests, fast SSD preferred. You can also copy some real files and see if you hit ENOSPC. If the test is on btrfs, you can easily snapshot or copy--reflink the original Mediafs.img and try various metadata balancing. And use the btrfs compression. > >> If you somehow manage to reduce the fs by lets say 100G and also the >> partition, you could install or copy a newer linux+kernel to >> partition(s) in that 100G space and boot from there. > > let me try finding the latest kernel then. There are backports. As suggested earlier, try kernel 4.4 (released today). People have reported that they weren't bugged anymore by ENOSPC aborts after switching from some older kernel. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
