Hi,
I can't figure out were 3TB on a 36 TB BTRFS volume (on LVM) are gone !
I know BTRFS can be tricky when speaking about space usage when using many physical drives in a RAID setup, but my conf is a very simple BTRFS volume without RAID(single Data type) using the whole disk (perhaps did I do something wrong with the LVM setup ?).
My BTRFS volume is mounted on /RAID01/.
There's only one folder in /RAID01/ shared with Samba, Windows also see a total of 28 TB used.
It only contains 443 files (big backup files created by Veeam), most of the file size is greater than 1GB and be be up to 5TB.
######> du -hs /RAID01/
28T /RAID01/
If I sum up the result of : ######> find . -printf '%s\n'
I also find 28TB.
I extracted btrfs binary from rpm version v4.9.1 and used ######> btrfs fi du
on each file and the result is 28TB.
OS : CentOS Linux release 7.3.1611 (Core)
btrfs-progs v4.4.1
######> ssm list
-------------------------------------------------------------------------
Device Free Used Total Pool Mount point
-------------------------------------------------------------------------
/dev/sda 36.39 TB PARTITIONED
/dev/sda1 200.00 MB /boot/efi
/dev/sda2 1.00 GB /boot
/dev/sda3 0.00 KB 36.32 TB 36.32 TB lvm_pool
/dev/sda4 0.00 KB 54.00 GB 54.00 GB cl_xxx-xxxamrepo-01
-------------------------------------------------------------------------
-------------------------------------------------------------------
Pool Type Devices Free Used Total
-------------------------------------------------------------------
cl_xxx-xxxamrepo-01 lvm 1 0.00 KB 54.00 GB 54.00 GB
lvm_pool lvm 1 0.00 KB 36.32 TB 36.32 TB
btrfs_lvm_pool-lvol001 btrfs 1 4.84 TB 36.32 TB 36.32 TB
-------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------
Volume Pool Volume size FS FS size Free Type Mount point
---------------------------------------------------------------------------------------------------------------------
/dev/cl_xxx-xxxamrepo-01/root cl_xxx-xxxamrepo-01 50.00 GB xfs 49.97 GB 48.50 GB linear /
/dev/cl_xxx-xxxamrepo-01/swap cl_xxx-xxxamrepo-01 4.00 GB linear
/dev/lvm_pool/lvol001 lvm_pool 36.32 TB linear /RAID01
btrfs_lvm_pool-lvol001 btrfs_lvm_pool-lvol001 36.32 TB btrfs 36.32 TB 4.84 TB btrfs /RAID01
/dev/sda1 200.00 MB vfat part /boot/efi
/dev/sda2 1.00 GB xfs 1015.00 MB 882.54 MB part /boot
---------------------------------------------------------------------------------------------------------------------
######> btrfs fi sh
Label: none uuid: df7ce232-056a-4c27-bde4-6f785d5d9f68
Total devices 1 FS bytes used 31.48TiB
devid 1 size 36.32TiB used 31.66TiB path /dev/mapper/lvm_pool-lvol001
######> btrfs fi df /RAID01/
Data, single: total=31.58TiB, used=31.44TiB
System, DUP: total=8.00MiB, used=3.67MiB
Metadata, DUP: total=38.00GiB, used=35.37GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
I tried to repair it :
######> btrfs check --repair -p /dev/mapper/lvm_pool-lvol001
enabling repair mode
Checking filesystem on /dev/mapper/lvm_pool-lvol001
UUID: df7ce232-056a-4c27-bde4-6f785d5d9f68
checking extents
Fixed 0 roots.
cache and super generation don't match, space cache will be invalidated
checking fs roots
checking csums
checking root refs
found 34600611349019 bytes used err is 0
total csum bytes: 33752513152
total tree bytes: 38037848064
total fs tree bytes: 583942144
total extent tree bytes: 653754368
btree space waste bytes: 2197658704
file data blocks allocated: 183716661284864 ?? what's this ??
referenced 30095956975616 = 27.3 TB !!
Tried the "new usage" display but the problem is the same : 31 TB used but total file size is 28TB
Overall:
Device size: 36.32TiB
Device allocated: 31.65TiB
Device unallocated: 4.67TiB
Device missing: 0.00B
Used: 31.52TiB
Free (estimated): 4.80TiB (min: 2.46TiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 512.00MiB (used: 0.00B)
Data,single: Size:31.58TiB, Used:31.45TiB
/dev/mapper/lvm_pool-lvol001 31.58TiB
Metadata,DUP: Size:38.00GiB, Used:35.37GiB
/dev/mapper/lvm_pool-lvol001 76.00GiB
System,DUP: Size:8.00MiB, Used:3.69MiB
/dev/mapper/lvm_pool-lvol001 16.00MiB
Unallocated:
/dev/mapper/lvm_pool-lvol001 4.67TiB
The only btrfs tool speaking about 28TB is btrfs check (but I'm not sure if it's bytes because it speaks about "referenced blocks" and I don't understand the meaning of "file data blocks allocated")
Code:
file data blocks allocated: 183716661284864 ?? what's this ??
referenced 30095956975616 = 27.3 TB !!
I also used the verbose option of https://github.com/knorrie/btrfs-heatmap/ to sum up the total size of all DATA EXTENT and found 32TB.
I did scrub, balance up to -dusage=90 (and also dusage=0) and ended up with 32TB used.
No snasphots nor subvolumes nor TB hidden under the mount point after unmounting the BTRFS volume
What did I do wrong or am I missing ?
Thanks in advance.
Frederic Larive.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
