Hello btrfs-enthusiasts,
I have a rather big btrfs RAID6 with currently 12 devices. It used to be
only 8 drives 4TB each, but I successfully added 4 more drives with 1TB
each at some point. What I am trying to find out, and that's my main
reason for posting this, is how to balance the data on the drives now.
I am wondering what I should read from this "btrfs filesystem show" output:
--- cut ---
Total devices 12 FS bytes used 19.23TiB
devid 1 size 3.64TiB used 3.64TiB path /dev/sdc
devid 2 size 3.64TiB used 3.64TiB path /dev/sdd
devid 3 size 3.64TiB used 3.64TiB path /dev/sde
devid 4 size 3.64TiB used 3.64TiB path /dev/sdf
devid 5 size 3.64TiB used 3.64TiB path /dev/sdh
devid 6 size 3.64TiB used 3.64TiB path /dev/sdi
devid 7 size 3.64TiB used 3.64TiB path /dev/sdj
devid 8 size 3.64TiB used 3.64TiB path /dev/sdb
devid 9 size 931.00GiB used 535.48GiB path /dev/sdg
devid 10 size 931.00GiB used 535.48GiB path /dev/sdk
devid 11 size 931.00GiB used 535.48GiB path /dev/sdl
devid 12 size 931.00GiB used 535.48GiB path /dev/sdm
btrfs-progs v4.1.2
--- cut ---
First of all I wonder why the first 8 disks are shown as "full" as "used
= size", but there is 5.3TB of free space for the fs shown by "df":
--- cut ---
Filesystem Size Used Avail Use% Mounted on
/dev/sdc 33T 20T 5.3T 79% /somemountpointsomewhere
--- cut ---
Also "btrfs filesystem df" doesn't give me any clues on the matter:
--- cut ---
btrfs filesystem df /srv/mirror/
Data, single: total=8.00MiB, used=0.00B
Data, RAID6: total=22.85TiB, used=19.19TiB
System, single: total=4.00MiB, used=0.00B
System, RAID6: total=12.00MiB, used=1.34MiB
Metadata, single: total=8.00MiB, used=0.00B
Metadata, RAID6: total=42.09GiB, used=38.42GiB
GlobalReserve, single: total=512.00MiB, used=1.58MiB
--- cut ---
What I am very certain about is that the "load" of I/O requests is not
equal yet, as iostat clearly shows:
--- cut ---
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s
avgrq-sz avgqu-sz await r_await w_await svctm %util
sdc 21.40 4.41 42.22 12.71 3626.12 940.79
166.29 3.82 69.38 42.83 157.60 5.98 32.82
sdb 22.35 4.45 41.29 12.71 3624.20 941.27
169.09 4.22 77.88 46.75 178.97 6.10 32.96
sdd 22.03 4.44 41.60 12.73 3623.76 943.22
168.13 3.79 69.45 42.53 157.48 6.05 32.85
sde 21.21 4.43 42.30 12.74 3621.39 943.36
165.88 3.82 69.28 42.99 156.62 5.98 32.90
sdf 22.19 4.42 41.42 12.75 3623.65 940.63
168.51 3.77 69.36 42.64 156.13 6.05 32.79
sdh 21.35 4.46 42.25 12.68 3623.12 940.28
166.14 3.95 71.72 43.61 165.40 6.02 33.06
sdi 21.92 4.38 41.67 12.79 3622.03 942.91
167.63 3.49 63.83 40.23 140.74 6.02 32.77
sdj 21.31 4.41 42.26 12.72 3625.32 941.50
166.12 3.99 72.25 44.50 164.44 6.00 33.01
sdg 8.90 4.97 12.53 21.16 1284.47 1630.08
173.02 0.83 24.61 27.31 23.02 1.77 5.95
sdk 9.14 4.94 12.30 21.19 1284.61 1630.02
174.07 0.79 23.41 26.59 21.57 1.76 5.91
sdl 8.88 4.95 12.58 21.19 1284.46 1630.06
172.62 0.80 23.80 25.68 22.68 1.78 6.00
sdm 9.07 4.85 12.35 21.29 1284.43 1630.01
173.26 0.79 23.57 26.57 21.83 1.77 5.94
--- cut ---
Should I run btrfs balance on the filesystem? If so, what FILTERS would
I then use in order for the data and therefore requests to be better
distributed?
With regards and thanks in advance,
Christian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
