W dniu 03.01.2020 o 20:02, Chris Murphy pisze:
> On Fri, Jan 3, 2020 at 7:39 AM Leszek Dubiel <leszek@xxxxxxxxx> wrote:
>>
>> ** number of files by given size
>>
>> root@wawel:/mnt/root/orion# cat disk_usage | perl -MData::Dumper -e
>> '$Data::Dumper::Sortkeys = 1; while (<>) { chomp; my ($byt, $nam) =
>> split /\t/, $_, -1; if (index("$las/", $nam) == 0) { $dir++; } else {
>> $filtot++; for $p (1 .. 99) { if ($byt < 10 ** $p) { $fil{"num of files
>> size <10^$p"}++; last; } } }; $las = $nam; }; print "\ndirectories:
>> $dir\ntotal num of files: $filtot\n", "\nnumber of files grouped by
>> size: \n", Dumper(\%fil) '
>>
>> directories: 1314246
>> total num of files: 10123960
>>
>> number of files grouped by size:
>> $VAR1 = {
>> 'num of files size <10^1' => 3325886,
>> 'num of files size <10^2' => 3709276,
>> 'num of files size <10^3' => 789852,
>> 'num of files size <10^4' => 1085927,
>> 'num of files size <10^5' => 650571,
>> 'num of files size <10^6' => 438717,
>> 'num of files size <10^7' => 116757,
>> 'num of files size <10^8' => 6638,
>> 'num of files size <10^9' => 323
>> 'num of files size <10^10' => 13,
>
> Is that really ~7.8 million files at or less than 1KiB?? (totalling
> the first three)
Yes. This is Workflow Management system in my company (bathroom mirorr
manufacturer).
System was made in 2004. Back then ReisierFs was great because it had
"tail packing" and put small pieces of data together with metadata, so
disks could read many pieces of data during one read request. Other
systems were not any close to ReiserFs when it came to speed with lots
of small files. That's why I'm testing BTFS for a few years now.
> Compression may not do much with such small files, and also I'm not
> sure which algorithm would do the best job. They all probably want a
> lot more than 1KiB to become efficient.
>
> But nodesize 64KiB might be a big deal...worth testing.
Yes -- I will have to read about nodesize.
Thank you for hint.
Current data:
root@wawel:~# btrfs inspect dump-super /dev/sdb2
superblock: bytenr=65536, device=/dev/sdb2
---------------------------------------------------------
csum_type 0 (crc32c)
csum_size 4
csum 0x0bd7280d [match]
bytenr 65536
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 44803366-3981-4ebb-853b-6c991380c8a6
metadata_uuid 44803366-3981-4ebb-853b-6c991380c8a6
label
generation 553943
root 17155128295424
sys_array_size 129
chunk_root_generation 553648
root_level 1
chunk_root 10136287444992
chunk_root_level 1
log_root 0
log_root_transid 0
log_root_level 0
total_bytes 23967879376896
bytes_used 5844982415360
sectorsize 4096
nodesize 16384 ---------------<<<
leafsize (deprecated) 16384
stripesize 4096
root_dir 6
num_devices 3
compat_flags 0x0
compat_ro_flags 0x0
incompat_flags 0x163
( MIXED_BACKREF |
DEFAULT_SUBVOL |
BIG_METADATA |
EXTENDED_IREF |
SKINNY_METADATA )
cache_generation 553943
uuid_tree_generation 594
dev_item.uuid 5f74e436-f8f9-43ba-95fc-44cdb2bc1838
dev_item.fsid 44803366-3981-4ebb-853b-6c991380c8a6 [match]
dev_item.type 0
dev_item.total_bytes 5992192409600
dev_item.bytes_used 2946381119488
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 3
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
