[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [ogfs-users]list directory content performance




I guess the reason for the poor ls performance is the way ogfs places dinode blocks onto disk. If you check an ext2 directory you will find that all inodes are placed to consecutive blocks on disk ( in the inode table section of block group) so once you have the dirent structure with one seek you can get several inodes depending on the read ahead settings. I checked the location of dinodes after I formatted an ogfs partition and copied 24 files to it. I found the next result in the order of readdir getting back dirent structures :

/eonstor1/100254.mpg is on dinode 96020
/eonstor1/100047.mpg is on dinode 152
/eonstor1/100238.mpg is on dinode 63902
/eonstor1/100046.mpg is on dinode 139
/eonstor1/100148.mpg is on dinode 31793
/eonstor1/100263.mpg is on dinode 128117
/eonstor1/100188.mpg is on dinode 63870
/eonstor1/100268.mpg is on dinode 128155
/eonstor1/100044.mpg is on dinode 119
/eonstor1/100133.mpg is on dinode 165
/eonstor1/100045.mpg is on dinode 126
/eonstor1/100269.mpg is on dinode 128180
/eonstor1/100189.mpg is on dinode 63877
/eonstor1/100142.mpg is on dinode 31767
/eonstor1/100260.mpg is on dinode 96035
/eonstor1/100271.mpg is on dinode 160294
/eonstor1/100236.mpg is on dinode 63884
/eonstor1/100147.mpg is on dinode 31780
/eonstor1/100270.mpg is on dinode 160269
/eonstor1/100265.mpg is on dinode 128130
/eonstor1/100253.mpg is on dinode 96000
/eonstor1/100135.mpg is on dinode 31754
/eonstor1/100043.mpg is on dinode 106
/eonstor1/100134.mpg is on dinode 31741

I guess this means a lot of slow seeks on disk and things getting worst if I increase the number of files in the directory. One solution could be to create unused dinodes during filesystem creation on consecutive blocks and place them to the on-disk link list of free dinodes in the resource group header.  Do you think this could help?

Lajos



Greg Freemyer <freemyer-ml@xxxxxxxxxxxxxxxxx>
Sent by: opengfs-users-admin@xxxxxxxxxxxxxxxxxxxxx

2004/04/27 18:26

Please respond to
opengfs-users@xxxxxxxxxxxxxxxxxxxxx

To
opengfs-users@xxxxxxxxxxxxxxxxxxxxx
cc
Subject
Re: [ogfs-users]list directory content performance





On Tue, 2004-04-27 at 08:53, Lajos.Okos@xxxxxx wrote:
> We have an OGFS filesystem around 1.7TB mounted on 3 nodes. Does
> anybody know how to speed up the response to the ls command? We
> mounted the filesystem with noatime,nodiratime option but it didn't
> help. Once I ask for the directory it scans the array for minutes
> before respond to the command.
>
> Thanks in advance,
>
> Lajos

This is the first time I recall seeing a OGFS filesystem above 1TB.

It could easily be a bug in the calculation for number of resource
groups.  (ie. based on filesize I believe.)

How many resource groups do you have?

Do you know if you have similar performance issues if you reduce your
filesystem to 1 TB or below.

Can you remake your filesystem with more resource groups.  (Use -R to
override the default calculation.)

Greg
--
Greg Freemyer



-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek
For a limited time only, get FREE Ground shipping on all orders of $35
or more. Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg=12297
_______________________________________________
Opengfs-users mailing list
Opengfs-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/opengfs-users



[Kernel]     [Security]     [Bugtraq]     [Photo]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Clusters]     [Linux RAID]     [Yosemite Hiking]     [Linux Resources]

Powered by Linux