| [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] |
Hi,
Recently we got for testing StoneFly iSCSI device with Axus Brownie RAID
attached. We also a have 16-node cluster, so test confuration is:
1) Each of 16 nodes is connected to a switch via Fast Ethernet
2) Frontend machine is connected to a switch via Gigabit Ethernet
3) StoneFly iSCSI device is connected to a switch via Gigabit Ethernet
We successfully setup OpenGFS filesystem and mounted it on frontend machine and
on all 16 nodes. After that we ran bonnie++ on a frontend machine and on one
of the nodes. The results are given below:
Frontend:
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
server 10G 11648 86 44102 27 17001 41 12971 94 28691 13 148.1 1
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
64 761 24 5771 67 393 11 929 19 9215 79 336 9
Node:
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
node-11 10G 7531 37 10770 6 5007 10 9140 45 9467 4 133.9 1
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
64 276 9 1564 22 252 14 229 6 1221 16 116 13
After that we tried to run bonnie++ simultaneously on two nodes. Unfortunately
we were not able to wait when tests are finished, though there are some
numbers:
1) Writing 1Gb file via putc() took about 35-50 minutes.
2) Block writing 1Gb file was fast enough - it took only 2-3 minutes.
3) Rewriting 1Gb was the most hard operation, it didn't finished. Looking
at vmstat output during rewriting we have seen the following:
- persistent input about 2-3Mb/sec
- occasional writes every 5 seconds about 140Kb
If we suspended one of the tests, another test was significantly accelerated,
so it seems there are some locking problems. BTW, memexpd on frontend machine
(Athlon MP 2000+) eated 20-25% CPU.
For comparison, below there are bonnie++ results from one of the nodes when
running tests simultaneously on two nodes on NFS filesystem (and exported
filesystem was reside on the same iSCSI device):
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
node-12 10G 9733 50 10584 10 2712 3 7892 39 8294 4 95.4 1
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
64 161 3 2332 15 979 11 163 3 2557 11 318 4
node-12,10G,9733,50,10584,10,2712,3,7892,39,8294,4,95.4,1,64,161,3,2332,15,979,11,163,3,2557,11,318,4
Are there any ideas what can be tuned to improve OpenGFS performance when
it is used concurrently by several nodes?
Best regards,
Andrey.
--
A right thing should be simple (tm)
-------------------------------------------------------
This SF.net email is sponsored by: ObjectStore.
If flattening out C++ or Java code to make your application fit in a
relational database is painful, don't do it! Check out ObjectStore.
Now part of Progress Software. http://www.objectstore.net/sourceforge
_______________________________________________
Opengfs-users mailing list
Opengfs-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opengfs-users