| [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] |
> > > I would like to have up to 40 nodes connected. Do you > think that OGFS > > > would perform better than lustre or coda? > > > > > That is really more nodes than ogfs is desgned for currently and > > possibly more than it has even been tested with. > [sophana ] really? Hmm I thought that many people would have > problem like > mine. There is a MAX_CLIENTS value, within the code for the memexp locking protocol, defined as 32. This is a locking-backend-specific value, that is rather independent from the filesystem itself. You may be able to tweak it (but no guarantees). > > > > If you are indeed compiling totally unrelated code and thus > have no file > > locking contention, then ogfs might do okay. There would still be > > locking issues based on rgrps. Others can talk more to > that if you want > > to pursue this option. > [sophana ] what are rgrps? Is it directory related? The jobs > are working in > the same filesystem tree. That is actualy what makes things > not as fast as > they should be. There should be no file lock contention and > very little > directory lock contention. > > Resource groups (rgrps) are contiguous groups of blocks within the filesystem, somewhat like "block groups" in ext2/3. One significant difference, however, is that OGFS keeps all block usage statistics within each rgrp, rather than in a single location in the superblock. All block allocation is done within the rgrp domain; you can think of the filesystem as being divided up into several mini-filesystems, rather than one big hunk. Ideally, each node selects a different rgrp from which to allocate a new directory, and leaf inodes under that directory are also allocated from the directory's rgrp (IIUC!). There are 3 different algorithms for choosing the target rgrp: single (assigned at mount time, used until the rgrp is full, then rgrp is incremented by 1), round-robin (starts like single, but each time a new directory is allocated, the node increments rgrp by 1), and random (each time a new directory is allocated, the node chooses an rgrp at random). If you are building separate projects, presumably under separate directories, then there *should* be fairly little contention at the rgrp level. You may want to experiment with the various algorithms. IMHO (and I've used OGFS only with 2 nodes), OGFS would be worth a try in your situation. Please realize that OGFS does not currently deliver great performance. We need to do some research to find the root cause, and hopefully fix it. We would also be extremely interested, if you *compare* various filesystems for your application, to hear about the results of your comparison. Another caution: Since the memexp locking backend uses a single lock storage server, there may be a *lot* of lock load/store traffic to that one server with so many (32, or 40) client nodes. BTW, I'm curious why you want to share the storage space, rather than use a disk for each node, since the jobs are separate? -- Ben -- ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Opengfs-users mailing list Opengfs-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opengfs-users