Re: Software RAID checksum performance on 24 disks not even close to kernel reported
|[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
On 06/04/2012 07:14 PM, Ole Tange wrote:
But I cannot explain why even the best performance (4600 MB/11s = 420 MB/s) is not even close to the checksum performance reported by the kernel at boot (6196 MB/s): Mar 13 16:02:42 server kernel: [ 35.120035] raid6: using algorithm sse2x4 (6196 MB/s) Can you explain why I only get 420 MB/s of real world checksumming instead of 6196 MB/s?
In the best possible case, you would get 22x 1-disk bandwidth, which would be ~120MB/s (assuming RAID6, and "infinite" speed of checksum computation). This is your "theoretical" upper bound on performance.
Your pragmatic upper bound on performance is reduced from the theoretical by many issues, including various hardware issues (controller, PCIe lanes, memory, disk, ...), as well as software (IO stack traversal, elevators, buffers/cache fills, ... etc.).
Aside from this, it is very rare that you will have a single application reading and writing at full stripe width all the time, which would be the optimal case for you. There are some, and we've play with a number for our customers. But they are the exception and not the rule.
Your real world IO performance for 24 disks is 420MB/s. So this particular setup is, from your numbers, operating at about 16% peak efficiency per disk +/- some. This is not uncommon for people's self-built systems. The checksumming isn't your rate limiting feature. Other things are.
-- Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email: landman@xxxxxxxxxxxxxxxxxxxxxxx web : http://scalableinformatics.com http://scalableinformatics.com/sicluster phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615 -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html