Re: builder io isue
|[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]|
On 12/25/2011 05:25 AM, Dennis Gilmore wrote:
When we were testing build were happening really fast, once we loaded up the build jobs things have become really slow http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=238672 started at 1:19 utc and at 5:18 utc four hours later the buildrequires are still being installed. australia is completely io bound. i think we need to see how we can spread the io load. ~30 hosts reading and writing to the 4 spindles just saturates the disk io.
Is your RAID aligned properly? I have found that making sure that your RAID and FS are aligned properly makes a big difference to performance, at least on my workloads. Have a look here:
http://www.altechnative.net/?p=96Just taking the default options when setting up the RAID and FS typically hammer the IOPS performance down to that of a single disk.
i guess options are find a way to add more spindles. move /var/lib/mock to sdcard, see if we can get some kind of san that has alot of smaller fast disks.
That will make it an order of magnitude worse. A typical class 10 SD card only manages about 20 random write IOPS.
get some 2.5" usb drives one for each builder. some other idea? is there anyway we could add 4-8 more disks to australia. the size of the matters little. gaining more iops by adding more spindles would help.
You could just replace the disks with some decent SSDs and be done with it (aligning the RAID and FS as explained on the above linked page still applies just the same). Especially since 15K rpm disks are now highly uncompetitively priced against SSDs.
Gordan _______________________________________________ arm mailing list arm@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/arm