Re: [RFC PATCH 0/6] Understanding delays due to throttling under very heavy write load

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, Feb 6, 2012 at 10:20 AM, Jim Schutt <jaschut@xxxxxxxxxx> wrote:
> On 02/06/2012 10:22 AM, Yehuda Sadeh Weinraub wrote:
>>
>> On Mon, Feb 6, 2012 at 8:20 AM, Jim Schutt<jaschut@xxxxxxxxxx>  wrote:
>
>
>>>
>>> The above suggests to me that the slowdown is a result
>>> of requests not getting submitted at the same rate as
>>> when things are running well.
>>>
>>
>> Yeah, it really looks like that. My suggestions wouldn't help there.
>>
>> I do see that when things go well the number of writes per device is
>> capped at ~200 writes per second and the throughput per device is
>> ~100MB/sec. Is 100MB/sec the expected device throughput?
>
>
> Pretty much, at least for the outer tracks on a drive.  I've seen
> ~108 MB/s with dd to a block device.  Also, I've got 8 drives per
> SAS adapter with 6 Gb/s links, so it seems unlikely to me that my
> disk subsystem is any sort of significant bottleneck.

Well, you might try changing your throttling settings on the OSDs.
ms_dispatch_throttle_bytes defaults to 100<<20 (100MB) and is used for
throttling dispatch; osd_max_client_bytes defaults to 500<<20 (500MB)
and is used to limit the amount of client data in memory (ie; messages
are included in this throttler for their entire lifetime, not just
while waiting for dispatch).
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux