Re: set mds message priority to MSG_PRIO_HIGH

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sat, Oct 19, 2013 at 7:14 AM, Yan, Zheng <ukernel@xxxxxxxxx> wrote:
> On Sat, Oct 19, 2013 at 8:58 PM, hjwsm1989-gmail <hjwsm1989@xxxxxxxxx> wrote:
>> Hi,
>> I'm testing ceph with samba.
>> I have 20 OSD nodes on 4 hosts
>> dy01:1MON, 5 OSDs, 1 samba server
>> dy02: 1MDS, 5OSDs, 1 samba server
>> dy03: 5 OSDs, 1 samba server
>> dy04:5 OSDs, 1 samba server
>> every samba server exports the ceph kclient mount point.
>>
>> When I set 3 samba clients connect to a samba server(dy01), and each write a
>> single 18GB file, gets 110MB/s speed each.
>> But when I use 4 clients, the cluster will report 'slow request' msg, and
>> this time I list files in samba client dir, it gets stuck.
>> And the test tool reports frame losing.
>> For the list operation related to MDS, I think the MDS op messages should
>> have higher priority than clients op.
>> Have I understood correctly?
>
> Hi,
>
> I think you are right, the MDS needs higher priority. could you modify
> the mds code
> and re-test.
>
> In src/ceph_mds.cc, add
> "messenger->set_default_send_priority(CEPH_MSG_PRIO_HIGH);"
> after
> "messenger->set_cluster_protocol(CEPH_MDS_PROTOCOL);"
>
> Regards
> Yan, Zheng

If this actually turns out to be useful, we'll want to briefly go
through the code base and adjust the other levels — a quick grep
indicates the MDS sets some traffic to LOW that should correspondingly
be bumped up to NORMAL; the OSDs are currently using HIGH for acks and
commits, but we probably want those messages to continue having a
higher priority than any clients (MDS included) are setting messages
at.
(I'm not sure it actually will; I suspect what's happening here is
just that 4 clients are passing some limit that 3 aren't.)
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux