Re: Ceph performance on Ubuntu Oneiric vs Ubuntu Precise

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Hi Greg,

Yep, 3 monitors each on their own node.


On 06/18/2012 01:04 PM, Gregory Farnum wrote:
Do I correctly assume that these nodes hosted only the OSDs, and the
monitors were on a separate node?

On Mon, Jun 18, 2012 at 10:56 AM, Mark Nelson<mark.nelson@xxxxxxxxxxx>  wrote:
Hi Guys,

I've been tracking down some performance issues over the past month with our
internal test nodes and believe I have narrowed it down to something related
to Ubuntu Oneiric.  Tests done on nodes running Ubuntu Precise are
significantly faster.

One of the major differences between the releases is the support for syncfs
in libc.  Theoretically this shouldn't have a big effect on btrfs so I'm not
totally sure that this is the culprit.  Having said that, previous tests
showed good SSD performance on Oneiric leading me to believe the lower
latency mitigates the effect.  Some of spinning disk seekwatcher results for
Oneiric are quite strange with long periods of inactivity on the OSD data

I wanted to post these results for those of you who have had performance
problems in the past.  If you are continuing to have issues, you may want to
try testing on precise and see if you notice any changes.  It is possible
that all of this could be specific to our internal testing nodes, so I
wouldn't mind hearing if other people have seen similar behavior.

These tests were done using rados bench with 16 concurrent requests. There
are two nodes that each have a single 7200rpm OSD data disk and journal on a
second 7200rpm disk.  Replication is set at the default level (2).  Kernel
is 3.4 in all cases.

Here's a run down (Numbers are MB/s)

4KB Requests

                        BTRFS   EXT4    XFS
Ceph 0.46/Oneiric:      0.073   0.694   0.723
Ceph 0.46/Precise:      2.15    2.031   1.546
Ceph 0.47.2/Oneiric:    1.072   0.836   0.749
Ceph 0.47.2/Precise:    2.566   2.579   1.498

128KB Requests:

                        BTRFS   EXT4    XFS
Ceph 0.46/Oneiric:      11.874  20.066  12.641
Ceph 0.46/Precise:      49.304  39.736  38.982
Ceph 0.47.2/Oneiric:    13.81   19.05   12.739
Ceph 0.47.2/Precise:    47.943  49.655  36.764

4MB Requests:

                        BTRFS   EXT4    XFS
Ceph 0.46/Oneiric:      110.202 26.58   15.445
Ceph 0.46/Precise:      135.975 128.759 106.426
Ceph 0.47.2/Oneiric:    91.337  46.277  23.897
Ceph 0.47.2/Precise:    136.906 134.955 106.545

I've posted seekwatcher results for all of the tests:

Ceph 0.46/Oneiric:
Ceph 0.46/Precise:
Ceph 0.47.2/Oneiric:
Ceph 0.47.2/Precise:

To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at

To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at

[CEPH Users]     [Information on CEPH]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Photo]     [Yosemite News]    [Yosemite Photos]    [Free Online Dating]     [Linux Kernel]     [Linux SCSI]     [XFree86]

Add to Google Powered by Linux