Re: efficient removal of old objects

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

On Tue, Jan 31, 2012 at 16:33, Sage Weil <sage@xxxxxxxxxxxx> wrote:
> Currently rgw logs objects it wants to delete after some period of time,
> and an radosgw-admin command comes back later to process the log.  It
> works, but is currently slow (one sync op at a time).
> A better approach would be to mark objects for later removal, and have the
> OSD do it in some more efficient way.  wip-objs-expire has a client side
> (librados) interface for this.

Is there some reason why this would be significantly more performant
when done by the OSD itself? It seems like the deletion times can be
bucketed by time nicely, then each bucket just contains a set of ids
-- a good fit for the map data type -- and the client for running this
deletion just streams the bucket contents over and issues delete
messages for everything. What makes that inherently slow?

> Should this be generalized to saying "do these osd ops at time X" instead
> of "delete at time X".  Then it could setxattr, remove, call into a class,
> whatever.

That sounds like a really complex API, for quite marginal gain.

To make my point even clearer: point me to another data store that has
that idiom.
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at

[CEPH Users]     [Information on CEPH]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Photo]     [Yosemite News]    [Yosemite Photos]    [Free Online Dating]     [Linux Kernel]     [Linux SCSI]     [XFree86]

Add to Google Powered by Linux