On Wed, Jul 30, 2014 at 2:31 PM, <ashford@xxxxxxxxxxxxx> wrote: > Nick, > >> On Wed, Jul 30, 2014 at 11:36 AM, <ashford@xxxxxxxxxxxxx> wrote: >>>> On Tue, Jul 29, 2014 at 11:54:20PM -0400, Nick Krause wrote: >>>>> Hey Guys , >>>>> I interested in helping improving the compression of btrfs by using a >>>>> set of threads using work queues like XFS >>>>> or reads and keeping the page cache after reading compressed blocks as >>>>> these seem to be a great way to improve >>>>> on compression performance mostly with large partitions of compressed >>>>> data. >>>> >>>> I suspect that this may be a medium-sized project, rather than a >>>> small one. My gut feeling (based on limited experience) is that the >>>> fallocate extensions project would be considerably simpler. >>>> >>>> Hugo. >>> >>> I may be in error, but I believe this may be a bit more complex than >>> just >>> routing all reads and writes through a decompression/compression work >>> queue. On the write side, it might be better to compress the >>> synchronous >>> requests in-line, instead of going through a work queue. Similarly, on >>> the read side, it might be better to decompress user-requested data (and >>> metadata) in-line, and have any system-generated read-ahead data be >>> decompressed in a work queue (the same work queue?). >>> >>> I believe that one of the significant concerns of the above is that the >>> compression and decompression routines will have to be verified to be >>> thread-safe. If BTRFS is using the same compression/decompression >>> routines that other file-systems use, then the core code is almost >>> certainly thread-safe. Any BTRFS wrapper code will have to be verified. >> >> Seems that the code from my reading is that the code is using the same >> compression >> routines as the other file systems. You seem to have some ideas on how >> to write this >> or improve it at least, if you want you can send me a list of ideas >> that you have. > > Sorry. It's been almost 20 years since I did any serious kernel work, and > that kernel (not Linux) didn't even know how to page. There have been too > many changes since then for me to be able to climb back in with under a > year of hard work. Even then, I would select a simpler task than this to > start on. > > What I DO remember is that the layer you're working on is fairly low in > the stack. The application, kernel and file-system have already had a > chance to prioritize the I/O requests as synchronous or asynchronous. > That prioritization must be properly handled, or the overall stability of > BTRFS will be at risk. > > Good luck. > > Peter Ashford > Peter, Seems like a challenge and I like challenges. Regards Nick -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
