Hi, On Mon, Jun 20, 2011 at 03:29:45PM +0400, Proskurin Kirill wrote: > What we have: > SL6 - kernel 2.6.32-131.2.1.el6.x86_64 > mdadm RAID5 with 8 HDD - 27T partition. btw .32 is very old > Mount options is "noatime,noacl,compress-force" > I use scribe daemon to copy log files from 200 hosts to that > partition for stress testing. > > But I found what compression ratio is really small. It is and it will be due to nature of compression method used and the constraints given: * compression has to be fast, only real-time methods can be considered which have natural speed/ratio tradeoff * you're using zlib (lzo was added recently), which is not that fast and does not have a great compression ratio * there is maximum length of compressed data per round, it's hardcoded to 128KB by now * zlib works in streaming mode, which compresses whole 128KB with dictionary reused * lzo (as implemented in btrfs) however does not reuse dictionary and compresses each page separately; resulting size is even bigger than it would be when full 128KB were compressed > Partition is full of regular log files - plain text. Most of them a > big ones(10-20Gb) and they grow in realtime. Realtime growth should not be a problem. The situation can be slightly improved if we use a method capable of streamed compression (ie. dictionary reuse). This is naturally supported by eg. quicklz and snappy. There are a few other realtime methods (fastlz, lzrw, lzjb) which may be extended to support streaming compression, but with careful evaluation of the result. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
