Re: Snappy compression algorithm for Reiser4.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Edward,

I would agree on most points, though the LZ4 algorithm is gaining some
interest.  Not to say we know if it works well or not ...
ISO images can be compressed as they are just a concatenation of files,
as well as DD created images (raw block)

If we were to take a vote, I would test LZ4 out in the general sense,
and if it's as good as they say (and fast enough for real-time
compression) I would vote that way.

Chris




On 09/25/13 15:52, Edward Shishkin wrote:
> On 09/22/2013 01:50 AM, Edward Shishkin wrote:
>> 21.09.2013 23:12, Evgeniy пишет:
>>> Hello all,
>>>
>>> Last few days I was playing a little bit around implementation Snappy
>>> algo in Reiser4. My code is dirty and should be optimized. But it's
>>> enough for testing.
>>> According to my tests Snappy shows near the same prefomance as LZO. In
>>> some kind of situations Snappy shows better compression with the less
>>> CPU usage.
>>> LZO has a few advantages over Snappy:
>>> 1) It's in kernel by default
>>> 2) It's optimized better then Snappy.
>>> Also a very significant fact is we're plaing with small blocks of
>>> data. In this case results of all algorithms looks near the same.
>>>
>>> Maybe in the future I'll reqrite a little bit BratSinot's LZ4 patch
>>> (http://sourceforge.net/p/reiser4/discussion/general/thread/780facb4/)
>>> to make it use LZ4 library which is build-in kernel. Friendly
>>> speaking, I'm not expect any big difference in comparison with LZO.
>>> But in any case, it funny :)
>>
>>
>> Support of a new compression algorithm means a format change.
>
>
> To be precise, format _upgrade_ (not change). Format change is a very
> bad thing inherent to systems, which don't possess any development
> model.
>
>
>>
>> So, people with roots on reiser4 will be forced to fsck their
>> partitions.
>> So, not so funny.  A visible advantage in some benchmarks is highly
>> desirable..
>
>
> It seems, it is hard to invent something more interesting than currently
> supported algorithms (lzo1 and gzip1). At least for file systems, where
> we can not afford a luxury to comress/decompress big chunks of data.
>
> Instead, I would suggest to take a look at the "intelligent" compression
> modes implemented by plugins of COMPRESSION_MODE interface. This is
> a set of hooks arranged at different levels, which decide, if compression
> should be turned off/on.
>
> The default mode ("conv") was suggested long ago by Hans. It's idea is
> that first, we try to compress the first 64K of the file and look at the
> result. If it is incompressible, then we turn compression off forever
> (pass
> management to unix-file plugin with formatting policy "extents only").
> Otherwise, stay with cryptcompress plugin, and perform "selective"
> compression on the "dynamic" lattice.
>
> I suspect that such heuristic doesn't work well for all kind of files.
> At the
> same time, we have never experimented in this area. The common idea is
> that it would be nice to understand by the beginning of a file,
> whether the
> whole file is incompressible. For example, badly compressible binary
> executables have special magics at the beginning, etc. Also, I think, it
> doesn't make sense to compress ISO images (no?)
>
> Anyway, a sanity check that file represents a binary executable won't be
> superfluous. If someone has other ideas, we'll be happy to discuss/encode
> them.
>
> Thanks,
> Edward.
> -- 
> To unsubscribe from this list: send the line "unsubscribe
> reiserfs-devel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux File System Development]     [Linux BTRFS]     [Linux NFS]     [Linux Filesystems]     [Ext4 Filesystem]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite Forum]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Device Mapper]     [Linux Resources]

  Powered by Linux