[Lzip-bug] size of decompression buffer

[John Reiser]

Some of the cases where I want to use lzip need more control
over the size of the decompression buffer.  Even if actually unused,
allocating or mapping large amounts of address space can be impossible,
or at least very expensive.  Here is a proposal.

Extend the interpretation of the current File_header.dictionary_size.
In the extension, this byte means:

  (037 & byte): code for length of decompression buffer [dictionary]
      0..11:  (file_size >> code)
     12..31:  (1 << code)

  (byte >> 5): code for location of file_size
        0:  8 bytes in File_trailer  [forced if input to compression
               was a stream, and not a file]
     1..7:  (1+code) bytes [little endian] immediately following header

Any current compressed output already agrees with the extension.
When the file_size is present at the beginning of the compressed output,
then it is omitted from File_trailer.  This saves 4 or 5 bytes of space
for typical files which occupy less than (1<<32) or (1<<24) bytes.

Knowing the length of the uncompressed data output at the beginning
of decompression is useful to bound the length of the decompression
buffer, and for noticing output overrun immediately.

Comments?

-- 
John Reiser