CodecLz4.jl

Transcoding codecs for compression and decompression with LZ4
Author JuliaIO
Popularity
9 Stars
Updated Last
1 Year Ago
Started In
January 2018

CodecLz4

Stable Latest Build Status CodeCov

Provides transcoding codecs for compression and decompression with LZ4. Source: LZ4 The compression algorithm is similar to the compression available through Blosc.jl, but uses the LZ4 Frame format as opposed to the standard LZ4 or LZ4_HC formats.

Codecs for the standard LZ4 and LZ4_HC formats are also provided as LZ4FastCompressor and LZ4HCCompressor. These codecs follow the LZ4 streaming examples, breaking the data into blocks and prepending each compressed block with a size. Data compressed with these codecs can be decompressed with LZ4SafeDecompressor.

Caution

LZ4FastCompressor and LZ4HCCompressor output is not compatible with lz4frame and is platform dependent.

Non-streaming functions are included via lz4_compress, lz4_hc_compress, and lz4_decompress. These should work with most other standard lz4 implementations.

Installation

Pkg.add("CodecLz4")

Usage

using CodecLz4 # Some text. text = """ Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sollicitudin mauris non nisi consectetur, a dapibus urna pretium. Vestibulum non posuere erat. Donec luctus a turpis eget aliquet. Cras tristique iaculis ex, eu malesuada sem interdum sed. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Etiam volutpat, risus nec gravida ultricies, erat ex bibendum ipsum, sed varius ipsum ipsum vitae dui. """ # Streaming API. stream = LZ4FrameCompressorStream(IOBuffer(text)) for line in eachline(LZ4FrameDecompressorStream(stream)) println(line) end close(stream) # Array API. compressed = transcode(LZ4FrameCompressor, text) @assert sizeof(compressed) < sizeof(text) @assert transcode(LZ4FrameDecompressor, compressed) == Vector{UInt8}(text)

The API is heavily based off of CodecZLib, and uses TranscodingStreams.jl. See those for details.