lz4/programs
Yann Collet 9443ed9332 working implementation of completionPorts
to support multithreading and asyncio on Windows
2024-07-05 14:33:14 -07:00
..
.gitignore make: fix lz4 mt compilation on linux/posix 2023-12-28 16:26:01 -08:00
bench.c benchmark results are displayed to stdout 2024-04-03 22:49:11 -07:00
bench.h can select validation of CRC during benchmark 2022-07-29 22:13:38 +02:00
COPYING Makefile : added capability to install libraries 2014-01-07 18:47:50 +00:00
lorem.c fixed minor conversion warnings 2024-02-20 15:42:45 -08:00
lorem.h fixed visual studio solution 2024-01-28 15:33:12 -08:00
lz4-exe.rc.in updated license & header dates 2020-11-25 14:45:14 -08:00
lz4.1 updated manuals 2023-12-29 14:04:10 -08:00
lz4.1.md updated manuals 2023-12-29 14:04:10 -08:00
lz4cli.c add test for bug #1374 2024-03-21 23:33:30 -07:00
lz4conf.h fix variable LZ4IO_MULTITHREAD 2024-03-10 17:57:26 -07:00
lz4io.c fixed tsan warnings 2024-07-05 13:52:35 +02:00
lz4io.h report nb of threads in verbose mode 2023-12-29 12:50:43 -08:00
Makefile fixed tsan warnings 2024-07-05 13:52:35 +02:00
platform.h Fix compiler preprocessor 2023-10-20 09:42:35 +00:00
README.md fix variable LZ4IO_MULTITHREAD 2024-03-10 17:57:26 -07:00
threadpool.c working implementation of completionPorts 2024-07-05 14:33:14 -07:00
threadpool.h added a simple (untested) threadpool implementation 2023-12-27 23:32:37 -08:00
timefn.c fixed MT incompatibility with --content-size 2023-12-29 12:18:52 -08:00
timefn.h lz4io: verbose compress/decompress operations display a summary 2023-12-28 18:04:01 -08:00
util.c fixed potential leak scenario detected by @t-mat 2023-12-29 19:33:01 -08:00
util.h better time measurement and reportin 2023-12-27 23:32:37 -08:00

Command Line Interface for LZ4 library

Build

The Command Line Interface (CLI) is generated using the make command, no additional parameter required.

The CLI generates and decodes LZ4-compressed frames.

For more control over the build process, the Makefile script supports all standard conventions, including standard targets (all, install, clean, etc.) and standard variables (CC, CFLAGS, CPPFLAGS, etc.).

The makefile offer several targets for various use cases:

  • lz4 : default CLI, with a command line syntax similar to gzip
  • lz4c : supports legacy lz4 commands (incompatible with gzip)
  • lz4c32 : Same as lz4c, but generates a 32-bits executable
  • unlz4, lz4cat : symlinks to lz4, default to decompression and cat compressed files
  • man : generates the man page, from lz4.1.md markdown source

C Preprocessor variables

These variables are read by the preprocessor at compilation time, they influence executable behavior, such as default starting values, and are exposed from programs/lz4conf.h. Assignment methods vary depending on environments. On a typical posix + gcc + make setup, they can be defined with CPPFLAGS=-DVARIABLE=value assignment.

  • LZ4IO_MULTITHREAD: enable multithreading support
  • LZ4_NBTHREADS_DEFAULT: default nb of threads in multithreading mode. Default is 0, which means "auto-determine" based on local cpu.
  • LZ4_BLOCKSIZEID_DEFAULT: default lz4 block size code. Valid values are [4-7].

Makefile Build variables

  • HAVE_PTHREAD : determines presences of <pthread> support. This is in turn used by make to determine multithreading support of lz4. Detection is automatic, but can be forced to 0 or 1 if needed.

Aggregation of parameters

The lz4 CLI supports aggregation for short commands. For example, -b1, -e18, and -i1 can be joined into -b1e18i1. This doesn't work for --long-commands, which must be separated.

Benchmark in Command Line Interface

CLI includes in-memory compression benchmark module for lz4. The benchmark is conducted using a given filename. The file is read into memory. It makes benchmark more precise as it eliminates I/O overhead.

The benchmark measures ratio, compressed size, compression and decompression speed. One can select compression levels starting from -b and ending with -e. The -i parameter selects a number of seconds used for each of tested levels.

Usage of Command Line Interface

The full list of commands can be obtained with -h or -H parameter:

Usage :
      lz4 [arg] [input] [output]

input   : a filename
          with no FILE, or when FILE is - or stdin, read standard input
Arguments :
 -1     : Fast compression (default)
 -9     : High compression
 -d     : decompression (default for .lz4 extension)
 -z     : force compression
 -D FILE: use FILE as dictionary
 -f     : overwrite output without prompting
 -k     : preserve source files(s)  (default)
--rm    : remove source file(s) after successful de/compression
 -h/-H  : display help/long help and exit

Advanced arguments :
 -V     : display Version number and exit
 -v     : verbose mode
 -q     : suppress warnings; specify twice to suppress errors too
 -c     : force write to standard output, even if it is the console
 -t     : test compressed file integrity
 -m     : multiple input files (implies automatic output filenames)
 -r     : operate recursively on directories (sets also -m)
 -l     : compress using Legacy format (Linux kernel compression)
 -B#    : cut file into blocks of size # bytes [32+]
                     or predefined block size [4-7] (default: 7)
 -BD    : Block dependency (improve compression ratio)
 -BX    : enable block checksum (default:disabled)
--no-frame-crc : disable stream checksum (default:enabled)
--content-size : compressed frame includes original size (default:not present)
--[no-]sparse  : sparse mode (default:enabled on file, disabled on stdout)
--favor-decSpeed: compressed files decompress faster, but are less compressed
--fast[=#]: switch to ultra fast compression level (default: 1)

Benchmark arguments :
 -b#    : benchmark file(s), using # compression level (default : 1)
 -e#    : test all compression levels from -bX to # (default : 1)
 -i#    : minimum evaluation time in seconds (default : 3s)```

License

All files in this directory are licensed under GPL-v2. See COPYING for details. The text of the license is also included at the top of each source file.