mirrors/zstd
0
0
Fork 0
mirror of https://github.com/facebook/zstd.git synced 2024-12-11 15:56:45 +08:00
Code Issues Packages Projects Releases Wiki Activity

removed some constants and _simpleArgs() from staging

Browse Source 
constants that *may* change in the future
will be accessed through functions instead
(to be created).

_simpleArgs() variant do not have (yet) a clear enough added value
to deserve "stable" status.
This commit is contained in:
Yann Collet 2018-11-19 17:38:15 -08:00
parent f07e13b729
commit 19e5f2a35b
2 changed files with 149 additions and 121 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

97
doc/zstd_manual.html
View File

@ -68,7 +68,7 @@
<a name="Chapter2"></a><h2>Version</h2><pre></pre>
<pre><b>unsigned ZSTD_versionNumber(void); </b>/**< useful to check dll version */<b>
<pre><b>unsigned ZSTD_versionNumber(void); </b>/**< to check runtime library version */<b>
</b></pre><BR>
<a name="Chapter3"></a><h2>Default constant</h2><pre></pre>
@ -310,8 +310,8 @@ size_t ZSTD_endStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output);
ZSTD_DStream objects can be re-used multiple times.
Use ZSTD_initDStream() to start a new decompression operation,
or ZSTD_initDStream_usingDict() if decompression requires a dictionary.
@return : recommended first input size
or ZSTD_initDStream_usingDict() if decompression requires a dictionary.
@return : recommended first input size
Use ZSTD_decompressStream() repetitively to consume your input.
The function will update both `pos` fields.
@ -385,7 +385,7 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
ZSTD_btopt=7,
ZSTD_btultra=8
</b>/* note : new strategies might be added in the future */<b>
} ZSTD_strategy;
} ZSTD_strategy;
</b></pre><BR>
<pre><b>typedef enum {
@ -510,7 +510,8 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
* enum. See the comments on that enum for an
* explanation of the feature.
*/
</b>/* Question : should rsyncable be categorized as experimental, or be candidate for stable ? */<b>
</b>/* Question : should rsyncable remain experimental, or be part of candidate for stable ? */<b>
ZSTD_p_rsyncable, </b>/* Enables rsyncable mode, which makes compressed<b>
* files more rsync friendly by adding periodic
* synchronization points to the compressed data.
@ -534,6 +535,16 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
*/
} ZSTD_cParameter;
</b></pre><BR>
<pre><b>unsigned ZSTD_cParam_lowerBound(ZSTD_cParameter cParam);
unsigned ZSTD_cParam_upperBound(ZSTD_cParameter cParam);
</b><p> All parameters must respect lower/upper bounds,
otherwise they will either trigger an error
or be automatically clamped.
@return : requested bound (inclusive)
note : if the request specifies a non-existing parameter, it will return 0.
</p></pre><BR>
<pre><b>size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, unsigned value);
</b><p> Set one compression parameter, selected by enum ZSTD_cParameter.
Setting a parameter is generally only possible during frame initialization (before starting compression).
@ -666,18 +677,6 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
</p></pre><BR>
<pre><b>size_t ZSTD_compress_generic_simpleArgs (
ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos,
ZSTD_EndDirective endOp);
</b><p> Same as ZSTD_compress_generic(),
but using only integral types as arguments.
This variant might be helpful for binders from dynamic languages
which have troubles handling structures containing memory pointers.
</p></pre><BR>
<pre><b> ZSTDLIB_API size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowSize);
</b><p> Refuses allocating internal buffers for frames requiring a window size larger than provided limit.
This protects a decoder context from reserving too much memory for itself (potential attack scenario).
@ -753,22 +752,11 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
</p></pre><BR>
<pre><b>size_t ZSTD_decompress_generic_simpleArgs (
ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos);
</b><p> Same as ZSTD_decompress_generic(),
but using only integral types as arguments.
This can be helpful for binders from dynamic languages
which have troubles handling structures containing memory pointers.
</p></pre><BR>
<a name="Chapter15"></a><h2>experimental API (static linking only)</h2><pre>
The following symbols and constants
are not planned to join "stable API" status anytime soon.
Some of them will never reach "stable", and are planned to remain in the static_only section indefinitely.
Some of them might even be removed in the future.
Some of them are planned to remain in the static_only section indefinitely.
Some of them might even be removed in the future (especially when redundant with existing stable functions)
<BR></pre>
@ -998,19 +986,20 @@ static ZSTD_customMem const ZSTD_defaultCMem = { NULL, NULL, NULL }; </b>/**< t
both values are optional, select `0` if unknown.
</p></pre><BR>
<pre><b>size_t ZSTD_compress_advanced (ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const void* dict,size_t dictSize,
ZSTD_parameters params);
</b><p> Same as ZSTD_compress_usingDict(), with fine-tune control over each compression parameter
<pre><b>size_t ZSTD_compress_advanced(ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const void* dict,size_t dictSize,
ZSTD_parameters params);
</b><p> Same as ZSTD_compress_usingDict(), with fine-tune control over compression parameters (by structure)
</p></pre><BR>
<pre><b>size_t ZSTD_compress_usingCDict_advanced(ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const ZSTD_CDict* cdict, ZSTD_frameParameters fParams);
</b><p> Same as ZSTD_compress_usingCDict(), with fine-tune control over frame parameters
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const ZSTD_CDict* cdict,
ZSTD_frameParameters fParams);
</b><p> Same as ZSTD_compress_usingCDict(), with fine-tune control over frame parameters
</p></pre><BR>
<pre><b>size_t ZSTD_CCtx_loadDictionary_byReference(ZSTD_CCtx* cctx, const void* dict, size_t dictSize);
@ -1098,6 +1087,18 @@ size_t ZSTD_freeCCtxParams(ZSTD_CCtx_params* params);
</p></pre><BR>
<pre><b>size_t ZSTD_compress_generic_simpleArgs (
ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos,
ZSTD_EndDirective endOp);
</b><p> Same as ZSTD_compress_generic(),
but using only integral types as arguments.
This variant might be helpful for binders from dynamic languages
which have troubles handling structures containing memory pointers.
</p></pre><BR>
<a name="Chapter19"></a><h2>Advanced decompression functions</h2><pre></pre>
<pre><b>unsigned ZSTD_isFrame(const void* buffer, size_t size);
@ -1164,7 +1165,21 @@ size_t ZSTD_freeCCtxParams(ZSTD_CCtx_params* params);
@return : 0, or an error code (which can be tested using ZSTD_isError()).
</p></pre><BR>
<a name="Chapter20"></a><h2>Advanced streaming functions</h2><pre></pre>
<pre><b>size_t ZSTD_decompress_generic_simpleArgs (
ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos);
</b><p> Same as ZSTD_decompress_generic(),
but using only integral types as arguments.
This can be helpful for binders from dynamic languages
which have troubles handling structures containing memory pointers.
</p></pre><BR>
<a name="Chapter20"></a><h2>Advanced streaming functions</h2><pre> Warning : most of these functions are now redundant with the Advanced API.
Once Advanced API reaches "stable" status,
redundant functions will be deprecated, and then at some point removed.
<BR></pre>
<h3>Advanced Streaming compression functions</h3><pre></pre><b><pre>size_t ZSTD_initCStream_srcSize(ZSTD_CStream* zcs, int compressionLevel, unsigned long long pledgedSrcSize); </b>/**< pledgedSrcSize must be correct. If it is not known at init time, use ZSTD_CONTENTSIZE_UNKNOWN. Note that, for compatibility with older programs, "0" also disables frame content size field. It may be enabled in the future. */<b>
size_t ZSTD_initCStream_usingDict(ZSTD_CStream* zcs, const void* dict, size_t dictSize, int compressionLevel); </b>/**< creates of an internal CDict (incompatible with static CCtx), except if dict == NULL or dictSize < 8, in which case no dict is used. Note: dict is loaded with ZSTD_dm_auto (treated as a full zstd dictionary if it begins with ZSTD_MAGIC_DICTIONARY, else as raw content) and ZSTD_dlm_byCopy.*/<b>

173
lib/zstd.h
View File

@ -74,13 +74,13 @@ extern "C" {
#define ZSTD_VERSION_RELEASE 8
#define ZSTD_VERSION_NUMBER (ZSTD_VERSION_MAJOR *100*100 + ZSTD_VERSION_MINOR *100 + ZSTD_VERSION_RELEASE)
ZSTDLIB_API unsigned ZSTD_versionNumber(void); /**< useful to check dll version */
ZSTDLIB_API unsigned ZSTD_versionNumber(void); /**< to check runtime library version */
#define ZSTD_LIB_VERSION ZSTD_VERSION_MAJOR.ZSTD_VERSION_MINOR.ZSTD_VERSION_RELEASE
#define ZSTD_QUOTE(str) #str
#define ZSTD_EXPAND_AND_QUOTE(str) ZSTD_QUOTE(str)
#define ZSTD_VERSION_STRING ZSTD_EXPAND_AND_QUOTE(ZSTD_LIB_VERSION)
ZSTDLIB_API const char* ZSTD_versionString(void); /* v1.3.0+ */
ZSTDLIB_API const char* ZSTD_versionString(void); /* since v1.3.0+ */
/***************************************
* Default constant
@ -354,8 +354,8 @@ ZSTDLIB_API size_t ZSTD_CStreamOutSize(void); /**< recommended size for output
* ZSTD_DStream objects can be re-used multiple times.
*
* Use ZSTD_initDStream() to start a new decompression operation,
* or ZSTD_initDStream_usingDict() if decompression requires a dictionary.
* @return : recommended first input size
* or ZSTD_initDStream_usingDict() if decompression requires a dictionary.
* @return : recommended first input size
*
* Use ZSTD_decompressStream() repetitively to consume your input.
* The function will update both `pos` fields.
@ -415,8 +415,13 @@ ZSTDLIB_API size_t ZSTD_DStreamOutSize(void); /*!< recommended size for output
* as the API is locked once reaching "stable" status.
* ***************************************************************************************/
/* === query limits === */
ZSTDLIB_API int ZSTD_minCLevel(void); /*!< minimum negative compression level allowed */
/* --- Constants ---*/
/* all magic numbers are supposed read/written to/from files/memory using little-endian convention */
@ -425,39 +430,9 @@ ZSTDLIB_API int ZSTD_minCLevel(void); /*!< minimum negative compression level a
#define ZSTD_MAGIC_SKIPPABLE_START 0x184D2A50 /* all 16 values, from 0x184D2A50 to 0x184D2A5F, signal the beginning of a skippable frame */
#define ZSTD_MAGIC_SKIPPABLE_MASK 0xFFFFFFF0
/* note : should this limit be smaller ? like 23 (8 MB) as suggested in the spec ?
* can it be different from zstd cli default limit, which is designed to match --ultra and --long default (27) ? */
#define ZSTD_WINDOWLOG_LIMIT_DEFAULT 27 /* by default, the streaming decoder will refuse any frame
* requiring larger than (1<<ZSTD_WINDOWLOG_LIMIT_DEFAULT) window size, to preserve memory.
* This limit can be overriden using ZSTD_DCtx_setMaxWindowSize().
* This limit does not apply to one-pass decoders (such as ZSTD_decompress()), since no additional memory is allocated */
#define ZSTD_BLOCKSIZELOG_MAX 17
#define ZSTD_BLOCKSIZE_MAX (1<<ZSTD_BLOCKSIZELOG_MAX)
/* compression parameter bounds */
#define ZSTD_WINDOWLOG_MAX_32 30
#define ZSTD_WINDOWLOG_MAX_64 31
#define ZSTD_WINDOWLOG_MAX ((unsigned)(sizeof(size_t) == 4 ? ZSTD_WINDOWLOG_MAX_32 : ZSTD_WINDOWLOG_MAX_64))
#define ZSTD_WINDOWLOG_MIN 10
#define ZSTD_HASHLOG_MAX ((ZSTD_WINDOWLOG_MAX < 30) ? ZSTD_WINDOWLOG_MAX : 30)
#define ZSTD_HASHLOG_MIN 6
#define ZSTD_CHAINLOG_MAX_32 29
#define ZSTD_CHAINLOG_MAX_64 30
#define ZSTD_CHAINLOG_MAX ((unsigned)(sizeof(size_t) == 4 ? ZSTD_CHAINLOG_MAX_32 : ZSTD_CHAINLOG_MAX_64))
#define ZSTD_CHAINLOG_MIN ZSTD_HASHLOG_MIN
#define ZSTD_SEARCHLOG_MAX (ZSTD_WINDOWLOG_MAX-1)
#define ZSTD_SEARCHLOG_MIN 1
#define ZSTD_SEARCHLENGTH_MAX 7 /* only for ZSTD_fast, other strategies are limited to 6 */
#define ZSTD_SEARCHLENGTH_MIN 3 /* only for ZSTD_btopt, other strategies are limited to 4 */
#define ZSTD_TARGETLENGTH_MAX ZSTD_BLOCKSIZE_MAX
#define ZSTD_TARGETLENGTH_MIN 0 /* note : comparing this constant to an unsigned results in a tautological test */
/* LDM parameter bounds */
#define ZSTD_LDM_MINMATCH_MAX 4096
#define ZSTD_LDM_MINMATCH_MIN 4
#define ZSTD_LDM_BUCKETSIZELOG_MAX 8
/***************************************
* Memory management
@ -503,7 +478,8 @@ typedef enum { ZSTD_fast=1,
ZSTD_btopt=7,
ZSTD_btultra=8
/* note : new strategies might be added in the future */
} ZSTD_strategy;
} ZSTD_strategy;
typedef enum {
@ -628,7 +604,8 @@ typedef enum {
* enum. See the comments on that enum for an
* explanation of the feature.
*/
/* Question : should rsyncable be categorized as experimental, or be candidate for stable ? */
/* Question : should rsyncable remain experimental, or be part of candidate for stable ? */
ZSTD_p_rsyncable, /* Enables rsyncable mode, which makes compressed
* files more rsync friendly by adding periodic
* synchronization points to the compressed data.
@ -653,6 +630,17 @@ typedef enum {
} ZSTD_cParameter;
/*! ZSTD_cParam_lowerBound() and ZSTD_cParam_upperBound() :
* All parameters must respect lower/upper bounds,
* otherwise they will either trigger an error
* or be automatically clamped.
* @return : requested bound (inclusive)
* note : if the request specifies a non-existing parameter, it will return 0.
*/
ZSTDLIB_API unsigned ZSTD_cParam_lowerBound(ZSTD_cParameter cParam);
ZSTDLIB_API unsigned ZSTD_cParam_upperBound(ZSTD_cParameter cParam);
/*! ZSTD_CCtx_setParameter() :
* Set one compression parameter, selected by enum ZSTD_cParameter.
* Setting a parameter is generally only possible during frame initialization (before starting compression).
@ -789,20 +777,6 @@ ZSTDLIB_API size_t ZSTD_compress_generic (ZSTD_CCtx* cctx,
ZSTD_EndDirective endOp);
/*! ZSTD_compress_generic_simpleArgs() :
* Same as ZSTD_compress_generic(),
* but using only integral types as arguments.
* This variant might be helpful for binders from dynamic languages
* which have troubles handling structures containing memory pointers.
*/
ZSTDLIB_API size_t ZSTD_compress_generic_simpleArgs (
ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos,
ZSTD_EndDirective endOp);
/* ============================== */
/* Advanced decompression API */
/* ============================== */
@ -892,38 +866,51 @@ ZSTDLIB_API size_t ZSTD_decompress_generic(ZSTD_DCtx* dctx,
ZSTD_inBuffer* input);
/*! ZSTD_decompress_generic_simpleArgs() :
* Same as ZSTD_decompress_generic(),
* but using only integral types as arguments.
* This can be helpful for binders from dynamic languages
* which have troubles handling structures containing memory pointers.
*/
ZSTDLIB_API size_t ZSTD_decompress_generic_simpleArgs (
ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos);
/****************************************************************************************
* experimental API (static linking only)
****************************************************************************************
* The following symbols and constants
* are not planned to join "stable API" status anytime soon.
* Some of them will never reach "stable", and are planned to remain in the static_only section indefinitely.
* Some of them might even be removed in the future.
* Some of them are planned to remain in the static_only section indefinitely.
* Some of them might even be removed in the future (especially when redundant with existing stable functions)
* ***************************************************************************************/
#define ZSTD_HASHLOG3_MAX 17
#define ZSTD_FRAMEHEADERSIZE_PREFIX 5 /* minimum input size required to query frame header size */
#define ZSTD_FRAMEHEADERSIZE_MIN 6
#define ZSTD_FRAMEHEADERSIZE_MAX 18 /* can be useful for static allocation */
#define ZSTD_SKIPPABLEHEADERSIZE 8
#define ZSTD_HASHLOG3_MAX 17
/* note : matches --ultra and --long default (27) ? */
#define ZSTD_WINDOWLOG_LIMIT_DEFAULT 27 /* by default, the streaming decoder will refuse any frame
* requiring larger than (1<<ZSTD_WINDOWLOG_LIMIT_DEFAULT) window size, to preserve memory.
* This limit can be overriden using ZSTD_DCtx_setMaxWindowSize().
* This limit does not apply to one-pass decoders (such as ZSTD_decompress()), since no additional memory is allocated */
/* compression parameter bounds */
#define ZSTD_WINDOWLOG_MAX_32 30
#define ZSTD_WINDOWLOG_MAX_64 31
#define ZSTD_WINDOWLOG_MAX ((unsigned)(sizeof(size_t) == 4 ? ZSTD_WINDOWLOG_MAX_32 : ZSTD_WINDOWLOG_MAX_64))
#define ZSTD_WINDOWLOG_MIN 10
#define ZSTD_HASHLOG_MAX ((ZSTD_WINDOWLOG_MAX < 30) ? ZSTD_WINDOWLOG_MAX : 30)
#define ZSTD_HASHLOG_MIN 6
#define ZSTD_CHAINLOG_MAX_32 29
#define ZSTD_CHAINLOG_MAX_64 30
#define ZSTD_CHAINLOG_MAX ((unsigned)(sizeof(size_t) == 4 ? ZSTD_CHAINLOG_MAX_32 : ZSTD_CHAINLOG_MAX_64))
#define ZSTD_CHAINLOG_MIN ZSTD_HASHLOG_MIN
#define ZSTD_SEARCHLOG_MAX (ZSTD_WINDOWLOG_MAX-1)
#define ZSTD_SEARCHLOG_MIN 1
#define ZSTD_SEARCHLENGTH_MAX 7 /* only for ZSTD_fast, other strategies are limited to 6 */
#define ZSTD_SEARCHLENGTH_MIN 3 /* only for ZSTD_btopt, other strategies are limited to 4 */
#define ZSTD_TARGETLENGTH_MAX ZSTD_BLOCKSIZE_MAX
#define ZSTD_TARGETLENGTH_MIN 0 /* note : comparing this constant to an unsigned results in a tautological test */
/* LDM parameter bounds */
#define ZSTD_LDM_MINMATCH_MAX 4096
#define ZSTD_LDM_MINMATCH_MIN 4
#define ZSTD_LDM_BUCKETSIZELOG_MAX 8
/* --- Advanced types --- */
@ -1197,19 +1184,20 @@ ZSTDLIB_API size_t ZSTD_checkCParams(ZSTD_compressionParameters params);
ZSTDLIB_API ZSTD_compressionParameters ZSTD_adjustCParams(ZSTD_compressionParameters cPar, unsigned long long srcSize, size_t dictSize);
/*! ZSTD_compress_advanced() :
* Same as ZSTD_compress_usingDict(), with fine-tune control over each compression parameter */
ZSTDLIB_API size_t ZSTD_compress_advanced (ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const void* dict,size_t dictSize,
ZSTD_parameters params);
* Same as ZSTD_compress_usingDict(), with fine-tune control over compression parameters (by structure) */
ZSTDLIB_API size_t ZSTD_compress_advanced(ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const void* dict,size_t dictSize,
ZSTD_parameters params);
/*! ZSTD_compress_usingCDict_advanced() :
* Same as ZSTD_compress_usingCDict(), with fine-tune control over frame parameters */
* Same as ZSTD_compress_usingCDict(), with fine-tune control over frame parameters */
ZSTDLIB_API size_t ZSTD_compress_usingCDict_advanced(ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const ZSTD_CDict* cdict, ZSTD_frameParameters fParams);
void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const ZSTD_CDict* cdict,
ZSTD_frameParameters fParams);
/*! ZSTD_CCtx_loadDictionary_byReference() :
@ -1298,6 +1286,18 @@ ZSTDLIB_API size_t ZSTD_CCtxParam_getParameter(ZSTD_CCtx_params* params, ZSTD_cP
ZSTDLIB_API size_t ZSTD_CCtx_setParametersUsingCCtxParams(
ZSTD_CCtx* cctx, const ZSTD_CCtx_params* params);
/*! ZSTD_compress_generic_simpleArgs() :
* Same as ZSTD_compress_generic(),
* but using only integral types as arguments.
* This variant might be helpful for binders from dynamic languages
* which have troubles handling structures containing memory pointers.
*/
ZSTDLIB_API size_t ZSTD_compress_generic_simpleArgs (
ZSTD_CCtx* cctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos,
ZSTD_EndDirective endOp);
/***************************************
* Advanced decompression functions
@ -1368,10 +1368,23 @@ ZSTDLIB_API size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx, const void* pre
* @return : 0, or an error code (which can be tested using ZSTD_isError()). */
ZSTDLIB_API size_t ZSTD_DCtx_setFormat(ZSTD_DCtx* dctx, ZSTD_format_e format);
/*! ZSTD_decompress_generic_simpleArgs() :
* Same as ZSTD_decompress_generic(),
* but using only integral types as arguments.
* This can be helpful for binders from dynamic languages
* which have troubles handling structures containing memory pointers.
*/
ZSTDLIB_API size_t ZSTD_decompress_generic_simpleArgs (
ZSTD_DCtx* dctx,
void* dst, size_t dstCapacity, size_t* dstPos,
const void* src, size_t srcSize, size_t* srcPos);
/********************************************************************
* Advanced streaming functions
* Warning : most of these functions are now redundant with the Advanced API.
* Once Advanced API reaches "stable" status,
* redundant functions will be deprecated, and then at some point removed.
********************************************************************/
/*===== Advanced Streaming compression functions =====*/