Commit Graph

10420 Commits

Author SHA1 Message Date
W. Felix Handte
9987d2f594 Unpoison Workspace Memory Before Freeing to Custom Free
MSAN is hooked into the system malloc, but when the user provides a custom
allocator, it may not provide the same cleansing behavior. So if we leave
memory poisoned and return it to the user's allocator, where it is re-used
elsewhere, our poisoning can blow up in some other context.
2023-08-16 12:09:12 -04:00
W. Felix Handte
5f5bdc1e5d Easy: Move Helper Functions Up 2023-08-16 12:08:52 -04:00
Nick Terrell
969e54f26e Improve dual license wording in README
We are licensed under BSD or GPLv2. It is clear in our headers, but not in the README.

Fixes #3717
2023-08-14 12:44:04 -04:00
Quentin Boswank
78dbba76b8 Updated Makefiles for full MSYS2 and Cygwin installation and testing support.
They are Linux-like environments under Windows and have all the tools needed to support staged installation and testing.

Beware: this only affects the make build system.
2023-08-13 19:44:15 +02:00
Yann Collet
d857369028
Merge pull request #3712 from alexsifivetw/fix_typo
Fixed typo
2023-08-02 11:11:31 -07:00
jysh1214
e99d554903 Fixed typo 2023-08-02 11:29:35 +08:00
Yann Collet
eb9f881eb8
Merge pull request #3701 from nikohoffren/grammar-fix
Fix typographical error in README.md
2023-07-31 03:25:17 -07:00
Yann Collet
65dbb0b77b
Merge pull request #3704 from void0red/dev
fileio_asyncio: handle malloc fails in AIO_ReadPool_create
2023-07-29 18:16:14 -07:00
void0red
4d267f3d4f
fileio_asyncio: handle malloc fails in AIO_ReadPool_create 2023-07-21 12:17:03 +08:00
Gianfranco Costamagna
de6b46dfc8 Update fileio.c: fix build failure with enabled LTO
For some reasons when LTO is enabled, the compiler complains about statbuf variable not being correctly initialized, even though the variable has an assert != NULL just few lines below (FIO_getDictFileStat)

This is the fixed build failure:
x86_64-linux-gnu-gcc -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fdebug-prefix-map=/<<PKGBUILDDIR>>=/usr/src/libzstd-1.5.5+dfsg2-1 -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wmissing-prototypes -Wc++-compat -g -Werror -Wa,--noexecstack -Wdate-time -D_FORTIFY_SOURCE=2 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=1 -DZSTD_LEGACY_SUPPORT=5 -DZSTD_MULTITHREAD -DZSTD_GZCOMPRESS -DZSTD_GZDECOMPRESS -DZSTD_LZMACOMPRESS -DZSTD_LZMADECOMPRESS -DZSTD_LZ4COMPRESS -DZSTD_LZ4DECOMPRESS -DZSTD_LEGACY_SUPPORT=5  -c -MT obj/conf_086c46a51a716b674719b8acb8484eb8/zstdcli_trace.o -MMD -MP -MF obj/conf_086c46a51a716b674719b8acb8484eb8/zstdcli_trace.d -o obj/conf_086c46a51a716b674719b8acb8484eb8/zstdcli_trace.o zstdcli_trace.c
In function ‘UTIL_isRegularFileStat’,
    inlined from ‘UTIL_getFileSizeStat’ at util.c:524:10,
    inlined from ‘FIO_createDResources’ at fileio.c:2230:30:
util.c:209:12: error: ‘statbuf.st_mode’ may be used uninitialized [-Werror=maybe-uninitialized]
  209 |     return S_ISREG(statbuf->st_mode) != 0;
      |            ^
fileio.c: In function ‘FIO_createDResources’:
fileio.c:2223:12: note: ‘statbuf’ declared here
 2223 |     stat_t statbuf;
      |            ^
lto1: all warnings being treated as errors
2023-07-20 19:00:13 -04:00
Jacob Greenfield
55ff3e4e17 Save one byte on the frame epilogue 2023-07-20 18:59:44 -04:00
Yann Collet
ef59cc39cd
Merge pull request #3665 from gjasny/fix-asm-for-xcode
Fix Intel Xcode builds with assembly
2023-07-20 01:48:39 -07:00
Niko Hoffrén
a1b9a5ad0e Fix typographical error in README.md 2023-07-19 21:49:43 +03:00
Elliot Gorokhovsky
ccd494860a
Merge pull request #3697 from facebook/dependabot/github_actions/github/codeql-action-2.20.3
Bump github/codeql-action from 2.20.1 to 2.20.3
2023-07-11 13:37:20 -07:00
dependabot[bot]
065ea9274f
Bump github/codeql-action from 2.20.1 to 2.20.3
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.20.1 to 2.20.3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](f6e388ebf0...46ed16ded9)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-10 05:37:59 +00:00
Yann Collet
118200f7b9
Merge pull request #3677 from facebook/detectOverflow
Changed the decoding loop to detect more invalid cases of corruption sooner
2023-07-05 00:59:08 -07:00
Yann Collet
25822342be
Merge pull request #3688 from nidhijaju/hide-asm-apple
Hide ASM symbols on Apple platforms
2023-06-29 19:40:37 -07:00
Elliot Gorokhovsky
68aa9a4ea3
Merge pull request #3689 from facebook/dependabot/github_actions/github/codeql-action-2.20.1
Bump github/codeql-action from 2.3.2 to 2.20.1
2023-06-27 18:05:52 -07:00
Elliot Gorokhovsky
8090c90491
Merge pull request #3690 from facebook/dependabot/github_actions/ossf/scorecard-action-2.2.0
Bump ossf/scorecard-action from 2.1.3 to 2.2.0
2023-06-27 18:05:32 -07:00
Elliot Gorokhovsky
52bbf1be54
Merge pull request #3686 from embg/ldm_error
Clean up a false error message in the LDM debug log
2023-06-26 13:43:20 -04:00
dependabot[bot]
2c97f5dbed
Bump ossf/scorecard-action from 2.1.3 to 2.2.0
Bumps [ossf/scorecard-action](https://github.com/ossf/scorecard-action) from 2.1.3 to 2.2.0.
- [Release notes](https://github.com/ossf/scorecard-action/releases)
- [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md)
- [Commits](80e868c13c...08b4669551)

---
updated-dependencies:
- dependency-name: ossf/scorecard-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-26 05:59:43 +00:00
dependabot[bot]
1a6278c82d
Bump github/codeql-action from 2.3.2 to 2.20.1
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.3.2 to 2.20.1.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](f3feb00acb...f6e388ebf0)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-26 05:59:38 +00:00
Nidhi Jaju
b1a30e2b4a hide asm functions on apple platforms 2023-06-26 00:07:30 +00:00
Yann Collet
e2cd353473
Merge pull request #3684 from emaste/dev
Update FreeBSD CI images to latest supported releases
2023-06-23 22:56:07 -07:00
Elliot Gorokhovsky
c6a888c073 suppress false error message in LDM mode 2023-06-21 19:19:02 -07:00
Ed Maste
f307493711 Update FreeBSD CI images to latest supported releases 2023-06-21 12:16:26 -04:00
Yann Collet
e4aeaebc20 fixed incorrect test in Win32 pthread wrapper
reported by @Banzai24-yht in #3683
2023-06-20 08:34:26 -07:00
Yann Collet
c123e69ad0 fixed static analyzer false positive regarding @sequence initialization
make a mock initialization to please the tool
2023-06-16 16:24:48 -07:00
Yann Collet
c60dcedcc9 adapted long decoder to new decodeSequences
removed older decodeSequences
2023-06-16 15:52:00 -07:00
Yann Collet
33fca19dd4 changed ZSTD_decompressSequences_bodySplitLitBuffer() decoding loop
to behave more like the regular decoding loop.
2023-06-16 15:32:07 -07:00
Yann Collet
84e898a76c removed _old variant from splitLit 2023-06-16 14:42:28 -07:00
Yann Collet
02134fad12 changed (partially) the decodeSequences flow logic
this allows detecting overflow events without a checksum.
2023-06-16 11:57:12 -07:00
Yann Collet
8b8b5f4d75
Merge pull request #3676 from facebook/overflow_zeroes
Bitstream produces only zeroes after an overflow event
2023-06-15 18:16:41 -07:00
Yann Collet
d9645327b3 fixed MEM_STATIC already defined in Linux Kernel mode 2023-06-14 20:07:18 -07:00
Yann Collet
74c901bbed fix : unused attribute for FORCE_INLINE functions
fix2 : reloadDStreamFast is used by decompress4x2,
modified the entry point, so that it works fine in this case too.
2023-06-14 16:32:51 -07:00
Yann Collet
ba50807029 make the bitstream generate only 0-value bits after an overflow 2023-06-14 15:42:37 -07:00
Yann Collet
bd00ec6804
Merge pull request #3674 from facebook/zeroSeq_noExtra
detect extraneous bytes in the Sequences section
2023-06-13 16:17:07 -07:00
Yann Collet
b46236278a detect extraneous bytes in the Sequences section
when nbSeq == 0.

Reported by @ip7z
2023-06-13 11:43:45 -07:00
Yann Collet
c8888d9bf4
Merge pull request #3671 from facebook/dependabot/github_actions/actions/checkout-3.5.3
Bump actions/checkout from 3.5.2 to 3.5.3
2023-06-12 09:58:53 -07:00
dependabot[bot]
6579f6c452
Bump actions/checkout from 3.5.2 to 3.5.3
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.2 to 3.5.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](8e5e7e5ab8...c85c95e3d7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-12 06:00:14 +00:00
Yann Collet
ba12054273
Merge pull request #3669 from facebook/zeroSeq_2B
fixed decoder behavior when nbSeqs==0 is encoded using 2 bytes
2023-06-06 00:06:15 -04:00
Yann Collet
3732a08f5b fixed decoder behavior when nbSeqs==0 is encoded using 2 bytes
The sequence section starts with a number, which tells how sequences are present in the section.
If this number if 0, the section automatically ends.

The number 0 can be represented using the 1 byte or the 2 bytes formats.
That's because the 2-bytes formats fully overlaps the 1 byte format.

However, when 0 is represented using the 2-bytes format,
the decoder was expecting the sequence section to continue,
and was looking for FSE tables, which is incorrect.

Fixed this behavior, in both the reference decoder and the educational behavior.

In practice, this behavior never happens,
because the encoder will always select the 1-byte format to represent 0,
since this is more efficient.

Completed the fix with a new golden sample for tests,
a clarification of the specification,
and a decoder errata paragraph.
2023-06-05 16:03:00 -07:00
Yann Collet
3e815f5b3a
Merge pull request #3664 from facebook/llu
changed LLU suffix into ULL for Visual 2012 and lower
2023-06-05 15:03:27 -04:00
Yann Collet
8030342eea
Merge pull request #3659 from facebook/fixHarness
Fixed a bug in the educational decoder
2023-06-05 15:03:14 -04:00
Yann Collet
f60100dd5f
Merge pull request #3668 from facebook/fix3667
fix a minor inefficiency in compress_superblock
2023-06-05 15:02:55 -04:00
Yann Collet
1f83b7cfc4 fix a minor inefficiency in compress_superblock
and in `decodecorpus`:
the specific case `nbSeq=127` can be represented using the 1-byte format.
Note that both the 1-byte and the 2-bytes formats are valid to represent this case,
so there was no "error", produced data remains valid,
it's just that the 1-byte format is more efficient.

fix #3667

Credit to @ip7z for finding this issue.
2023-06-05 09:51:52 -07:00
Gregor Jasny
7e09f07b32 Fix Intel Xcode builds with assembly
When forcing the source file language to `C`, Xcode enforces
the file to be compiled as `C` by appending `-x c` to the
compiler command line.

For now try to limit the damage and only enforce the language
if the ASM and C compilers differ.

Reproducer (CMake `3.26.4`, Xcode `14.3`):
```
cmake -S build/cmake -B _b -GXcode -DCMAKE_OSX_ARCHITECTURES=x86_64
cmake --build _b
```

Fix: #3622
2023-06-02 12:43:59 +02:00
Yann Collet
94a2f2791f changed LLU suffix into ULL for Visual 2012 and lower
both suffixes are supposed to be valid,
but for some reason, Visual 2012 and lower only support ULL.
2023-05-31 13:29:53 -07:00
Yann Collet
5108c9ac97 Fixed a bug in the educational decoder
Credit to Igor Pavlov
2023-05-27 11:22:30 -07:00
Tao He
5059618295 Add options in Makefile to cmake
Signed-off-by: Tao He <sighingnow@gmail.com>
2023-05-26 13:45:21 -07:00