Commit Graph

10420 Commits

Author SHA1 Message Date
Yann Collet
a3f1b2448c
Merge pull request #3902 from facebook/fuzz_debuglevel
Fix fuzz issue 5131069967892480
2024-02-09 13:07:46 -08:00
Yann Collet
b0e8580dc7 fix fuzz issue 5131069967892480 2024-02-08 16:38:20 -08:00
Yann Collet
695d154cac fuzz: control debuglevel from Makefile
and make the compilation faster
2024-02-08 16:23:52 -08:00
Yann Collet
54107e263f
Merge pull request #3900 from facebook/fix5921623844651008
fix 5921623844651008
2024-02-07 12:27:03 -08:00
Sergey Andreenko
a844455095
Merge pull request #3899 from sandreenko/dev
[Zstd] Less verbose log for patch mode.
2024-02-06 18:01:15 -05:00
Yann Collet
22574d848d fix issue 5921623844651008
ossfuzz managed to create a scenario which triggers an `assert`.
This fixes it, by giving +1 more space for the backward search pass.
2024-02-06 13:01:14 -08:00
sergeyandreenko
1f87c88ecf [Zstd] Less verbose log for patch mode. 2024-02-06 12:07:11 -08:00
Yann Collet
06b5b37b8d
Merge pull request #3895 from facebook/fix_3793
Improve compression of Arrays of Integers (High compression mode)
2024-02-05 20:12:24 -08:00
Yann Collet
b88c593d8f added or updated code comments
as suggested by @terrelln,
to make the code of the optimal parser a bit more understandable.
2024-02-05 18:32:25 -08:00
Yann Collet
887f5b62ae update compression results for regression tests 2024-02-05 01:27:22 -08:00
Yann Collet
6c35fb2e8c fix msan warnings 2024-02-05 01:21:06 -08:00
Yann Collet
641749fc09 fix uasan dictionary_stream_round_trip fuzz test 2024-02-05 00:36:10 -08:00
Yann Collet
5a08f58df2
Merge pull request #3897 from facebook/dependabot/github_actions/microsoft/setup-msbuild-2.0.0
Bump microsoft/setup-msbuild from 1.3.2 to 2.0.0
2024-02-04 23:49:56 -08:00
Yann Collet
0d9fb5dc33
fix msbuild action version number 2024-02-04 22:45:18 -08:00
dependabot[bot]
9fed5ef108
Bump microsoft/setup-msbuild from 1.3.2 to 2.0.0
Bumps [microsoft/setup-msbuild](https://github.com/microsoft/setup-msbuild) from 1.3.2 to 2.0.0.
- [Release notes](https://github.com/microsoft/setup-msbuild/releases)
- [Changelog](https://github.com/microsoft/setup-msbuild/blob/main/building-release.md)
- [Commits](031090342a...6fb0222098)

---
updated-dependencies:
- dependency-name: microsoft/setup-msbuild
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-05 05:43:28 +00:00
Yann Collet
fe2e2ad36d use ZSTD_memcpy()
which can be redirected in Linux kernel mode
2024-02-03 19:57:38 -08:00
Yann Collet
0ae21d8c31 removed trace control 2024-02-03 19:32:59 -08:00
Yann Collet
5474edbe60 fixed wrong assert
by introducing ZSTD_OPT_SIZE
2024-02-03 19:31:53 -08:00
Yann Collet
9ae3bf5ee2 update compression results
good news: there are only improvements
2024-02-03 17:52:50 -08:00
Yann Collet
e5af24c5fa fixed wrong assert 2024-02-03 17:48:29 -08:00
Yann Collet
8168a451e5 minor optimization, mostly for clarity 2024-02-03 17:26:47 -08:00
Yann Collet
d31018e223 finally, a version that generalizes well
While it's not always strictly a win,
it's a win for files that see a noticeably compression ratio increase,
while it's a very small noise for other files.

Downside is, this patch is less efficient for 32-bit arrays of integer
than the previous patch which was introducing losses for other files,
but it's still a net improvement on this scenario.
2024-02-03 14:26:18 -08:00
Felix Handte
7afd9db8db
Merge pull request #3893 from felixhandte/gh-actions-cet-test-update-dep
Update Dependency in Intel CET Test; Re-Enable Test
2024-02-01 14:49:37 -05:00
W. Felix Handte
04a6c8cbe2 Update Dependency in Intel CET Test; Re-Enable Test
Motivated by #3884.
2024-02-01 10:24:42 -08:00
Yann Collet
79e9459efc
Merge pull request #3890 from facebook/lorem
add a lorem ipsum generator
2024-02-01 10:15:57 -08:00
Yann Collet
0166b2ba80 modification: differentiate literal update at pos+1
helps when litlen==1 is cheaper than litlen==0

works great on pathological arr[u32] examples
but doesn't generalize well on other files.

silesia/x-ray is amoung the most negatively affected ones.
2024-01-31 11:20:43 -08:00
Yann Collet
81bfe3655e
Merge pull request #3886 from facebook/sparc64
add sparc64 compilation test
2024-01-31 10:55:11 -08:00
Yann Collet
4683667785 refactor optimal parser
store stretches as intermediate solution instead of sequences.
makes it possible to link a solution to a predecessor.
2024-01-31 02:51:46 -08:00
Yann Collet
de10f56be2 improve high compression ratio for file like #3793
this works great for 32-bit arrays,
notably the synthetic ones, with extreme regularity,
unfortunately, it's not universal,
and in some cases, it's a loss.
Crucially, on average, it's a loss on silesia.
The most negatively impacted file is x-ray.
It deserves an investigation before suggesting it as an evolution.
2024-01-29 23:25:24 -08:00
Yann Collet
fd03971252 blindfix meson recipe
note: absence of GLOB capability within meson makes its maintenance more painful.
2024-01-29 15:50:21 -08:00
Yann Collet
befcec1788 fix cmake recipe 2024-01-29 15:45:16 -08:00
Yann Collet
3ce4c6e046 fix Visual Studio solutions
note: we probably don't want to maintain VS2008 solution anymore.
Its successor VS2010 is > 10 years old,
which is more or less the limit after which we can stop supporting old compilers.
2024-01-29 15:24:42 -08:00
Yann Collet
a261375996 fix paramgrill Makefile recipe 2024-01-29 15:18:27 -08:00
Yann Collet
d0b7da30e2 add a lorem ipsum generator
this generator replaces the statistical generator
for the general case when no statistic is requested.

Generated data features a compression level speed / ratio curve
which is more in line with expectation.
2024-01-29 15:00:32 -08:00
Yann Collet
a7b4dafa03
Merge pull request #3888 from facebook/dependabot/github_actions/microsoft/setup-msbuild-1.3.2
Bump microsoft/setup-msbuild from 1.3.1 to 1.3.2
2024-01-29 14:34:53 -08:00
Yann Collet
4a74eb8cfb
Merge pull request #3887 from facebook/dependabot/github_actions/actions/upload-artifact-4.3.0
Bump actions/upload-artifact from 4.2.0 to 4.3.0
2024-01-29 09:49:36 -08:00
dependabot[bot]
c485b57bc7
Bump microsoft/setup-msbuild from 1.3.1 to 1.3.2
Bumps [microsoft/setup-msbuild](https://github.com/microsoft/setup-msbuild) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/microsoft/setup-msbuild/releases)
- [Changelog](https://github.com/microsoft/setup-msbuild/blob/main/building-release.md)
- [Commits](1ff57057b5...031090342a)

---
updated-dependencies:
- dependency-name: microsoft/setup-msbuild
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-29 05:53:45 +00:00
dependabot[bot]
163e9b6637
Bump actions/upload-artifact from 4.2.0 to 4.3.0
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.2.0 to 4.3.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](694cdabd8b...26f96dfa69)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-29 05:53:39 +00:00
Yann Collet
e1ef81a3ae add sparc64 compilation test 2024-01-28 20:25:04 -08:00
Yann Collet
60829445bf
Merge pull request #3880 from facebook/dependabot/github_actions/actions/upload-artifact-4.2.0
Bump actions/upload-artifact from 4.1.0 to 4.2.0
2024-01-27 21:49:29 -08:00
Yann Collet
798034aeae
Merge pull request #3883 from facebook/cmake_test
improve cmake test
2024-01-27 18:18:15 -08:00
Yann Collet
2fc7248412 fix cmakebuild test
write it in a way which is more compatible with older versions of cmake (<3.13)

Also:
fix pzstd compilation (notably on macos)
2024-01-27 17:30:06 -08:00
Yann Collet
799b904628
Merge pull request #3884 from facebook/disable_intelcet
disable Intel CET Compatibility tests
2024-01-27 17:29:17 -08:00
Yann Collet
c7611d6964 disable Intel CET Compatibility tests
The binary blob that must be downloaded from intel.com is no longer available
2024-01-27 15:37:29 -08:00
dependabot[bot]
ee2efb634e
Bump actions/upload-artifact from 4.1.0 to 4.2.0
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.1.0 to 4.2.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](1eb3cb2b3e...694cdabd8b)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-22 05:21:32 +00:00
Yann Collet
1a860c8737
Merge pull request #3877 from facebook/oldgrep
minor: playTests.sh more compatible with older versions of grep
2024-01-21 11:31:58 -08:00
Yann Collet
81f444f4f9 made playTests.sh more compatible with older versions of grep
replaced `\+` by `*`.
`\+` means `[1-N]`,
while `*` means `[0-N]`,
so it's not strictly equivalent
but `\+` happens to be badly supported on some flavors of grep,
and for the purpose of these tests, `*` is good enough.
2024-01-16 12:14:35 -08:00
Yann Collet
050fec5c37
Merge pull request #3865 from facebook/grep_e
playTests.sh does no longer needs grep -E
2024-01-15 12:44:46 -08:00
Yann Collet
e6f4b46493 playTests.sh does no longer needs grep -E
it makes the test script more portable across posix systems
because `grep -E` is not guaranteed
while `grep` is fairly common.
2024-01-15 11:16:46 -08:00
Yann Collet
d2ea3fb5ef
Merge pull request #3863 from facebook/dependabot/github_actions/github/codeql-action-3.23.0
Bump github/codeql-action from 2.21.4 to 3.23.0
2024-01-15 09:44:19 -08:00