mirrors/zstd
0
0
Fork 0
mirror of https://github.com/facebook/zstd.git synced 2024-11-27 11:46:47 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,431 Commits 29 Branches 75 Tags 55 MiB
1e240af30a
Commit Graph

10431 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet
1e240af30a fix datagen size control 2024-02-20 18:06:56 -08:00
Yann Collet
7a225c0c46 internal benchmark: can select size of generated synthetic sample 2024-02-20 15:47:09 -08:00
Yann Collet
83598aa106 datagen generates lorem ipsum by default 2024-02-20 15:24:25 -08:00
Yann Collet
7003c9905e increase word dictionary 
for higher variety of messages.
Now, level 5 compresses better than level 4 (by a hair).
 2024-02-20 13:27:36 -08:00
Yann Collet
3dbd861b7d runtime weight distribution table 
and made small words a bit more common.
 2024-02-20 12:26:37 -08:00
Yann Collet
5a1bb4a4e0 add question marks 
and (slightly) longer sentences.
 2024-02-20 00:37:21 -08:00
Yann Collet
40874d4aea enriched vocabulary again 
using real latin sentences from Cicero.

Compression ratio lower again, closer to "real" text,

now level 6 is way better than level 4.

level 5 is still lower than level 4,
but at least it's now higher than level 3.
 2024-02-20 00:30:29 -08:00
Yann Collet
1e046ce7fa increase vocabulary size 
makes compression a bit less good,
hence a bit more comparable with real text (though still too easy to compress).
level 6 is now stronger than level 4, by a hair.
However, there is still a ratio dip at level 5.
 2024-02-20 00:12:32 -08:00
Felix Handte
889392dac8
Merge pull request #3909 from felixhandte/security-md-notif-list-ad 
Advertise Availability of Security Vulnerability Notifications
 2024-02-13 12:38:18 -05:00
W. Felix Handte
e13d099bf8 Advertise Availability of Security Vulnerability Notifications 2024-02-13 11:54:59 -05:00
W. Felix Handte
b6805c54d6 Add SECURITY.md File 
This just adds a copy of the Meta default SECURITY.md that we can then modify.
 2024-02-13 11:50:55 -05:00
Yann Collet
a3f1b2448c
Merge pull request #3902 from facebook/fuzz_debuglevel 
Fix fuzz issue 5131069967892480
 2024-02-09 13:07:46 -08:00
Yann Collet
b0e8580dc7 fix fuzz issue 5131069967892480 2024-02-08 16:38:20 -08:00
Yann Collet
695d154cac fuzz: control debuglevel from Makefile 
and make the compilation faster
 2024-02-08 16:23:52 -08:00
Yann Collet
54107e263f
Merge pull request #3900 from facebook/fix5921623844651008 
fix 5921623844651008
 2024-02-07 12:27:03 -08:00
Sergey Andreenko
a844455095
Merge pull request #3899 from sandreenko/dev 
[Zstd] Less verbose log for patch mode.
 2024-02-06 18:01:15 -05:00
Yann Collet
22574d848d fix issue 5921623844651008 
ossfuzz managed to create a scenario which triggers an `assert`.
This fixes it, by giving +1 more space for the backward search pass.
 2024-02-06 13:01:14 -08:00
sergeyandreenko
1f87c88ecf [Zstd] Less verbose log for patch mode. 2024-02-06 12:07:11 -08:00
Yann Collet
06b5b37b8d
Merge pull request #3895 from facebook/fix_3793 
Improve compression of Arrays of Integers (High compression mode)
 2024-02-05 20:12:24 -08:00
Yann Collet
b88c593d8f added or updated code comments 
as suggested by @terrelln,
to make the code of the optimal parser a bit more understandable.
 2024-02-05 18:32:25 -08:00
Yann Collet
887f5b62ae update compression results for regression tests 2024-02-05 01:27:22 -08:00
Yann Collet
6c35fb2e8c fix msan warnings 2024-02-05 01:21:06 -08:00
Yann Collet
641749fc09 fix uasan dictionary_stream_round_trip fuzz test 2024-02-05 00:36:10 -08:00
Yann Collet
5a08f58df2
Merge pull request #3897 from facebook/dependabot/github_actions/microsoft/setup-msbuild-2.0.0 
Bump microsoft/setup-msbuild from 1.3.2 to 2.0.0
 2024-02-04 23:49:56 -08:00
Yann Collet
0d9fb5dc33
fix msbuild action version number 2024-02-04 22:45:18 -08:00
dependabot[bot]
9fed5ef108
Bump microsoft/setup-msbuild from 1.3.2 to 2.0.0 
Bumps [microsoft/setup-msbuild](https://github.com/microsoft/setup-msbuild) from 1.3.2 to 2.0.0.
- [Release notes](https://github.com/microsoft/setup-msbuild/releases)
- [Changelog](https://github.com/microsoft/setup-msbuild/blob/main/building-release.md)
- [Commits](031090342a...6fb0222098)

---
updated-dependencies:
- dependency-name: microsoft/setup-msbuild
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-02-05 05:43:28 +00:00
Yann Collet
fe2e2ad36d use ZSTD_memcpy() 
which can be redirected in Linux kernel mode
 2024-02-03 19:57:38 -08:00
Yann Collet
0ae21d8c31 removed trace control 2024-02-03 19:32:59 -08:00
Yann Collet
5474edbe60 fixed wrong assert 
by introducing ZSTD_OPT_SIZE
 2024-02-03 19:31:53 -08:00
Yann Collet
9ae3bf5ee2 update compression results 
good news: there are only improvements
 2024-02-03 17:52:50 -08:00
Yann Collet
e5af24c5fa fixed wrong assert 2024-02-03 17:48:29 -08:00
Yann Collet
8168a451e5 minor optimization, mostly for clarity 2024-02-03 17:26:47 -08:00
Yann Collet
d31018e223 finally, a version that generalizes well 
While it's not always strictly a win,
it's a win for files that see a noticeably compression ratio increase,
while it's a very small noise for other files.

Downside is, this patch is less efficient for 32-bit arrays of integer
than the previous patch which was introducing losses for other files,
but it's still a net improvement on this scenario.
 2024-02-03 14:26:18 -08:00
Felix Handte
7afd9db8db
Merge pull request #3893 from felixhandte/gh-actions-cet-test-update-dep 
Update Dependency in Intel CET Test; Re-Enable Test
 2024-02-01 14:49:37 -05:00
W. Felix Handte
04a6c8cbe2 Update Dependency in Intel CET Test; Re-Enable Test 
Motivated by #3884.
 2024-02-01 10:24:42 -08:00
Yann Collet
79e9459efc
Merge pull request #3890 from facebook/lorem 
add a lorem ipsum generator
 2024-02-01 10:15:57 -08:00
Yann Collet
0166b2ba80 modification: differentiate literal update at pos+1 
helps when litlen==1 is cheaper than litlen==0

works great on pathological arr[u32] examples
but doesn't generalize well on other files.

silesia/x-ray is amoung the most negatively affected ones.
 2024-01-31 11:20:43 -08:00
Yann Collet
81bfe3655e
Merge pull request #3886 from facebook/sparc64 
add sparc64 compilation test
 2024-01-31 10:55:11 -08:00
Yann Collet
4683667785 refactor optimal parser 
store stretches as intermediate solution instead of sequences.
makes it possible to link a solution to a predecessor.
 2024-01-31 02:51:46 -08:00
Yann Collet
de10f56be2 improve high compression ratio for file like #3793 
this works great for 32-bit arrays,
notably the synthetic ones, with extreme regularity,
unfortunately, it's not universal,
and in some cases, it's a loss.
Crucially, on average, it's a loss on silesia.
The most negatively impacted file is x-ray.
It deserves an investigation before suggesting it as an evolution.
 2024-01-29 23:25:24 -08:00
Yann Collet
fd03971252 blindfix meson recipe 
note: absence of GLOB capability within meson makes its maintenance more painful.
 2024-01-29 15:50:21 -08:00
Yann Collet
befcec1788 fix cmake recipe 2024-01-29 15:45:16 -08:00
Yann Collet
3ce4c6e046 fix Visual Studio solutions 
note: we probably don't want to maintain VS2008 solution anymore.
Its successor VS2010 is > 10 years old,
which is more or less the limit after which we can stop supporting old compilers.
 2024-01-29 15:24:42 -08:00
Yann Collet
a261375996 fix paramgrill Makefile recipe 2024-01-29 15:18:27 -08:00
Yann Collet
d0b7da30e2 add a lorem ipsum generator 
this generator replaces the statistical generator
for the general case when no statistic is requested.

Generated data features a compression level speed / ratio curve
which is more in line with expectation.
 2024-01-29 15:00:32 -08:00
Yann Collet
a7b4dafa03
Merge pull request #3888 from facebook/dependabot/github_actions/microsoft/setup-msbuild-1.3.2 
Bump microsoft/setup-msbuild from 1.3.1 to 1.3.2
 2024-01-29 14:34:53 -08:00
Yann Collet
4a74eb8cfb
Merge pull request #3887 from facebook/dependabot/github_actions/actions/upload-artifact-4.3.0 
Bump actions/upload-artifact from 4.2.0 to 4.3.0
 2024-01-29 09:49:36 -08:00
dependabot[bot]
c485b57bc7
Bump microsoft/setup-msbuild from 1.3.1 to 1.3.2 
Bumps [microsoft/setup-msbuild](https://github.com/microsoft/setup-msbuild) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/microsoft/setup-msbuild/releases)
- [Changelog](https://github.com/microsoft/setup-msbuild/blob/main/building-release.md)
- [Commits](1ff57057b5...031090342a)

---
updated-dependencies:
- dependency-name: microsoft/setup-msbuild
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-29 05:53:45 +00:00
dependabot[bot]
163e9b6637
Bump actions/upload-artifact from 4.2.0 to 4.3.0 
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.2.0 to 4.3.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](694cdabd8b...26f96dfa69)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-29 05:53:39 +00:00
Yann Collet
e1ef81a3ae add sparc64 compilation test 2024-01-28 20:25:04 -08:00