Commit Graph

215 Commits

Author SHA1 Message Date
Patrick Steinhardt
13420028e5 global: convert trivial usages of test <expr> -a/-o <expr>
Our coding guidelines say to not use `test` with `-a` and `-o` because
it can easily lead to bugs. Convert trivial cases where we still use
these to instead instead concatenate multiple invocations of `test` via
`&&` and `||`, respectively.

While not all of the converted instances can cause ambiguity, it is
worth getting rid of all of them regardless:

    - It becomes easier to reason about the code as we do not have to
      argue why one use of `-a`/`-o` is okay while another one isn't.

    - We don't encourage people to use these expressions.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-11 09:21:00 +09:00
Andreas Herrmann
fb8f7269c2 configure.ac: always save NO_ICONV to config.status
In case 'configure --with-iconv=no' is used, NO_ICONV is not saved to
config.status and thus git is built with iconv support.

Always save NO_ICONV to config.status to honor what user selected
during configure step.

Signed-off-by: Andreas Herrmann <aherrmann@suse.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-19 10:07:55 -07:00
Andreas Herrmann
92d8f00a11 configure.ac: don't overwrite NO_CURL option
Even if 'configure --with-curl=no' was run, curl support is used,
because library detection overwrites it. Avoid this overwrite.
Configure should obey what the user has specified.

Signed-off-by: Andreas Herrmann <aherrmann@suse.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-19 10:07:55 -07:00
Andreas Herrmann
0dd79e0d49 configure.ac: don't overwrite NO_EXPAT option
Even if 'configure --with-expat=no' was run, expat support is used,
because library detection overwrites it. Avoid this overwrite.
Configure should obey what the user has specified.

Signed-off-by: Andreas Herrmann <aherrmann@suse.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-19 10:07:55 -07:00
Ævar Arnfjörð Bjarmason
9dc523aa0e Makefile + hash.h: remove PPC_SHA1 implementation
Remove the PPC_SHA1 implementation added in a6ef3518f9 ([PATCH] PPC
assembly implementation of SHA1, 2005-04-22). When this was added
Apple consumer hardware used the PPC architecture, and the
implementation was intended to improve SHA-1 speed there.

Since it was added we've moved to using sha1collisiondetection by
default, and anyone wanting hard-rolled non-DC SHA-1 implementation
can use OpenSSL's via the OPENSSL_SHA1 knob.

The PPC_SHA1 originally originally targeted 32 bit PPC, and later the
64 bit PPC 970 (a.k.a. Apple PowerPC G5). See 926172c5e4 (block-sha1:
improve code on large-register-set machines, 2009-08-10) for a
reference about the performance on G5 (a comment in block-sha1/sha1.c
being removed here).

I can't get it to do anything but segfault on both the BE and LE POWER
machines in the GCC compile farm[1]. Anyone who's concerned about
performance on PPC these days is likely to be using the IBM POWER
processors.

There have been proposals to entirely remove non-sha1collisiondetection
implementations from the tree[2]. I think per [3] that would be a bit
overzealous. I.e. there are various set-ups git's speed is going to be
more important than the relatively implausible SHA-1 collision attack,
or where such attacks are entirely mitigated by other means (e.g. by
incoming objects being checked with DC_SHA1).

But that really doesn't apply to PPC_SHA1 in particular, which seems
to have outlived its usefulness.

As this gets rid of the only in-tree *.S assembly file we can remove
the small bits of logic from the Makefile needed to build objects
from *.S (as opposed to *.c)

The code being removed here was also throwing warnings with the
"-pedantic" flag, it could have been fixed as 544d93bc3b (block-sha1:
remove use of obsolete x86 assembly, 2022-03-10) did for block-sha1/*,
but as noted above let's remove it instead.

1. https://cfarm.tetaneutral.net/machines/list/
   Tested on gcc{110,112,135,203}, a mixture of POWER [789] ppc64 and
   ppc64le. All segfault in anything needing object
   hashing (e.g. t/t1007-hash-object.sh) when compiled with
   PPC_SHA1=Y.
2. https://lore.kernel.org/git/20200223223758.120941-1-mh@glandium.org/
3. https://lore.kernel.org/git/20200224044732.GK1018190@coredump.intra.peff.net/

Acked-by: brian m. carlson" <sandals@crustytoothpaste.net>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-31 14:37:31 -07:00
Junio C Hamano
88cbd17e87 Merge branch 'ab/misc-cleanup'
Code clean-up.

* ab/misc-cleanup:
  alloc.[ch]: remove alloc_report() function
  object-store.h: remove unused has_sha1_file*()
  pack-bitmap-write: remove unused bitmap_reset() function
  xdiff/xmacros.h: remove unused XDL_PTRFREE
  configure.ac: remove USE_PIC comment
  run-command.h: remove always unused "clean_on_exit_handler_cbdata"
2022-05-10 17:41:10 -07:00
Adam Dinwoodie
2e37594797 configure.ac: fix HAVE_SYNC_FILE_RANGE definition
If sync_file_range is not available when building the configure script,
there is a cosmetic bug when running that script reporting
"HAVE_SYNC_FILE_RANGE: command not found".  Remove that error message by
defining HAVE_SYNC_FILE_RANGE to an empty string, rather than generating
a script where that appears as a bare command.

Signed-off-by: Adam Dinwoodie <adam@dinwoodie.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-06 09:36:19 -07:00
Ævar Arnfjörð Bjarmason
7146f271c6 configure.ac: remove USE_PIC comment
Remove a comment about a Makefile knob that was removed in
f7661ce0b8 (Remove -fPIC which was only needed for Git.xs,
2006-09-29). The comment had been copied over to configure.ac in
633b423961 (Copy description of build configuration variables to
configure.ac, 2006-07-08).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 10:16:05 -07:00
Neeraj Singh
abf38abec2 core.fsyncmethod: add writeout-only mode
This commit introduces the `core.fsyncMethod` configuration
knob, which can currently be set to `fsync` or `writeout-only`.

The new writeout-only mode attempts to tell the operating system to
flush its in-memory page cache to the storage hardware without issuing a
CACHE_FLUSH command to the storage controller.

Writeout-only fsync is significantly faster than a vanilla fsync on
common hardware, since data is written to a disk-side cache rather than
all the way to a durable medium. Later changes in this patch series will
take advantage of this primitive to implement batching of hardware
flushes.

When git_fsync is called with FSYNC_WRITEOUT_ONLY, it may fail and the
caller is expected to do an ordinary fsync as needed.

On Apple platforms, the fsync system call does not issue a CACHE_FLUSH
directive to the storage controller. This change updates fsync to do
fcntl(F_FULLFSYNC) to make fsync actually durable. We maintain parity
with existing behavior on Apple platforms by setting the default value
of the new core.fsyncMethod option.

Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-10 15:10:22 -08:00
Ævar Arnfjörð Bjarmason
07564773c2 compat: auto-detect if zlib has uncompress2()
We have a copy of uncompress2() implementation in compat/ so that we
can build with an older version of zlib that lack the function, and
the build procedure selects if it is used via the NO_UNCOMPRESS2
$(MAKE) variable.  This is yet another "annoying" knob the porters
need to tweak on platforms that are not common enough to have the
default set in the config.mak.uname file.

Attempt to instead ask the system header <zlib.h> to decide if we
need the compatibility implementation.  This is a deviation from the
way we have been handling the "compatiblity" features so far, and if
it can be done cleanly enough, it could work as a model for features
that need compatibility definition we discover in the future.  With
that goal in mind, avoid expedient but ugly hacks, like shoving the
code that is conditionally compiled into an unrelated .c file, which
may not work in future cases---instead, take an approach that uses a
file that is independently compiled and stands on its own.

Compile and link compat/zlib-uncompress2.c file unconditionally, but
conditionally hide the implementation behind #if/#endif when zlib
version is 1.2.9 or newer, and unconditionally archive the resulting
object file in the libgit.a to be picked up by the linker.

There are a few things to note in the shape of the code base after
this change:

 - We no longer use NO_UNCOMPRESS2 knob; if the system header
   <zlib.h> claims a version that is more cent than the library
   actually is, this would break, but it is easy to add it back when
   we find such a system.

 - The object file compat/zlib-uncompress2.o is always compiled and
   archived in libgit.a, just like a few other compat/ object files
   already are.

 - The inclusion of <zlib.h> is done in <git-compat-util.h>; we used
   to do so from <cache.h> which includes <git-compat-util.h> as the
   first thing it does, so from the *.c codes, there is no practical
   change.

 - Until objects in libgit.a that is already used gains a reference
   to the function, the reftable code will be the only one that
   wants it, so libgit.a on the linker command line needs to appear
   once more at the end to satisify the mutual dependency.

 - Beat found a trick used by OpenSSL to avoid making the
   conditionally-compiled object truly empty (apparently because
   they had to deal with compilers that do not want to see an
   effectively empty input file).  Our compat/zlib-uncompress2.c
   file borrows the same trick for portabilty.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Helped-by: Beat Bolli <dev+git@drbeat.li>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-26 09:05:55 -08:00
Junio C Hamano
a4bbd13be3 Merge branch 'hn/reftable'
The "reftable" backend for the refs API, without integrating into
the refs subsystem, has been added.

* hn/reftable:
  Add "test-tool dump-reftable" command.
  reftable: add dump utility
  reftable: implement stack, a mutable database of reftable files.
  reftable: implement refname validation
  reftable: add merged table view
  reftable: add a heap-based priority queue for reftable records
  reftable: reftable file level tests
  reftable: read reftable files
  reftable: generic interface to tables
  reftable: write reftable files
  reftable: a generic binary tree implementation
  reftable: reading/writing blocks
  Provide zlib's uncompress2 from compat/zlib-compat.c
  reftable: (de)serialization for the polymorphic record type.
  reftable: add blocksource, an abstraction for random access reads
  reftable: utility functions
  reftable: add error related functionality
  reftable: add LICENSE
  hash.h: provide constants for the hash IDs
2021-12-15 09:39:45 -08:00
Ævar Arnfjörð Bjarmason
ebeb39faad git-sh-setup: remove "sane_grep", it's not needed anymore
Remove the sane_grep() shell function in git-sh-setup. The two reasons
for why it existed don't apply anymore:

1. It was added due to GNU grep supporting GREP_OPTIONS. See
   e1622bfcba (Protect scripted Porcelains from GREP_OPTIONS insanity,
   2009-11-23).

   Newer versions of GNU grep ignore that, but even on older versions
   its existence won't matter, none of these sane_grep() uses care
   about grep's output, they're merely using it to check if a string
   exists in a file or stream. We also don't care about the "LC_ALL=C"
   that "sane_grep" was using, these greps for fixed or ASCII strings
   will behave the same under any locale.

2. The SANE_TEXT_GREP added in 71b401032b (sane_grep: pass "-a" if
   grep accepts it, 2016-03-08) isn't needed either, none of these grep
   uses deal with binary data.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-21 16:17:57 -07:00
Han-Wen Nienhuys
a322920d0b Provide zlib's uncompress2 from compat/zlib-compat.c
This will be needed for reading reflog blocks in reftable.

Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08 10:45:48 -07:00
Ævar Arnfjörð Bjarmason
7599730b7e Remove support for v1 of the PCRE library
Remove support for using version 1 of the PCRE library. Its use has
been discouraged by upstream for a long time, and it's in a
bugfix-only state.

Anyone who was relying on v1 in particular got a nudge to move to v2
in e6c531b808 (Makefile: make USE_LIBPCRE=YesPlease mean v2, not v1,
2018-03-11), which was first released as part of v2.18.0.

With this the LIBPCRE2 test prerequisites is redundant to PCRE. But
I'm keeping it for self-documentation purposes, and to avoid conflict
with other in-flight PCRE patches.

I'm also not changing all of our own "pcre2" names to "pcre", i.e. the
inverse of 6d4b5747f0 (grep: change internal *pcre* variable &
function names to be *pcre1*, 2017-05-25). I don't see the point, and
it makes the history/blame harder to read. Maybe if there's ever a
PCRE v3...

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-23 21:15:43 -08:00
Junio C Hamano
6511cb33c9 Merge branch 'dd/sequencer-utf8'
Handling of commit objects that use non UTF-8 encoding during
"rebase -i" has been improved.

* dd/sequencer-utf8:
  sequencer: reencode commit message for am/rebase --show-current-patch
  sequencer: reencode old merge-commit message
  sequencer: reencode squashing commit's message
  sequencer: reencode revert/cherry-pick's todo list
  sequencer: reencode to utf-8 before arrange rebase's todo list
  t3900: demonstrate git-rebase problem with multi encoding
  configure.ac: define ICONV_OMITS_BOM if necessary
  t0028: eliminate non-standard usage of printf
2019-12-01 09:04:36 -08:00
Doan Tran Cong Danh
1ba6e7aecd configure.ac: define ICONV_OMITS_BOM if necessary
From commit 79444c9294, ("utf8: handle systems that don't write BOM for
UTF-16", 2019-02-12), we're supporting those systems with iconv that
omits BOM with:

    make ICONV_OMITS_BOM=Yes

However, configure script wasn't taught to detect those systems.

Teach configure to do so.

Signed-off-by: Doan Tran Cong Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-11 09:43:47 +09:00
Elijah Newren
15beaaa3d1 Fix spelling errors in code comments
Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-10 16:00:54 +09:00
Michael Osipov
9a1699c88e configure: Detect linking style for HP aCC on HP-UX
HP aCC does not accept any of the previously tested CC_LD_DYNPATH
formats, but only its own[1] "-Wl,+b" format. Add it to configure.ac.

1. http://nixdoc.net/man-pages/hp-ux/man1/ld_pa.1.html

Signed-off-by: Michael Osipov <michael.osipov@siemens.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-07 10:06:11 -07:00
Vadim Kochan
b71e56a683 autoconf: #include <libintl.h> when checking for gettext()
Some libc implementations like uclibc or musl provides
gettext stubs via libintl library but this case is not checked
by AC_CHECK_LIB(c, gettext ...) because gcc has gettext as builtin
which passess the check.

So check it with included libintl.h where gettext may unfold into
libintl_gettext which will cause check to fail if libintl_gettext are
needed to be linked with -lintl.

Signed-off-by: Vadim Kochan <vadim4j@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 13:57:23 +09:00
Junio C Hamano
1c6e646235 Merge branch 'dd/poll-dot-h'
A build update.

* dd/poll-dot-h:
  git-compat-util: prefer poll.h to sys/poll.h
2018-11-19 16:24:41 +09:00
Đoàn Trần Công Danh
2648ccc231 git-compat-util: prefer poll.h to sys/poll.h
POSIX specifies that <poll.h> is the correct header for poll(2)
whereas <sys/poll.h> is only needed for some old libc.

Let's follow the POSIX way by default.

This effectively eliminates musl's warning:

    warning redirecting incorrect #include <sys/poll.h> to <poll.h>

Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14 16:32:24 +09:00
James Knight
23c4bbe28e build: link with curl-defined linker flags
Adjusting the build process to rely more on curl-config to populate
linker flags instead of manually populating flags based off detected
features.

Originally, a configure-invoked build would check for SSL-support in the
target curl library. If enabled, NEEDS_SSL_WITH_CURL would be set and
used in the Makefile to append additional libraries to link against. As
for systems building solely with make, the defines NEEDS_IDN_WITH_CURL
and NEEDS_SSL_WITH_CURL could be set to indirectly enable respective
linker flags. Since both configure.ac and Makefile already rely on
curl-config utility to provide curl-related build information, adjusting
the respective assets to populate required linker flags using the
utility (unless explicitly configured).

Signed-off-by: James Knight <james.d.knight@live.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-05 10:19:25 +09:00
Junio C Hamano
8b22d13243 Merge branch 'es/fread-reads-dir-autoconf-fix'
Small fix to the autoconf build procedure.

* es/fread-reads-dir-autoconf-fix:
  configure.ac: fix botched FREAD_READS_DIRECTORIES check
2018-04-25 13:29:01 +09:00
Eric Sunshine
249482daf0 configure.ac: fix botched FREAD_READS_DIRECTORIES check
3adf9fdecf (configure.ac: loosen FREAD_READS_DIRECTORIES test program,
2017-06-14) broke the test program for the FREAD_READS_DIRECTORIES check
by making it syntactically invalid (a dangling ")") and by botching the
type returned from 'main' (a FILE* rather than int). As a consequence,
the test program won't even compile, thus the check fails
unconditionally. Fix these problems.

Reported-by: Jonathan Primrose <jprimros@gmail.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-09 18:50:35 +09:00
Ævar Arnfjörð Bjarmason
e6c531b808 Makefile: make USE_LIBPCRE=YesPlease mean v2, not v1
Change the USE_LIBPCRE flag from being an alias for USE_LIBPCRE1 to
being an alias for USE_LIBPCRE2.

When support for v2 was added in my 94da9193a6 ("grep: add support for
PCRE v2", 2017-06-01) the existing USE_LIBPCRE flag was left as
meaning v1, with a note that this would likely change in a future
release. That optional support for v2 first made it into Git version
2.14.0.

The PCRE v2 support has been shown to be stable, and the upstream PCRE
project is highly encouraging downstream users to move to v2, so it
makes sense to give packagers of Git who haven't heard the news about
PCRE v2 a further nudge to move to v2.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14 15:27:50 -07:00
Ævar Arnfjörð Bjarmason
a363f981ec configure: detect redundant --with-libpcre & --with-libpcre1
The --with-libpcre option is a synonym for the --with-libpcre1 flag,
but the configure script allowed for redundantly specifying both.

Nothing broke as a result of this, but it's confusing, so let's
disallow it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14 15:27:47 -07:00
Ævar Arnfjörð Bjarmason
a91b113320 configure: fix a regression in PCRE v1 detection
Change the check for PCRE v1 to disable the --with-libpcre1 option if
the pcre_version() function can't be found in the pcre library. I
unintentionally changed this in my 94da9193a6 ("grep: add support for
PCRE v2", 2017-06-01) while renaming moving some variables.

The intent of this check ever since it was added in
a119f91e57 ("configure: Check for libpcre", 2011-05-09) is to
second-guess the user and turn off an explicitly provided
--with-libpcre if the library can't be found.

I don't think that behavior makes any sense, we shouldn't be
second-guessing the user with an auto-detection, but changing that
needs a bigger refactoring of this script, and only has marginal
benefits. So let's fix it to work as it was intended to work again.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14 15:27:36 -07:00
Ævar Arnfjörð Bjarmason
edb6a17c36 Makefile: NO_OPENSSL=1 should no longer imply BLK_SHA1=1
Use the collision detecting SHA-1 implementation by default even when
NO_OPENSSL is set.

Setting NO_OPENSSL=UnfortunatelyYes has implied BLK_SHA1=1 ever since
the former was introduced in dd53c7ab29 (Support for NO_OPENSSL,
2005-07-29).  That implication should have been removed when the
default SHA-1 implementation changed from OpenSSL to DC_SHA1 in
e6b07da278 (Makefile: make DC_SHA1 the default, 2017-03-17).  Finish
what that commit started by removing the BLK_SHA1 fallback setting so
the default DC_SHA1 implementation will be used.

Helped-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-28 11:55:56 -08:00
Junio C Hamano
df7fd961a9 Merge branch 'nd/fopen-errors'
Hotfix for a topic that is already in 'master'.

* nd/fopen-errors:
  configure.ac: loosen FREAD_READS_DIRECTORIES test program
2017-06-22 14:15:20 -07:00
Jeff King
3adf9fdecf configure.ac: loosen FREAD_READS_DIRECTORIES test program
We added an FREAD_READS_DIRECTORIES Makefile knob long ago
in cba22528f (Add compat/fopen.c which returns NULL on
attempt to open directory, 2008-02-08) to handle systems
where reading from a directory returned garbage. This works
by catching the problem at the fopen() stage and returning
NULL.

More recently, we found that there is a class of systems
(including Linux) where fopen() succeeds but fread() fails.
Since the solution is the same (having fopen return NULL),
they use the same Makefile knob as of e2d90fd1c
(config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and
FreeBSD, 2017-05-03).

This works fine except for one thing: the autoconf test in
configure.ac to set FREAD_READS_DIRECTORIES actually checks
whether fread succeeds. Which means that on Linux systems,
the knob isn't set (and we even override the config.mak.uname
default). t1308 catches the failure.

We can fix this by tweaking the autoconf test to cover both
cases. In theory we might care about the distinction between
the traditional "fread reads directories" case and the new
"fopen opens directories". But since our solution catches
the problem at the fopen stage either way, we don't actually
need to know the difference. The "fopen" case is a superset.

This does mean the FREAD_READS_DIRECTORIES name is slightly
misleading. Probably FOPEN_OPENS_DIRECTORIES would be more
accurate. But it would be disruptive to simply change the
name (people's existing build configs would fail), and it's
not worth the complexity of handling both. Let's just add a
comment in the knob description.

Reported-by: Øyvind A. Holm <sunny@sunbase.org>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-15 14:14:33 -07:00
Ævar Arnfjörð Bjarmason
94da9193a6 grep: add support for PCRE v2
Add support for v2 of the PCRE API. This is a new major version of
PCRE that came out in early 2015[1].

The regular expression syntax is the same, but while the API is
similar, pretty much every function is either renamed or takes
different arguments. Thus using it via entirely new functions makes
sense, as opposed to trying to e.g. have one compile_pcre_pattern()
that would call either PCRE v1 or v2 functions.

Git can now be compiled with either USE_LIBPCRE1=YesPlease or
USE_LIBPCRE2=YesPlease, with USE_LIBPCRE=YesPlease currently being a
synonym for the former. Providing both is a compile-time error.

With earlier patches to enable JIT for PCRE v1 the performance of the
release versions of both libraries is almost exactly the same, with
PCRE v2 being around 1% slower.

However after I reported this to the pcre-dev mailing list[2] I got a
lot of help with the API use from Zoltán Herczeg, he subsequently
optimized some of the JIT functionality in v2 of the library.

Running the p7820-grep-engines.sh performance test against the latest
Subversion trunk of both, with both them and git compiled as -O3, and
the test run against linux.git, gives the following results. Just the
/perl/ tests shown:

    $ GIT_PERF_REPEAT_COUNT=30 GIT_PERF_LARGE_REPO=~/g/linux GIT_PERF_MAKE_COMMAND='grep -q LIBPCRE2 Makefile && make -j8 USE_LIBPCRE2=YesPlease CC=~/perl5/installed/bin/gcc NO_R_TO_GCC_LINKER=YesPlease CFLAGS=-O3 LIBPCREDIR=/home/avar/g/pcre2/inst LDFLAGS=-Wl,-rpath,/home/avar/g/pcre2/inst/lib || make -j8 USE_LIBPCRE=YesPlease CC=~/perl5/installed/bin/gcc NO_R_TO_GCC_LINKER=YesPlease CFLAGS=-O3 LIBPCREDIR=/home/avar/g/pcre/inst LDFLAGS=-Wl,-rpath,/home/avar/g/pcre/inst/lib' ./run HEAD~5 HEAD~ HEAD p7820-grep-engines.sh
    [...]
    Test                                            HEAD~5            HEAD~                    HEAD
    -----------------------------------------------------------------------------------------------------------------
    7820.3: perl grep 'how.to'                      0.31(1.10+0.48)   0.21(0.35+0.56) -32.3%   0.21(0.34+0.55) -32.3%
    7820.7: perl grep '^how to'                     0.56(2.70+0.40)   0.24(0.64+0.52) -57.1%   0.20(0.28+0.60) -64.3%
    7820.11: perl grep '[how] to'                   0.56(2.66+0.38)   0.29(0.95+0.45) -48.2%   0.23(0.45+0.54) -58.9%
    7820.15: perl grep '(e.t[^ ]*|v.ry) rare'       1.02(5.77+0.42)   0.31(1.02+0.54) -69.6%   0.23(0.50+0.54) -77.5%
    7820.19: perl grep 'm(ú|u)lt.b(æ|y)te'          0.38(1.57+0.42)   0.27(0.85+0.46) -28.9%   0.21(0.33+0.57) -44.7%

See commit ("perf: add a comparison test of grep regex engines",
2017-04-19) for details on the machine the above test run was executed
on.

Here HEAD~2 is git with PCRE v1 without JIT, HEAD~ is PCRE v1 with
JIT, and HEAD is PCRE v2 (also with JIT). See previous commits of mine
mentioning p7820-grep-engines.sh for more details on the test setup.

For ease of readability, a different run just of HEAD~ (PCRE v1 with
JIT v.s. PCRE v2), again with just the /perl/ tests shown:

    [...]
    Test                                            HEAD~             HEAD
    ----------------------------------------------------------------------------------------
    7820.3: perl grep 'how.to'                      0.21(0.42+0.52)   0.21(0.31+0.58) +0.0%
    7820.7: perl grep '^how to'                     0.25(0.65+0.50)   0.20(0.31+0.57) -20.0%
    7820.11: perl grep '[how] to'                   0.30(0.90+0.50)   0.23(0.46+0.53) -23.3%
    7820.15: perl grep '(e.t[^ ]*|v.ry) rare'       0.30(1.19+0.38)   0.23(0.51+0.51) -23.3%
    7820.19: perl grep 'm(ú|u)lt.b(æ|y)te'          0.27(0.84+0.48)   0.21(0.34+0.57) -22.2%

I.e. the two are either neck-to-neck, but PCRE v2 usually pulls ahead,
when it does it's around 20% faster.

A brief note on thread safety: As noted in pcre2api(3) & pcre2jit(3)
the compiled pattern can be shared between threads, but not some of
the JIT context, however the grep threading support does all pattern &
JIT compilation in separate threads, so this code doesn't need to
concern itself with thread safety.

See commit 63e7e9d8b6 ("git-grep: Learn PCRE", 2011-05-09) for the
initial addition of PCRE v1. This change follows some of the same
patterns it did (and which were discussed on list at the time),
e.g. mocking up types with typedef instead of ifdef-ing them out when
USE_LIBPCRE2 isn't defined. This adds some trivial memory use to the
program, but makes the code look nicer.

1. https://lists.exim.org/lurker/message/20150105.162835.0666407a.en.html
2. https://lists.exim.org/lurker/thread/20170419.172322.833ee099.en.html

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-02 08:29:05 +09:00
Ævar Arnfjörð Bjarmason
072473e659 Makefile & configure: reword inaccurate comment about PCRE
Reword an outdated & inaccurate comment which suggests that only
git-grep can use PCRE.

This comment was added back when PCRE support was initially added in
commit 63e7e9d8b6 ("git-grep: Learn PCRE", 2011-05-09), and was true
at the time.

It hasn't been telling the full truth since git-log learned to use
PCRE with --grep in commit 727b6fc3ed ("log --grep: accept
--basic-regexp and --perl-regexp", 2012-10-03), and more importantly
is likely to get more inaccurate over time as more use is made of PCRE
in other areas.

Reword it to be more future-proof, and to more clearly explain that
this enables user-initiated runtime behavior.

Copy/pasting this so much in configure.ac is lame, these Makefile-like
flags aren't even used by autoconf, just the corresponding
--with[out]-* options. But copy/pasting the comments that make sense
for the Makefile to configure.ac where they make less sense is the
pattern everything else follows in that file. I'm not going to war
against that as part of this change, just following the existing
pattern.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-21 08:25:37 +09:00
Ramsay Jones
b2d593a779 wrapper.c: remove unused gitmkstemps() function
The last call to the mkstemps() function was removed in commit 659488326
("wrapper.c: delete dead function git_mkstemps()", 22-04-2016). In order
to support platforms without mkstemps(), this functionality was provided,
along with a Makefile build variable (NO_MKSTEMPS), by the gitmkstemps()
function. Remove the dead code, along with the defunct build machinery.

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-28 11:54:21 -08:00
Junio C Hamano
9424bf2707 Merge branch 'js/regexec-buf'
* js/regexec-buf:
  configure.ac: improve description of NO_REGEX test
2016-10-17 13:25:21 -07:00
Jakub Narębski
842a516cb0 configure.ac: improve description of NO_REGEX test
The commit 2f8952250a ("regex: add regexec_buf() that can work on a
non NUL-terminated string", 2016-09-21) changed description of
NO_REGEX build config variable to be more neutral, and actually say
that it is about support for REG_STARTEND.  Change description in
configure.ac to match.

Change also the test message and variable name to match.  The test
just checks that REG_STARTEND is #defined.

Issue-found-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Jakub Narębski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10 16:15:15 -07:00
Junio C Hamano
667c6d591e Merge branch 'dp/autoconf-curl-ssl'
The ./configure script generated from configure.ac was taught how
to detect support of SSL by libcurl better.

* dp/autoconf-curl-ssl:
  ./configure.ac: detect SSL in libcurl using curl-config
2016-10-10 14:03:48 -07:00
Junio C Hamano
7b01ab562a Merge branch 'ew/autoconf-pthread'
Existing autoconf generated test for the need to link with pthread
library did not check all the functions from pthread libraries;
recent FreeBSD has some functions in libc but not others, and we
mistakenly thought linking with libc is enough when it is not.

* ew/autoconf-pthread:
  configure.ac: stronger test for pthread linkage
2016-07-25 14:13:46 -07:00
Eric Wong
a9b02de8b7 configure.ac: stronger test for pthread linkage
We need to test linkage of pthread_create and pthread_join,
as pthread_mutex_* and pthread_key_* functions do not need
extra linkage under FreeBSD 10.3, leading to a false-positive
of the empty case.

Signed-off-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-18 11:22:35 -07:00
Дилян Палаузов
924b7eb1c9 ./configure.ac: detect SSL in libcurl using curl-config
The API of libcurl does not mention Curl_ssl_init() and when curl is
built with -flto, the Curl_ssl_init symbol is not exported.

https://curl.haxx.se/libcurl/using/ suggests calling

  curl-config --feature | grep SSL

to see, if the installed curl has SSL support.  Another approach
would be calling curl_version_info and checking the returned struct.

This patch removes the check for the Curl_ssl_init exported symbol
from libcurl and uses curl-config to detect SSL support in libcurl.

Signed-off-by: Дилян Палаузов <git-dpa@aegee.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-28 09:25:06 -07:00
Junio C Hamano
a0c9cf51c0 Merge branch 'ky/imap-send-openssl-1.1.0' into maint
Upcoming OpenSSL 1.1.0 will break compilation b updating a few APIs
we use in imap-send, which has been adjusted for the change.

* ky/imap-send-openssl-1.1.0:
  configure: remove checking for HMAC_CTX_cleanup
  imap-send: avoid deprecated TLSv1_method()
  imap-send: check NULL return of SSL_CTX_new()
  imap-send: use HMAC() function provided by OpenSSL
2016-05-06 14:53:24 -07:00
Junio C Hamano
33e4ec89d9 Merge branch 'ky/imap-send-openssl-1.1.0'
Upcoming OpenSSL 1.1.0 will break compilation b updating a few APIs
we use in imap-send, which has been adjusted for the change.

* ky/imap-send-openssl-1.1.0:
  configure: remove checking for HMAC_CTX_cleanup
  imap-send: avoid deprecated TLSv1_method()
  imap-send: check NULL return of SSL_CTX_new()
  imap-send: use HMAC() function provided by OpenSSL
2016-04-22 15:45:08 -07:00
Kazuki Yamaguchi
1245c74936 configure: remove checking for HMAC_CTX_cleanup
We don't need it, as we no longer use HMAC_CTX_cleanup() directly.

Signed-off-by: Kazuki Yamaguchi <k@rhe.jp>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-08 11:46:36 -07:00
Junio C Hamano
2df13639e7 Merge branch 'jc/sane-grep'
Recent versions of GNU grep is pickier than before to decide if a
file is "binary" and refuse to give line-oriented hits when we
expect it to, unless explicitly told with "-a" option.  As our
scripted Porcelains use sane_grep wrapper for line-oriented data,
even when the line may contain non-ASCII payload we took from
end-user data, use "grep -a" to implement sane_grep wrapper when
using an implementation of "grep" that takes the "-a" option.

* jc/sane-grep:
  rebase-i: clarify "is this commit relevant?" test
  sane_grep: pass "-a" if grep accepts it
2016-03-16 13:16:54 -07:00
Junio C Hamano
71b401032b sane_grep: pass "-a" if grep accepts it
Newer versions of GNU grep is reported to be pickier when we feed a
non-ASCII input and break some Porcelain scripts.  As we know we do
not feed random binary file to our own sane_grep wrapper, allow us
to always pass "-a" by setting SANE_TEXT_GREP=-a Makefile variable
to work it around.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-10 15:35:43 -08:00
Jeff King
cd76778049 Merge branch 'rc/configure-use-libs-when-checking-a-lib' into maint
The "configure" script did not test for -lpthread correctly, which
upset some linkers.

* rc/configure-use-libs-when-checking-a-lib:
  configure.ac: use $LIBS not $CFLAGS when testing -lpthread
2015-12-01 17:19:06 -05:00
Jeff King
a3824e7145 Merge branch 'rc/configure-use-libs-when-checking-a-lib'
The "configure" script did not test for -lpthread correctly, which
upset some linkers.

* rc/configure-use-libs-when-checking-a-lib:
  configure.ac: use $LIBS not $CFLAGS when testing -lpthread
2015-11-24 19:06:33 -05:00
Rainer M. Canavan
bac2c5bf1a configure.ac: use $LIBS not $CFLAGS when testing -lpthread
Some linkers, namely the one on IRIX are rather strict concerning
the order or arguments for symbol resolution, i.e. no libraries
listed before objects or other libraries on the command line are
considered for symbol resolution.  Therefore, -lpthread can't work
if it's put in CFLAGS, because it will not be considered for
resolving pthread_key_create in conftest.o. Use $LIBS instead.

Signed-off-by: Rainer Canavan <git@canavan.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-06 09:41:08 -08:00
Junio C Hamano
9627b0a49f Merge branch 'rp/link-curl-before-ssl'
The linkage order of libraries was wrong in places around libcurl.

* rp/link-curl-before-ssl:
  configure.ac: detect ssl need with libcurl
  Makefile: make curl-config path configurable
  Makefile: link libcurl before zlib
2015-10-29 13:59:20 -07:00
Remi Pommarel
7e91e8d73a configure.ac: detect ssl need with libcurl
When libcurl has been statically compiled with openssl support they both
need to be linked in everytime libcurl is used.

During configuration this can be detected by looking for Curl_ssl_init
function symbol in libcurl, which will only be present if libcurl has been
compiled statically built with openssl.

configure.ac checks for Curl_ssl_init function in libcurl and if such function
exists; it sets NEEDS_SSL_WITH_CURL that is used by the Makefile to include
-lssl alongside with -lcurl.

Signed-off-by: Remi Pommarel <repk@triplefau.lt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-21 12:44:44 -07:00
Remi Pommarel
f89158760d Makefile: make curl-config path configurable
There are situations, e.g. during cross compilation, where curl-config
program is not present in the PATH.

Make the makefile use a configurable curl-config program passed through
CURL_CONFIG variable which can be set through config.mak.

Also make this variable tunable through use of autoconf/configure. Configure
will set CURL_CONFIG variable in config.mak.autogen to whatever value has been
passed to ac_cv_prog_CURL_CONFIG.

Signed-off-by: Remi Pommarel <repk@triplefau.lt>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-21 12:43:31 -07:00