Commit Graph

149 Commits

Author SHA1 Message Date
Patrick Steinhardt
91839a8827 ci: create script to set up Git for Windows SDK
In order to build and test Git, we have to first set up the Git for
Windows SDK, which contains various required tools and libraries. The
SDK is basically a clone of [1], but that repository is quite large due
to all the binaries it contains. We thus use both shallow clones and
sparse checkouts to speed up the setup. To handle this complexity we use
a GitHub action that is hosted externally at [2].

Unfortunately, this makes it rather hard to reuse the logic for CI
platforms other than GitHub Actions. After chatting with Johannes
Schindelin we came to the conclusion that it would be nice if the Git
for Windows SDK would regularly publish releases that one can easily
download and extract, thus moving all of the complexity into that single
step. Like this, all that a CI job needs to do is to fetch and extract
the resulting archive. This published release comes in the form of a new
"ci-artifacts" tag that gets updated regularly [3].

Implement a new script that knows how to fetch and extract that script
and convert GitHub Actions to use it.

[1]: https://github.com/git-for-windows/git-sdk-64/
[2]: https://github.com/git-for-windows/setup-git-for-windows-sdk/
[3]: https://github.com/git-for-windows/git-sdk-64/releases/tag/ci-artifacts/

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-10-09 11:33:04 -07:00
Junio C Hamano
aeda40b96e Merge branch 'jk/ci-linux32-update'
CI updates

* jk/ci-linux32-update:
  ci: add Ubuntu 16.04 job to GitLab CI
  ci: use regular action versions for linux32 job
  ci: use more recent linux32 image
  ci: unify ubuntu and ubuntu32 dependencies
  ci: drop run-docker scripts
2024-09-16 15:27:08 -07:00
Junio C Hamano
f9fff154d3 Merge branch 'jc/ci-upload-artifact-and-linux32'
CI started failing completely for linux32 jobs, as the step to
upload failed test directory uses GitHub actions that is deprecated
and is now disabled.  Remove the step so at least we will know if
the tests are passing.

* jc/ci-upload-artifact-and-linux32:
  ci: remove 'Upload failed tests' directories' step from linux32 jobs
2024-09-16 15:27:08 -07:00
Jeff King
9c261856c9 ci: use regular action versions for linux32 job
The linux32 job runs inside a docker container with a 32-bit libc, etc.
This breaks any GitHub Actions scripts that are implemented in
javascript, because they ship with their own 64-bit version of Node.js
that's dynamically linked. They'll fail with a message like:

    exec /__e/node20/bin/node: no such file or directory

because they can't find the runtime linker.

This hasn't been a problem until recently because we special-case older,
non-javascript versions of these actions for the linux32 job. But it
recently became an issue when our old version of actions/upload-artifact
was deprecated, causing the job to fail. We worked around that in
90f2c7240c (ci: remove 'Upload failed tests' directories' step from
linux32 jobs, 2024-09-09), but it meant a loss of functionality for that
job. And we may eventually run into the same deprecation problem with
actions/checkout, which can't just be removed.

We can solve the linking issue by installing the 64-bit libc and stdc++
packages before doing anything else. Coupled with the switch to a more
recent image in the previous patch, that lets us remove the
special-casing of the action scripts entirely.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-12 10:21:10 -07:00
Jeff King
9ce2e99c7d ci: use more recent linux32 image
The Xenial image we're using was released more than 8 years ago. This is
a problem for using some recent GitHub Actions scripts, as they require
Node.js 20, and all of the binaries they ship need glibc 2.28 or later.
We're not using them yet, but moving forward prepares us for a future
patch which will.

Xenial was actually the last official 32-bit Ubuntu release, but you can
still find i386 images for more recent releases. This patch uses Focal,
which was released in 2020 (and is the oldest one with glibc 2.28).

There are two small downsides here:

  - while Xenial is pretty old, it is still in LTS support until April
    2026. So there's probably some value in testing with such an old
    system, and we're losing that.

  - there are no i386 subversion packages in the Focal repository. So we
    won't be able to test that (OTOH, we had never tested it until the
    previous patch which unified the 32/64-bit dependency code).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-12 10:21:10 -07:00
Junio C Hamano
90f2c7240c ci: remove 'Upload failed tests' directories' step from linux32 jobs
Linux32 jobs seem to be getting:

    Error: This request has been automatically failed because it uses a
    deprecated version of `actions/upload-artifact: v1`. Learn more:
    https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/

before doing anything useful.  For now, disable the step.

Ever since actions/upload-artifact@v1 got disabled, mentioning the
offending version of it seems to stop anything from happening.  At
least this should run the same build and test.

See

    https://github.com/git/git/actions/runs/10780030750/job/29894867249

for example.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-09 16:00:53 -07:00
Junio C Hamano
80ccd8a260 Sync with 'maint' for Windows+VS build jobs used at CI 2024-08-20 14:24:57 -07:00
Johannes Schindelin
9f39e2fa26 ci(win+VS): download the vcpkg artifacts using a dedicated GitHub Action
The Git for Windows project provides a GitHub Action to download and
cache Azure Pipelines artifacts (such as the `vcpkg` artifacts), hiding
gnarly internals, and also providing some robustness against network
glitches. Let's use it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-08-20 08:24:28 -07:00
Johannes Schindelin
46cbfd3f7e ci: bump microsoft/setup-msbuild from v1 to v2
The main benefit: The new version uses a node.js version that is not yet
deprecated.

Links:
- [Release notes](https://github.com/microsoft/setup-msbuild/releases)
- [Changelog](https://github.com/microsoft/setup-msbuild/blob/main/building-release.md)
- [Commits](https://github.com/microsoft/setup-msbuild/compare/v1...v2)

This patch was originally by GitHub's Dependabot, but I cannot attribute
that bot properly because it has no dedicated email address. Probably
because it hasn't reached legal age yet, or something.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-08-20 08:24:27 -07:00
Karthik Nayak
bce7e52d4e ci: run style check on GitHub and GitLab
We don't run style checks on our CI, even though we have a
'.clang-format' setup in the repository. Let's add one, the job will
validate only against the new commits added and will only run on merge
requests. Since we're introducing it for the first time, let's allow
this job to fail, so we can validate if this is useful and eventually
enforce it.

For GitHub, we allow the job to pass by adding 'continue-on-error: true'
to the workflow. This means the job would show as passed, even if the
style check failed. To know the status of the job, users have to
manually check the logs.

For GitLab, we allow the job to pass by adding 'allow_failure: true', to
the job. Unlike GitHub, here the job will show as failed with a yellow
warning symbol, but the pipeline would still show as passed.

Also for GitLab, we use the 'CI_MERGE_REQUEST_TARGET_BRANCH_SHA'
variable by default to obtain the base SHA of the merged pipeline (which
is only available for merged pipelines [1]). Otherwise we use the
'CI_MERGE_REQUEST_DIFF_BASE_SHA' variable.

[1]: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html#predefined-variables-for-merge-request-pipelines

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-07-23 09:56:50 -07:00
Junio C Hamano
3fc99d037f Merge branch 'jt/port-ci-whitespace-check-to-gitlab'
The "whitespace check" task that was enabled for GitHub Actions CI
has been ported to GitLab CI.

* jt/port-ci-whitespace-check-to-gitlab:
  gitlab-ci: add whitespace error check
  ci: make the whitespace report optional
  ci: separate whitespace check script
  github-ci: fix link to whitespace error
  ci: pre-collapse GitLab CI sections
2024-05-15 09:52:54 -07:00
Junio C Hamano
537f17ec8b Merge branch 'jk/ci-test-with-jgit-fix'
CI fix.

* jk/ci-test-with-jgit-fix:
  ci: update coverity runs_on_pool reference
2024-05-13 10:19:47 -07:00
Junio C Hamano
6cb0bd7fc3 Merge branch 'jk/ci-macos-gcc13-fix'
CI fix.

* jk/ci-macos-gcc13-fix:
  ci: stop installing "gcc-13" for osx-gcc
  ci: avoid bare "gcc" for osx-gcc job
  ci: drop mention of BREW_INSTALL_PACKAGES variable
2024-05-13 10:19:47 -07:00
Jeff King
7df2405b38 ci: stop installing "gcc-13" for osx-gcc
Our osx-gcc job explicitly asks to install gcc-13. But since the GitHub
runner image already comes with gcc-13 installed, this is mostly doing
nothing (or in some cases it may install an incremental update over the
runner image). But worse, it recently started causing errors like:

    ==> Fetching gcc@13
    ==> Downloading https://ghcr.io/v2/homebrew/core/gcc/13/blobs/sha256:fb2403d97e2ce67eb441b54557cfb61980830f3ba26d4c5a1fe5ecd0c9730d1a
    ==> Pouring gcc@13--13.2.0.ventura.bottle.tar.gz
    Error: The `brew link` step did not complete successfully
    The formula built, but is not symlinked into /usr/local
    Could not symlink bin/c++-13
    Target /usr/local/bin/c++-13
    is a symlink belonging to gcc. You can unlink it:
      brew unlink gcc

which cause the whole CI job to bail.

I didn't track down the root cause, but I suspect it may be related to
homebrew recently switching the "gcc" default to gcc-14. And it may even
be fixed when a new runner image is released. But if we don't need to
run brew at all, it's one less thing for us to worry about.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-09 09:58:08 -07:00
Jeff King
11c7001e3d ci: avoid bare "gcc" for osx-gcc job
On macOS, a bare "gcc" (without a version) will invoke a wrapper for
clang, not actual gcc. Even when gcc is installed via homebrew, that
only provides version-specific links in /usr/local/bin (like "gcc-13"),
and never a version-agnostic "gcc" wrapper.

As far as I can tell, this has been the case for a long time, and this
osx-gcc job has largely been doing nothing. We can point it at "gcc-13",
which will pick up the homebrew-installed version.

The fix here is specific to the github workflow file, as the gitlab one
does not have a matching job.

It's a little unfortunate that we cannot just ask for the latest version
of gcc which homebrew provides, but as far as I can tell there is no
easy alias (you'd have to find the highest number gcc-* in
/usr/local/bin yourself).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-09 09:57:32 -07:00
Jeff King
157ed03c83 ci: update coverity runs_on_pool reference
Commit 2d65e5b6a6 (ci: rename "runs_on_pool" to "distro", 2024-04-12)
renamed this variable for the main CI workflow, as well as in the ci/
scripts. Because the coverity workflow also relies on those scripts to
install dependencies, it needs to be updated, too. Without this patch,
the coverity build fails because we lack libcurl.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-09 09:38:43 -07:00
Justin Tobler
66820fb7bf ci: separate whitespace check script
The `check-whitespace` CI job is only available as a GitHub action. To
help enable this job with other CI providers, first separate the logic
performing the whitespace check into its own script. In subsequent
commits, this script is further generalized allowing its reuse.

Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-03 12:11:49 -07:00
Justin Tobler
ecaacbc7a2 github-ci: fix link to whitespace error
When the `check-whitespace` CI job detects whitespace errors, a
formatted summary of the issue is generated. This summary contains links
to the commits and blobs responsible for the whitespace errors. The
generated links for blobs do not work and result in a 404.

Instead of using the reference name in the link, use the commit ID
directly. This fixes the broken link and also helps enable future
generalization of the script for other CI providers by removing one of
the GitHub specific CI variables used.

Signed-off-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-03 12:11:49 -07:00
Patrick Steinhardt
9cdeb34b96 ci: merge scripts which install dependencies
We have two different scripts which install dependencies, one for
dockerized jobs and one for non-dockerized ones. Naturally, these
scripts have quite some duplication. Furthermore, either of these
scripts is missing some test dependencies that the respective other
script has, thus reducing test coverage.

Merge those two scripts such that there is a single source of truth for
test dependencies, only.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-12 08:47:50 -07:00
Patrick Steinhardt
ab2b3aadf3 ci: expose distro name in dockerized GitHub jobs
Expose a distro name in dockerized jobs. This will be used in a
subsequent commit where we merge the installation scripts for dockerized
and non-dockerized jobs.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-12 08:47:50 -07:00
Patrick Steinhardt
2d65e5b6a6 ci: rename "runs_on_pool" to "distro"
The "runs_on_pool" environment variable is used by our CI scripts to
distinguish the different kinds of operating systems. It is quite
specific to GitHub Actions though and not really a descriptive name.

Rename the variable to "distro" to clarify its intent.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-12 08:47:49 -07:00
Junio C Hamano
b0b43e3b1a Merge branch 'pb/ci-win-artifact-names-fix'
CI update.

* pb/ci-win-artifact-names-fix:
  ci(github): make Windows test artifacts name unique
2024-03-21 14:55:13 -07:00
Philippe Blain
e1aaf309db ci(github): make Windows test artifacts name unique
If several jobs in the windows-test or vs-test matrices fail, the
upload-artifact action in each job tries to upload the test directories
of the failed tests as "failed-tests-windows.zip", which fails for all
jobs except the one which finishes first with the following error:

    Error: Failed to CreateArtifact: Received non-retryable error:
    Failed request: (409) Conflict: an artifact with this name
    already exists on the workflow run

Make the artifacts name unique by using the 'matrix.nr' token, and
disambiguate the vs-test artifacts from the windows-test ones.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-03-11 10:13:03 -07:00
Junio C Hamano
b4385bf016 Merge branch 'ps/reftable-backend'
Integrate the reftable code into the refs framework as a backend.

* ps/reftable-backend:
  refs/reftable: fix leak when copying reflog fails
  ci: add jobs to test with the reftable backend
  refs: introduce reftable backend
2024-02-26 18:10:23 -08:00
Jiang Xin
1bb7fcbffc l10n: ci: disable cache for setup-go to suppress warnings
After we upgraded actions/setup-go to v5, the following warning message
was reported every time we ran the CI.

    Restore cache failed: Dependencies file is not found ...

Disable cache to suppress warning messages as described in the solution
below.

    https://github.com/actions/setup-go/issues/427

Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2024-02-16 11:51:19 +08:00
Jiang Xin
4d733f09f0 l10n: ci: remove unused param for add-pr-comment@v2
When we upgraded GitHub Actions "mshick/add-pr-comment" to v2, the
following warning message was reported every time we ran the CI.

    Unexpected input(s) 'repo-token-user-login', valid inputs ...

Removed the obsolete parameter "repo-token-user-login" to suppress
warning messages.

Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2024-02-16 11:40:58 +08:00
Jiang Xin
f98643fcb2 Merge branch 'master' of github.com:git/git
* 'master' of github.com:git/git: (51 commits)
  Hopefully the last batch of fixes before 2.44 final
  Git 2.43.2
  A few more fixes before -rc1
  write-or-die: fix the polarity of GIT_FLUSH environment variable
  A few more topics before -rc1
  completion: add and use __git_compute_second_level_config_vars_for_section
  completion: add and use __git_compute_first_level_config_vars_for_section
  completion: complete 'submodule.*' config variables
  completion: add space after config variable names also in Bash 3
  receive-pack: use find_commit_header() in check_nonce()
  ci(linux32): add a note about Actions that must not be updated
  ci: bump remaining outdated Actions versions
  unit-tests: do show relative file paths on non-Windows, too
  receive-pack: use find_commit_header() in check_cert_push_options()
  prune: mark rebase autostash and orig-head as reachable
  sequencer: unset GIT_CHERRY_PICK_HELP for 'exec' commands
  ref-filter.c: sort formatted dates by byte value
  ssh signing: signal an error with a negative return value
  bisect: document command line arguments for "bisect start"
  bisect: document "terms" subcommand more fully
  ...
2024-02-15 09:48:25 +08:00
Junio C Hamano
c2914d4677 Merge branch 'js/github-actions-update'
Update remaining GitHub Actions jobs to avoid warnings against
using deprecated version of Node.js.

* js/github-actions-update:
  ci(linux32): add a note about Actions that must not be updated
  ci: bump remaining outdated Actions versions
2024-02-13 14:31:11 -08:00
Junio C Hamano
133a7b08dc Merge branch 'jc/github-actions-update'
Squelch node.js 16 deprecation warnings from GitHub Actions CI
by updating actions/github-script and actions/checkout that use
node.js 20.

* jc/github-actions-update:
  GitHub Actions: update to github-script@v7
  GitHub Actions: update to checkout@v4
2024-02-13 14:31:11 -08:00
Junio C Hamano
32c5ab6ee4 Merge branch 'pb/template-for-single-commit-pr'
Doc update.

* pb/template-for-single-commit-pr:
  .github/PULL_REQUEST_TEMPLATE.md: add a note about single-commit PRs
2024-02-12 13:16:11 -08:00
Johannes Schindelin
20e0ff8835 ci(linux32): add a note about Actions that must not be updated
The Docker container used by the `linux32` job comes without Node.js,
and therefore the `actions/checkout` and `actions/upload-artifact`
Actions cannot be upgraded to the latest versions (because they use
Node.js).

One time too many, I accidentally tried to update them, where
`actions/checkout` at least fails immediately, but the
`actions/upload-artifact` step is only used when any test fails, and
therefore the CI run usually passes even though that Action was updated
to a version that is incompatible with the Docker container in which
this job runs.

So let's add a big fat warning, mainly for my own benefit, to avoid
running into the very same issue over and over again.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-12 08:48:22 -08:00
Johannes Schindelin
820a340085 ci: bump remaining outdated Actions versions
After activating automatic Dependabot updates in the
git-for-windows/git repository, Dependabot noticed a couple
of yet-unaddressed updates.  They avoid "Node.js 16 Actions"
deprecation messages by bumping the following Actions'
versions:

- actions/upload-artifact from 3 to 4
- actions/download-artifact from 3 to 4
- actions/cache from 3 to 4

Helped-by: Matthias Aßhauer <mha1993@live.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-12 08:47:38 -08:00
Johannes Schindelin
6032aee65e l10n: bump Actions versions in l10n.yml
This avoids the "Node.js 16 Actions are deprecated" warnings.

Original-commits-by: dependabot[bot] <support@github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2024-02-11 12:47:51 +01:00
Patrick Steinhardt
c0350cb964 ci: add jobs to test with the reftable backend
Add CI jobs for both GitHub Workflows and GitLab CI to run Git with the
new reftable backend.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-07 08:28:37 -08:00
Philippe Blain
78307f1a89 .github/PULL_REQUEST_TEMPLATE.md: add a note about single-commit PRs
Contributors using Gitgitgadget continue to send single-commit PRs with
their commit message text duplicated below the three-dash line,
increasing the signal-to-noise ratio for reviewers.

This is because Gitgitgadget copies the pull request description as an
in-patch commentary, for single-commit PRs, and _GitHub_ defaults to
prefilling the pull request description with the commit message, for
single-commit PRs (followed by the content of the pull request
template).

Add a note in the pull request template mentioning that for
single-commit PRs, the PR description should thus be kept empty, in the
hope that contributors read it and act on it.

This partly addresses:
https://github.com/gitgitgadget/gitgitgadget/issues/340

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-06 12:22:55 -08:00
Junio C Hamano
dcce2bda21 Merge branch 'jc/maint-github-actions-update' into jc/github-actions-update
This contains an evil merge to tell the fuzz-smoke-test job to
also use checkout@v4; the job has been added since the master
track diverged from the maintenance track.

* jc/maint-github-actions-update:
  GitHub Actions: update to github-script@v7
  GitHub Actions: update to checkout@v4
2024-02-02 13:03:30 -08:00
Junio C Hamano
c4ddbe043e GitHub Actions: update to github-script@v7
We seem to be getting "Node.js 16 actions are deprecated." warnings
for jobs that use github-script@v6.  Update to github-script@v7,
which is said to use Node.js 20.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-02 13:00:46 -08:00
Junio C Hamano
e94dec0c1d GitHub Actions: update to checkout@v4
We seem to be getting "Node.js 16 actions are deprecated." warnings
for jobs that use checkout@v3.  Except for the i686 containers job
that is kept at checkout@v1 [*], update to checkout@v4, which is
said to use Node.js 20.

[*] 6cf4d908 (ci(main): upgrade actions/checkout to v3, 2022-12-05)
    refers to https://github.com/actions/runner/issues/2115 and
    explains why container jobs are kept at checkout@v1.  We may
    want to check the current status of the issue and move it to the
    same version as other jobs, but that is outside the scope of
    this step.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-02-02 13:00:34 -08:00
Josh Steadmon
c4a9cf1df3 ci: build and run minimal fuzzers in GitHub CI
To prevent bitrot, we would like to regularly exercise the fuzz tests in
order to make sure they still link & run properly. We already compile
the fuzz test objects as part of the default `make` target, but we do
not link the executables due to the fuzz tests needing specific
compilers and compiler features. This has lead to frequent build
breakages for the fuzz tests.

To remedy this, we can add a CI step to actually link the fuzz
executables, and run them (with finite input rather than the default
infinite random input mode) to verify that they execute properly.

Since the main use of the fuzz tests is via OSS-Fuzz [1], and OSS-Fuzz
only runs tests on Linux [2], we only set up a CI test for the fuzzers
on Linux.

[1] https://github.com/google/oss-fuzz
[2] https://google.github.io/oss-fuzz/further-reading/fuzzer-environment/

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-01-19 14:29:25 -08:00
Johannes Schindelin
682a868f67 ci: upgrade to using macos-13
In April, GitHub announced that the `macos-13` pool is available:
https://github.blog/changelog/2023-04-24-github-actions-macos-13-is-now-available/.
It is only a matter of time until the `macos-12` pool is going away,
therefore we should switch now, without pressure of a looming deadline.

Since the `macos-13` runners no longer include Python2, we also drop
specifically testing with Python2 and switch uniformly to Python3, see
https://github.com/actions/runner-images/blob/HEAD/images/macos/macos-13-Readme.md
for details about the software available on the `macos-13` pool's
runners.

Also, on macOS 13, Homebrew seems to install a `gcc@9` package that no
longer comes with a regular `unistd.h` (there seems only to be a
`ssp/unistd.h`), and hence builds would fail with:

    In file included from base85.c:1:
    git-compat-util.h:223:10: fatal error: unistd.h: No such file or directory
      223 | #include <unistd.h>
          |          ^~~~~~~~~~
    compilation terminated.

The reason why we install GCC v9.x explicitly is historical, and back in
the days it was because it was the _newest_ version available via
Homebrew: 176441bfb5 (ci: build Git with GCC 9 in the 'osx-gcc' build
job, 2019-11-27).

To reinstate the spirit of that commit _and_ to fix that build failure,
let's switch to the now-newest GCC version: v13.x.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-03 18:52:02 +09:00
Junio C Hamano
0510d06b56 Merge branch 'jk/ci-retire-allow-ref' into maint-2.42
CI update.

* jk/ci-retire-allow-ref:
  ci: deprecate ci/config/allow-ref script
  ci: allow branch selection through "vars"
2023-11-02 16:53:23 +09:00
Junio C Hamano
ec7cc187d4 Merge branch 'jc/ci-skip-same-commit' into maint-2.42
Tweak GitHub Actions CI so that pushing the same commit to multiple
branch tips at the same time will not waste building and testing
the same thing twice.

* jc/ci-skip-same-commit:
  ci: avoid building from the same commit in parallel
2023-11-02 16:53:15 +09:00
Johannes Schindelin
3349520e1a coverity: detect and report when the token or project is incorrect
When trying to obtain the MD5 of the Coverity Scan Tool (in order to
decide whether a cached version can be used or a new version has to be
downloaded), it is possible to get a 401 (Authorization required) due to
either an incorrect token, or even more likely due to an incorrect
Coverity project name.

Seeing an authorization failure that is caused by an incorrect project
name was somewhat surprising to me when developing the Coverity
workflow, as I found such a failure suggestive of an incorrect token
instead.

So let's provide a helpful error message about that specifically when
encountering authentication issues.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-10-05 11:45:46 -07:00
Johannes Schindelin
c13d2adf8b coverity: allow running on macOS
For completeness' sake, let's add support for submitting macOS builds to
Coverity Scan.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-25 10:12:49 -07:00
Johannes Schindelin
d3c3ffa624 coverity: support building on Windows
By adding the repository variable `ENABLE_COVERITY_SCAN_ON_OS` with a
value, say, `["windows-latest"]`, this GitHub workflow now runs on
Windows, allowing to analyze Windows-specific issues.

This allows, say, the Git for Windows fork to submit Windows builds to
Coverity Scan instead of Linux builds.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-25 10:12:49 -07:00
Johannes Schindelin
7bc49e8f55 coverity: allow overriding the Coverity project
By default, the builds are submitted to the `git` project at
https://scan.coverity.com/projects/git.

The Git for Windows project would like to use this workflow, too,
though, and needs the builds to be submitted to the `git-for-windows`
Coverity project.

To that end, allow configuring the Coverity project name via the
repository variable, you guessed it, `COVERITY_PROJECT`. The default if
that variable is not configured or has an empty value is still `git`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-25 10:12:49 -07:00
Johannes Schindelin
002e5e9ad1 coverity: cache the Coverity Build Tool
It would add a 1GB+ download for every run, better cache it.

This is inspired by the GitHub Action `vapier/coverity-scan-action`,
however, it uses the finer-grained `restore`/`save` method to be able to
cache the Coverity Build Tool even if an unrelated step in the GitHub
workflow fails later on.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-25 10:12:48 -07:00
Johannes Schindelin
a56b6230d0 ci: add a GitHub workflow to submit Coverity scans
Coverity is a static analysis tool that detects and generates reports on
various security and code quality issues.

It is particularly useful when diagnosing memory safety issues which may
be used as part of exploiting a security vulnerability.

Coverity's website provides a service that accepts "builds" (which
contains the object files generated during a standard build as well as a
database generated by Coverity's scan tool).

Let's add a GitHub workflow to automate all of this. To avoid running it
without appropriate Coverity configuration (e.g. the token required to
use Coverity's services), the job only runs when the repository variable
"ENABLE_COVERITY_SCAN_FOR_BRANCHES" has been configured accordingly (see
https://docs.github.com/en/actions/learn-github-actions/variables for
details how to configure repository variables): It is expected to be a
valid JSON array of branch strings, e.g. `["main", "next"]`.

In addition, this workflow requires two repository secrets:

- COVERITY_SCAN_EMAIL: the email to send the report to, and

- COVERITY_SCAN_TOKEN: the Coverity token (look in the Project Settings
  tab of your Coverity project).

Note: The initial version of this patch used
`vapier/coverity-scan-action` to benefit from that Action's caching of
the Coverity tool, which is rather large. Sadly, that Action only
supports Linux, and we want to have the option of building on Windows,
too. Besides, in the meantime Coverity requires `cov-configure` to be
runantime, and that Action was not adjusted accordingly, i.e. it seems
not to be maintained actively. Therefore it would seem prudent to
implement the steps manually instead of using that Action.

Initial-patch-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-09-25 10:12:48 -07:00
Jeff King
edf80d23f1 ci: deprecate ci/config/allow-ref script
Now that we have the CI_BRANCHES mechanism, there is no need for anybody
to use the ci/config/allow-ref mechanism. In the long run, we can
hopefully remove it and the whole "config" job, as it consumes CPU and
adds to the end-to-end latency of the whole workflow. But we don't want
to do that immediately, as people need time to migrate until the
CI_BRANCHES change has made it into the workflow file of every branch.

So let's issue a warning, which will appear in the "annotations" section
below the workflow result in GitHub's web interface. And let's remove
the sample allow-refs script, as we don't want to encourage anybody to
use it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-08-30 15:56:11 -07:00
Jeff King
21c82dcd62 ci: allow branch selection through "vars"
When we added config to skip CI for certain branches in e76eec3554 (ci:
allow per-branch config for GitHub Actions, 2020-05-07), there wasn't
any way to avoid spinning up a VM just to check the config. From the
developer's perspective this isn't too bad, as the "skipped" branches
complete successfully after running the config job (the workflow result
is "success" instead of "skipped", but that is a minor lie).

But we are still wasting time and GitHub's CPU to spin up a VM just to
check the result of a short shell script. At the time there wasn't any
way to avoid this. But they've since introduced repo-level variables
that should let us do the same thing:

  https://github.blog/2023-01-10-introducing-required-workflows-and-configuration-variables-to-github-actions/#configuration-variables

This is more efficient, and as a bonus is probably less confusing to
configure (the existing system requires sticking your config on a magic
ref).

See the included docs for how to configure it.

The code itself is pretty simple: it checks the variable and skips the
config job if appropriate (and everything else depends on the config job
already). There are two slight inaccuracies here:

  - we don't insist on branches, so this likewise applies to tag names
    or other refs. I think in practice this is OK, and keeping the code
    (and docs) short is more important than trying to be more exact. We
    are targeting developers of git.git and their limited workflows.

  - the match is done as a substring (so if you want to run CI for
    "foobar", then branch "foo" will accidentally match). Again, this
    should be OK in practice, as anybody who uses this is likely to only
    specify a handful of well-known names. If we want to be more exact,
    we can have the code check for adjoining spaces. Or even move to a
    more general CI_CONFIG variable formatted as JSON. I went with this
    scheme for the sake of simplicity.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-08-30 15:56:09 -07:00