2019-10-04 23:09:35 +08:00
|
|
|
variables:
|
|
|
|
|
Agent.Source.Git.ShallowFetchDepth: 1
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
|
|
|
|
|
jobs:
|
ci: parallelize testing on Windows
The fact that Git's test suite is implemented in Unix shell script that
is as portable as we can muster, combined with the fact that Unix shell
scripting is foreign to Windows (and therefore has to be emulated),
results in pretty abysmal speed of the test suite on that platform, for
pretty much no other reason than that language choice.
For comparison: while the Linux build & test is typically done within
about 8 minutes, the Windows build & test typically lasts about 80
minutes in Azure Pipelines.
To help with that, let's use the Azure Pipeline feature where you can
parallelize jobs, make jobs depend on each other, and pass artifacts
between them.
The tests are distributed using the following heuristic: listing all
test scripts ordered by size in descending order (as a cheap way to
estimate the overall run time), every Nth script is run (where N is the
total number of parallel jobs), starting at the index corresponding to
the parallel job. This slicing is performed by a new function that is
added to the `test-tool`.
To optimize the overall runtime of the entire Pipeline, we need to move
the Windows jobs to the beginning (otherwise there would be a very
decent chance for the Pipeline to be run only the Windows build, while
all the parallel Windows test jobs wait for this single one).
We use Azure Pipelines Artifacts for both the minimal Git for Windows
SDK as well as the built executables, as deduplication and caching close
to the agents makes that really fast. For comparison: while downloading
and unpacking the minimal Git for Windows SDK via PowerShell takes only
one minute (down from anywhere between 2.5 to 7 when using a shallow
clone), uploading it as Pipeline Artifact takes less than 30s and
downloading and unpacking less than 20s (sometimes even as little as
only twelve seconds).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:38 +08:00
|
|
|
- job: windows_build
|
|
|
|
|
displayName: Windows Build
|
ci: add a Windows job to the Azure Pipelines definition
Previously, we did not have robust support for Windows in our CI
definition, simply because Travis cannot accommodate our needs (even
after Travis added experimental Windows support very recently, it takes
longer than Travis' 50 minute timeout to build Git and run the test
suite on Windows). Instead, we used a hack that started a dedicated
Azure Pipeline from Travis and waited for the output, often timing out
(which is quite fragile, as we found out).
With this commit, we finally have first-class support for Windows in our
CI definition (in the Azure Pipelines one, that is).
Due to our reliance on Unix shell scripting in the test suite, combined
with the challenges on executing such scripts on Windows, the Windows
job currently takes a whopping ~1h20m to complete. Which is *far* longer
than the next-longest job takes (linux-gcc, ~35m).
Now, Azure Pipelines's free tier for open source projects (such as Git)
offers up to 10 concurrent jobs for free, meaning that the overall run
time will be dominated by the slowest job(s).
Therefore, it makes sense to start the Windows job first, to minimize
the time the entire build takes from start to end (which is now pretty
safely the run time of the Windows job).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:30 +08:00
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: windows-latest
|
ci: add a Windows job to the Azure Pipelines definition
Previously, we did not have robust support for Windows in our CI
definition, simply because Travis cannot accommodate our needs (even
after Travis added experimental Windows support very recently, it takes
longer than Travis' 50 minute timeout to build Git and run the test
suite on Windows). Instead, we used a hack that started a dedicated
Azure Pipeline from Travis and waited for the output, often timing out
(which is quite fragile, as we found out).
With this commit, we finally have first-class support for Windows in our
CI definition (in the Azure Pipelines one, that is).
Due to our reliance on Unix shell scripting in the test suite, combined
with the challenges on executing such scripts on Windows, the Windows
job currently takes a whopping ~1h20m to complete. Which is *far* longer
than the next-longest job takes (linux-gcc, ~35m).
Now, Azure Pipelines's free tier for open source projects (such as Git)
offers up to 10 concurrent jobs for free, meaning that the overall run
time will be dominated by the slowest job(s).
Therefore, it makes sense to start the Windows job first, to minimize
the time the entire build takes from start to end (which is now pretty
safely the run time of the Windows job).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:30 +08:00
|
|
|
timeoutInMinutes: 240
|
|
|
|
|
steps:
|
|
|
|
|
- powershell: |
|
|
|
|
|
if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
|
|
|
|
|
net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no
|
|
|
|
|
cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\
|
|
|
|
|
}
|
|
|
|
|
displayName: 'Mount test-cache'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- powershell: |
|
ci: use git-sdk-64-minimal build artifact
Instead of a shallow fetch followed by a sparse checkout, we are
better off by using a separate, dedicated Pipeline that bundles
the SDK as a build artifact, and then consuming that build artifact
here.
In fact, since this artifact will be used a lot, we spent substantial
time on figuring out a minimal subset of the Git for Windows SDK, just
enough to build and test Git. The result is a size reduction from around
1GB (compressed) to around 55MB (compressed). This also comes with the
change where we now call `usr\bin\bash.exe` directly, as `git-cmd.exe`
is not included in the minimal SDK.
That reduces the time to initialize Git for Windows' SDK from anywhere
between 2m30s-7m to a little over 1m.
Note: in theory, we could also use the DownloadBuildArtifacts@0 task
here. However, restricted permissions that are in effect when building
from forks would let this fail for PR builds, defeating the whole
purpose of the Azure Pipelines support for git.git.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:31 +08:00
|
|
|
$urlbase = "https://dev.azure.com/git-for-windows/git/_apis/build/builds"
|
|
|
|
|
$id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=22&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id
|
|
|
|
|
$downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[1].resource.downloadUrl
|
|
|
|
|
(New-Object Net.WebClient).DownloadFile($downloadUrl,"git-sdk-64-minimal.zip")
|
|
|
|
|
Expand-Archive git-sdk-64-minimal.zip -DestinationPath . -Force
|
|
|
|
|
Remove-Item git-sdk-64-minimal.zip
|
ci: add a Windows job to the Azure Pipelines definition
Previously, we did not have robust support for Windows in our CI
definition, simply because Travis cannot accommodate our needs (even
after Travis added experimental Windows support very recently, it takes
longer than Travis' 50 minute timeout to build Git and run the test
suite on Windows). Instead, we used a hack that started a dedicated
Azure Pipeline from Travis and waited for the output, often timing out
(which is quite fragile, as we found out).
With this commit, we finally have first-class support for Windows in our
CI definition (in the Azure Pipelines one, that is).
Due to our reliance on Unix shell scripting in the test suite, combined
with the challenges on executing such scripts on Windows, the Windows
job currently takes a whopping ~1h20m to complete. Which is *far* longer
than the next-longest job takes (linux-gcc, ~35m).
Now, Azure Pipelines's free tier for open source projects (such as Git)
offers up to 10 concurrent jobs for free, meaning that the overall run
time will be dominated by the slowest job(s).
Therefore, it makes sense to start the Windows job first, to minimize
the time the entire build takes from start to end (which is now pretty
safely the run time of the Windows job).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:30 +08:00
|
|
|
|
|
|
|
|
# Let Git ignore the SDK and the test-cache
|
ci: use git-sdk-64-minimal build artifact
Instead of a shallow fetch followed by a sparse checkout, we are
better off by using a separate, dedicated Pipeline that bundles
the SDK as a build artifact, and then consuming that build artifact
here.
In fact, since this artifact will be used a lot, we spent substantial
time on figuring out a minimal subset of the Git for Windows SDK, just
enough to build and test Git. The result is a size reduction from around
1GB (compressed) to around 55MB (compressed). This also comes with the
change where we now call `usr\bin\bash.exe` directly, as `git-cmd.exe`
is not included in the minimal SDK.
That reduces the time to initialize Git for Windows' SDK from anywhere
between 2m30s-7m to a little over 1m.
Note: in theory, we could also use the DownloadBuildArtifacts@0 task
here. However, restricted permissions that are in effect when building
from forks would let this fail for PR builds, defeating the whole
purpose of the Azure Pipelines support for git.git.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:31 +08:00
|
|
|
"/git-sdk-64-minimal/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude"
|
|
|
|
|
displayName: 'Download git-sdk-64-minimal'
|
ci: add a Windows job to the Azure Pipelines definition
Previously, we did not have robust support for Windows in our CI
definition, simply because Travis cannot accommodate our needs (even
after Travis added experimental Windows support very recently, it takes
longer than Travis' 50 minute timeout to build Git and run the test
suite on Windows). Instead, we used a hack that started a dedicated
Azure Pipeline from Travis and waited for the output, often timing out
(which is quite fragile, as we found out).
With this commit, we finally have first-class support for Windows in our
CI definition (in the Azure Pipelines one, that is).
Due to our reliance on Unix shell scripting in the test suite, combined
with the challenges on executing such scripts on Windows, the Windows
job currently takes a whopping ~1h20m to complete. Which is *far* longer
than the next-longest job takes (linux-gcc, ~35m).
Now, Azure Pipelines's free tier for open source projects (such as Git)
offers up to 10 concurrent jobs for free, meaning that the overall run
time will be dominated by the slowest job(s).
Therefore, it makes sense to start the Windows job first, to minimize
the time the entire build takes from start to end (which is now pretty
safely the run time of the Windows job).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:30 +08:00
|
|
|
- powershell: |
|
ci: use git-sdk-64-minimal build artifact
Instead of a shallow fetch followed by a sparse checkout, we are
better off by using a separate, dedicated Pipeline that bundles
the SDK as a build artifact, and then consuming that build artifact
here.
In fact, since this artifact will be used a lot, we spent substantial
time on figuring out a minimal subset of the Git for Windows SDK, just
enough to build and test Git. The result is a size reduction from around
1GB (compressed) to around 55MB (compressed). This also comes with the
change where we now call `usr\bin\bash.exe` directly, as `git-cmd.exe`
is not included in the minimal SDK.
That reduces the time to initialize Git for Windows' SDK from anywhere
between 2m30s-7m to a little over 1m.
Note: in theory, we could also use the DownloadBuildArtifacts@0 task
here. However, restricted permissions that are in effect when building
from forks would let this fail for PR builds, defeating the whole
purpose of the Azure Pipelines support for git.git.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:31 +08:00
|
|
|
& git-sdk-64-minimal\usr\bin\bash.exe -lc @"
|
ci: parallelize testing on Windows
The fact that Git's test suite is implemented in Unix shell script that
is as portable as we can muster, combined with the fact that Unix shell
scripting is foreign to Windows (and therefore has to be emulated),
results in pretty abysmal speed of the test suite on that platform, for
pretty much no other reason than that language choice.
For comparison: while the Linux build & test is typically done within
about 8 minutes, the Windows build & test typically lasts about 80
minutes in Azure Pipelines.
To help with that, let's use the Azure Pipeline feature where you can
parallelize jobs, make jobs depend on each other, and pass artifacts
between them.
The tests are distributed using the following heuristic: listing all
test scripts ordered by size in descending order (as a cheap way to
estimate the overall run time), every Nth script is run (where N is the
total number of parallel jobs), starting at the index corresponding to
the parallel job. This slicing is performed by a new function that is
added to the `test-tool`.
To optimize the overall runtime of the entire Pipeline, we need to move
the Windows jobs to the beginning (otherwise there would be a very
decent chance for the Pipeline to be run only the Windows build, while
all the parallel Windows test jobs wait for this single one).
We use Azure Pipelines Artifacts for both the minimal Git for Windows
SDK as well as the built executables, as deduplication and caching close
to the agents makes that really fast. For comparison: while downloading
and unpacking the minimal Git for Windows SDK via PowerShell takes only
one minute (down from anywhere between 2.5 to 7 when using a shallow
clone), uploading it as Pipeline Artifact takes less than 30s and
downloading and unpacking less than 20s (sometimes even as little as
only twelve seconds).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:38 +08:00
|
|
|
ci/make-test-artifacts.sh artifacts
|
|
|
|
|
"@
|
|
|
|
|
if (!$?) { exit(1) }
|
|
|
|
|
displayName: Build
|
|
|
|
|
env:
|
|
|
|
|
HOME: $(Build.SourcesDirectory)
|
|
|
|
|
MSYSTEM: MINGW64
|
|
|
|
|
DEVELOPER: 1
|
|
|
|
|
NO_PERL: 1
|
|
|
|
|
- task: PublishPipelineArtifact@0
|
|
|
|
|
displayName: 'Publish Pipeline Artifact: test artifacts'
|
|
|
|
|
inputs:
|
|
|
|
|
artifactName: 'windows-artifacts'
|
|
|
|
|
targetPath: '$(Build.SourcesDirectory)\artifacts'
|
|
|
|
|
- task: PublishPipelineArtifact@0
|
|
|
|
|
displayName: 'Publish Pipeline Artifact: git-sdk-64-minimal'
|
|
|
|
|
inputs:
|
|
|
|
|
artifactName: 'git-sdk-64-minimal'
|
|
|
|
|
targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal'
|
|
|
|
|
- powershell: |
|
|
|
|
|
if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
|
|
|
|
|
cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
|
|
|
|
|
}
|
|
|
|
|
displayName: 'Unmount test-cache'
|
|
|
|
|
condition: true
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
|
|
|
|
|
- job: windows_test
|
|
|
|
|
displayName: Windows Test
|
|
|
|
|
dependsOn: windows_build
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: windows-latest
|
ci: parallelize testing on Windows
The fact that Git's test suite is implemented in Unix shell script that
is as portable as we can muster, combined with the fact that Unix shell
scripting is foreign to Windows (and therefore has to be emulated),
results in pretty abysmal speed of the test suite on that platform, for
pretty much no other reason than that language choice.
For comparison: while the Linux build & test is typically done within
about 8 minutes, the Windows build & test typically lasts about 80
minutes in Azure Pipelines.
To help with that, let's use the Azure Pipeline feature where you can
parallelize jobs, make jobs depend on each other, and pass artifacts
between them.
The tests are distributed using the following heuristic: listing all
test scripts ordered by size in descending order (as a cheap way to
estimate the overall run time), every Nth script is run (where N is the
total number of parallel jobs), starting at the index corresponding to
the parallel job. This slicing is performed by a new function that is
added to the `test-tool`.
To optimize the overall runtime of the entire Pipeline, we need to move
the Windows jobs to the beginning (otherwise there would be a very
decent chance for the Pipeline to be run only the Windows build, while
all the parallel Windows test jobs wait for this single one).
We use Azure Pipelines Artifacts for both the minimal Git for Windows
SDK as well as the built executables, as deduplication and caching close
to the agents makes that really fast. For comparison: while downloading
and unpacking the minimal Git for Windows SDK via PowerShell takes only
one minute (down from anywhere between 2.5 to 7 when using a shallow
clone), uploading it as Pipeline Artifact takes less than 30s and
downloading and unpacking less than 20s (sometimes even as little as
only twelve seconds).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:38 +08:00
|
|
|
timeoutInMinutes: 240
|
|
|
|
|
strategy:
|
|
|
|
|
parallel: 10
|
|
|
|
|
steps:
|
|
|
|
|
- powershell: |
|
|
|
|
|
if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
|
|
|
|
|
net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no
|
|
|
|
|
cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\
|
|
|
|
|
}
|
|
|
|
|
displayName: 'Mount test-cache'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- task: DownloadPipelineArtifact@0
|
|
|
|
|
displayName: 'Download Pipeline Artifact: test artifacts'
|
|
|
|
|
inputs:
|
|
|
|
|
artifactName: 'windows-artifacts'
|
|
|
|
|
targetPath: '$(Build.SourcesDirectory)'
|
|
|
|
|
- task: DownloadPipelineArtifact@0
|
|
|
|
|
displayName: 'Download Pipeline Artifact: git-sdk-64-minimal'
|
|
|
|
|
inputs:
|
|
|
|
|
artifactName: 'git-sdk-64-minimal'
|
|
|
|
|
targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal'
|
|
|
|
|
- powershell: |
|
|
|
|
|
& git-sdk-64-minimal\usr\bin\bash.exe -lc @"
|
|
|
|
|
test -f artifacts.tar.gz || {
|
|
|
|
|
echo No test artifacts found\; skipping >&2
|
|
|
|
|
exit 0
|
|
|
|
|
}
|
|
|
|
|
tar xf artifacts.tar.gz || exit 1
|
|
|
|
|
|
|
|
|
|
# Let Git ignore the SDK and the test-cache
|
|
|
|
|
printf '%s\n' /git-sdk-64-minimal/ /test-cache/ >>.git/info/exclude
|
ci: add a Windows job to the Azure Pipelines definition
Previously, we did not have robust support for Windows in our CI
definition, simply because Travis cannot accommodate our needs (even
after Travis added experimental Windows support very recently, it takes
longer than Travis' 50 minute timeout to build Git and run the test
suite on Windows). Instead, we used a hack that started a dedicated
Azure Pipeline from Travis and waited for the output, often timing out
(which is quite fragile, as we found out).
With this commit, we finally have first-class support for Windows in our
CI definition (in the Azure Pipelines one, that is).
Due to our reliance on Unix shell scripting in the test suite, combined
with the challenges on executing such scripts on Windows, the Windows
job currently takes a whopping ~1h20m to complete. Which is *far* longer
than the next-longest job takes (linux-gcc, ~35m).
Now, Azure Pipelines's free tier for open source projects (such as Git)
offers up to 10 concurrent jobs for free, meaning that the overall run
time will be dominated by the slowest job(s).
Therefore, it makes sense to start the Windows job first, to minimize
the time the entire build takes from start to end (which is now pretty
safely the run time of the Windows job).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:30 +08:00
|
|
|
|
ci: parallelize testing on Windows
The fact that Git's test suite is implemented in Unix shell script that
is as portable as we can muster, combined with the fact that Unix shell
scripting is foreign to Windows (and therefore has to be emulated),
results in pretty abysmal speed of the test suite on that platform, for
pretty much no other reason than that language choice.
For comparison: while the Linux build & test is typically done within
about 8 minutes, the Windows build & test typically lasts about 80
minutes in Azure Pipelines.
To help with that, let's use the Azure Pipeline feature where you can
parallelize jobs, make jobs depend on each other, and pass artifacts
between them.
The tests are distributed using the following heuristic: listing all
test scripts ordered by size in descending order (as a cheap way to
estimate the overall run time), every Nth script is run (where N is the
total number of parallel jobs), starting at the index corresponding to
the parallel job. This slicing is performed by a new function that is
added to the `test-tool`.
To optimize the overall runtime of the entire Pipeline, we need to move
the Windows jobs to the beginning (otherwise there would be a very
decent chance for the Pipeline to be run only the Windows build, while
all the parallel Windows test jobs wait for this single one).
We use Azure Pipelines Artifacts for both the minimal Git for Windows
SDK as well as the built executables, as deduplication and caching close
to the agents makes that really fast. For comparison: while downloading
and unpacking the minimal Git for Windows SDK via PowerShell takes only
one minute (down from anywhere between 2.5 to 7 when using a shallow
clone), uploading it as Pipeline Artifact takes less than 30s and
downloading and unpacking less than 20s (sometimes even as little as
only twelve seconds).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:38 +08:00
|
|
|
ci/run-test-slice.sh `$SYSTEM_JOBPOSITIONINPHASE `$SYSTEM_TOTALJOBSINPHASE || {
|
ci: add a Windows job to the Azure Pipelines definition
Previously, we did not have robust support for Windows in our CI
definition, simply because Travis cannot accommodate our needs (even
after Travis added experimental Windows support very recently, it takes
longer than Travis' 50 minute timeout to build Git and run the test
suite on Windows). Instead, we used a hack that started a dedicated
Azure Pipeline from Travis and waited for the output, often timing out
(which is quite fragile, as we found out).
With this commit, we finally have first-class support for Windows in our
CI definition (in the Azure Pipelines one, that is).
Due to our reliance on Unix shell scripting in the test suite, combined
with the challenges on executing such scripts on Windows, the Windows
job currently takes a whopping ~1h20m to complete. Which is *far* longer
than the next-longest job takes (linux-gcc, ~35m).
Now, Azure Pipelines's free tier for open source projects (such as Git)
offers up to 10 concurrent jobs for free, meaning that the overall run
time will be dominated by the slowest job(s).
Therefore, it makes sense to start the Windows job first, to minimize
the time the entire build takes from start to end (which is now pretty
safely the run time of the Windows job).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:30 +08:00
|
|
|
ci/print-test-failures.sh
|
|
|
|
|
exit 1
|
|
|
|
|
}
|
|
|
|
|
"@
|
|
|
|
|
if (!$?) { exit(1) }
|
ci: parallelize testing on Windows
The fact that Git's test suite is implemented in Unix shell script that
is as portable as we can muster, combined with the fact that Unix shell
scripting is foreign to Windows (and therefore has to be emulated),
results in pretty abysmal speed of the test suite on that platform, for
pretty much no other reason than that language choice.
For comparison: while the Linux build & test is typically done within
about 8 minutes, the Windows build & test typically lasts about 80
minutes in Azure Pipelines.
To help with that, let's use the Azure Pipeline feature where you can
parallelize jobs, make jobs depend on each other, and pass artifacts
between them.
The tests are distributed using the following heuristic: listing all
test scripts ordered by size in descending order (as a cheap way to
estimate the overall run time), every Nth script is run (where N is the
total number of parallel jobs), starting at the index corresponding to
the parallel job. This slicing is performed by a new function that is
added to the `test-tool`.
To optimize the overall runtime of the entire Pipeline, we need to move
the Windows jobs to the beginning (otherwise there would be a very
decent chance for the Pipeline to be run only the Windows build, while
all the parallel Windows test jobs wait for this single one).
We use Azure Pipelines Artifacts for both the minimal Git for Windows
SDK as well as the built executables, as deduplication and caching close
to the agents makes that really fast. For comparison: while downloading
and unpacking the minimal Git for Windows SDK via PowerShell takes only
one minute (down from anywhere between 2.5 to 7 when using a shallow
clone), uploading it as Pipeline Artifact takes less than 30s and
downloading and unpacking less than 20s (sometimes even as little as
only twelve seconds).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:38 +08:00
|
|
|
displayName: 'Test (parallel)'
|
ci: add a Windows job to the Azure Pipelines definition
Previously, we did not have robust support for Windows in our CI
definition, simply because Travis cannot accommodate our needs (even
after Travis added experimental Windows support very recently, it takes
longer than Travis' 50 minute timeout to build Git and run the test
suite on Windows). Instead, we used a hack that started a dedicated
Azure Pipeline from Travis and waited for the output, often timing out
(which is quite fragile, as we found out).
With this commit, we finally have first-class support for Windows in our
CI definition (in the Azure Pipelines one, that is).
Due to our reliance on Unix shell scripting in the test suite, combined
with the challenges on executing such scripts on Windows, the Windows
job currently takes a whopping ~1h20m to complete. Which is *far* longer
than the next-longest job takes (linux-gcc, ~35m).
Now, Azure Pipelines's free tier for open source projects (such as Git)
offers up to 10 concurrent jobs for free, meaning that the overall run
time will be dominated by the slowest job(s).
Therefore, it makes sense to start the Windows job first, to minimize
the time the entire build takes from start to end (which is now pretty
safely the run time of the Windows job).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:30 +08:00
|
|
|
env:
|
|
|
|
|
HOME: $(Build.SourcesDirectory)
|
|
|
|
|
MSYSTEM: MINGW64
|
ci: parallelize testing on Windows
The fact that Git's test suite is implemented in Unix shell script that
is as portable as we can muster, combined with the fact that Unix shell
scripting is foreign to Windows (and therefore has to be emulated),
results in pretty abysmal speed of the test suite on that platform, for
pretty much no other reason than that language choice.
For comparison: while the Linux build & test is typically done within
about 8 minutes, the Windows build & test typically lasts about 80
minutes in Azure Pipelines.
To help with that, let's use the Azure Pipeline feature where you can
parallelize jobs, make jobs depend on each other, and pass artifacts
between them.
The tests are distributed using the following heuristic: listing all
test scripts ordered by size in descending order (as a cheap way to
estimate the overall run time), every Nth script is run (where N is the
total number of parallel jobs), starting at the index corresponding to
the parallel job. This slicing is performed by a new function that is
added to the `test-tool`.
To optimize the overall runtime of the entire Pipeline, we need to move
the Windows jobs to the beginning (otherwise there would be a very
decent chance for the Pipeline to be run only the Windows build, while
all the parallel Windows test jobs wait for this single one).
We use Azure Pipelines Artifacts for both the minimal Git for Windows
SDK as well as the built executables, as deduplication and caching close
to the agents makes that really fast. For comparison: while downloading
and unpacking the minimal Git for Windows SDK via PowerShell takes only
one minute (down from anywhere between 2.5 to 7 when using a shallow
clone), uploading it as Pipeline Artifact takes less than 30s and
downloading and unpacking less than 20s (sometimes even as little as
only twelve seconds).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:38 +08:00
|
|
|
NO_SVN_TESTS: 1
|
|
|
|
|
GIT_TEST_SKIP_REBASE_P: 1
|
ci: add a Windows job to the Azure Pipelines definition
Previously, we did not have robust support for Windows in our CI
definition, simply because Travis cannot accommodate our needs (even
after Travis added experimental Windows support very recently, it takes
longer than Travis' 50 minute timeout to build Git and run the test
suite on Windows). Instead, we used a hack that started a dedicated
Azure Pipeline from Travis and waited for the output, often timing out
(which is quite fragile, as we found out).
With this commit, we finally have first-class support for Windows in our
CI definition (in the Azure Pipelines one, that is).
Due to our reliance on Unix shell scripting in the test suite, combined
with the challenges on executing such scripts on Windows, the Windows
job currently takes a whopping ~1h20m to complete. Which is *far* longer
than the next-longest job takes (linux-gcc, ~35m).
Now, Azure Pipelines's free tier for open source projects (such as Git)
offers up to 10 concurrent jobs for free, meaning that the overall run
time will be dominated by the slowest job(s).
Therefore, it makes sense to start the Windows job first, to minimize
the time the entire build takes from start to end (which is now pretty
safely the run time of the Windows job).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:30 +08:00
|
|
|
- powershell: |
|
|
|
|
|
if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
|
|
|
|
|
cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
|
|
|
|
|
}
|
|
|
|
|
displayName: 'Unmount test-cache'
|
|
|
|
|
condition: true
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- task: PublishTestResults@2
|
|
|
|
|
displayName: 'Publish Test Results **/TEST-*.xml'
|
|
|
|
|
inputs:
|
|
|
|
|
mergeTestResults: true
|
|
|
|
|
testRunTitle: 'windows'
|
|
|
|
|
platform: Windows
|
|
|
|
|
publishRunAttachments: false
|
|
|
|
|
condition: succeededOrFailed()
|
|
|
|
|
- task: PublishBuildArtifacts@1
|
|
|
|
|
displayName: 'Publish trash directories of failed tests'
|
|
|
|
|
condition: failed()
|
|
|
|
|
inputs:
|
|
|
|
|
PathtoPublish: t/failed-test-artifacts
|
|
|
|
|
ArtifactName: failed-test-artifacts
|
|
|
|
|
|
2019-10-04 23:09:36 +08:00
|
|
|
- job: vs_build
|
|
|
|
|
displayName: Visual Studio Build
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: windows-latest
|
2019-10-04 23:09:36 +08:00
|
|
|
timeoutInMinutes: 240
|
|
|
|
|
steps:
|
|
|
|
|
- powershell: |
|
|
|
|
|
if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
|
|
|
|
|
net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no
|
|
|
|
|
cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\
|
|
|
|
|
}
|
|
|
|
|
displayName: 'Mount test-cache'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- powershell: |
|
|
|
|
|
$urlbase = "https://dev.azure.com/git-for-windows/git/_apis/build/builds"
|
|
|
|
|
$id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=22&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id
|
|
|
|
|
$downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[1].resource.downloadUrl
|
|
|
|
|
(New-Object Net.WebClient).DownloadFile($downloadUrl,"git-sdk-64-minimal.zip")
|
|
|
|
|
Expand-Archive git-sdk-64-minimal.zip -DestinationPath . -Force
|
|
|
|
|
Remove-Item git-sdk-64-minimal.zip
|
|
|
|
|
|
|
|
|
|
# Let Git ignore the SDK and the test-cache
|
|
|
|
|
"/git-sdk-64-minimal/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude"
|
|
|
|
|
displayName: 'Download git-sdk-64-minimal'
|
|
|
|
|
- powershell: |
|
|
|
|
|
& git-sdk-64-minimal\usr\bin\bash.exe -lc @"
|
2019-10-22 03:59:57 +08:00
|
|
|
make NDEBUG=1 DEVELOPER=1 vcxproj
|
2019-10-04 23:09:36 +08:00
|
|
|
"@
|
|
|
|
|
if (!$?) { exit(1) }
|
|
|
|
|
displayName: Generate Visual Studio Solution
|
|
|
|
|
env:
|
|
|
|
|
HOME: $(Build.SourcesDirectory)
|
|
|
|
|
MSYSTEM: MINGW64
|
|
|
|
|
DEVELOPER: 1
|
|
|
|
|
NO_PERL: 1
|
|
|
|
|
GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
|
|
|
|
|
- powershell: |
|
|
|
|
|
$urlbase = "https://dev.azure.com/git/git/_apis/build/builds"
|
|
|
|
|
$id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=9&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id
|
|
|
|
|
$downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[0].resource.downloadUrl
|
|
|
|
|
(New-Object Net.WebClient).DownloadFile($downloadUrl, "compat.zip")
|
|
|
|
|
Expand-Archive compat.zip -DestinationPath . -Force
|
|
|
|
|
Remove-Item compat.zip
|
|
|
|
|
displayName: 'Download vcpkg artifacts'
|
|
|
|
|
- task: MSBuild@1
|
|
|
|
|
inputs:
|
|
|
|
|
solution: git.sln
|
|
|
|
|
platform: x64
|
|
|
|
|
configuration: Release
|
|
|
|
|
maximumCpuCount: 4
|
2020-02-27 21:23:13 +08:00
|
|
|
msbuildArguments: /p:PlatformToolset=v142
|
2019-10-04 23:09:36 +08:00
|
|
|
- powershell: |
|
|
|
|
|
& compat\vcbuild\vcpkg_copy_dlls.bat release
|
|
|
|
|
if (!$?) { exit(1) }
|
|
|
|
|
& git-sdk-64-minimal\usr\bin\bash.exe -lc @"
|
|
|
|
|
mkdir -p artifacts &&
|
|
|
|
|
eval \"`$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts | grep ^tar)\"
|
|
|
|
|
"@
|
|
|
|
|
if (!$?) { exit(1) }
|
|
|
|
|
displayName: Bundle artifact tar
|
|
|
|
|
env:
|
|
|
|
|
HOME: $(Build.SourcesDirectory)
|
|
|
|
|
MSYSTEM: MINGW64
|
|
|
|
|
DEVELOPER: 1
|
|
|
|
|
NO_PERL: 1
|
|
|
|
|
MSVC: 1
|
|
|
|
|
VCPKG_ROOT: $(Build.SourcesDirectory)\compat\vcbuild\vcpkg
|
|
|
|
|
- powershell: |
|
|
|
|
|
$tag = (Invoke-WebRequest -UseBasicParsing "https://gitforwindows.org/latest-tag.txt").content
|
|
|
|
|
$version = (Invoke-WebRequest -UseBasicParsing "https://gitforwindows.org/latest-version.txt").content
|
|
|
|
|
$url = "https://github.com/git-for-windows/git/releases/download/${tag}/PortableGit-${version}-64-bit.7z.exe"
|
|
|
|
|
(New-Object Net.WebClient).DownloadFile($url,"PortableGit.exe")
|
|
|
|
|
& .\PortableGit.exe -y -oartifacts\PortableGit
|
|
|
|
|
# Wait until it is unpacked
|
|
|
|
|
while (-not @(Remove-Item -ErrorAction SilentlyContinue PortableGit.exe; $?)) { sleep 1 }
|
|
|
|
|
displayName: Download & extract portable Git
|
|
|
|
|
- task: PublishPipelineArtifact@0
|
|
|
|
|
displayName: 'Publish Pipeline Artifact: MSVC test artifacts'
|
|
|
|
|
inputs:
|
|
|
|
|
artifactName: 'vs-artifacts'
|
|
|
|
|
targetPath: '$(Build.SourcesDirectory)\artifacts'
|
|
|
|
|
- powershell: |
|
|
|
|
|
if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
|
|
|
|
|
cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
|
|
|
|
|
}
|
|
|
|
|
displayName: 'Unmount test-cache'
|
|
|
|
|
condition: true
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
|
|
|
|
|
- job: vs_test
|
|
|
|
|
displayName: Visual Studio Test
|
|
|
|
|
dependsOn: vs_build
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: windows-latest
|
2019-10-04 23:09:36 +08:00
|
|
|
timeoutInMinutes: 240
|
|
|
|
|
strategy:
|
|
|
|
|
parallel: 10
|
|
|
|
|
steps:
|
|
|
|
|
- powershell: |
|
|
|
|
|
if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
|
|
|
|
|
net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no
|
|
|
|
|
cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\
|
|
|
|
|
}
|
|
|
|
|
displayName: 'Mount test-cache'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- task: DownloadPipelineArtifact@0
|
|
|
|
|
displayName: 'Download Pipeline Artifact: VS test artifacts'
|
|
|
|
|
inputs:
|
|
|
|
|
artifactName: 'vs-artifacts'
|
|
|
|
|
targetPath: '$(Build.SourcesDirectory)'
|
|
|
|
|
- powershell: |
|
|
|
|
|
& PortableGit\git-cmd.exe --command=usr\bin\bash.exe -lc @"
|
|
|
|
|
test -f artifacts.tar.gz || {
|
|
|
|
|
echo No test artifacts found\; skipping >&2
|
|
|
|
|
exit 0
|
|
|
|
|
}
|
|
|
|
|
tar xf artifacts.tar.gz || exit 1
|
|
|
|
|
|
|
|
|
|
# Let Git ignore the SDK and the test-cache
|
|
|
|
|
printf '%s\n' /PortableGit/ /test-cache/ >>.git/info/exclude
|
|
|
|
|
|
|
|
|
|
cd t &&
|
|
|
|
|
PATH=\"`$PWD/helper:`$PATH\" &&
|
ci(visual-studio): actually run the tests in parallel
Originally, the CI/PR builds that build and test using Visual Studio
were implemented imitating `linux-clang`, i.e. still using the
`Makefile`-based build infrastructure.
Later (but still before the patches made their way into git.git's
`master`), however, this was changed to generate Visual Studio project
files and build the binaries using `MSBuild`, as this reflects more
accurately how Visual Studio users would want to build Git (internally,
Visual Studio uses `MSBuild`, or at least something very similar).
During that transition, we needed to implement a new way to run the test
suite in parallel, as Visual Studio users typically will only have a Git
Bash available (which does not ship with `make` nor with support for
`prove`): we simply implemented a new test helper to run the test suite.
This helper even knows how to run the tests in parallel, but due to a
mistake on this developer's part, it was never turned on in the CI/PR
builds. This results in 2x-3x longer run times of the test phase.
Let's use the `--jobs=10` option to fix this.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-22 03:59:58 +08:00
|
|
|
test-tool.exe run-command testsuite --jobs=10 -V -x --write-junit-xml \
|
2019-10-04 23:09:36 +08:00
|
|
|
`$(test-tool.exe path-utils slice-tests \
|
|
|
|
|
`$SYSTEM_JOBPOSITIONINPHASE `$SYSTEM_TOTALJOBSINPHASE t[0-9]*.sh)
|
|
|
|
|
"@
|
|
|
|
|
if (!$?) { exit(1) }
|
|
|
|
|
displayName: 'Test (parallel)'
|
|
|
|
|
env:
|
|
|
|
|
HOME: $(Build.SourcesDirectory)
|
|
|
|
|
MSYSTEM: MINGW64
|
|
|
|
|
NO_SVN_TESTS: 1
|
|
|
|
|
GIT_TEST_SKIP_REBASE_P: 1
|
|
|
|
|
- powershell: |
|
|
|
|
|
if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
|
|
|
|
|
cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
|
|
|
|
|
}
|
|
|
|
|
displayName: 'Unmount test-cache'
|
|
|
|
|
condition: true
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- task: PublishTestResults@2
|
|
|
|
|
displayName: 'Publish Test Results **/TEST-*.xml'
|
|
|
|
|
inputs:
|
|
|
|
|
mergeTestResults: true
|
|
|
|
|
testRunTitle: 'vs'
|
|
|
|
|
platform: Windows
|
|
|
|
|
publishRunAttachments: false
|
|
|
|
|
condition: succeededOrFailed()
|
|
|
|
|
- task: PublishBuildArtifacts@1
|
|
|
|
|
displayName: 'Publish trash directories of failed tests'
|
|
|
|
|
condition: failed()
|
|
|
|
|
inputs:
|
|
|
|
|
PathtoPublish: t/failed-test-artifacts
|
|
|
|
|
ArtifactName: failed-vs-test-artifacts
|
|
|
|
|
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
- job: linux_clang
|
|
|
|
|
displayName: linux-clang
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: ubuntu-latest
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
steps:
|
|
|
|
|
- bash: |
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
|
|
|
|
|
|
|
|
|
|
sudo apt-get update &&
|
|
|
|
|
sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin &&
|
|
|
|
|
|
|
|
|
|
export CC=clang || exit 1
|
|
|
|
|
|
|
|
|
|
ci/install-dependencies.sh || exit 1
|
|
|
|
|
ci/run-build-and-tests.sh || {
|
|
|
|
|
ci/print-test-failures.sh
|
|
|
|
|
exit 1
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
|
|
|
|
|
displayName: 'ci/run-build-and-tests.sh'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- task: PublishTestResults@2
|
|
|
|
|
displayName: 'Publish Test Results **/TEST-*.xml'
|
|
|
|
|
inputs:
|
|
|
|
|
mergeTestResults: true
|
|
|
|
|
testRunTitle: 'linux-clang'
|
|
|
|
|
platform: Linux
|
|
|
|
|
publishRunAttachments: false
|
|
|
|
|
condition: succeededOrFailed()
|
|
|
|
|
- task: PublishBuildArtifacts@1
|
|
|
|
|
displayName: 'Publish trash directories of failed tests'
|
|
|
|
|
condition: failed()
|
|
|
|
|
inputs:
|
|
|
|
|
PathtoPublish: t/failed-test-artifacts
|
|
|
|
|
ArtifactName: failed-test-artifacts
|
|
|
|
|
|
|
|
|
|
- job: linux_gcc
|
|
|
|
|
displayName: linux-gcc
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: ubuntu-latest
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
steps:
|
|
|
|
|
- bash: |
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
|
|
|
|
|
|
|
|
|
|
sudo add-apt-repository ppa:ubuntu-toolchain-r/test &&
|
|
|
|
|
sudo apt-get update &&
|
|
|
|
|
sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2 language-pack-is git-svn gcc-8 || exit 1
|
|
|
|
|
|
|
|
|
|
ci/install-dependencies.sh || exit 1
|
|
|
|
|
ci/run-build-and-tests.sh || {
|
|
|
|
|
ci/print-test-failures.sh
|
|
|
|
|
exit 1
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
|
|
|
|
|
displayName: 'ci/run-build-and-tests.sh'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- task: PublishTestResults@2
|
|
|
|
|
displayName: 'Publish Test Results **/TEST-*.xml'
|
|
|
|
|
inputs:
|
|
|
|
|
mergeTestResults: true
|
|
|
|
|
testRunTitle: 'linux-gcc'
|
|
|
|
|
platform: Linux
|
|
|
|
|
publishRunAttachments: false
|
|
|
|
|
condition: succeededOrFailed()
|
|
|
|
|
- task: PublishBuildArtifacts@1
|
|
|
|
|
displayName: 'Publish trash directories of failed tests'
|
|
|
|
|
condition: failed()
|
|
|
|
|
inputs:
|
|
|
|
|
PathtoPublish: t/failed-test-artifacts
|
|
|
|
|
ArtifactName: failed-test-artifacts
|
|
|
|
|
|
|
|
|
|
- job: osx_clang
|
|
|
|
|
displayName: osx-clang
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: macOS-latest
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
steps:
|
|
|
|
|
- bash: |
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
|
|
|
|
|
|
|
|
|
|
export CC=clang
|
|
|
|
|
|
|
|
|
|
ci/install-dependencies.sh || exit 1
|
|
|
|
|
ci/run-build-and-tests.sh || {
|
|
|
|
|
ci/print-test-failures.sh
|
|
|
|
|
exit 1
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
|
|
|
|
|
displayName: 'ci/run-build-and-tests.sh'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- task: PublishTestResults@2
|
|
|
|
|
displayName: 'Publish Test Results **/TEST-*.xml'
|
|
|
|
|
inputs:
|
|
|
|
|
mergeTestResults: true
|
|
|
|
|
testRunTitle: 'osx-clang'
|
|
|
|
|
platform: macOS
|
|
|
|
|
publishRunAttachments: false
|
|
|
|
|
condition: succeededOrFailed()
|
|
|
|
|
- task: PublishBuildArtifacts@1
|
|
|
|
|
displayName: 'Publish trash directories of failed tests'
|
|
|
|
|
condition: failed()
|
|
|
|
|
inputs:
|
|
|
|
|
PathtoPublish: t/failed-test-artifacts
|
|
|
|
|
ArtifactName: failed-test-artifacts
|
|
|
|
|
|
|
|
|
|
- job: osx_gcc
|
|
|
|
|
displayName: osx-gcc
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: macOS-latest
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
steps:
|
|
|
|
|
- bash: |
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
|
|
|
|
|
|
|
|
|
|
ci/install-dependencies.sh || exit 1
|
|
|
|
|
ci/run-build-and-tests.sh || {
|
|
|
|
|
ci/print-test-failures.sh
|
|
|
|
|
exit 1
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
|
|
|
|
|
displayName: 'ci/run-build-and-tests.sh'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- task: PublishTestResults@2
|
|
|
|
|
displayName: 'Publish Test Results **/TEST-*.xml'
|
|
|
|
|
inputs:
|
|
|
|
|
mergeTestResults: true
|
|
|
|
|
testRunTitle: 'osx-gcc'
|
|
|
|
|
platform: macOS
|
|
|
|
|
publishRunAttachments: false
|
|
|
|
|
condition: succeededOrFailed()
|
|
|
|
|
- task: PublishBuildArtifacts@1
|
|
|
|
|
displayName: 'Publish trash directories of failed tests'
|
|
|
|
|
condition: failed()
|
|
|
|
|
inputs:
|
|
|
|
|
PathtoPublish: t/failed-test-artifacts
|
|
|
|
|
ArtifactName: failed-test-artifacts
|
|
|
|
|
|
|
|
|
|
- job: gettext_poison
|
|
|
|
|
displayName: GETTEXT_POISON
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: ubuntu-latest
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
steps:
|
|
|
|
|
- bash: |
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
|
|
|
|
|
|
|
|
|
|
sudo apt-get update &&
|
|
|
|
|
sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev &&
|
|
|
|
|
|
|
|
|
|
export jobname=GETTEXT_POISON || exit 1
|
|
|
|
|
|
|
|
|
|
ci/run-build-and-tests.sh || {
|
|
|
|
|
ci/print-test-failures.sh
|
|
|
|
|
exit 1
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
|
|
|
|
|
displayName: 'ci/run-build-and-tests.sh'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- task: PublishTestResults@2
|
|
|
|
|
displayName: 'Publish Test Results **/TEST-*.xml'
|
|
|
|
|
inputs:
|
|
|
|
|
mergeTestResults: true
|
|
|
|
|
testRunTitle: 'gettext-poison'
|
|
|
|
|
platform: Linux
|
|
|
|
|
publishRunAttachments: false
|
|
|
|
|
condition: succeededOrFailed()
|
|
|
|
|
- task: PublishBuildArtifacts@1
|
|
|
|
|
displayName: 'Publish trash directories of failed tests'
|
|
|
|
|
condition: failed()
|
|
|
|
|
inputs:
|
|
|
|
|
PathtoPublish: t/failed-test-artifacts
|
|
|
|
|
ArtifactName: failed-test-artifacts
|
|
|
|
|
|
|
|
|
|
- job: linux32
|
|
|
|
|
displayName: Linux32
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: ubuntu-latest
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
steps:
|
|
|
|
|
- bash: |
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
|
|
|
|
|
|
|
|
|
|
res=0
|
|
|
|
|
sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1
|
|
|
|
|
|
|
|
|
|
sudo chmod a+r t/out/TEST-*.xml
|
|
|
|
|
test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts
|
|
|
|
|
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1
|
|
|
|
|
exit $res
|
|
|
|
|
displayName: 'ci/run-linux32-docker.sh'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
- task: PublishTestResults@2
|
|
|
|
|
displayName: 'Publish Test Results **/TEST-*.xml'
|
|
|
|
|
inputs:
|
|
|
|
|
mergeTestResults: true
|
|
|
|
|
testRunTitle: 'linux32'
|
|
|
|
|
platform: Linux
|
|
|
|
|
publishRunAttachments: false
|
|
|
|
|
condition: succeededOrFailed()
|
|
|
|
|
- task: PublishBuildArtifacts@1
|
|
|
|
|
displayName: 'Publish trash directories of failed tests'
|
|
|
|
|
condition: failed()
|
|
|
|
|
inputs:
|
|
|
|
|
PathtoPublish: t/failed-test-artifacts
|
|
|
|
|
ArtifactName: failed-test-artifacts
|
|
|
|
|
|
|
|
|
|
- job: static_analysis
|
|
|
|
|
displayName: StaticAnalysis
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: ubuntu-latest
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
steps:
|
|
|
|
|
- bash: |
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
|
|
|
|
|
|
|
|
|
|
sudo apt-get update &&
|
2019-10-01 19:16:26 +08:00
|
|
|
sudo apt-get install -y coccinelle libcurl4-openssl-dev libssl-dev libexpat-dev gettext &&
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
|
|
|
|
|
export jobname=StaticAnalysis &&
|
|
|
|
|
|
|
|
|
|
ci/run-static-analysis.sh || exit 1
|
|
|
|
|
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
|
|
|
|
|
displayName: 'ci/run-static-analysis.sh'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|
|
|
|
|
|
|
|
|
|
- job: documentation
|
|
|
|
|
displayName: Documentation
|
|
|
|
|
condition: succeeded()
|
2020-02-27 21:23:13 +08:00
|
|
|
pool:
|
|
|
|
|
vmImage: ubuntu-latest
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
steps:
|
|
|
|
|
- bash: |
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
|
|
|
|
|
|
|
|
|
|
sudo apt-get update &&
|
Documentation: fix build with Asciidoctor 2
Our documentation toolchain has traditionally been built around DocBook
4.5. This version of DocBook is the last DTD-based version of DocBook.
In 2009, DocBook 5 was introduced using namespaces and its syntax is
expressed in RELAX NG, which is more expressive and allows a wider
variety of syntax forms.
Asciidoctor, one of the alternatives for building our documentation,
moved support for DocBook 4.5 out of core in its recent 2.0 release and
now only supports DocBook 5 in the main release. The DocBoook 4.5
converter is still available as a separate component, but this is not
available in most distro packages. This would not be a problem but for
the fact that we use xmlto, which is still stuck in the DocBook 4.5 era.
xmlto performs DTD validation as part of the build process. This is not
problematic for DocBook 4.5, which has a valid DTD, but it clearly
cannot work for DocBook 5, since no DTD can adequately express its full
syntax. In addition, even if xmlto did support RELAX NG validation,
that wouldn't be sufficient because it uses the libxml2-based xmllint to
do so, which has known problems with validating interleaves in RELAX NG.
Fortunately, there's an easy way forward: ask Asciidoctor to use its
DocBook 5 backend and tell xmlto to skip validation. Asciidoctor has
supported DocBook 5 since v0.1.4 in 2013 and xmlto has supported
skipping validation for probably longer than that.
We also need to teach xmlto how to use the namespaced DocBook XSLT
stylesheets instead of the non-namespaced ones it usually uses.
Normally these stylesheets are interchangeable, but the non-namespaced
ones have a bug that causes them not to strip whitespace automatically
from certain elements when namespaces are in use. This results in
additional whitespace at the beginning of list elements, which is
jarring and unsightly.
We can do this by passing a custom stylesheet with the -x option that
simply imports the namespaced stylesheets via a URL. Any system with
support for XML catalogs will automatically look this URL up and
reference a local copy instead without us having to know where this
local copy is located. We know that anyone using xmlto will already
have catalogs set up properly since the DocBook 4.5 DTD used during
validation is also looked up via catalogs. All major Linux
distributions distribute the necessary stylesheets and have built-in
catalog support, and Homebrew does as well, albeit with a requirement to
set an environment variable to enable catalog support.
On the off chance that someone lacks support for catalogs, it is
possible for xmlto (via xmllint) to download the stylesheets from the
URLs in question, although this will likely perform poorly enough to
attract attention. People still have the option of using the prebuilt
documentation that we ship, so happily this should not be an impediment.
Finally, we need to filter out some messages from other stylesheets that
occur when invoking dblatex in the CI job. This tool strips namespaces
much like the unnamespaced DocBook stylesheets and prints similar
messages. If we permit these messages to be printed to standard error,
our documentation CI job will fail because we check standard error for
unexpected output. Due to dblatex's reliance on Python 2, we may need
to revisit its use in the future, in which case this problem may go
away, but this can be delayed until a future patch.
The final message we filter is due to libxslt on modern Debian and
Ubuntu. The patch which they use to implement reproducible ID
generation also prints messages about the ID generation. While this
doesn't affect our current CI images since they use Ubuntu 16.04 which
lacks this patch, if we upgrade to Ubuntu 18.04 or a modern Debian,
these messages will appear and, like the above messages, cause a CI
failure.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-09-16 06:43:32 +08:00
|
|
|
sudo apt-get install -y asciidoc xmlto asciidoctor docbook-xsl-ns &&
|
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.
The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.
Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).
To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).
Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 22:19:29 +08:00
|
|
|
|
|
|
|
|
export ALREADY_HAVE_ASCIIDOCTOR=yes. &&
|
|
|
|
|
export jobname=Documentation &&
|
|
|
|
|
|
|
|
|
|
ci/test-documentation.sh || exit 1
|
|
|
|
|
|
|
|
|
|
test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
|
|
|
|
|
displayName: 'ci/test-documentation.sh'
|
|
|
|
|
env:
|
|
|
|
|
GITFILESHAREPWD: $(gitfileshare.pwd)
|