All the post-merge pipelines are failing to be created with this error:
'shader-db' job needs 'debian-build-testing' job, but 'debian-build-testing' is not in any previous stage
Like all the other test jobs, shader-db should never run in post-merge
pipelines.
Fixes: 0724b00dad ("ci: run shader-db only when one of the drivers it tests is modified")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31758>
If you're emitting a section header under set -x, you will see:
+ section_start foo "foo"
+ x_off
[the section header]
This is kind of annoying. Instead of trying to squash it everywhere by
dancing around local set +x management, play some extremely stupid
tricks with shells to make sure we never emit it.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31602>
The date isn't interesting, since you can tell if it wraps around
anyway. Just print the time down to the millisecond, which should be
plenty enough. We also don't need to print 'R SERIAL_CPU> ' when almost
every line is reading from the CPU serial buffer, so make that the
default.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31602>
All the foreground colours pass 1 to ANSI SGR, which sets bold. The
other arguments are either a colour from 30-37 (passed directly), or
38;5;nnn, where nnn is an extended RGB colour. It looks like most of the
definitions were cargo-culted from FG_RED, which correctly sets an
extended colour, because the arguments there were being parsed as
setting blinking, followed by 197 which was ignored as unknown.
Fix them to just set the original definition.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31602>
It's really tedious having to run shellcheck in a loop to find every
failure; go through them all and print them all at once.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31602>
ANGLE has a waiver for certain XFB tests, but this wasn't properly applied
on Alder Lake and these tests weren't skipped there.
Add a global angle-skips.txt file so that we don't have to keep copy-pasting
these skips.
Signed-off-by: Valentine Burley <valentine.burley@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31721>
Our current install tarball is 120ish MB zstd, and over 500MB unpacked.
A lot of this size was two static libraries which we'll obviously never
use at runtime.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31735>
ANGLE currently pulls absolutely loads of stuff that we don't need. Fix
it up so we don't need to do that anymore, so it's much faster to build.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31716>
These skip files contain tests that exceed one minute in execution time,
which are better suited for nightly jobs. This change aims to optimize
execution time and prevent delays in merge pipelines under time
constraints.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31699>
The Vulkan caselist is gigantic - over 400MB - and only growing over
time. Luckily zstd gets that down to about 9MB, with absolutely
negligible runtime overhead.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31595>
Currently, we:
* copy the case list to a temporary file
* have sed read it in, decimate for fraction, write it out again
* have sed read it in, decimate for shard, write it out again
Flatten this out to have a single-pass sed covering both fraction and
shard that reads the source caselist in directly and writes the final
one.
This comes at the cost of an expression that's unreadable even for sed,
but with the Vulkan mustpass over 400MB and growing, it's a good
improvement.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31595>
The validation layers being installed are somehow 206MB in size. Slim
them down to 26 MB by building the dependencies in release mode, and
stripping on install.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31595>
We previously only had two traces (one for Raven, one for lavapipe)
running via Wine. To support this, the Vulkan container had Debian's
full Wine installation, one from DXVK, and then a third from Proton
which is used by the b2c jobs.
At 600MB each, this was pretty unsustainable. Remove everything but the
Proton ones.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31595>
Since deqp-runner is already being built in the debian base image for test
jobs, it’s not necessary to build it subsequently in GL and VK debian
test images. Update the image tag instructions for uprevving deqp-runner.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31429>
Since the 'git clone --branch' option only accepts branch names or tags as
arguments, it’s not currently possible to build deqp-runner directly from
a git commit hash.
Revise the deqp-runner build script so that it can build from a commit
even if it doesn’t have a tag or version number.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31429>
The 'yaml-toml-shell-test' job has a task to review the CI shell files. So,
we need this job present when there is a change on any of those shells.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31627>
Some merge requests are failing due to `.gitlab-ci/run-shellcheck.sh` failing
on the 'yaml-toml-shell-test' because an 'or', on an elif, in the meson's
build needs to be reformulated.
Fixes: afab416be4 ("ci: enable IPO/LTO only for nightly builds")
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Acked-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Reviewed-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31627>
prepare-artifacts-python.sh fails in drm-ci, since
src directory is not present in linux. So copy files
only if src directory is present.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31586>
1. nightly takes a long, the extra few minutes for linking don't matter
2. nightly will run faster, since where it's CPU dependent it's at least +5% perf
3. it may reveal some painful areas of common code or driver.
4. for some jobs (not enabled yet, it generates ugly errors, disable
there
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Signed-off-by: David Heidelberg <david@ixit.cz>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28684>
Bring it up to parity with the LAVA and bare-metal containers by
stripping things we don't need at runtime. There is a lot of stuff we
don't need in container images we only use to execute tests, including
but not limited to the system Mesa which can only cause problems. Call
the same strip-rootfs we already run for LAVA to make sure that this
doesn't happen, as well as slimming down the container image.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31281>
Set exit_code to 1 in case of an exception; otherwise,
the job exits with 0, and GitLab shows the job as successful.
Fixes: b9cee06f9e ("ci/lava: handle non-zero exit codes")
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31556>
Following the merge of Merge Request #31151, we encountered an issue
where the performance jobs were failing silently. Although these
failures did not cause the pipeline to fail, they resulted in warnings
for all merge requests that ran the .*-traces-performance jobs, putting
critical performance data for the [Mesa Performance Driver
dashboard](https://ci-stats-grafana.freedesktop.org/goto/G3xkvykHg?orgId=1)
at risk.
To resolve this issue, this commit updates the LAVA performance jobs to
utilize the Pyutils artifact package, which is now the only required
artifact for the jobs that run the LAVA job submitter.
Fixes: dd5d737e6c ("ci/lava: Use new pyutils container")
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31553>
Add firmware for Gen10 Arm Mali GPUs directly to rootfs since
it is not available from debian package. For the panthor kernel
driver to be able to execute jobs on the GPU it needs this firmware.
Update the kernel, which contains a fix to build the panthor driver
as module and use HWCI_KERNEL_MODULES to load panthor module.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31342>
Enable build testing for zink and llvmpipe in the debian-no-libdrm job.
EGL is still excluded due to additional dependencies on libdrm.
Signed-off-by: Valentine Burley <valentine.burley@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31430>
The `.common-rules` need to be executed first, before all the "does this
file exist? is it modified?" logic in the farm rules (and in
api/driver/etc. rules after).
The custom override in windows jobs was bypassing this, resulting in
windows container jobs something being missing, breaking the pipelines.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31390>
The problem with radeonsi+ACO is that UBO loads from vec4 uniforms using
only 1 component always load all 4 components. This fixes that.
We are only interested in shrinking UBO and SSBO loads, but I added more
intrinsics because why not.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29384>
`shader-db` uses its output, and with its 8-10min runtime it can't be
ignored and should be counted with the other test jobs.
This makes the critical build jobs a bit slower but makes the overall
pipeline shorter when the test jobs respect the 15min deadline.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31327>
When debugging piglit job failure in CI, it is sometimes useful to pass options to
the runner, allow to do this even in guest when using crosvm.
Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31284>
None of the test jobs were referencing our 'common-rules', so we weren't
marking test jobs as manual when they should be.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Fixes: 07202111a6 ("ci/rules: make every job exist as manual in fork pipelines")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31270>
The fastboot/poe/cros-servo scripts always exits with code 1,
regardless of the HWCI_TEST_SCRIPT's exit code. This commit
fixes these scripts to exit with the actual code returned by
the test.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31189>
The LAVA job submitter always exits with code 1, regardless
of the HWCI_TEST_SCRIPT's exit code. This commit fixes the
LAVA job submitter to exit with the actual code returned by
the test.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31189>
The HWCI result output now includes the exit code from
the test script to help in debugging. Also Update
B2C_JOB_SUCCESS_REGEX with the exit_code for ci-tron.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31189>
Instead of using a tee to the log, step the verbosity down - we already
know by this point that it's working pretty well. Passing --output-file
directly also lets us avoid a messy 'mv'.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31110>
Make sure that ci-kdl is built directly into the destination path; the
venv documentation explicitly states that venvs cannot be relocated.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31110>
During init-stage2 (used for hardware jobs) and setup-test-env (used
for running directly on shared runners), make sure we always create a
results directory.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31110>
When we're running VirGL/Venus, we sometimes want to invert our
parallelism. As some commands can serialise at the host level, we don't
always want to launch as many test clients as we have CPU cores.
Instead, we want to use our parallelism for llvmpipe's rendering, and
launch only a single test at a time.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31110>
Instead of unpacking the x86_64_build container and its billion build
dependencies every time, switch to using only what's in the minimal
pyutils container, and the Python scripts we get as an artifact from the
python-test job. Pulling the artifacts from S3 rather than using GitLab
is also much more efficient.
This should substantially reduce the runtime required to get to testing.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31151>
Currently, our jobs which only want to run a little bit of python -
python-test and the LAVA jobs - pull the entire x86_64-build image,
which is both massive, and massively unnecessary.
Create a separate image which only carries what we need to run our
Python tests and utilities, and switch python-test to using that.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31151>
It looks like the compute reconvergence tests are slamming us into OOM
on stoney at least; backport a commit which makes this less horrible.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
SC2319 is actually helpful! It's telling us that in:
foo="$(bar)"
[ -n "$foo" ] && ret=$?
ret actually gets us the return code from [ which is by definition zero,
instead of the return code from bar that we wanted.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
This was the only thing running shellcheck and pytest. Unsurprisingly,
tons of them regressed whilst it was gone. Since these have now been
moved to other jobs, we can just bring back debian-build-testing in the
non-critical-path stage if we drop LTO, so let's do that as there hasn't
been any movement towards fixing it.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
Without EXTRA_CARGO_ARGS='--target armv7-unknown-linux-gnueabihf',
deqp-runner binary was built for aarch64 architecture for arm32
builds and causing below failures on arm32 target,
/usr/bin/deqp-runner: cannot execute binary file: Exec format error
So pass EXTRA_CARGO_ARGS to cargo to fix this.
Fixes: 83d9cfa5 ("ci/deqp-runner: build from git checkout even on linux")
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31071>
this existed for historical reasons, but realistically now it should
be possible to build mesa with dri3 always enabled. additionally,
this check was often used as a substitute for having drm functionality,
which is sort of similar but also not really a direct match
this simplifies a bunch of conditionals and prevents users from footgunnning
themselves into orbit
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30952>
Placing mtl-fw.json in src/intel/ci/mtl-fw.json works for the
mesa build, but it fails to fetch in drm-ci. Move it to the
.gitlab-ci directory so it is included in the artifacts used
for building the kernel/rootfs in drm-ci.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30947>
Sometimes not all of the jobs execute. For instance, Windows build jobs
will not trigger on AMD-only MRs. Use the 'optional' keyword to ignore
jobs which don't exist in our pipeline.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Fixes: 310e3bb026 ("ci: do not start build-only jobs until the critical build-for-tests jobs are done")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30926>
