Commit Graph

2757 Commits

Author SHA1 Message Date
Daniel Stone
dd5d737e6c ci/lava: Use new pyutils container
Instead of unpacking the x86_64_build container and its billion build
dependencies every time, switch to using only what's in the minimal
pyutils container, and the Python scripts we get as an artifact from the
python-test job. Pulling the artifacts from S3 rather than using GitLab
is also much more efficient.

This should substantially reduce the runtime required to get to testing.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31151>
2024-09-12 18:17:52 +00:00
Daniel Stone
4ee55568b7 ci/python: Allow empty PYTHONPATH
$PYTHONPATH doesn't have to be set, so let it expand to nothing if
there's nothing there.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31151>
2024-09-12 18:17:52 +00:00
Daniel Stone
51c2707b8e ci: Add pyutils container
Currently, our jobs which only want to run a little bit of python -
python-test and the LAVA jobs - pull the entire x86_64-build image,
which is both massive, and massively unnecessary.

Create a separate image which only carries what we need to run our
Python tests and utilities, and switch python-test to using that.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31151>
2024-09-12 18:17:52 +00:00
Sil Vilerino
702bd657cc ci: Bump DirectX-Headers and Agility SDK dependencies to 1.614.1
Reviewed-By: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31092>
2024-09-12 15:01:18 +00:00
David Rosca
dba3454717 ci: Disable building OMX driver
Acked-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30865>
2024-09-09 18:14:49 +00:00
Daniel Stone
ed4dc4261e ci/vulkan: Backport dEQP commit to make it less OOMy
It looks like the compute reconvergence tests are slamming us into OOM
on stoney at least; backport a commit which makes this less horrible.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
b943f42ef5 ci/intel-gpu-freq: Fix shellcheck warning
SC2319 is actually helpful! It's telling us that in:
    foo="$(bar)"
    [ -n "$foo" ] && ret=$?
ret actually gets us the return code from [ which is by definition zero,
instead of the return code from bar that we wanted.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
37c22dd403 ci: Bring back shader-db and drop LTO
This was the only thing running shellcheck and pytest. Unsurprisingly,
tons of them regressed whilst it was gone. Since these have now been
moved to other jobs, we can just bring back debian-build-testing in the
non-critical-path stage if we drop LTO, so let's do that as there hasn't
been any movement towards fixing it.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
cc4db4bc36 ci: Run pytest once for both LAVA and bin/ci
There's no need for these to be totally separate, and also for the LAVA
tests to have been disabled for months.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
28aafcfaef ci: Bring back yamllint and shellcheck
These have been disabled for a while, but shouldn't have been.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
5aea59c37e ci: Fix shellcheck warnings in firmware download
Signed-off-by: Daniel Stone <daniels@collabora.com>
Fixes: 0441202d6b ("ci: add firmware files to rootfs")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
9937ea5c1c ci/alpine: Fix shellcheck errors
Signed-off-by: Daniel Stone <daniels@collabora.com>
Fixes: 34753cefd8 ("ci/alpine: use llvm variables")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
873622e4ac ci/lava: Fix fastboot tests
We changed the args without changing the tests.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
6608b5ee46 ci/lava: Fix pytest not passing farm value
This was throwing an exception as a required argument was missing.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
1a87439518 ci/lava: Allow passing more args to pytest
Using -v or -vv is really helpful to debug failing tests.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
0a2606cf84 ci/lava: Don't dump venv into .venv
We could have a lot of venvs around. Pick a slightly more isolated one
for LAVA's pytest.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
3fd72266cd ci/lava: Make it easier to run pytest locally
If we don't have ${CI_PROJECT_DIR} set, just find it from where we're
running the script.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
eae6e122ab ci/lava: Fix path to structured logger
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Daniel Stone
f5b0699efa ci/angle: Clean up stray open-coded path
Somehow the sed seemed to miss this.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31059>
2024-09-09 12:54:34 +00:00
Vignesh Raman
c6a9a9c3bc ci/deqp-runner: pass EXTRA_CARGO_ARGS
Without EXTRA_CARGO_ARGS='--target armv7-unknown-linux-gnueabihf',
deqp-runner binary was built for aarch64 architecture for arm32
builds and causing below failures on arm32 target,
/usr/bin/deqp-runner: cannot execute binary file: Exec format error

So pass EXTRA_CARGO_ARGS to cargo to fix this.

Fixes: 83d9cfa5 ("ci/deqp-runner: build from git checkout even on linux")
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31071>
2024-09-09 08:26:47 +00:00
Mike Blumenkrantz
8f6fca89aa meson: delete dri3 build option
this existed for historical reasons, but realistically now it should
be possible to build mesa with dri3 always enabled. additionally,
this check was often used as a substitute for having drm functionality,
which is sort of similar but also not really a direct match

this simplifies a bunch of conditionals and prevents users from footgunnning
themselves into orbit

Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30952>
2024-09-04 14:48:17 +00:00
Vignesh Raman
1eb98bc457 ci: move mtl-fw.json to .gitlab-ci directory
Placing mtl-fw.json in src/intel/ci/mtl-fw.json works for the
mesa build, but it fails to fetch in drm-ci. Move it to the
.gitlab-ci directory so it is included in the artifacts used
for building the kernel/rootfs in drm-ci.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30947>
2024-09-03 19:25:49 +00:00
Daniel Stone
e833589acf ci/windows: Don't upload shader cache to artifacts
We don't need to keep this, and it only makes the upload slower.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30993>
2024-09-03 18:41:50 +00:00
Collabora's Gfx CI Team
b32b15f0df Uprev Piglit to 93b4bd2e0aaab1c22ae3e1a23f9e057a8f7451b2
a3826de3c2...93b4bd2e0a

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30863>
2024-09-02 08:30:51 +00:00
Eric Engestrom
2d10233f2f ci: add virgl & venus to the s390 build
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30954>
2024-08-31 18:55:15 +00:00
Daniel Stone
43d65e0ff0 ci: Make per-build dependencies optional
Sometimes not all of the jobs execute. For instance, Windows build jobs
will not trigger on AMD-only MRs. Use the 'optional' keyword to ignore
jobs which don't exist in our pipeline.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Fixes: 310e3bb026 ("ci: do not start build-only jobs until the critical build-for-tests jobs are done")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30926>
2024-08-29 17:50:16 +00:00
Eric Engestrom
310e3bb026 ci: do not start build-only jobs until the critical build-for-tests jobs are done
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30808>
2024-08-29 14:33:15 +00:00
Eric Engestrom
27fba5ccdf ci: merge build-x86_64 & build-misc
The split doesn't really make sense anymore, and with the next commit,
the build-misc jobs would get delayed until the build-x86_64 jobs are
done, which is undesirable.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30808>
2024-08-29 14:33:15 +00:00
Eric Engestrom
c9e847e51f ci: move build jobs that are used by tests to an earlier stage
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30808>
2024-08-29 14:33:15 +00:00
Jesse Natalie
9f88b6025b ci/windows: Bump image tags to rebuild deps with v143 build tools
Acked-by: David Heidelberg <david@ixit.cz>
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30877>
2024-08-29 01:46:01 +00:00
Jesse Natalie
1a93b1e217 ci/windows: Update VK-GL-CTS to d48899f85b486a70d090af59a1453763458611d9
This is needed to work around an MSVC compiler bug which causes test failures

Acked-by: David Heidelberg <david@ixit.cz>
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30877>
2024-08-29 01:46:01 +00:00
Jesse Natalie
5053ac02ed ci/windows: Use MSVC v143 build tools
MSVC finally gained support for __typeof__ and we want to start requiring
it. Let's bump the version of the build tools we use to v143.

Acked-by: David Heidelberg <david@ixit.cz>
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30877>
2024-08-29 01:46:01 +00:00
Eric Engestrom
74f7b370ea ci/build: untangle debian-s390x from debian-ppc64el
The only thing inherited and not overridden was the `BUILDTYPE: debug`.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30800>
2024-08-29 01:03:45 +00:00
Eric Engestrom
f01825569e ci/bare-metal/poe: change the default TEST_PHASE_TIMEOUT_MINUTES to CI_JOB_TIMEOUT - 5min
The 5min margin is configurable as `TEST_SETUP_AND_UPLOAD_MARGIN_MINUTES`.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30800>
2024-08-29 01:03:45 +00:00
Eric Engestrom
e5852bd41f ci/vkd3d: check for tests listed in fails but not actually failing
Meaning either it is not executed or passes.

Catches typos, which I would've merged without this check.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24739>
2024-08-29 00:18:54 +00:00
Eric Engestrom
5434aa79f5 ci: run only one vkd3d test at a time
Both radv and nvk seem to be having a lot of random failures when there
are several tests running in parallel.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24739>
2024-08-29 00:18:54 +00:00
Eric Engestrom
23e9cbc623 ci/vkd3d: use upstream test-runner.sh for process isolation
Once again, making sure the input (*-vkd3d-*.txt) have the same format
as deqp-runner so that users don't have to care which one they're using,
and the output is also in the same format so that tools automatically
handle everything.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24739>
2024-08-29 00:18:54 +00:00
Eric Engestrom
2019ccfc49 ci/vkd3d: make the job output less spammy, focus on what is useful to print
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24739>
2024-08-29 00:18:54 +00:00
Deborah Brouwer
18f15da94d ci/intel: add i915/MTL firmware to rootfs
Add Meteor Lake firmware directly to rootfs since it is not available
from debian package.

Signed-off-by: Deborah Brouwer <deborah.brouwer@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30770>
2024-08-28 04:31:10 +00:00
Deborah Brouwer
0441202d6b ci: add firmware files to rootfs
Currently only package versions of firmware files are available in the
rootfs.

This commit allows firmware files to be pulled directly from a specific
git hash of a remote repository.

Signed-off-by: Deborah Brouwer <deborah.brouwer@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30770>
2024-08-28 04:31:10 +00:00
Eric Engestrom
c0762e88f3 ci/build: fix ppc64le and s390x jobs rules
I think these were written with the idea of making it "(build rules) &&
(any relevant driver), but instead the driver rules are bypassing the
build rules because
1) it's not an AND, it's an OR; any line that matches applies, and
2) the driver rules are `when: on_success` when these need to be `when:
   manual` like the rest of the build jobs.

Let's stop trying to be special and simply behave like all the other
build jobs.

We can always try making complex rules later, but once we're on a base
that works.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30871>
2024-08-28 00:11:26 +00:00
Eric Engestrom
f6eeb3c6d1 ci/image-tags: re-generate all the images building deqp-runner
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30866>
2024-08-27 22:26:17 +00:00
Eric Engestrom
8a95129aee ci/deqp-runner: add infra to apply patches
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30866>
2024-08-27 22:26:17 +00:00
Eric Engestrom
83d9cfa58d ci/deqp-runner: build from git checkout even on linux
This allows things like patching deqp-runner, and unifies linux and
android.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30866>
2024-08-27 22:26:17 +00:00
Eric Engestrom
03e50318ff ci/deqp-runner: be less verbose in the loop printing the deqp builds info
The bash stuff printed in the middle makes it unnecessarily hard to read
the useful part of the output.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30866>
2024-08-27 22:26:17 +00:00
Eric Engestrom
3b0c527b56 ci/deqp: simplify command to list local deqp patches
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30866>
2024-08-27 22:26:17 +00:00
Eric Engestrom
ca7fde8761 ci/deqp-runner: restore CC after temporarily overriding it
Fixes: 6edfb09dda ("ci/deqp-runner: unset CC for arm32 cross-compilation")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30866>
2024-08-27 22:26:17 +00:00
Eric Engestrom
07202111a6 ci/rules: make every job exist as manual in fork pipelines
In forks, every job exists, but no job is automatically run.

Should help with the problem of users clicking on that tempting "play"
button GitLab offers and that has been causing a lot of issues with
wasted resources.

Downside is, users now have to use the `bin/ci/ci_run_n_monitor.sh`
script to be able to run a bunch of jobs at once.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25252>
2024-08-26 11:49:34 +00:00
Collabora's Gfx CI Team
e54440d15e Uprev Piglit to a3826de3c26a279599d15b018a9a3e75ca46f4f8
6533b66878...a3826de3c2

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30825>
2024-08-26 11:01:58 +00:00
Collabora's Gfx CI Team
e3113ffcfd Uprev Piglit to 6533b66878c8cb9361c554b7ff940518ab12c816
Fix 038b3c24d70762071d5ed2a219fe1a99a4ba46c6 image build and some expectation files updated.

Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30687>
2024-08-23 12:48:30 +00:00