mirrors/mesa
0
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2024-11-27 12:14:10 +08:00
Code Issues Packages Projects Releases Wiki Activity
26754f658e
mesa/.gitlab-ci.yml

319 lines
9.8 KiB
YAML
Raw Normal View History

gitlab-ci: add ubuntu container Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
2019-01-20 19:21:45 +08:00
variables:
ci: Update the ci-templates commit. There has been a big rename of variables in the upstream repo to make it clear what's being handed to ci-templates. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4099>
2020-03-07 05:23:20 +08:00
FDO_UPSTREAM_REPO: mesa/mesa
ci: Uprev ci-templates Uprev it to: d5aa3941aa03 ("freebsd: Move from 13.0 to 13.1") Reviewed-by: David Heidelberg <david.heidelberg@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18467>
2022-09-07 19:38:36 +08:00
MESA_TEMPLATES_COMMIT: &ci-templates-commit d5aa3941aa03c2f716595116354fb81eb8012acb
CI: reduce bandwidth for git pull Over the last 7 days, git pulls represented a total of 1.7 TB. On those 1.7 TB, we can see: - ~300 GB for the CI farm on hetzner - ~730 GB for the CI farm on packet.net - ~680 GB for the rest of the world We can not really change the rest of the world*, but we can certainly reduce the egress costs towards our CI farms. Right now, the gitlab runners are not doing a good job at caching the git trees for the various jobs we make, and we end up with a lot of cache-misses. A typical pipeline ends up with a good 2.8GB of git pull data. (a compressed archive of the mesa folder accounts for 280MB) In this patch, we implemented what was suggested in https://gitlab.com/gitlab-org/gitlab/-/issues/215591#note_334642576 - we host a brand new MinIO server on packet - jobs can upload files on 2 locations: * git-cache/<namespace>/<project>/<branch-name>.tar.gz * artifacts/<namespace>/<project>/<pipeline-id>/ - the authorization is handled by gitlab with short tokens valid only for the time of the job is running - whenever a job runs, the runner are configured to execute (eval) $CI_PRE_CLONE_SCRIPT - this variable is set globally to download the current cache from the MinIO packet server, unpack it and replace the possibly out of date cache found on the runner - then git fetch is run by the runner, and only the delta between the upstream tree and the local tree gets pulled. We can rebuild the git cache in a schedule job (once a day seems sufficient), and then we can stop the cache miss entirely. First results showed that instead of pulling 280MB of data in my fork, I got a pull of only 250KB. That should help us. * arguably, there are other farms in the rest of the world, so hopefully we can change those too. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5428>
2020-06-11 23:16:28 +08:00
CI_PRE_CLONE_SCRIPT: |-
set -o xtrace
CI: Don't stream wget directly into bash If our environment has pipefail set, bash could exit early before wget has completed, and we will die with a broken pipe. Work around this by first downloading from wget, and then executing from bash. Signed-off-by: Daniel Stone <daniels@collabora.com> Acked-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14036>
2021-12-03 15:07:47 +08:00
wget -q -O download-git-cache.sh ${CI_PROJECT_URL}/-/raw/${CI_COMMIT_SHA}/.gitlab-ci/download-git-cache.sh
bash download-git-cache.sh
rm download-git-cache.sh
CI: reduce bandwidth for git pull Over the last 7 days, git pulls represented a total of 1.7 TB. On those 1.7 TB, we can see: - ~300 GB for the CI farm on hetzner - ~730 GB for the CI farm on packet.net - ~680 GB for the rest of the world We can not really change the rest of the world*, but we can certainly reduce the egress costs towards our CI farms. Right now, the gitlab runners are not doing a good job at caching the git trees for the various jobs we make, and we end up with a lot of cache-misses. A typical pipeline ends up with a good 2.8GB of git pull data. (a compressed archive of the mesa folder accounts for 280MB) In this patch, we implemented what was suggested in https://gitlab.com/gitlab-org/gitlab/-/issues/215591#note_334642576 - we host a brand new MinIO server on packet - jobs can upload files on 2 locations: * git-cache/<namespace>/<project>/<branch-name>.tar.gz * artifacts/<namespace>/<project>/<pipeline-id>/ - the authorization is handled by gitlab with short tokens valid only for the time of the job is running - whenever a job runs, the runner are configured to execute (eval) $CI_PRE_CLONE_SCRIPT - this variable is set globally to download the current cache from the MinIO packet server, unpack it and replace the possibly out of date cache found on the runner - then git fetch is run by the runner, and only the delta between the upstream tree and the local tree gets pulled. We can rebuild the git cache in a schedule job (once a day seems sufficient), and then we can stop the cache miss entirely. First results showed that instead of pulling 280MB of data in my fork, I got a pull of only 250KB. That should help us. * arguably, there are other farms in the rest of the world, so hopefully we can change those too. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5428>
2020-06-11 23:16:28 +08:00
set +o xtrace
ci: Set CI_JOB_JWT_FILE to a fixed path outside /tmp Having CI_JOB_JWT_FILE pointing to path on /tmp makes it difficult to be managed in VM contexts (e.g. crosvm) because the /tmp mountpoint usually refers to a local filesystem rather than the host one. Additionally, there is another restriction for 'piglit-traces-test' job to have the file available on the root filesystem. To avoid amending all the jobs that might be affected, let's just set the variable to a fixed path '/minio_jwt'. Note we also need to do this in the 'variables:' section instead of 'before_script:' in order to be able to reference it in CI job variables, e.g. PIGLIT_REPLAY_EXTRA_ARGS Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15208>
2022-02-23 23:44:33 +08:00
CI_JOB_JWT_FILE: /minio_jwt
CI: convert to use the new S3 server instead of the legacy minio We don't need to login anymore, but we can't use plain minio commands now. `ci-fairy` got a helper as `s3cp` to keep an almost identical API. Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19076>
2022-10-14 03:25:38 +08:00
MINIO_HOST: s3.freedesktop.org
ci: Make PIPELINE_ARTIFACTS_BASE a common variable $minio/artifacts/$project/$pipeline/ is common between all our CI. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 23:24:48 +08:00
# per-pipeline artifact storage on MinIO
PIPELINE_ARTIFACTS_BASE: ${MINIO_HOST}/artifacts/${CI_PROJECT_PATH}/${CI_PIPELINE_ID}
ci: Add JOB_ARTIFACTS_BASE variable Used for both LAVA (uploading results to MinIO because we don't yet have non-ephemeral NFS storage) and Piglit (for the Tracie dashboard). Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 23:29:39 +08:00
# per-job artifact storage on MinIO
JOB_ARTIFACTS_BASE: ${PIPELINE_ARTIFACTS_BASE}/${CI_JOB_ID}
ci/piglit: Fix path to uploaded images Missed this bit when I was reworking environment variables. Signed-off-by: Daniel Stone <daniels@collabora.com> Reported-by: Emma Anholt <emma@anholt.net> Ref: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329#note_956187 Acked-by: Martin Peres <martin.peres@mupuf.org> Acked-by: Emma Anholt <emma@anholt.net> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>
2021-06-12 17:19:36 +08:00
# reference images stored for traces
PIGLIT_REPLAY_REFERENCE_IMAGES_BASE: "${MINIO_HOST}/mesa-tracie-results/$FDO_UPSTREAM_REPO"
freedreno/ci: Add a status variable for CI farm Make it a one-line change to disable freedreno CI jobs if the farm is offline. Signed-off-by: Rob Clark <robdclark@chromium.org> Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12602>
2021-08-28 01:06:16 +08:00
# Individual CI farm status, set to "offline" to disable jobs
# running on a particular CI farm (ie. for outages, etc):
Revert "ci: disable the freedreno farm." It's been moved back to the lab network, which hopefully has been stabilized. This reverts commit 13f36d66ad5ee581740ec13297a33312863e1c56. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18505>
2022-09-09 02:26:15 +08:00
FD_FARM: "online"
Revert "ci: disable Collabora's LAVA lab for maintance" This reverts commit 60d7e15a7e61c6d51c3fa8e26839793782a12fb4. Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20903>
2023-02-06 22:14:30 +08:00
COLLABORA_FARM: "online"
Revert "ci: Windows runner is experiencing DNS issues; disable Microsoft farm" Let's try this again, landed couple config changes on the runners. This reverts commit c52b7aa732e78d2ed93ef95a42262fae270c2027. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20424>
2022-12-23 21:33:40 +08:00
MICROSOFT_FARM: "online"
Revert "CI: Lima farm is offline" This reverts commit 0733aafa2271fee6a6724467ec7f2e50754d5a9d. Signed-off-by: Erico Nunes <nunes.erico@gmail.com> Acked-by: Vasily Khoruzhick <anarsoul@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21035>
2023-02-01 04:43:52 +08:00
LIMA_FARM: "online"
Revert "CI: Igalia farm is down" This reverts commit aa405b789ebf1797fdb3a1891fac49d0cca21c42. Igalia farm is up again. Switch died and it needed to be replaced. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19102>
2022-10-17 08:02:01 +08:00
IGALIA_FARM: "online"
ci: Add support for Jetson TK1. This is a farm of 5 (6, but one fails) TK1 boards for nouveau testing, hosted and maintained by me. Currently it runs GLES dEQP. I've been using ./.gitlab-ci/bin/ci_run_n_monitor.py --stress --target gk20a to test it and am pretty confident of the skips/flakes list. Last night it ran 318 jobs without fail, and prior to that there were two sets of runs in the 100-200 range where only the one failing runner failed any jobs. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18497>
2022-05-17 22:05:36 +08:00
ANHOLT_FARM: "online"
ci: introduce the VALVE_FARM variable This mirrors all the other *_FARM variables, and allows developers to quickly disable all the jobs that would otherwise run on Valve's CI infrastructure by setting it to the 'offline' value. To this end, .radv_rules gets split into .radv-collabora-rules and .radv-valve-rules, since the driver will be testable in two different test farms. Every radv job is then made to inherit from the right farm. Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19556>
2022-11-06 23:49:09 +08:00
VALVE_FARM: "online"
ci: Allow disabling the whole of austriancoder's farm Add a global-level variable that allows disabling all jobs that would have gone to my lab, to be used in case of outages. Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com> Reviewed-by: David Heidelberg <david.heidelberg@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18997>
2022-10-07 19:31:22 +08:00
AUSTRIANCODER_FARM: "online" # only etnaviv GPUs
CI: use wayland ci-templates repo to create the base image There shouldn't be a difference for users, but this way we do manage all of our containers from freedesktop.org note: compared to the provious Dockerfile, we need to manually add gcc, g++ and python*-wheel Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Reviewed-by: Michel Dänzer <michel.daenzer@amd.com> Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-04-02 15:24:00 +08:00
ci: Use ci-fairy minio login via token file For every CI job, put JWT content into a file and unset CI_JOB_JWT environment var ======= * virgl jobs: - Share JWT token file to crosvm instance - Keep using `export -p` due to high complexity in the scripts of these jobs. At least, the CI_JOB_JWT will not be leaked, since it is being unset at the `before_script` phase of each Mesa CI job. * iris jobs: Update lava_job_submitter to take token file as argument - generate-env with CI_JOB_JWT_TOKEN_FILE - create token file during baremetal init stage * baremetal jobs: Copy token file to bare-metal NFS Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14004>
2021-12-02 21:13:10 +08:00
default:
before_script:
- echo -e "\e[0Ksection_start:$(date +%s):unset_env_vars_section[collapsed=true]\r\e[0KUnsetting vulnerable environment variables"
- echo -n "${CI_JOB_JWT}" > "${CI_JOB_JWT_FILE}"
- unset CI_JOB_JWT
- echo -e "\e[0Ksection_end:$(date +%s):unset_env_vars_section\r\e[0K"
after_script:
- >
set +x
test -e "${CI_JOB_JWT_FILE}" &&
export CI_JOB_JWT="$(<${CI_JOB_JWT_FILE})" &&
rm "${CI_JOB_JWT_FILE}"
ci: Make the retry policy default for all jobs. We had to make sure to enable .ci-run-policy from every job to get the retry, but we can just put it in the default section. Acked-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17386>
2022-07-08 02:32:45 +08:00
# Retry build or test jobs up to twice when the gitlab-runner itself fails somehow.
retry:
max: 2
when:
- runner_system_failure
CI: use wayland ci-templates repo to create the base image There shouldn't be a difference for users, but this way we do manage all of our containers from freedesktop.org note: compared to the provious Dockerfile, we need to manually add gcc, g++ and python*-wheel Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Reviewed-by: Michel Dänzer <michel.daenzer@amd.com> Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-04-02 15:24:00 +08:00
include:
ci: Update the ci-templates commit. There has been a big rename of variables in the upstream repo to make it clear what's being handed to ci-templates. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4099>
2020-03-07 05:23:20 +08:00
- project: 'freedesktop/ci-templates'
ci: bump ci-fairy with session support (robust downloads) Signed-off-by: David Heidelberg <david.heidelberg@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20788>
2023-01-19 23:09:13 +08:00
ref: 16bc29078de5e0a067ff84a1a199a3760d3b3811
ci: Drop ci-templates-sha anchor As of GitLab 13.6, multiple files can be included for a single ref. Suggested-by: Daniel Stone <daniel@fooishbar.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7676>
2020-11-25 00:02:13 +08:00
file:
ci: Use ci-fairy docker image instead of local git_archive one The ci-fairy image contains pip and ci-fairy. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7676>
2020-11-19 01:23:29 +08:00
- '/templates/ci-fairy.yml'
ci: Go back to previous ci-templates commit for debian.yml The newer commit broke the arm64_test container build. Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3910 v2: * Also bump all image tags which had been bumped in the meantime Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7883>
2020-12-03 00:37:16 +08:00
- project: 'freedesktop/ci-templates'
ci: Fix MESA_TEMPLATES_COMMIT value The intention was to set it to the commit used for debian.yml, but I accidentally set it to the ci-fairy.yml commit instead. v2: * While we're at it, put two dashes between the base tag and the ci-templates commit hash, making them easier to tell apart. v3: * While we're at it, fix the year of some recently bumped tags. Fixes: 48f78dfd1ac1 "ci: Define global variable MESA_TEMPLATES_COMMIT for ci-templates commit" Reviewed-by: Eric Anholt <eric@anholt.net> # v2 Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8690>
2020-12-16 01:02:04 +08:00
ref: *ci-templates-commit
ci: Go back to previous ci-templates commit for debian.yml The newer commit broke the arm64_test container build. Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3910 v2: * Also bump all image tags which had been bumped in the meantime Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7883>
2020-12-03 00:37:16 +08:00
file:
ci/alpine: build Mesa on top of Alpine 3.16 Alpine is lightweight distribution with musl-libc. We currently testing with Debian and Fedora, which are both Glibc based distro, so this option should be able give us chance to catch more build-system bugs. Acked-by: Yonggang Luo <luoyonggang@gmail.com> Signed-off-by: David Heidelberg <david.heidelberg@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19588>
2022-10-14 10:19:31 +08:00
- '/templates/alpine.yml'
ci: Drop ci-templates-sha anchor As of GitLab 13.6, multiple files can be included for a single ref. Suggested-by: Daniel Stone <daniel@fooishbar.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7676>
2020-11-25 00:02:13 +08:00
- '/templates/debian.yml'
ci: Add Fedora 34 based x86 build docker image v2: * Do not install weak dependencies in Fedora docker image. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11412>
2021-05-12 22:42:37 +08:00
- '/templates/fedora.yml'
ci: Move common variables out into a separate file Moving common variables out allows for other projects like virglrenderer to be able to reuse Mesa CI's containers Signed-off-by: Rohan Garg <rohan.garg@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> [cristian: fixed conflicts while rebasing on latest main; updated tags] Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14413>
2021-11-10 05:38:33 +08:00
- local: '.gitlab-ci/image-tags.yml'
ci/lava: Move LAVA files to lava/ Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 18:10:10 +08:00
- local: '.gitlab-ci/lava/lava-gitlab-ci.yml'
ci: Move most stuff out of root .gitlab-ci.yml This file was getting a bit hard to navigate. Split container, build and test jobs to their own files. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15891>
2022-04-08 19:29:04 +08:00
- local: '.gitlab-ci/container/gitlab-ci.yml'
- local: '.gitlab-ci/build/gitlab-ci.yml'
- local: '.gitlab-ci/test/gitlab-ci.yml'
gitlab-ci: rules:changes to test on tested drivers changes For now tests only use these drivers: * llvmpipe * softpipe * freedreno * lima * etnaviv * panfrost So using rules:changes gitlab feature to run the tests when the changes made are potentially affecting these drivers. A few notes: * the following code: .piglit-test: extends: - .test-gl - .llvmpipe-rules makes gitlab replace .test-gl "rules:changes" values by the one from ".llvmpipe-rules". * rules:changes always matches for non-MR new branches so jobs will always be created (and they'll be run if their dependencies are run). For pushes to existing branches the files changed by the push are used to match the rules:changes path. * the same gitlab feature could be used for some build jobs Acked-by: Eric Engestrom <eric@engestrom.ch> Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2569> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2569>
2019-10-31 03:39:08 +08:00
- local: '.gitlab-ci/test-source-dep.yml'
ci: Move specific driver testing to separate files in separate dirs. The top-level gitlab-ci.yml is big and unwieldy when one wants to work on CI for a single driver. Move the drivers to separate include files for ease of finding all your driver's tests, and also to pave the way for work on a single driver's CI to not retest all other drivers. Reviewed-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Juan A. Suarez <jasuarez@igalia.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9139>
2021-02-19 03:12:56 +08:00
- local: 'src/amd/ci/gitlab-ci.yml'
Revert "ci: disable Broadcom CI" Our POE switch rebooted with factory settings. So devices weren't turning on/off because of failing SMNP auth. This reverts commit 87cdcf56455b78b0ca799adccb20c7d23d1b3a51. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11906>
2021-07-15 19:11:17 +08:00
- local: 'src/broadcom/ci/gitlab-ci.yml'
ci/bare-metal: add etnaviv Add deqp gles2 CI run for GC2000. Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com> Acked-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>
2020-06-12 19:23:44 +08:00
- local: 'src/etnaviv/ci/gitlab-ci.yml'
ci: Move specific driver testing to separate files in separate dirs. The top-level gitlab-ci.yml is big and unwieldy when one wants to work on CI for a single driver. Move the drivers to separate include files for ease of finding all your driver's tests, and also to pave the way for work on a single driver's CI to not retest all other drivers. Reviewed-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Juan A. Suarez <jasuarez@igalia.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9139>
2021-02-19 03:12:56 +08:00
- local: 'src/freedreno/ci/gitlab-ci.yml'
ci/crocus: Add support for manual CI runs on my G41. Uses a shared runner at my house so we have an easy way to minimally test crocus. Reviewed-by: Adam Jackson <ajax@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14238>
2021-11-24 03:25:41 +08:00
- local: 'src/gallium/drivers/crocus/ci/gitlab-ci.yml'
CI/windows: Move D3D12 test YML to D3D12 driver folder Reviewed-by: Enrico Galli <enrico.galli@intel.com> Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13902>
2021-11-21 22:38:20 +08:00
- local: 'src/gallium/drivers/d3d12/ci/gitlab-ci.yml'
ci/i915g: Introduce manual testing of i915g using anholt's runner. Now you can mash the button to see how badly i915g's broken these days. Reviewed-by: Adam Jackson <ajax@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10874>
2021-05-18 06:06:34 +08:00
- local: 'src/gallium/drivers/i915/ci/gitlab-ci.yml'
ci/lava: Move the per-driver gitlab-ci.yml to each driver. Follow-up to !9139, will cause less testing of other drivers when changing the CI configuration for a single driver. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9353>
2021-03-02 04:50:51 +08:00
- local: 'src/gallium/drivers/lima/ci/gitlab-ci.yml'
ci: Move specific driver testing to separate files in separate dirs. The top-level gitlab-ci.yml is big and unwieldy when one wants to work on CI for a single driver. Move the drivers to separate include files for ease of finding all your driver's tests, and also to pave the way for work on a single driver's CI to not retest all other drivers. Reviewed-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Juan A. Suarez <jasuarez@igalia.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9139>
2021-02-19 03:12:56 +08:00
- local: 'src/gallium/drivers/llvmpipe/ci/gitlab-ci.yml'
ci/nouveau: Add a manual run for the Jetson Nano (GM20B). The test suite is full of flakes around transform feedback, atomics, and tess. But, I hope it can be useful for regression testing core Mesa reworks. This required updating the kernel to 5.16.12 to get a more stable boot process. That kernel rebuild caused an update of the container with piglit which that was missed in a previous MR, so we got new xfails in x86 swrast. Acked-by: Ilia Mirkin <imirkin@alum.mit.edu> (nouveau) Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15201>
2021-12-03 08:14:30 +08:00
- local: 'src/gallium/drivers/nouveau/ci/gitlab-ci.yml'
ci: Move specific driver testing to separate files in separate dirs. The top-level gitlab-ci.yml is big and unwieldy when one wants to work on CI for a single driver. Move the drivers to separate include files for ease of finding all your driver's tests, and also to pave the way for work on a single driver's CI to not retest all other drivers. Reviewed-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Juan A. Suarez <jasuarez@igalia.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9139>
2021-02-19 03:12:56 +08:00
- local: 'src/gallium/drivers/softpipe/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/virgl/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/zink/ci/gitlab-ci.yml'
ci/lavapipe: split out lavapipe ci into lavapipe dir there's no need to run llvmpipe jobs if only lavapipe has changed Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9695>
2021-03-19 03:58:04 +08:00
- local: 'src/gallium/frontends/lavapipe/ci/gitlab-ci.yml'
anv/ci: Test with deqp-vk on Tiger Lake Run half of the CTS in 10 Volteer Chromebook devices. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14121>
2021-12-06 17:01:52 +08:00
- local: 'src/intel/ci/gitlab-ci.yml'
CI/windows: Move SPIRV-to-DXIL test YML to microsoft folder Reviewed-by: Enrico Galli <enrico.galli@intel.com> Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13902>
2021-11-21 22:41:37 +08:00
- local: 'src/microsoft/ci/gitlab-ci.yml'
panfrost/ci: Move CI files to src/panfrost In preparation for testing panvk. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Acked-by: Alyssa Rosenzweig <alyssa@collabora.com> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13016>
2021-07-05 15:25:57 +08:00
- local: 'src/panfrost/ci/gitlab-ci.yml'
ci: Add vulkan dEQP tests for venus Co-authored-by: Corentin Noël <corentin.noel@collabora.com> Signed-off-by: Omar Akkila <omar.akkila@collabora.com> Acked-by: David Heidelberg <david.heidelberg@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15692>
2021-10-25 15:29:07 +08:00
- local: 'src/virtio/ci/gitlab-ci.yml'
gitlab-ci: Run dEQP on devices with Panfrost Include Panfrost's gitlab.ci.yml file from Mesa's main .gitlab-ci.yml so we test on devices with Panfrost. This uses LAVA to schedule jobs in the devices and will be the base for testing Etnaviv, Lima, etc. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Eric Anholt <eric@anholt.net>
2019-09-18 22:03:36 +08:00
gitlab-ci: add ubuntu container Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
2019-01-20 19:21:45 +08:00
stages:
ci: Move sanity stage to the beginning of the pipeline This is possible now that it uses the external ci-fairy docker image. This allows dropping the "check mr" job from needs: of other jobs, the container stage jobs will only become available once the sanity stage has passed. This also allows simplifying the "check mr" job rules and script, since the job only needs to exist in pre-merge pipelines for MRs anymore. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7676>
2020-11-19 01:32:05 +08:00
- sanity
ci: Move test-docs job to deploy stage We put it in the first container stage to prevent it from waiting for jobs in previous stages, but empty needs: works for that as well. Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6209>
2020-08-06 23:10:08 +08:00
- container
CI: reduce bandwidth for git pull Over the last 7 days, git pulls represented a total of 1.7 TB. On those 1.7 TB, we can see: - ~300 GB for the CI farm on hetzner - ~730 GB for the CI farm on packet.net - ~680 GB for the rest of the world We can not really change the rest of the world*, but we can certainly reduce the egress costs towards our CI farms. Right now, the gitlab runners are not doing a good job at caching the git trees for the various jobs we make, and we end up with a lot of cache-misses. A typical pipeline ends up with a good 2.8GB of git pull data. (a compressed archive of the mesa folder accounts for 280MB) In this patch, we implemented what was suggested in https://gitlab.com/gitlab-org/gitlab/-/issues/215591#note_334642576 - we host a brand new MinIO server on packet - jobs can upload files on 2 locations: * git-cache/<namespace>/<project>/<branch-name>.tar.gz * artifacts/<namespace>/<project>/<pipeline-id>/ - the authorization is handled by gitlab with short tokens valid only for the time of the job is running - whenever a job runs, the runner are configured to execute (eval) $CI_PRE_CLONE_SCRIPT - this variable is set globally to download the current cache from the MinIO packet server, unpack it and replace the possibly out of date cache found on the runner - then git fetch is run by the runner, and only the delta between the upstream tree and the local tree gets pulled. We can rebuild the git cache in a schedule job (once a day seems sufficient), and then we can stop the cache miss entirely. First results showed that instead of pulling 280MB of data in my fork, I got a pull of only 250KB. That should help us. * arguably, there are other farms in the rest of the world, so hopefully we can change those too. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5428>
2020-06-11 23:16:28 +08:00
- git-archive
ci: Rename Debian based build jobs from meson-* to debian-* meson has been the only build system in tree for some time, so the meson- prefix was a bit meaningless. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11412>
2021-06-16 00:23:33 +08:00
- build-x86_64
CI: Collapse SCons & meson-misc stages into one This is now called 'build-misc'. Signed-off-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Acked-by: Eric Anholt <eric@anholt.net> Acked-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7873>
2020-12-02 21:50:38 +08:00
- build-misc
ci: rusticl with llvmpipe Signed-off-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Emma Anholt <emma@anholt.net> Reviewed-by: Karol Herbst <kherbst@redhat.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15439>
2022-08-12 00:11:07 +08:00
- lint
CI: Collapse radv & radeonsi stages into one This is now called 'amd'. Signed-off-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Acked-by: Eric Anholt <eric@anholt.net> Acked-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7873>
2020-12-02 21:50:38 +08:00
- amd
gitlab-ci: add intel APL and GLK devices with manual triggers This add support for the Intel Apollo Lake and Gemini Lake families, however the job will be disabled by default unless the developer manually hit play for the iris-apl-traces and iris-glk-traces jobs in GitLab CI. These devices are still under experimental level support in the Lava lab and are not guaranteed to work reliably yet. Once they become reliable and more resilient we will enable them by default in MesaCI. Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com> Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8162>
2020-12-18 11:40:02 +08:00
- intel
ci/nouveau: Add a manual run for the Jetson Nano (GM20B). The test suite is full of flakes around transform feedback, atomics, and tess. But, I hope it can be useful for regression testing core Mesa reworks. This required updating the kernel to 5.16.12 to get a more stable boot process. That kernel rebuild caused an update of the container with piglit which that was missed in a previous MR, so we got new xfails in x86 swrast. Acked-by: Ilia Mirkin <imirkin@alum.mit.edu> (nouveau) Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15201>
2021-12-03 08:14:30 +08:00
- nouveau
CI: Collapse lima & panfrost stages into one This is now called 'arm'; even though they're two separate drivers, it does help cut down on some of the more egregious horizontal scrolling. Signed-off-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Acked-by: Eric Anholt <eric@anholt.net> Acked-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7873>
2020-12-02 21:50:38 +08:00
- arm
ci/vc4: rename stage to Broadcom It fits better with what other drivers has done. Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com> Acked-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8702>
2021-01-25 19:33:52 +08:00
- broadcom
CI: Collapse lima & panfrost stages into one This is now called 'arm'; even though they're two separate drivers, it does help cut down on some of the more egregious horizontal scrolling. Signed-off-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Acked-by: Eric Anholt <eric@anholt.net> Acked-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7873>
2020-12-02 21:50:38 +08:00
- freedreno
ci/bare-metal: add etnaviv Add deqp gles2 CI run for GC2000. Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com> Acked-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>
2020-06-12 19:23:44 +08:00
- etnaviv
CI: Reorder non-hardware stages last Signed-off-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Acked-by: Eric Anholt <eric@anholt.net> Acked-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7873>
2020-12-02 21:57:47 +08:00
- software-renderer
CI: Collapse virgl & d3d12 stages into one This is now called 'layered-backends'. Signed-off-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Acked-by: Eric Anholt <eric@anholt.net> Acked-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7873>
2020-12-02 21:50:38 +08:00
- layered-backends
ci: Move deploy stage to the end of the pipeline This should make more important jobs visible without scrolling on pipeline pages. The deploy stage jobs only depend on the sanity job or none at all, so this has no impact on when the former can run. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7926>
2020-12-05 17:15:38 +08:00
- deploy
gitlab-ci: add ubuntu container Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
2019-01-20 19:21:45 +08:00
gitlab-ci: do not run full CI on scheduled pipelines Currently, scheduled pipelines are only used to rebuild the git-cache archive daily. There is no point in rebuilding eveything, so ensure that any normal jobs are removed from the scheduled pipelines. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Acked-by: Daniel Stone <daniels@collabora.com> Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5804>
2020-07-07 21:02:35 +08:00
ci: Add "is master branch of main project" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 18:20:39 +08:00
# YAML anchors for rule conditions
# --------------------------------
.rules-anchors:
rules:
ci: Use $CI_COMMIT_BRANCH This was recently added to indicate pipelines for branches. v2: * Modify .gitlab-ci/test-source-dep.yml as well. Acked-by: Emma Anholt <emma@anholt.net> # v1 Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14225>
2021-12-16 19:03:18 +08:00
# Pipeline for forked project branch
- if: &is-forked-branch '$CI_COMMIT_BRANCH && $CI_PROJECT_NAMESPACE != "mesa"'
ci: Add "is forked branch" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 23:52:24 +08:00
when: manual
gitlab-ci: rule anchor for experimental devices as manual in MRs We want to give developers the option to run their jobs on devices that are still being stabilized in the CI infrastructure. These jobs should be optional and not prevent merging from happening. The is-forked-branch-or-pre-merge anchor was not being used anywhere, so it was changed to is-forked-branch-or-pre-merge-not-for-marge to create this new rule. Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com> Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10405>
2021-04-22 19:17:40 +08:00
# Forked project branch / pre-merge pipeline not for Marge bot
ci: Use $CI_PIPELINE_SOURCE Instead of $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME. Shorter and clearer. Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14225>
2021-12-16 19:09:10 +08:00
- if: &is-forked-branch-or-pre-merge-not-for-marge '$CI_PROJECT_NAMESPACE != "mesa" || ($GITLAB_USER_LOGIN != "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event")'
ci: Add "is forked branch or pre-merge pipeline" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 23:30:49 +08:00
when: manual
.gitlab-ci.yml: Use main branch for gitlab ci Reworks: * Fix mesa/mesamaster typo to mesa/mesa main (anholt) * Use $CI_DEFAULT_BRANCH (eric_engestrom) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2021-04-27 04:16:55 +08:00
# Pipeline runs for the main branch of the upstream Mesa project
ci: Use $CI_COMMIT_BRANCH This was recently added to indicate pipelines for branches. v2: * Modify .gitlab-ci/test-source-dep.yml as well. Acked-by: Emma Anholt <emma@anholt.net> # v1 Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14225>
2021-12-16 19:03:18 +08:00
- if: &is-mesa-main '$CI_PROJECT_NAMESPACE == "mesa" && $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH && $CI_COMMIT_BRANCH'
ci: Add "is master branch of main project" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 18:20:39 +08:00
when: always
ci: Add "is post-merge pipeline" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 23:47:18 +08:00
# Post-merge pipeline
ci: Use $CI_COMMIT_BRANCH This was recently added to indicate pipelines for branches. v2: * Modify .gitlab-ci/test-source-dep.yml as well. Acked-by: Emma Anholt <emma@anholt.net> # v1 Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14225>
2021-12-16 19:03:18 +08:00
- if: &is-post-merge '$CI_PROJECT_NAMESPACE == "mesa" && $CI_COMMIT_BRANCH'
ci: Add "is post-merge pipeline" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 23:47:18 +08:00
when: on_success
ci: Add "is post-merge pipeline, not for Marge Bot" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 18:36:11 +08:00
# Post-merge pipeline, not for Marge Bot
ci: Use $CI_COMMIT_BRANCH This was recently added to indicate pipelines for branches. v2: * Modify .gitlab-ci/test-source-dep.yml as well. Acked-by: Emma Anholt <emma@anholt.net> # v1 Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14225>
2021-12-16 19:03:18 +08:00
- if: &is-post-merge-not-for-marge '$CI_PROJECT_NAMESPACE == "mesa" && $GITLAB_USER_LOGIN != "marge-bot" && $CI_COMMIT_BRANCH'
ci: Add "is post-merge pipeline, not for Marge Bot" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 18:36:11 +08:00
when: on_success
ci: Add "is pre-merge pipeline" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 23:58:32 +08:00
# Pre-merge pipeline
ci: Use $CI_PIPELINE_SOURCE Instead of $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME. Shorter and clearer. Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14225>
2021-12-16 19:09:10 +08:00
- if: &is-pre-merge '$CI_PIPELINE_SOURCE == "merge_request_event"'
ci: Add "is pre-merge pipeline" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 23:58:32 +08:00
when: on_success
ci: Add "is pre-merge pipeline for Marge Bot" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 18:31:08 +08:00
# Pre-merge pipeline for Marge Bot
ci: Use $CI_PIPELINE_SOURCE Instead of $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME. Shorter and clearer. Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14225>
2021-12-16 19:09:10 +08:00
- if: &is-pre-merge-for-marge '$GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event"'
ci: Add "is pre-merge pipeline for Marge Bot" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 18:31:08 +08:00
when: on_success
ci: Add "is master branch of main project" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 18:20:39 +08:00
ci: test docs for non-master builds This ensures that we test on CI before merge-requests gets merged. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
2020-06-22 17:10:29 +08:00
.docs-base:
docs/ci: allow overriding sphinx builder Reviewed-by: Eric Engestrom <eric@igalia.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19574>
2022-11-04 23:13:20 +08:00
variables:
BUILDER: html
ci: Use ci-fairy docker image instead of local git_archive one The ci-fairy image contains pip and ci-fairy. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7676>
2020-11-19 01:23:29 +08:00
extends:
- .fdo.ci-fairy
ci: Rename .ci-run-policy rules to .build-rules. ... and explain what they're doing, compared to the test rules in test-source-dep.yml. Unfortunately, we can't really pull them into test-source-dep.yml with other source deps, because of various '&'-'*' references. Acked-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17386>
2022-07-08 02:36:05 +08:00
- .build-rules
ci/docs: Always expose the built docs. When testing my docs changes outside of an MR, there was no way to look at the result. Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19733>
2022-11-15 05:22:53 +08:00
artifacts:
expose_as: 'Documentation preview'
paths:
- public/
gitlab-ci: build and deploy docs Dunno if alpine is a good idea. It's what the gitlab docs use for most of their examples, so that's what I've gone with... Can probably be changed to something else if wanted. Reviewed-by: Eric Engestrom <eric@engestrom.ch> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4630>
2019-05-27 23:12:10 +08:00
script:
docs: Begin documenting ISL This commit mostly just adds the framework required to scrape documentation out of the ISL sources and headers. The method chosen here is a combination of doxygen and breathe (a sphinx extension for doxygen integration). I'll freely admit that doxygen is pretty terrible but it seems like the best option we have available to us today. Acked-by: Emma Anholt <emma@anholt.net> Acked-by: Daniel Stone <daniels@collabora.com> Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11366>
2021-06-15 05:55:21 +08:00
- apk --no-cache add graphviz doxygen
ci/docs: Stick to specific version of python packages Signed-off-by: Yonggang Luo <luoyonggang@gmail.com> Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Acked-by: Eric Engestrom <eric@igalia.com> Cc: mesa-stable Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18833>
2022-09-27 10:23:42 +08:00
- pip3 install sphinx===5.1.1 breathe===4.34.0 mako===1.2.3 sphinx_rtd_theme===1.0.0
docs: Begin documenting ISL This commit mostly just adds the framework required to scrape documentation out of the ISL sources and headers. The method chosen here is a combination of doxygen and breathe (a sphinx extension for doxygen integration). I'll freely admit that doxygen is pretty terrible but it seems like the best option we have available to us today. Acked-by: Emma Anholt <emma@anholt.net> Acked-by: Daniel Stone <daniels@collabora.com> Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11366>
2021-06-15 05:55:21 +08:00
- docs/doxygen-wrapper.py --out-dir=docs/doxygen_xml
docs/ci: allow overriding sphinx builder Reviewed-by: Eric Engestrom <eric@igalia.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19574>
2022-11-04 23:13:20 +08:00
- sphinx-build -W -b $BUILDER docs public
ci: test docs for non-master builds This ensures that we test on CI before merge-requests gets merged. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
2020-06-22 17:10:29 +08:00
pages:
extends: .docs-base
ci: move test-docs to container stage While we're at it, rename it to reflect that we're now also testing docs here. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
2020-06-22 17:10:40 +08:00
stage: deploy
ci: Add empty needs: to pages job So it doesn't need to wait for jobs in earlier stages. Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6209>
2020-08-06 23:12:11 +08:00
needs: []
gitlab-ci: Extend .ci-run-policy template for docs jobs Requires using rules: in the pages job as well, so it doesn't inherit the rules from the template. v2: * Add comment explaining that cases not covered by explicit rules default to "when: never". Reviewed-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
2020-06-22 17:21:06 +08:00
rules:
ci: Use "!references" to manage scheduled pipeline rules. Because !references merging happens after yaml parsing, this lets us remove a duplicated definition between .test-source-dep.yml and .gitlab-ci.yml. Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Acked-by: Eric Engestrom <eric@engestrom.ch> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17287>
2022-06-29 01:43:34 +08:00
- !reference [.no_scheduled_pipelines-rules, rules]
.gitlab-ci.yml: Use main branch for gitlab ci Reworks: * Fix mesa/mesamaster typo to mesa/mesa main (anholt) * Use $CI_DEFAULT_BRANCH (eric_engestrom) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2021-04-27 04:16:55 +08:00
- if: *is-mesa-main
gitlab-ci: Extend .ci-run-policy template for docs jobs Requires using rules: in the pages job as well, so it doesn't inherit the rules from the template. v2: * Add comment explaining that cases not covered by explicit rules default to "when: never". Reviewed-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
2020-06-22 17:21:06 +08:00
changes: &docs-or-ci
- docs/**/*
- .gitlab-ci.yml
ci: Use "when: always" for pages job "when: on_success" meant that that the job wouldn't run automatically until all jobs of all earlier stages passed, and would be skipped if any of them failed. But we need to always run this job if any documentation files were modified. Fixes: 8e2cb8ef276b "gitlab-ci: Extend .ci-run-policy template for docs jobs" Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5711>
2020-07-01 23:41:06 +08:00
when: always
gitlab-ci: Extend .ci-run-policy template for docs jobs Requires using rules: in the pages job as well, so it doesn't inherit the rules from the template. v2: * Add comment explaining that cases not covered by explicit rules default to "when: never". Reviewed-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
2020-06-22 17:21:06 +08:00
# Other cases default to never
gitlab-ci: add ubuntu container Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
2019-01-20 19:21:45 +08:00
ci: test docs for non-master builds This ensures that we test on CI before merge-requests gets merged. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
2020-06-22 17:10:29 +08:00
test-docs:
extends: .docs-base
ci: Do not mark container / pages jobs as interruptible If another MR was merged while these were still running for the main project, the result could be no updated images in the main project registry (forcing a rebuild of the new images in all forked projects) or an outdated Mesa website. Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6011>
2020-07-21 22:13:37 +08:00
# Cancel job if a newer commit is pushed to the same branch
interruptible: true
ci: Move test-docs job to deploy stage We put it in the first container stage to prevent it from waiting for jobs in previous stages, but empty needs: works for that as well. Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6209>
2020-08-06 23:10:08 +08:00
stage: deploy
ci: Run sanity job only in pre-merge pipelines It's more of a nuisance than useful for forked branches. This means the test-docs job can no longer have a direct dependency on sanity for forked branches, so split it up into two jobs: one for pre-merge pipelines, one for forked branches. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7926>
2020-12-04 01:25:54 +08:00
needs: []
rules:
ci: Use "!references" to manage scheduled pipeline rules. Because !references merging happens after yaml parsing, this lets us remove a duplicated definition between .test-source-dep.yml and .gitlab-ci.yml. Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Acked-by: Eric Engestrom <eric@engestrom.ch> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17287>
2022-06-29 01:43:34 +08:00
- !reference [.no_scheduled_pipelines-rules, rules]
ci: Run sanity job only in pre-merge pipelines It's more of a nuisance than useful for forked branches. This means the test-docs job can no longer have a direct dependency on sanity for forked branches, so split it up into two jobs: one for pre-merge pipelines, one for forked branches. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7926>
2020-12-04 01:25:54 +08:00
- if: *is-forked-branch
changes: *docs-or-ci
when: manual
# Other cases default to never
test-docs-mr:
extends:
- test-docs
ci: Make test-docs job depend on sanity job Without this, the test-docs job could end up waiting for manual action after the sanity job failed, which prevented the pipeline as a whole from having failed status. (This means the test-docs job will no longer exist in one corner case where it did before, when pushing directly to a non-master branch of the main repository. That should be fine, since the docs are only deployed from master.) Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7676>
2020-11-20 18:05:56 +08:00
needs:
- sanity
gitlab-ci: Use rules: instead of except:/only: for test-docs job Only run the job automatically for Marge Bot, otherwise let it be triggered manually. v2: * Never run this job for the main project, since it's only needed in pre-merge pipelines. * Add comment explaining that cases not covered by explicit rules default to "when: never". Reviewed-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
2020-06-22 17:13:05 +08:00
rules:
ci: Run sanity job only in pre-merge pipelines It's more of a nuisance than useful for forked branches. This means the test-docs job can no longer have a direct dependency on sanity for forked branches, so split it up into two jobs: one for pre-merge pipelines, one for forked branches. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7926>
2020-12-04 01:25:54 +08:00
- if: *is-pre-merge
ci: Do not create manual test-docs job in post-merge pipelines Only in pre-merge pipelines for MRs, or in pipelines for forked project branches. Having the manual job in post-merge pipelines prevented the pages job from running automatically as well, which could prevent the public website from getting updated. Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6534>
2020-09-01 17:44:54 +08:00
changes: *docs-or-ci
CI: always expose docs artifacts This makes it easier to preview docs changes in merge-requests. Also make sure we build the docs right away, rather than waiting for when marge merges. This allows us to see the artifacts right away. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8398>
2021-01-09 05:08:56 +08:00
when: on_success
gitlab-ci: Use rules: instead of except:/only: for test-docs job Only run the job automatically for Marge Bot, otherwise let it be triggered manually. v2: * Never run this job for the main project, since it's only needed in pre-merge pipelines. * Add comment explaining that cases not covered by explicit rules default to "when: never". Reviewed-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
2020-06-22 17:13:05 +08:00
# Other cases default to never
ci: test docs for non-master builds This ensures that we test on CI before merge-requests gets merged. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5469>
2020-06-22 17:10:29 +08:00
docs: add linkcheck job This should make it easier to keep links in our docs up-to-date. But, because links can die behind our backs, we can't really enable this all over the place, or we'll risk blocking merge-requests due to unrelated changes. So let's just make this a periodic job on the main branch instead. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Reviewed-by: Eric Engestrom <eric@igalia.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19574>
2022-11-07 22:24:27 +08:00
lincheck-docs:
extends: .docs-base
# Cancel job if a newer commit is pushed to the same branch
interruptible: true
stage: deploy
needs: []
rules:
- !reference [.scheduled_pipeline-rules, rules]
allow_failure: true
variables:
BUILDER: linkcheck
ci: Rename .ci-run-policy rules to .build-rules. ... and explain what they're doing, compared to the test rules in test-source-dep.yml. Unfortunately, we can't really pull them into test-source-dep.yml with other source deps, because of various '&'-'*' references. Acked-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17386>
2022-07-08 02:36:05 +08:00
# When to automatically run the CI for build jobs
.build-rules:
gitlab-ci: Overhaul job run policy Use new rules: instead of only: For container stage jobs: * In the main Mesa project, run them by default. * In merge requests, run them by default if any files affecting pipeline results are changed. * In all other cases (in particular branches in personal projects), don't run them by default but allow triggering them manually. build & test stage jobs are left at the default (when: on_success), so they will run automatically once all their dependencies are satisified. (Using the same rules as above would require these jobs to be manually triggered as well, which is only possible once all dependency jobs have passed) Please be considerate of CI runner resources and cancel unneeded jobs on personal branches with no corresponding merge requests (this can be done before the jobs start running). In summary: No more special branch names. Unnecessary job runs are avoided by default, but jobs which don't run by default can be triggered manually. v2: * Split out LAVA changes to separate commit * Clarify commit log a little, in particular WRT build/test stage jobs Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> # v1 Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> # v1 Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> # v1 Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-09-26 15:27:27 +08:00
rules:
gitlab-ci: Don't require triggering build/test jobs manually Let them run automatically once all their dependencies have passed. Reviewed-by: Adam Jackson <ajax@redhat.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
2020-04-03 18:50:11 +08:00
# If any files affecting the pipeline are changed, build/test jobs run
# automatically once all dependency jobs have passed
- changes: &all_paths
gitlab-ci: Stop using manual jobs for merge requests They were causing trouble with Marge Bot: The project settings require that the pipeline succeeds before a merge request (MR) can be merged, otherwise Marge doesn't wait for the pipeline to succeed before merging an MR assigned to her. But Marge can't start manual jobs, so she would always time out waiting for pipelines with manual jobs. To avoid this, use these rules: * Run the pipeline by default for MRs and main project branches changing any files affecting it. * For other MRs, run a single dummy job which always succeeds. * Don't run any jobs for main project branch changes (e.g. from an MR having been merged) not affecting the pipeline. * Allow jobs to be started manually on branches of forked projects, as before. Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Acked-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Eric Engestrom <eric@engestrom.ch> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3361> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3361>
2020-01-13 16:45:57 +08:00
- VERSION
gitlab-ci: exclude scripts that don't affect the build All the other files in bin/ are not used by any build system and as such cannot affect the build. I've been working on maintainer tools lately and it's frustrating to have the CI wait for 45 minutes to rebuild everything and not even read/run the files in the MR when it could've just been merged and moved on to the next MR 45 minutes ago. Signed-off-by: Eric Engestrom <eric@engestrom.ch> Acked-by: Tapani Pälli <tapani.palli@intel.com> Reviewed-by: Dylan Baker <dylan@pnwbakers.com> Reviewed-by: Jose Fonseca <jfonseca@vmware.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5046>
2020-05-15 04:51:38 +08:00
- bin/git_sha1_gen.py
- bin/install_megadrivers.py
- bin/meson_get_version.py
- bin/symbols-check.py
gitlab-ci: Stop using manual jobs for merge requests They were causing trouble with Marge Bot: The project settings require that the pipeline succeeds before a merge request (MR) can be merged, otherwise Marge doesn't wait for the pipeline to succeed before merging an MR assigned to her. But Marge can't start manual jobs, so she would always time out waiting for pipelines with manual jobs. To avoid this, use these rules: * Run the pipeline by default for MRs and main project branches changing any files affecting it. * For other MRs, run a single dummy job which always succeeds. * Don't run any jobs for main project branch changes (e.g. from an MR having been merged) not affecting the pipeline. * Allow jobs to be started manually on branches of forked projects, as before. Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Acked-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Eric Engestrom <eric@engestrom.ch> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3361> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3361>
2020-01-13 16:45:57 +08:00
# GitLab CI
- .gitlab-ci.yml
- .gitlab-ci/**/*
# Meson
- meson*
- build-support/**/*
- subprojects/**/*
# Source code
- include/**/*
- src/**/*
when: on_success
ci: Remove the trailing "when: never"s from rules. This avoids the risks of concatenating our rulesets missing out on some file list because of a "never" in the middle. Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Acked-by: Eric Engestrom <eric@engestrom.ch> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17287>
2022-06-29 06:15:54 +08:00
# Otherwise, build/test jobs won't run because no rule matched.
gitlab-ci: always run the containers build If the first time a fork was created, the job creating the containers was manually cancelled, this would have left the fork unable to use the CI (until the next automatic regeneration of the container). Avoid this by always running the container-generation job, even though 99% of the time it will spin up, see that the container exists and shut down. Signed-off-by: Eric Engestrom <eric.engestrom@intel.com> Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
2019-02-22 23:52:08 +08:00
gitlab-ci: Stop using manual jobs for merge requests They were causing trouble with Marge Bot: The project settings require that the pipeline succeeds before a merge request (MR) can be merged, otherwise Marge doesn't wait for the pipeline to succeed before merging an MR assigned to her. But Marge can't start manual jobs, so she would always time out waiting for pipelines with manual jobs. To avoid this, use these rules: * Run the pipeline by default for MRs and main project branches changing any files affecting it. * For other MRs, run a single dummy job which always succeeds. * Don't run any jobs for main project branch changes (e.g. from an MR having been merged) not affecting the pipeline. * Allow jobs to be started manually on branches of forked projects, as before. Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Acked-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Eric Engestrom <eric@engestrom.ch> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3361> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3361>
2020-01-13 16:45:57 +08:00
gitlab-ci: Use multiple inheritance instead of YAML references Support for multiple inheritance was added to GitLab recently. Reviewed-by: Eric Engestrom <eric.engestrom@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net>
2019-09-06 23:35:52 +08:00
.ci-deqp-artifacts:
gitlab-ci: Run the GLES2 CTS on llvmpipe. This is the start of doing CTS tests on merges to Mesa master. We use the surfaceless platform so that we don't need to bother bringing up weston or X11. The surface size is kept low to reduce runtime, but this comes at the cost of many rendering tests skipping due to too-small render targets (as we see the impact of Mesa on the shared runner pool, we can reevaluate this and what set of CTS tests we want to run). We split the job up across 4 runners (each at 4 llvmpipe threads), so that the job can load-balance across our shared runners and finish sooner (since dEQP is very single-thread-performance bound). Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-06-29 07:35:32 +08:00
artifacts:
gitlab-ci: Always name artifacts archive after the job producing it This will help determine which artifacts generate how much traffic. v2: * Add "mesa_" prefix to make it obvious which project the artifacts are from. Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Reviewed-by: Eric Anholt <eric@anholt.net> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4085> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4085>
2020-03-06 19:35:17 +08:00
name: "mesa_${CI_JOB_NAME}"
gitlab-ci: Run the GLES2 CTS on llvmpipe. This is the start of doing CTS tests on merges to Mesa master. We use the surfaceless platform so that we don't need to bother bringing up weston or X11. The surface size is kept low to reduce runtime, but this comes at the cost of many rendering tests skipping due to too-small render targets (as we see the impact of Mesa on the shared runner pool, we can reevaluate this and what set of CTS tests we want to run). We split the job up across 4 runners (each at 4 llvmpipe threads), so that the job can load-balance across our shared runners and finish sooner (since dEQP is very single-thread-performance bound). Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-06-29 07:35:32 +08:00
when: always
untracked: false
paths:
# Watch out! Artifacts are relative to the build dir.
# https://gitlab.com/gitlab-org/gitlab-ce/commit/8788fb925706cad594adf6917a6c5f6587dd1521
- artifacts
ci: Ensure that jobs inherting the ci-deqp jobs artifact meson logs Since gitlab-ci cannot merge artifact lists, and the last entry a job extends from overwrites the artifact list, we should ensure that we artifact meson logs via the .ci-deqp-artifacts job. Signed-off-by: Rohan Garg <rohan.garg@collabora.com> Acked-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8937>
2021-02-10 00:43:35 +08:00
- _build/meson-logs/*.txt
ci: Run meson tests in strace if it's available and can be used Keep the strace logs in job artifacts for tests which timed out. This can be useful for figuring out why a timeout occurred. strace cannot be used in jobs where ASAN is enabled, because ASAN's leak checker also uses ptrace(), which isn't possible within strace. Suggested-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Adam Jackson <ajax@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9472>
2021-03-04 19:58:56 +08:00
- _build/meson-logs/strace
gitlab-ci: always run the containers build If the first time a fork was created, the job creating the containers was manually cancelled, this would have left the fork unable to use the CI (until the next automatic regeneration of the container). Avoid this by always running the container-generation job, even though 99% of the time it will spin up, see that the container exists and shut down. Signed-off-by: Eric Engestrom <eric.engestrom@intel.com> Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
2019-02-22 23:52:08 +08:00
ci: Move most stuff out of root .gitlab-ci.yml This file was getting a bit hard to navigate. Split container, build and test jobs to their own files. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15891>
2022-04-08 19:29:04 +08:00
.container-rules:
gitlab-ci: Don't require triggering build/test jobs manually Let them run automatically once all their dependencies have passed. Reviewed-by: Adam Jackson <ajax@redhat.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
2020-04-03 18:50:11 +08:00
rules:
gitlab-ci: Automatically run pipelines for Marge Bot pre-merge only Marge only merges an MR if the pipeline passed. Running the pipeline again after merging is redundant. v2: * Add rule to ensure docker images are up to date in the main project registry (Eric Anholt) Reviewed-by: Eric Anholt <eric@anholt.net> Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5680>
2020-06-29 17:33:13 +08:00
# Run pipeline by default in the main project if any CI pipeline
# configuration files were changed, to ensure docker images are up to date
ci: Add "is post-merge pipeline" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 23:47:18 +08:00
- if: *is-post-merge
gitlab-ci: Automatically run pipelines for Marge Bot pre-merge only Marge only merges an MR if the pipeline passed. Running the pipeline again after merging is redundant. v2: * Add rule to ensure docker images are up to date in the main project registry (Eric Anholt) Reviewed-by: Eric Anholt <eric@anholt.net> Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5680>
2020-06-29 17:33:13 +08:00
changes:
- .gitlab-ci.yml
- .gitlab-ci/**/*
when: on_success
gitlab-ci: Run merge request pipelines automatically only for Marge Bot MR pipelines not triggered by Marge Bot can still be triggered manually. Motivation: The main & forked Mesa project CI pipelines combined are currently generating over 1 TB of egress traffic per week. ~80% of this is from pre-merge pipelines. Assuming this corresponds to 4 pre-merge and one post-merge pipeline per MR on average, this change could potentially eliminate up to ~60% of the overall traffic (by preventing 3 of the 4 pre-merge pipelines from running automatically). (Of course, this could be subverted if all jobs of the other pipelines were triggered manually anyway... In most cases, manually triggering just a few jobs should suffice) v2: * $GITLAB_USER_NAME was the wrong variable, $GITLAB_USER_LOGIN should do the trick. Suggested-by: Marek Olšák <maraeo@gmail.com> Reviewed-by: Adam Jackson <ajax@redhat.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
2020-04-03 17:46:12 +08:00
# Run pipeline by default if it was triggered by Marge Bot, is for a
gitlab-ci: Automatically run pipelines for Marge Bot pre-merge only Marge only merges an MR if the pipeline passed. Running the pipeline again after merging is redundant. v2: * Add rule to ensure docker images are up to date in the main project registry (Eric Anholt) Reviewed-by: Eric Anholt <eric@anholt.net> Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5680>
2020-06-29 17:33:13 +08:00
# merge request, and any files affecting the pipeline were changed
ci: Add "is pre-merge pipeline for Marge Bot" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 18:31:08 +08:00
- if: *is-pre-merge-for-marge
gitlab-ci: Don't require triggering build/test jobs manually Let them run automatically once all their dependencies have passed. Reviewed-by: Adam Jackson <ajax@redhat.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
2020-04-03 18:50:11 +08:00
changes:
*all_paths
when: on_success
gitlab-ci: Automatically run pipelines for Marge Bot pre-merge only Marge only merges an MR if the pipeline passed. Running the pipeline again after merging is redundant. v2: * Add rule to ensure docker images are up to date in the main project registry (Eric Anholt) Reviewed-by: Eric Anholt <eric@anholt.net> Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5680>
2020-06-29 17:33:13 +08:00
# Run pipeline by default in the main project if it was not triggered by
# Marge Bot, and any files affecting the pipeline were changed
ci: Add "is post-merge pipeline, not for Marge Bot" YAML anchor Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6566>
2020-09-08 18:36:11 +08:00
- if: *is-post-merge-not-for-marge
gitlab-ci: Don't require triggering build/test jobs manually Let them run automatically once all their dependencies have passed. Reviewed-by: Adam Jackson <ajax@redhat.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
2020-04-03 18:50:11 +08:00
changes:
*all_paths
when: on_success
gitlab-ci: Automatically run pipelines for Marge Bot pre-merge only Marge only merges an MR if the pipeline passed. Running the pipeline again after merging is redundant. v2: * Add rule to ensure docker images are up to date in the main project registry (Eric Anholt) Reviewed-by: Eric Anholt <eric@anholt.net> Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5680>
2020-06-29 17:33:13 +08:00
# Allow triggering jobs manually in other cases if any files affecting the
# pipeline were changed
gitlab-ci: Run merge request pipelines automatically only for Marge Bot MR pipelines not triggered by Marge Bot can still be triggered manually. Motivation: The main & forked Mesa project CI pipelines combined are currently generating over 1 TB of egress traffic per week. ~80% of this is from pre-merge pipelines. Assuming this corresponds to 4 pre-merge and one post-merge pipeline per MR on average, this change could potentially eliminate up to ~60% of the overall traffic (by preventing 3 of the 4 pre-merge pipelines from running automatically). (Of course, this could be subverted if all jobs of the other pipelines were triggered manually anyway... In most cases, manually triggering just a few jobs should suffice) v2: * $GITLAB_USER_NAME was the wrong variable, $GITLAB_USER_LOGIN should do the trick. Suggested-by: Marek Olšák <maraeo@gmail.com> Reviewed-by: Adam Jackson <ajax@redhat.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
2020-04-03 17:46:12 +08:00
- changes:
gitlab-ci: Don't require triggering build/test jobs manually Let them run automatically once all their dependencies have passed. Reviewed-by: Adam Jackson <ajax@redhat.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
2020-04-03 18:50:11 +08:00
*all_paths
when: manual
ci: Remove the trailing "when: never"s from rules. This avoids the risks of concatenating our rulesets missing out on some file list because of a "never" in the middle. Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Acked-by: Eric Engestrom <eric@engestrom.ch> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17287>
2022-06-29 06:15:54 +08:00
# Otherwise, container jobs won't run because no rule matched.
CI: reduce bandwidth for git pull Over the last 7 days, git pulls represented a total of 1.7 TB. On those 1.7 TB, we can see: - ~300 GB for the CI farm on hetzner - ~730 GB for the CI farm on packet.net - ~680 GB for the rest of the world We can not really change the rest of the world*, but we can certainly reduce the egress costs towards our CI farms. Right now, the gitlab runners are not doing a good job at caching the git trees for the various jobs we make, and we end up with a lot of cache-misses. A typical pipeline ends up with a good 2.8GB of git pull data. (a compressed archive of the mesa folder accounts for 280MB) In this patch, we implemented what was suggested in https://gitlab.com/gitlab-org/gitlab/-/issues/215591#note_334642576 - we host a brand new MinIO server on packet - jobs can upload files on 2 locations: * git-cache/<namespace>/<project>/<branch-name>.tar.gz * artifacts/<namespace>/<project>/<pipeline-id>/ - the authorization is handled by gitlab with short tokens valid only for the time of the job is running - whenever a job runs, the runner are configured to execute (eval) $CI_PRE_CLONE_SCRIPT - this variable is set globally to download the current cache from the MinIO packet server, unpack it and replace the possibly out of date cache found on the runner - then git fetch is run by the runner, and only the delta between the upstream tree and the local tree gets pulled. We can rebuild the git cache in a schedule job (once a day seems sufficient), and then we can stop the cache miss entirely. First results showed that instead of pulling 280MB of data in my fork, I got a pull of only 250KB. That should help us. * arguably, there are other farms in the rest of the world, so hopefully we can change those too. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5428>
2020-06-11 23:16:28 +08:00
# Git archive
make git archive:
ci: Use ci-fairy docker image instead of local git_archive one The ci-fairy image contains pip and ci-fairy. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7676>
2020-11-19 01:23:29 +08:00
extends:
- .fdo.ci-fairy
CI: reduce bandwidth for git pull Over the last 7 days, git pulls represented a total of 1.7 TB. On those 1.7 TB, we can see: - ~300 GB for the CI farm on hetzner - ~730 GB for the CI farm on packet.net - ~680 GB for the rest of the world We can not really change the rest of the world*, but we can certainly reduce the egress costs towards our CI farms. Right now, the gitlab runners are not doing a good job at caching the git trees for the various jobs we make, and we end up with a lot of cache-misses. A typical pipeline ends up with a good 2.8GB of git pull data. (a compressed archive of the mesa folder accounts for 280MB) In this patch, we implemented what was suggested in https://gitlab.com/gitlab-org/gitlab/-/issues/215591#note_334642576 - we host a brand new MinIO server on packet - jobs can upload files on 2 locations: * git-cache/<namespace>/<project>/<branch-name>.tar.gz * artifacts/<namespace>/<project>/<pipeline-id>/ - the authorization is handled by gitlab with short tokens valid only for the time of the job is running - whenever a job runs, the runner are configured to execute (eval) $CI_PRE_CLONE_SCRIPT - this variable is set globally to download the current cache from the MinIO packet server, unpack it and replace the possibly out of date cache found on the runner - then git fetch is run by the runner, and only the delta between the upstream tree and the local tree gets pulled. We can rebuild the git cache in a schedule job (once a day seems sufficient), and then we can stop the cache miss entirely. First results showed that instead of pulling 280MB of data in my fork, I got a pull of only 250KB. That should help us. * arguably, there are other farms in the rest of the world, so hopefully we can change those too. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5428>
2020-06-11 23:16:28 +08:00
stage: git-archive
gitlab-ci: do not run full CI on scheduled pipelines Currently, scheduled pipelines are only used to rebuild the git-cache archive daily. There is no point in rebuilding eveything, so ensure that any normal jobs are removed from the scheduled pipelines. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Acked-by: Daniel Stone <daniels@collabora.com> Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5804>
2020-07-07 21:02:35 +08:00
rules:
ci: Use "!references" to manage scheduled pipeline rules. Because !references merging happens after yaml parsing, this lets us remove a duplicated definition between .test-source-dep.yml and .gitlab-ci.yml. Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Acked-by: Eric Engestrom <eric@engestrom.ch> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17287>
2022-06-29 01:43:34 +08:00
- !reference [.scheduled_pipeline-rules, rules]
CI: reduce bandwidth for git pull Over the last 7 days, git pulls represented a total of 1.7 TB. On those 1.7 TB, we can see: - ~300 GB for the CI farm on hetzner - ~730 GB for the CI farm on packet.net - ~680 GB for the rest of the world We can not really change the rest of the world*, but we can certainly reduce the egress costs towards our CI farms. Right now, the gitlab runners are not doing a good job at caching the git trees for the various jobs we make, and we end up with a lot of cache-misses. A typical pipeline ends up with a good 2.8GB of git pull data. (a compressed archive of the mesa folder accounts for 280MB) In this patch, we implemented what was suggested in https://gitlab.com/gitlab-org/gitlab/-/issues/215591#note_334642576 - we host a brand new MinIO server on packet - jobs can upload files on 2 locations: * git-cache/<namespace>/<project>/<branch-name>.tar.gz * artifacts/<namespace>/<project>/<pipeline-id>/ - the authorization is handled by gitlab with short tokens valid only for the time of the job is running - whenever a job runs, the runner are configured to execute (eval) $CI_PRE_CLONE_SCRIPT - this variable is set globally to download the current cache from the MinIO packet server, unpack it and replace the possibly out of date cache found on the runner - then git fetch is run by the runner, and only the delta between the upstream tree and the local tree gets pulled. We can rebuild the git cache in a schedule job (once a day seems sufficient), and then we can stop the cache miss entirely. First results showed that instead of pulling 280MB of data in my fork, I got a pull of only 250KB. That should help us. * arguably, there are other farms in the rest of the world, so hopefully we can change those too. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5428>
2020-06-11 23:16:28 +08:00
# ensure we are running on packet
tags:
- packet.net
script:
ci: Run git gc before creating Git cache tarball Since the job which creates the cache tarball starts from the previous cache, the cache kept accumulating cruft and growing bigger. This cuts the size of the tarball in half (from almost 600M to under 300M), which can translate to significant time savings when downloading it on some runners. v2: * Use git gc --aggressive (Eric Anholt) Reviewed-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7920>
2020-12-04 21:55:54 +08:00
# Compactify the .git directory
- git gc --aggressive
CI: reduce bandwidth for git pull Over the last 7 days, git pulls represented a total of 1.7 TB. On those 1.7 TB, we can see: - ~300 GB for the CI farm on hetzner - ~730 GB for the CI farm on packet.net - ~680 GB for the rest of the world We can not really change the rest of the world*, but we can certainly reduce the egress costs towards our CI farms. Right now, the gitlab runners are not doing a good job at caching the git trees for the various jobs we make, and we end up with a lot of cache-misses. A typical pipeline ends up with a good 2.8GB of git pull data. (a compressed archive of the mesa folder accounts for 280MB) In this patch, we implemented what was suggested in https://gitlab.com/gitlab-org/gitlab/-/issues/215591#note_334642576 - we host a brand new MinIO server on packet - jobs can upload files on 2 locations: * git-cache/<namespace>/<project>/<branch-name>.tar.gz * artifacts/<namespace>/<project>/<pipeline-id>/ - the authorization is handled by gitlab with short tokens valid only for the time of the job is running - whenever a job runs, the runner are configured to execute (eval) $CI_PRE_CLONE_SCRIPT - this variable is set globally to download the current cache from the MinIO packet server, unpack it and replace the possibly out of date cache found on the runner - then git fetch is run by the runner, and only the delta between the upstream tree and the local tree gets pulled. We can rebuild the git cache in a schedule job (once a day seems sufficient), and then we can stop the cache miss entirely. First results showed that instead of pulling 280MB of data in my fork, I got a pull of only 250KB. That should help us. * arguably, there are other farms in the rest of the world, so hopefully we can change those too. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5428>
2020-06-11 23:16:28 +08:00
# compress the current folder
- tar -cvzf ../$CI_PROJECT_NAME.tar.gz .
CI: convert to use the new S3 server instead of the legacy minio We don't need to login anymore, but we can't use plain minio commands now. `ci-fairy` got a helper as `s3cp` to keep an almost identical API. Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19076>
2022-10-14 03:25:38 +08:00
- ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" ../$CI_PROJECT_NAME.tar.gz https://$MINIO_HOST/git-cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/$CI_PROJECT_NAME.tar.gz
CI: reduce bandwidth for git pull Over the last 7 days, git pulls represented a total of 1.7 TB. On those 1.7 TB, we can see: - ~300 GB for the CI farm on hetzner - ~730 GB for the CI farm on packet.net - ~680 GB for the rest of the world We can not really change the rest of the world*, but we can certainly reduce the egress costs towards our CI farms. Right now, the gitlab runners are not doing a good job at caching the git trees for the various jobs we make, and we end up with a lot of cache-misses. A typical pipeline ends up with a good 2.8GB of git pull data. (a compressed archive of the mesa folder accounts for 280MB) In this patch, we implemented what was suggested in https://gitlab.com/gitlab-org/gitlab/-/issues/215591#note_334642576 - we host a brand new MinIO server on packet - jobs can upload files on 2 locations: * git-cache/<namespace>/<project>/<branch-name>.tar.gz * artifacts/<namespace>/<project>/<pipeline-id>/ - the authorization is handled by gitlab with short tokens valid only for the time of the job is running - whenever a job runs, the runner are configured to execute (eval) $CI_PRE_CLONE_SCRIPT - this variable is set globally to download the current cache from the MinIO packet server, unpack it and replace the possibly out of date cache found on the runner - then git fetch is run by the runner, and only the delta between the upstream tree and the local tree gets pulled. We can rebuild the git cache in a schedule job (once a day seems sufficient), and then we can stop the cache miss entirely. First results showed that instead of pulling 280MB of data in my fork, I got a pull of only 250KB. That should help us. * arguably, there are other farms in the rest of the world, so hopefully we can change those too. Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5428>
2020-06-11 23:16:28 +08:00
ci: Add jobs running ci-fairy checks These can catch various common issues in MR settings and Git commit logs. The "check mr" job only exists in pre-merge pipelines for MRs, and runs automatically. The "check commits" job only exists in pre-merge pipelines for MRs and in pipelines for forked branches. It runs automatically in the former case and can be manually triggered in the latter. v2: * Use git_archive docker image (Daniel Stone) * Use a single sanity stage for both jobs (Tomeu Vizoso) Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6209>
2020-08-06 23:37:33 +08:00
# Sanity checks of MR settings and commit logs
ci: Squash "check mr/commits" jobs into a single sanity job The "check commits" job can work again with current ci-templates, but there's no need to keep it as a separate job. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7676>
2020-11-19 01:48:47 +08:00
sanity:
ci: Use ci-fairy docker image instead of local git_archive one The ci-fairy image contains pip and ci-fairy. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7676>
2020-11-19 01:23:29 +08:00
extends:
- .fdo.ci-fairy
ci: Add jobs running ci-fairy checks These can catch various common issues in MR settings and Git commit logs. The "check mr" job only exists in pre-merge pipelines for MRs, and runs automatically. The "check commits" job only exists in pre-merge pipelines for MRs and in pipelines for forked branches. It runs automatically in the former case and can be manually triggered in the latter. v2: * Use git_archive docker image (Daniel Stone) * Use a single sanity stage for both jobs (Tomeu Vizoso) Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6209>
2020-08-06 23:37:33 +08:00
stage: sanity
rules:
ci: Only run the sanity job if there's a MR As it will fail right away if there isn't, and that prevents the CI to run on people's branches. $ ci-fairy check-merge-request --require-allow-collaboration --junit-xml=check-merge-request.xml ERROR: No open merge request against mesa/mesa with sha 9f6aba4be0ceb59d35732651cc44f676e93b70b5 Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Fixes: d4151f2e ("ci: Run sanity job only in pre-merge pipelines") Reviewed-by: Michel Dänzer <mdaenzer@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8032>
2020-12-10 19:48:32 +08:00
- if: *is-pre-merge
ci: Add jobs running ci-fairy checks These can catch various common issues in MR settings and Git commit logs. The "check mr" job only exists in pre-merge pipelines for MRs, and runs automatically. The "check commits" job only exists in pre-merge pipelines for MRs and in pipelines for forked branches. It runs automatically in the former case and can be manually triggered in the latter. v2: * Use git_archive docker image (Daniel Stone) * Use a single sanity stage for both jobs (Tomeu Vizoso) Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6209>
2020-08-06 23:37:33 +08:00
when: on_success
# Other cases default to never
ci: sanity job doesn't need the Git tree This should fix the sanity job taking >= 1 minute in some cases. Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7904>
2020-12-04 01:58:09 +08:00
variables:
GIT_STRATEGY: none
ci: Add jobs running ci-fairy checks These can catch various common issues in MR settings and Git commit logs. The "check mr" job only exists in pre-merge pipelines for MRs, and runs automatically. The "check commits" job only exists in pre-merge pipelines for MRs and in pipelines for forked branches. It runs automatically in the former case and can be manually triggered in the latter. v2: * Use git_archive docker image (Daniel Stone) * Use a single sanity stage for both jobs (Tomeu Vizoso) Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6209>
2020-08-06 23:37:33 +08:00
script:
ci: disable check-commits Otherwise the CI sanity check fails if the commit title exceed 80 characters Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7852>
2020-12-01 21:27:42 +08:00
# ci-fairy check-commits --junit-xml=check-commits.xml
ci: Squash "check mr/commits" jobs into a single sanity job The "check commits" job can work again with current ci-templates, but there's no need to keep it as a separate job. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7676>
2020-11-19 01:48:47 +08:00
- ci-fairy check-merge-request --require-allow-collaboration --junit-xml=check-merge-request.xml
artifacts:
when: on_failure
reports:
junit: check-*.xml
ci: Add jobs running ci-fairy checks These can catch various common issues in MR settings and Git commit logs. The "check mr" job only exists in pre-merge pipelines for MRs, and runs automatically. The "check commits" job only exists in pre-merge pipelines for MRs and in pipelines for forked branches. It runs automatically in the former case and can be manually triggered in the latter. v2: * Use git_archive docker image (Daniel Stone) * Use a single sanity stage for both jobs (Tomeu Vizoso) Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6209>
2020-08-06 23:37:33 +08:00
ci: Add manual rules variations to disable irrelevant driver jobs. If you're only affecting one or a couple of drivers, it would be nice if your pipeline buttons on the web UI weren't full of manual run buttons for all the other drivers. This is a bunch of duplicated lines, but less than it could have been now that we have !references. In some of these cases (i915g, nouveau, etnaviv), we have no non-manual jobs for those drivers, so I could have just rewritten the original "driver-rules" to "driver-manual-rules". I decided to keep things consistent between drivers, though, because this is all esoteric enough to readers already without making different drivers' rules look different. Fixes: #4891 Acked-by: David Heidelberg <david.heidelberg@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17445>
2022-07-10 11:22:35 +08:00
# Temporary rules for tests that should not block merging, but should be
# available to optionally run with the "play" button in the UI in pre-merge
# non-marge pipelines. This should appear in "extends:" after any includes of
ci: Remove the trailing "when: never"s from rules. This avoids the risks of concatenating our rulesets missing out on some file list because of a "never" in the middle. Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Acked-by: Eric Engestrom <eric@engestrom.ch> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17287>
2022-06-29 06:15:54 +08:00
# test-source-dep.yml rules, so that these rules replace those.
ci: Add manual rules variations to disable irrelevant driver jobs. If you're only affecting one or a couple of drivers, it would be nice if your pipeline buttons on the web UI weren't full of manual run buttons for all the other drivers. This is a bunch of duplicated lines, but less than it could have been now that we have !references. In some of these cases (i915g, nouveau, etnaviv), we have no non-manual jobs for those drivers, so I could have just rewritten the original "driver-rules" to "driver-manual-rules". I decided to keep things consistent between drivers, though, because this is all esoteric enough to readers already without making different drivers' rules look different. Fixes: #4891 Acked-by: David Heidelberg <david.heidelberg@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17445>
2022-07-10 11:22:35 +08:00
#
# If a job is going to be manual for a long period of time, then please use a
# driver-specific manual rules set from test-source-dep (see
# freedreno-manual-rules, for example)
gitlab-ci: rule anchor for experimental devices as manual in MRs We want to give developers the option to run their jobs on devices that are still being stabilized in the CI infrastructure. These jobs should be optional and not prevent merging from happening. The is-forked-branch-or-pre-merge anchor was not being used anywhere, so it was changed to is-forked-branch-or-pre-merge-not-for-marge to create this new rule. Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com> Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10405>
2021-04-22 19:17:40 +08:00
.test-manual-mr:
rules:
ci: Use "!references" to manage scheduled pipeline rules. Because !references merging happens after yaml parsing, this lets us remove a duplicated definition between .test-source-dep.yml and .gitlab-ci.yml. Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Acked-by: Eric Engestrom <eric@engestrom.ch> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17287>
2022-06-29 01:43:34 +08:00
- !reference [.no_scheduled_pipelines-rules, rules]
gitlab-ci: rule anchor for experimental devices as manual in MRs We want to give developers the option to run their jobs on devices that are still being stabilized in the CI infrastructure. These jobs should be optional and not prevent merging from happening. The is-forked-branch-or-pre-merge anchor was not being used anywhere, so it was changed to is-forked-branch-or-pre-merge-not-for-marge to create this new rule. Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com> Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10405>
2021-04-22 19:17:40 +08:00
- if: *is-forked-branch-or-pre-merge-not-for-marge
changes:
*all_paths
when: manual
variables:
ci: Let manual LAVA jobs have a longer timeout than others So far only LAVA jobs make use of it, but I guess baremetal could be extended to have these timeouts as well. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13441>
2021-10-22 22:23:21 +08:00
JOB_TIMEOUT: 80
gitlab-ci: rule anchor for experimental devices as manual in MRs We want to give developers the option to run their jobs on devices that are still being stabilized in the CI infrastructure. These jobs should be optional and not prevent merging from happening. The is-forked-branch-or-pre-merge anchor was not being used anywhere, so it was changed to is-forked-branch-or-pre-merge-not-for-marge to create this new rule. Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com> Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10405>
2021-04-22 19:17:40 +08:00
Reference in New Issue Copy Permalink