Commit Graph

32 Commits

Author SHA1 Message Date
Guilherme Gallo
655cf2f553 ci: Simplify filter_env_vars using indirect expansion
Avoid parsing variable values manually by using `${!varname@A}`, which
returns the exact declaration as stored in the bash session.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35421>
2025-06-10 18:19:21 -03:00
Valentine Burley
0ccfa323bc ci: Add and use find_s3_project_artifact (and curl-with-retry) helpers
Introduce find_s3_project_artifact (and curl-with-retry) helpers to
simplify locating S3 artifacts across upstream mesa/mesa and forks.

Use these helpers in the Fluster build process and for fetching LAVA
rootfs artifacts.

Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35167>
2025-06-10 06:33:09 +00:00
Antonio Ospite
b16ed16d75 ci: set up XDG_RUNTIME_DIR to be under /tmp which is tmpfs
Set up XDG_RUNTIME_DIR path to be under /tmp.

This might improve speed as /tmp is under tmpfs while the older path
might have been under NFS.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35126>
2025-06-04 15:43:40 +00:00
Valentine Burley
4986c650f6 ci: Don't forward long environment variables
Filter out the CI_COMMIT_DESCRIPTION, CI_COMMIT_MESSAGE, and
CI_MERGE_REQUEST_DESCRIPTION variables, which were causing issues in LAVA.

Fixes: cc83b3db5f ("ci: Forward all environment variables to DUTs and crosvm")
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35196>
2025-05-28 18:44:16 +00:00
Valentine Burley
a6e1926a8e ci: Fix date parsing with BusyBox on Alpine
This fixes the timestamps in Alpine containers.

Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35051>
2025-05-26 15:30:47 +00:00
Valentine Burley
cc83b3db5f ci: Forward all environment variables to DUTs and crosvm
Instead of the current allowlist in export-gitlab-job-env-for-dut.sh,
filter out unwanted environment variables and forward the rest to
bare-metal and LAVA DUTs, as well as crosvm.

Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35051>
2025-05-26 15:30:47 +00:00
Vignesh Raman
6c33b1e8c1 ci: replace s3_upload wrapper with ci-fairy s3cp
Now that ci-fairy s3cp works, replace the s3_upload curl wrapper with
ci-fairy s3cp command.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34550>
2025-04-22 11:17:16 +00:00
Vignesh Raman
7959250d1e s3_upload: improve url validation and error message
Ensure s3_upload correctly validates the S3 folder url by requiring it
to end with /. This prevents wrong uploads to invalid paths, such as
file urls. Also improve the error message.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34255>
2025-04-04 13:32:45 +00:00
Eric Engestrom
7178425ccf ci: replace broken s3cp command with a simple curl call
The current `s3cp` implementation does not work anymore after the
migration, and instead of fixing it and propagating the fix down to us,
it's simpler to directly use `curl`.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34120>
2025-03-24 20:45:52 +00:00
Guilherme Gallo
5798f5d05f Revert "ci: setup-test-env: Prefer functions over aliases"
This reverts commit 1cc2c738bb

We originally changed some aliases into functions so scripts could use
them without needing to be sourced, keeping the environment cleaner.

However, this broke `x_off`, which is supposed to stop debug logs
(xtrace output) from showing in the console. The function version still
triggered xtrace before disabling it, while the alias correctly
redirected the logs to `/dev/null`.

It also fixes the `bin/ci/update_tag.py` script to be able to reuse the
aliases via double sourcing the setup-test-env.sh and the respective
build script.

Reported-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33863>
2025-03-12 20:04:20 +00:00
Guilherme Gallo
f13b95ad5c ci: add support for structural tagging
Make structural tagging functions available for both test and build
scripts.

Introduces the update_tag.sh helper for listing, checking, and updating
deterministic tags.

Also adds the ci_tag_build_time_check and ci_tag_test_time_check
functions to validate tags during build and test phases, ensuring
consistent component versioning.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33421>
2025-02-28 01:33:34 +00:00
Guilherme Gallo
775c2c3254 ci: add _error_msg for internal messaging
So it can be reused in functions that wants to print errors without
dealing with GitLab sections.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33421>
2025-02-28 01:33:34 +00:00
Guilherme Gallo
1cc2c738bb ci: setup-test-env: Prefer functions over aliases
Alias are not exportable, in the current situation of the build scripts,
we use alias to deal with sections, but it infers that all build scripts
will be included in a bigger one that has already included
`setup-test-env.sh`.

With the structured tagging, we do a dry run of all build scripts, to
early check if the tagging is valid, before building stuff.

So changing alias to functions will not have an effect on the current
setup, but it also removes the need to reinclude library bash scripts in
some situations, as described above.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33421>
2025-02-28 01:33:34 +00:00
Deborah Brouwer
b5b2515f86 ci: Remove duplicate slash before $RESULTS_DIR
The RESULTS_DIR variable is defined by reference to the present
working directory, but if the pwd is the root directory then the
$RESULTS_DIR begins with two slashes instead of one like this: //results.

This is harmless but not necessary, so remove it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32051>
2024-11-09 08:40:48 +00:00
Eric Engestrom
8d2620569c ci: make error handling quieter
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32054>
2024-11-08 20:56:46 +00:00
Eric Engestrom
e5708ab2b4 ci: use quiet alias for commands
And set x_off again when nesting these functions but we're not done and
we have more after.

Fixes: d69bd58365 ("ci: consistently restore `-x` after temporarily disabling it")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32054>
2024-11-08 20:56:46 +00:00
Eric Engestrom
5cd054ebe5 ci: move error handling functions at the end
So that everything is defined by the time we use it in here.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32054>
2024-11-08 20:56:46 +00:00
Eric Engestrom
b76b53cbbb ci: rename state_x to previous_state_x to be clear about what it contains
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31939>
2024-11-05 20:30:22 +00:00
Eric Engestrom
d69bd58365 ci: consistently restore -x after temporarily disabling it
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31939>
2024-11-05 20:30:22 +00:00
Eric Engestrom
4a19f7d53d ci: initialize state_x (tracking set -x)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31939>
2024-11-05 20:30:21 +00:00
Eric Engestrom
4fa1730cdd ci: only end current section if there is a current section
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31939>
2024-11-05 20:30:21 +00:00
Daniel Stone
ef175ea594 ci: Make section emission really quiet
If you're emitting a section header under set -x, you will see:
+ section_start foo "foo"
+ x_off
[the section header]

This is kind of annoying. Instead of trying to squash it everywhere by
dancing around local set +x management, play some extremely stupid
tricks with shells to make sure we never emit it.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31602>
2024-10-20 11:32:43 +01:00
Daniel Stone
4ae2d97c36 ci: Make setup-test-env safe for set -u
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31281>
2024-10-10 11:05:56 +00:00
Daniel Stone
9b6d14aed1 ci: Always create results dir from init
During init-stage2 (used for hardware jobs) and setup-test-env (used
for running directly on shared runners), make sure we always create a
results directory.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31110>
2024-09-13 10:12:09 +01:00
Daniel Stone
28aafcfaef ci: Bring back yamllint and shellcheck
These have been disabled for a while, but shouldn't have been.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30978>
2024-09-09 16:27:07 +00:00
Eric Engestrom
9be380c6da ci: simplify unnecessarily complex printf
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28344>
2024-03-25 13:04:15 +00:00
Eric Engestrom
ed45e373bf ci: convert the job start date into a timestamp only once
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28344>
2024-03-25 13:04:15 +00:00
Eric Engestrom
775eca9449 ci: deduplicate converting the current job runtime into %M:%S
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28344>
2024-03-25 13:04:15 +00:00
David Heidelberg
5941fc8ccd ci: enable shellcheck on whole .gitlab-ci
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21977>
2023-05-25 16:06:53 +02:00
David Heidelberg
fb3632d24f ci: distribute XDG_RUNTIME_DIR with setup-test-env script
Fixes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8615
Fixes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8646

Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22004>
2023-03-19 15:39:12 +01:00
David Heidelberg
d79d62c638 ci: do not exit when an error happens inside the section
Let CI handle that case as it did before. The only thing which needs to be done by the error() function is to get out of the section.

Fixes the issue where a job is not saving artifacts after failure.
Fixes: 4cc0cec473 ("ci: implement unified sections")

Acked-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21715>
2023-03-06 11:59:42 +00:00
David Heidelberg
4cc0cec473 ci: implement unified sections
in after_script, variable $SCRIPTS_DIR is lost

Acked-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20272>
2023-03-01 12:02:24 +00:00