The job definitions for lava-related jobs are encapsulated in a directory,
while the other two farm managers were in the generic test directory. Having a
directory for ci-tron places it side by side with other farm managers. For
bare-metal, it has another advantage, as this encapsulates elements related to
this farm manager in a single place.
To maintain simplicity and consistency in file naming, the gitlab-ci file in
the lava directory is renamed as it has a prefix that corresponds with the
directory hosting it. The other farm equivalent files don't include this
duplication as a prefix, and there isn't such a prefix in any other case of
the CI.
Signed-off-by: Sergi Blanch-Torne <sergi.blanch.torne@collabora.com>
Acked-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35427>
Structured tagging ensures that we are building and testing the current
component version specified in the commit by matching the checksum of
the related build script file.
In this case, it is worthy to isolate the Android CTS version part,
because we don't need to rebuild the entire test-android container when
we change the CTS version or the CTS modules filtering.
PS: actually the new file `build-android-cts.sh` is not building
anything, it is just downloads, filters, compress and reupload the
stripped version to S3. The `build-` prefix is to make it work
transparently with `bin/ci/update_tag.py` script.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35596>
Remove use of `eval` when writing `dut-job-env-vars.sh`, as it's
unnecessary. The script only needs to declare variables, not evaluate
them.
Using `eval` introduces parsing issues when variables contain both
single and double quotes, such as in commit titles. Example:
https://gitlab.freedesktop.org/mesa/mesa/-/jobs/77995175#L3188
This job failed to parse `CI_COMMIT_TITLE` and `CI_MERGE_REQUEST_TITLE`
correctly due to mixed quoting in:
Revert "ci: disable Collabora's farm due to maintenance"
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35421>
For pre-merge pipelines, we know that we don't need to run the
container-check jobs unless image-tags.yml has been modified. In this
case, we can just skip the container jobs, because we 'know' that the
containers are present in the upstream repo.
For downstream or off-main pipelines, we still need to run the check
job, because the changed-file tracking is not reliable on other
branches, and because we may need to copy the container image from the
upstream container storage into the user's namespace.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35203>
Remove the /vkd3d-proton directories from the core LAVA rootfs archive
and upload vkd3d-proton as a separate archive for use as an optional LAVA
overlay.
This reduces the size of the core lava-rootfs tarball and ensures
vkd3d-proton is only deployed when the job needs it.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35167>
Time drift can occur during LAVA job execution due to transitions
between three different clocks.
The process begins in the GitLab job [1], using the CI_JOB_STARTED_AT
variable. If SSH is enabled, we then connect to the DUT through an
Alpine-based SSH client container inside the LAVA dispatcher [2], where
some GitLab-related steps are timestamped by lava_job_submitter.
Finally, the DUT [3] runs and uses the setup-test-env.sh helper to
handle GitLab sections, potentially using a third distinct clock.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35222>
Compared to the existing Debian-based x86_64_pyutils container, this
Alpine-based variant reduces the image size by approximately 83%.
Include all the necessary python artifacts, including lava_job_submitter
in the container to avoid having to download them at the start of each
test job.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34980>
Firmware packages continue to grow in size, so stop installing them in
the test-base image.
The necessary firmware is now collected and uploaded per vendor in an
external repository.
LAVA devices can opt into optional firmware by specifying the name of the
archive via LAVA_FIRMWARE.
For bare-metal, Qualcomm firmware required for DUTs in the Google lab is
included in the baremetal image.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13051
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34861>
We can have jobs with a lower job timeout values, given by
CI_JOB_TIMEOUT environment variable, such as the pytest ones.
The previously hardcoded burst length of 1000 messages at a simulated
rate of 1 msg/sec caused tests to exceed these timeouts and fail
unexpectedly on specific job timeouts.
Reported-by: Eric Engestrom <eric@igalia.com>
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34907>
Some jobs, like those using pytest, have lower `CI_JOB_TIMEOUT` values.
This change ensures that the estimated LAVA overhead (in minutes) is
compatible with the actual job timeout, failing early with an assertion
and also avoiding mismatches and unintended timeouts.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34907>
The EXTERNAL_KERNEL_TAG variable is no longer needed.
For LAVA and bare-metal, we can override the KERNEL_TAG variable to fetch
both the kernel image and modules from a different tag than the default
mainline gfx-ci/linux kernel.
For LAVA, this also avoids the issue where jobs using EXTERNAL_KERNEL_TAG
would still have mainline kernel modules downloaded by the LAVA overlay.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34873>
Add new job definitions using the new debian/x86_64-test-video container,
and convert the radeonsi-raven-va and radeonsi-raven-vaapi-fluster jobs
to use the rootfs derived from this container.
Now that we are no longer downloading the Fluster vectors at runtime, the
parallelsim of the radeonsi-raven-vaapi-fluster job can be greatly
decreased.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34451>
Downloading them at runtime proved too slow and unreliable.
Keep the logic to upload the vectors to S3 as it's faster, the build job
takes around 5-10 minutes when downloading from S3, but 30+ minutes to
using Fluster to download the vectors.
Move this to the build-fluster.sh script to allow re-using it in the
test-video container.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34451>
Structured tagging (cf. mesa/mesa!33421) captures a checksum of the
thing we think we're building, and verifies this through the chain.
When we run container builds, we check that the tag we've captured in
the CI variables matches the calculated checksum, to make sure the
declared tags are consistent and we always have traceability.
When we run tests, we check the tags again between what was declared in
the CI variables and what we're actually running from the test
container. This makes sure that we're always testing what we think we're
testing.
As a side advantage, the rule inheritance we need to make this work
means that we can start doing more optional downloads via overlays,
instead of pulling a whole container full of stuff we might not ever
use.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34539>
Enforce a default job timeout of 1 second, to make jobs which don't
explicitly specify a timeout insta-fail, rather than potentially hanging
around for an hour.
Container builds get the full hour as they can run long and are not run
in pre-merge context, and LAVA jobs also get the full hour as they have
multiple internal timeout mechanisms which aim to fast-fail jobs once
they actually start. However, as they just queue jobs to an external
host (shared with other projects like KernelCI), these timeouts aren't
reflected into the GitLab CI definitions.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34280>
