Structured tagging ensures that we are building and testing the current
component version specified in the commit by matching the checksum of
the related build script file.
In this case, it is worthy to isolate the Android CTS version part,
because we don't need to rebuild the entire test-android container when
we change the CTS version or the CTS modules filtering.
PS: actually the new file `build-android-cts.sh` is not building
anything, it is just downloads, filters, compress and reupload the
stripped version to S3. The `build-` prefix is to make it work
transparently with `bin/ci/update_tag.py` script.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35596>
For pre-merge pipelines, we know that we don't need to run the
container-check jobs unless image-tags.yml has been modified. In this
case, we can just skip the container jobs, because we 'know' that the
containers are present in the upstream repo.
For downstream or off-main pipelines, we still need to run the check
job, because the changed-file tracking is not reliable on other
branches, and because we may need to copy the container image from the
upstream container storage into the user's namespace.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35203>
Compared to the existing Debian-based x86_64_pyutils container, this
Alpine-based variant reduces the image size by approximately 83%.
Include all the necessary python artifacts, including lava_job_submitter
in the container to avoid having to download them at the start of each
test job.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34980>
Firmware packages continue to grow in size, so stop installing them in
the test-base image.
The necessary firmware is now collected and uploaded per vendor in an
external repository.
LAVA devices can opt into optional firmware by specifying the name of the
archive via LAVA_FIRMWARE.
For bare-metal, Qualcomm firmware required for DUTs in the Google lab is
included in the baremetal image.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13051
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34861>
Introduce per-component CI_BUILD_COMPONENTS_* variables to allow
more flexible composition of CI_BUILD_COMPONENTS, and add these to
every container job by listing them in .container.
Also rework the -arm64 and -x86_64 definitions, which were still
based on the old kernel+rootfs layout. Split them into -gl and -vk,
but keep -arm32, which builds fewer components.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34813>
Including the kernel image in test-base requires rebuilding every
x86_64 container image on each kernel uprev, even though only the
crosvm jobs use this kernel.
Move the download to runtime and update the distribution tags to avoid
triggering container rebuilds on kernel changes.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34813>
Structured tagging is used to verify the checksum of the component we're
building. In Android's case, this is currently only used for ANGLE.
Move the build-time check to the debian/x86_64_test-android container,
where ANGLE is built.
Previously, having this definition in .android-variables meant that every
Android test job inherited the ANGLE_TAG variable, making it impossible to
use Mesa as the GLES driver in CI.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34572>
Show how to add more Android CTS tests cases, using --include-filters
Only CtsNativeHardwareTestCases and CtsSkQPTestCases are actually
enabled for now, because the android-angle-lavapipe-cts job is part of
the pre-merge pipeline and these modules would not be too expensive to
run.
The container size increases by about 60Mb and the test time from 4m to
7 min on a local system, so it's an acceptable compromise to show how
multiple modules can be tested.
A similar mechanism will be used to add CtsDeqpTestCases tests in the
future, probably in nightly or weekly jobs, because that would require
more space in the containers and a lot more time to run the tests.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34479>
Structured tagging (cf. mesa/mesa!33421) captures a checksum of the
thing we think we're building, and verifies this through the chain.
When we run container builds, we check that the tag we've captured in
the CI variables matches the calculated checksum, to make sure the
declared tags are consistent and we always have traceability.
When we run tests, we check the tags again between what was declared in
the CI variables and what we're actually running from the test
container. This makes sure that we're always testing what we think we're
testing.
As a side advantage, the rule inheritance we need to make this work
means that we can start doing more optional downloads via overlays,
instead of pulling a whole container full of stuff we might not ever
use.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34539>
Enforce a default job timeout of 1 second, to make jobs which don't
explicitly specify a timeout insta-fail, rather than potentially hanging
around for an hour.
Container builds get the full hour as they can run long and are not run
in pre-merge context, and LAVA jobs also get the full hour as they have
multiple internal timeout mechanisms which aim to fast-fail jobs once
they actually start. However, as they just queue jobs to an external
host (shared with other projects like KernelCI), these timeouts aren't
reflected into the GitLab CI definitions.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34280>
This affects:
- generic jobs (sanity, rustfmt, shader-db, docs, etc.)
- linux image builds
- linux mesa builds
- software renderer tests
- android tests
- virgl & venus tests
Marge pipelines get high priority, nightly pipelines get low priority,
and everything else is in between.
(Hardware test farms have their own mechanisms.)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34264>
We also want to run Android CTS in the Android jobs.
Since the Android CTS is quite large, download it and strip it down to
only contain the interesting tests, so to reduce the space taken in the
container image.
Eventually we might want to have android-cts be run via deqp-runner
itself, but for now add a proof-of-concept mechanism which calls the
android-cts directly and uses an ad-hoc handling of expectations.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33499>
LLVM 15 is pretty old, and notably not supported by either ANGLE nor
Skia anymore. So let's move up to LLVM 19 using packages provided by
LLVM themselves, apart from PPC and ARMv7 which don't have builds.
The Skia build now requires a bunch of new warning exclusions; hopefully
most of these are no longer needed when we can upgrade Skia shortly.
The ci-deb-repo revision has also been bumped to get us a new version of
xtensor which builds with LLVM 19, and a version of spirv-tools which
also works with LLVM 19.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Closes: mesa/mesa#11538
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33137>
