A new container registry was added recently in the fd.o infrastructure.
It is located in a datacenter that does not charge for bandwidth usage,
and aims to become the canonical registry in the future. It is however
currently configured to be a partial mirror of registry.fd.o, where
only the most-downloaded container images can be found.
The address of the new registry is specified in mesa/mesa's CI
variables, which means forks will default to registry.fd.o. However,
Valve Infra DUTs do not have access to the internet, and instead use
another cache proxy hosted locally on the CI gateways. This forced
Benjamin to overwrite the registry URL from harbor.freedesktop.org to
registry.freedesktop.org in 0bd9a062e1 ("CI: Overwrite valve infra's
registry").
After adding support for the new registry in valve infra, then
deploying the update, we are now able to make use of the new container
registry. This commit simply rewrites the URL harbor.freedesktop.org
into `{{ harbor_fdo_registry }}`. This variable is set in the valve
infra to point to the `host:ip` of the service, which means changes to
the way we deploy this service can be done on the valve-infra side
without needing to make changes to every project that makes use of our
machines.
Related: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7913
Acked-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20450>
gitlab-runner reuses containers, and since we don't pull git, the working
directory doesn't get cleaned automatically. You don't want to have stale
files from previous builds, particularly if someone's testing changes of
build options that might disable a driver.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20302>
Every barebone arm or aarch64 job depends on these.
When experimenting with CI, this job drags both kernel+rootfs_arm64 and armhf,
even when the latter is not used.
This should speed up development and relieve one instance for about 30 minutes
and overlap between finishing kernel+rootfs_arm64 and armhf jobs per change.
Here are some size stats:
old arm_test 904M (32+64)
new arm64_test 579M (64bit)
new armhf_test 402M (32bit)
---
both ~ 981M (overhead is 81M overall, but never used together so it's only cost "at rest")
Runtime of the arm*_test jobs is ~ 2 minutes.
Acked-by: Martin Roukala <martin.roukala@mupuf.org>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: "Sergi Blanch Torné" <sergi.blanch.torne@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19550>
If you accidentally re-included your test job core definition after your
driver-specific ruleset, you'd end up running the driver job on every
source code change. This had happened with a630_gles_asan: it included
.baremetal-test-arm64-asan (and thus .baremetal-test) after including
.a630-test, to override .baremetal-test-arm64's depednencies to use asan
artifacts instead.
Acked-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17386>
... and explain what they're doing, compared to the test rules in
test-source-dep.yml.
Unfortunately, we can't really pull them into test-source-dep.yml with
other source deps, because of various '&'-'*' references.
Acked-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17386>
We had it set up for arm64 asan already, do it for everyone else too. In
cleaning up the duplication, this fixes a pasteo in rpi3 which had the
"artifacts: false" on the wrong job, causing it to do a slow download of
the mesa build from gitlab.
Doing this required also moving the ".use-debian/arm_test" in as well, so
that its "needs:" didn't overwrite ours if it appeared after us in the
consumer's "extends:"
Should save about 20 seconds on rpi3 jobs.
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17146>
This is not one of the valid stages in the top level "stages:"
declaration, you're supposed to get your stage from your
test-source-dep.yml include. Avoids issues when .baremetal-test gets
included after your test-source-dep.
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17146>
This reverts commit 4824396572.
The newest version of b2c has changed the mount point of the cache
device, which is has proven to be working very well in my local
testing, but ends up confusing podman when the cache device had been
initialized using an older version of b2c. The end result is that we
end up using tmpfs to run our jobs, and some machines just run out of
RAM before the end of the job...
We'll be fixing this issue, cut a new b2c release, then re-introduce
in Mesa.
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17120>
Until now, we have been extracing the install.tar image on the gitlab
runner before sharing it to the test machine through a MINIO bucket.
It turns out that hardlinks and symlinks do not get shared, so let's
just extract the tarball directly on the test machine to fix the issue.
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16968>
I am not super happy about doing this, as this means we will
potentially be introducing regressions because CI would have papered
over a rare race condition... But the situation is that this is already
happening, and improving the reliability will likely come from tracking
the occurence rate of hangs in an external system.
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16472>
The `IMAGE_UNDER_TEST` variable set in `.b2c-test` got broken with
the merge of 7d474c1 (ci: Move most stuff out of root .gitlab-ci.yml).
During the shuffling, the `MESA_BASE_TAG` and `MESA_IMAGE_TAG`
variables were dropped, leading to `IMAGE_UNDER_TEST` being an
unexisting container.
To make this issue less likely to happen in the future, this patch
drops the code duplication that led to `IMAGE_UNDER_TEST` to be
the same as `MESA_IMAGE` and instead re-uses .use-debian/x86_test-vk
to generate `MESA_IMAGE`, which we then use verbatim in
`IMAGE_UNDER_TEST`.
The renaming is `MESA_IMAGE` into `IMAGE_UNDER_TEST` there to make the
distinction clear between the image run by gitlab-runner (what is
usually called `MESA_IMAGE` but we instead hardcode to valve-infra's
trigger container), and the image we are running on the test machines.
Fixes: 7d474c1 (ci: Move most stuff out of root .gitlab-ci.yml)
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Reviewed-by: Charlie Turner <cturner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15555>
