Downloading the android artifacts separately is really a peculiarity of
cuttlefish jobs, where we need mesa artifacts for both the host and the
guest.
So move the separate download of android artifacts to
cuttlefish-runner.sh
This aligns with the .test-android definition being cuttlefish specific,
and it also matches the original comment in that definition regarding
S3_ANDROID_ARTIFACT_NAME.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35380>
The soft limits cannot go above the hard limits values, so the hard
limits should be configured too, just in case default limits values set
by the kernel or the init process are too small.
E.g. when booting with `init=/bin/sh` we have:
$ cat /proc/1/limits
Limit Soft Limit Hard Limit Units
...
Max open files 1024 4096 files
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35126>
We only have 8GB on the Brya Chromebook used for venus testing, and the
job is unstable because using all the available RAM causes OOM kills.
So reduce launch_cvd memory limit to 4GB by default, still allowing to
change the default by setting the CUTTLEFISH_MEMORY variable in jobs
that might want a different value.
This change has been inspired by CROSVM_MEMORY, but a new variable is
used because technically cuttlefish can also use qemu in some setups,
not only crosvm.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35126>
The venus GPU_MODEs require some patches to the Android 14 kernel to
work, so custom built bzImage and initramfs.img need to be passed to
launch_cvd.
However specifying a custom kernel triggers some image repacking which
is quite expensive in terms of I/O; so, since the custom kernel is only
needed for the venus GPU_MODEs, avoid specifying it for the other modes
to speed up launching cuttlefish in those cases.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34468>
Some of the commands in cuttlefish-runner.sh, like updating mesa and
ANGLE, are not specific to cuttlefish, in general they can be executed
on any Android device under test.
So split those commands out of cuttlefish-runner.sh and put them into an
android-runner.sh script.
For example, when testing a physical Android device instead of a virtual
device, a mesa-ci job will call android-runner.sh directly instead of
cuttlefish-runner.sh
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34111>
We also want to run Android CTS in the Android jobs.
Since the Android CTS is quite large, download it and strip it down to
only contain the interesting tests, so to reduce the space taken in the
container image.
Eventually we might want to have android-cts be run via deqp-runner
itself, but for now add a proof-of-concept mechanism which calls the
android-cts directly and uses an ad-hoc handling of expectations.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33499>
To run deqp-runner in cuttlefish we do something similar to
deqp-runner.sh but adapted to be used on Android via adb.
Isolate those adapted commands in an android-deqp-runner.sh script so
that in the future it will be easier to compare with deqp-runner.sh and
evaluate if deqp-runner.sh and android-deqp-runner.sh could be possibly
consolidated into a single script.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33499>
In .gitlab-ci/cuttlefish-runner.sh 32bit libraries were removed but they
were not being replaced with newer ones, however this caused some
problems because by default the x86_64 target in AOSP is still
multi-library and for example the 32bit zygote process ended up crashing
because of the missing 32bit libraries, causing a general system
instability.
Since the CI is only building 64bit libraries for the android target,
use an x86_64_only cuttlefish product which only has components and
libraries built for the 64bit target, this avoids dealing with 32bit
EGL/Vulkan libraries at all, preventing any possible cause of
instability.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33499>
Disable the modem simulator in cuttlefish, it is not needed for testing
the graphics subsystem and avoids opening a few vsock ports which
reduces the chance of collisions in case of multiple instances of
cuttlefish running concurrently.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33499>
Cleaning up and stopping cuttlefish before launching it is not strictly
necessary when using gitlab shared runners.
This can be added back later when we have a better justification.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33499>
Now that every ANGLE use is covered by tag consistency checks
(structured tagging), we don't need the USE_ANGLE flag anymore, because
if we have ANGLE_TAG set, it means that ANGLE is required in this job.
In detail, it means that the test job has inherited ANGLE_TAG from
`.container-builds-angle`.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33421>
Make setting the ANGLE_ARCH variable optional by providing a default
based on DEBIAN_ARCH, while keeping it possible to override it, which is
expected to be necessary for the Android-arm64 build.
Exclude unnecessary third party dependencies in the .gclient file, which
allows us to delete our first local patch. Thanks to Yuly Novikov for the
suggestion.
Use -no-history for gclient sync, which is equivalent to git's --depth=1
argument. This greatly speeds up the process of fetching sources.
Thanks to this speedup fetching third_party/catapult is no longer an
issue, allowing us to remove our second local patch.
Since we're no longer applying local patches, use ANGLE_REV and
/angle/version as the base for our version check on Android.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Reviewed-by: Antonio Ospite <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33467>
After commit 44bda7c258 (dri: put shared-glapi into libgallium.*.so,
2024-12-26) the mesa Android build does not have a separate libglapi.so
object anymore in the install dir, so stop pushing it to the Android
device in cuttlefish-runner.sh
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33261>
Add a trailing dot to the remote directoyy in the `adb pull` command to
make sure to recursively pull only the **content** of the directory and
not the directory itself.
This prevents having `results/results/` in the artifacts.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33261>
Cuttlefish performs some checks on the host system before launching,
setting VK_DRIVER_FILES to the path of the host artifacts makes sure
that a vulkan library can be found on the host side.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Set XDG_CACHE_HOME when invoking deqp-runner so that mesa can create the
shader cache and avoid this error:
-----------------------------------------------------------------------
ERROR - dEQP error: Failed to create //.cache for shader cache (Read-only file system)---disabling.
-----------------------------------------------------------------------
And also pass --shader-cache-dir to deqp-runner for vulkan tests.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Also copy mesa vulkan libraries to the Android guest, this makes it
possible to test venus with the latest upstream mesa build.
While at it also keep the *_angle libraries as they will be used by the
venus_guest_angle GPU_MODE.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Prefer libraries from /cuttlefish when loading binaries, in particular
this solves some loading problem when launching
/cuttlefish/bin/virgl_render_server
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
The /etc/init.d/cuttlefish-host-resources wnants to perform some
operations that are not really desirable on a gitlab shared runner, like
loading specific kernel modules.
Since the script does not seem to be strictly necessary for basic
operation of cuttlefish, don't call it at all, avoiding the kernel
modules problem.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Commit 4143199be7 (ci/android: Use common $RESULTS_DIR for cuttlefish,
2024-08-21) clarified the separation between the result dir used on the
host and shipped in the artifacts, and the result dir used in the
Android guest to collect test results, however the latter was not
correctly passed to deqp-runner.
Fix that so that deqp-runner can finally run.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Fix warning when using chown:
-----------------------------------------------------------------------
+ chown root.kvm /dev/kvm
chown: warning: '.' should be ':': 'root.kvm'
-----------------------------------------------------------------------
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Use a custom kernel when launching cuttlefish, this is needed for
-gpu_mode=venus.
And while at it also clean up the state from previous runs, this seems
to be required when passing a custom kernel.
In this change also pass -enable_bootanimation=false which makes
drm_virgl work, and pass -enable_minimal_mode=true to avoid using some
features which are not really needed.
Note that the file number limit has to be increased too in
.gitlab-ci/cuttlefish-runner.sh for cuttlefish to work properly.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Clarify that artifacts from the Android guest are downloaded explicitly
and put them under /mesa-android directory so that they don't clash with
the Linux host artifacts downloaded by gitlab under
"${CI_PROJECT_DIR}/install".
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Different Android images may contain different graphic libraries, for
example images from Google using an old mesa version may have
libGLES_mesa.so, while newer Android images built with mesa from
upstream have libGLESv1_CM_mesa.so, etc.
Use `rm -f` instead of just `rm` to avoid running into errors when
a file does not exist, resulting in a failing job.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
