This will be helpful for DUTs that do not boot reliably due to firmware
issue as it will allow us to specify a boot retry count that only
counts towards booting linux and not other issues found during test
execution (like hitting a GPU hang).
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32572>
This unifies the behaviour between the LAVA, baremetal, and CI-Tron
farms by ensuring every job has access and runs the same scripts.
The init-* scripts are however still sourced from outside the build
artifact, hopefully not for too long.
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33026>
When cross-building for Android meson fails to find supported compiler
arguments for the C++ cross-compiler, e.g.:
```
Compiler for C++ supports arguments -Wno-array-bounds: NO (cached)
Compiler for C++ supports arguments -Wno-overflow: NO
Compiler for C++ supports arguments -Wno-c++11-narrowing: NO (cached)
Compiler for C++ supports arguments -Wno-vla-cxx-extension: NO (cached)
```
This is due to an **unrelated** and more generic compilation failure
when testing for the supported arguments, e.g.:
```
Command line: `/tmp/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android34-clang++ -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -static-libstdc++ /home/ao2/Collabora/GOO0042/mesa/_build/meson-private/tmpv6_hke9l/testfile.cpp -o /home/ao2/Collabora/GOO0042/mesa/_build/meson-private/tmpv6_hke9l/output.obj -c -Wno-error=c99-designator -Wno-error=unused-variable -Wno-error=unused-but-set-variable -Wno-error=self-assign -D_FILE_OFFSET_BITS=64 -O0 -fpermissive -Werror=implicit-function-declaration -Werror=unknown-warning-option -Werror=unused-command-line-argument -Werror=ignored-optimization-argument -Wvla-cxx-extension -Wno-vla-cxx-extension` -> 1
stderr:
clang++: error: argument unused during compilation: '-static-libstdc++' [-Werror,-Wunused-command-line-argument]
-----------
Compiler for C++ supports arguments -Wno-vla-cxx-extension: NO
```
The issue is caused by how the cross compiler is set up by
.gitlab-ci/container/create-android-cross-file.sh
Allow the cross compiler to still start even when the
`-Werror,-Wunused-command-line-argument` error occurs in order to be
able to actually detect other arguments:
```
Command line: `/tmp/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android34-clang++ -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables --start-no-unused-arguments -static-libstdc++ --end-no-unused-arguments /home/ao2/Collabora/GOO0042/mesa/_build/meson-private/tmpm6eolxed/testfile.cpp -o /home/ao2/Collabora/GOO0042/mesa/_build/meson-private/tmpm6eolxed/output.obj -c -Wno-error=c99-designator -Wno-error=unused-variable -Wno-error=unused-but-set-variable -Wno-error=self-assign -D_FILE_OFFSET_BITS=64 -O0 -fpermissive -Werror=implicit-function-declaration -Werror=unknown-warning-option -Werror=unused-command-line-argument -Werror=ignored-optimization-argument -Wvla-cxx-extension -Wno-vla-cxx-extension` -> 0
Compiler for C++ supports arguments -Wno-vla-cxx-extension: YES
```
This makes argument detection work again, e.g:
```
Compiler for C++ supports arguments -Wno-array-bounds: YES (cached)
Compiler for C++ supports arguments -Wno-overflow: YES
Compiler for C++ supports arguments -Wno-c++11-narrowing: YES (cached)
Compiler for C++ supports arguments -Wno-vla-cxx-extension: YES (cached)
```
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12441
Reviewed-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Valentine Burley <valentine.burley@collabora.com>
Reviewed-by: Rob Clark <robclark@freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33013>
Also add a note in .gitlab-ci/container/debian/x86_64_test-android.sh
that DEBIAN_TEST_ANDROID_TAG needs to be updated when that file changes
too.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Cuttlefish performs some checks on the host system before launching,
setting VK_DRIVER_FILES to the path of the host artifacts makes sure
that a vulkan library can be found on the host side.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Set XDG_CACHE_HOME when invoking deqp-runner so that mesa can create the
shader cache and avoid this error:
-----------------------------------------------------------------------
ERROR - dEQP error: Failed to create //.cache for shader cache (Read-only file system)---disabling.
-----------------------------------------------------------------------
And also pass --shader-cache-dir to deqp-runner for vulkan tests.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Also copy mesa vulkan libraries to the Android guest, this makes it
possible to test venus with the latest upstream mesa build.
While at it also keep the *_angle libraries as they will be used by the
venus_guest_angle GPU_MODE.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Prefer libraries from /cuttlefish when loading binaries, in particular
this solves some loading problem when launching
/cuttlefish/bin/virgl_render_server
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
The /etc/init.d/cuttlefish-host-resources wnants to perform some
operations that are not really desirable on a gitlab shared runner, like
loading specific kernel modules.
Since the script does not seem to be strictly necessary for basic
operation of cuttlefish, don't call it at all, avoiding the kernel
modules problem.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Commit 4143199be7 (ci/android: Use common $RESULTS_DIR for cuttlefish,
2024-08-21) clarified the separation between the result dir used on the
host and shipped in the artifacts, and the result dir used in the
Android guest to collect test results, however the latter was not
correctly passed to deqp-runner.
Fix that so that deqp-runner can finally run.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Fix warning when using chown:
-----------------------------------------------------------------------
+ chown root.kvm /dev/kvm
chown: warning: '.' should be ':': 'root.kvm'
-----------------------------------------------------------------------
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Use a custom kernel when launching cuttlefish, this is needed for
-gpu_mode=venus.
And while at it also clean up the state from previous runs, this seems
to be required when passing a custom kernel.
In this change also pass -enable_bootanimation=false which makes
drm_virgl work, and pass -enable_minimal_mode=true to avoid using some
features which are not really needed.
Note that the file number limit has to be increased too in
.gitlab-ci/cuttlefish-runner.sh for cuttlefish to work properly.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Clarify that artifacts from the Android guest are downloaded explicitly
and put them under /mesa-android directory so that they don't clash with
the Linux host artifacts downloaded by gitlab under
"${CI_PROJECT_DIR}/install".
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Different Android images may contain different graphic libraries, for
example images from Google using an old mesa version may have
libGLES_mesa.so, while newer Android images built with mesa from
upstream have libGLESv1_CM_mesa.so, etc.
Use `rm -f` instead of just `rm` to avoid running into errors when
a file does not exist, resulting in a failing job.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
To test mesa on cuttlefish some special images tailored for upstream
mesa are needed, in particular these images may set properties or add
functionality that enable new use cases for upstream mesa on Android,
like for instance using venus via the vulkan.virtio mesa driver.
So instead of downloading the standard images from Google, get those
custom images from s3.freedesktop.org.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
The ./tools/buildutils/build_packages.sh script used by the
android-cuttlefish project built in debian/x86_64_test-android.sh uses
sudo, so install that as an EPHEMERAL dependency.
Also set SUDO_FORCE_REMOVE=yes when removing sudo to avoid some errors.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Save some logs and stop cuttlefish more cleanly when
.gitlab-ci/cuttlefish-runner.sh exits prematurely because of some
errors.
This makes it possible to have the logs available as artifacts even if
the script didn't run to completion.
This approach also leaves the system in a cleaner state and makes it
possible to relaunch the script and have launch_cvd work even if it had
been run in the past both with or without errors.
This is especially useful for local experiments.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
After copying the newly built mesa libraries to the Android vendor
partition the services have to be restarted to make sure that the new
libraries are used, in particular by Surfaceflinger, so call 'stop'
and 'start' for that.
Also print the GLES implementation used by Surfaceflinger before and
after restarting, to be able to compare and check that the new libraries
are effectively picked up.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
For the current tests the DRI drivers do not seem to be needed on
Android, so remove the ANDROID_DRIVER variable and stop copying the DRI
libraries to the guest file system, only copy libgallium_dri.so which is
really needed.
While at it, also copy libGLESv1_CM.so and libGLESv2.so
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
In order to build the mesa llvmpipe driver for Android, llvm libraries
for Android are needed; force meson to use a fallback for llvm and
provide prebuilt libraries making them available to meson thanks to
a wrap file.
Enable other software drivers for the x86_64-linux-android build, but
also disable the amd and intel hardware drivers for now which don't
build correctly on Android.
Download the prebuilt LLVM libraries for Android in the container job
rather than in the build job, to avoid downloading some big artifacts
for every mesa version built for Android.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
In commit ad42be50c9 (ci/deqp: fully isolate deqp builds, 2024-03-17)
the deqp module version file was changed from something like
`/deqp/version-$deqp_api` to something like `/deqp-$deqp_api/version`
and this seems to cause some issues when building for Android:
-----------------------------------------------------------------------
[439/1301] Building CXX object external/amber/src/src/vulkan/CMakeFiles/libamberenginevulkan.dir/blas.cc.o
FAILED: external/amber/src/src/vulkan/CMakeFiles/libamberenginevulkan.dir/blas.cc.o
/tmp/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ --target=x86_64-none-linux-android28 --sysroot=/tmp/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DAMBER_CTS_VULKAN_HEADER=1 -DAMBER_ENABLE_CLSPV=0 -DAMBER_ENABLE_DXC=0 -DAMBER_ENABLE_LODEPNG=1 -DAMBER_ENABLE_RTTI=1 -DAMBER_ENABLE_SHADERC=0 -DAMBER_ENABLE_SPIRV_TOOLS=0 -DAMBER_ENGINE_DAWN=0 -DAMBER_ENGINE_VULKAN=1 -DDE_ANDROID_API=28 -DDE_ASSERT_FAILURE_CALLBACK -DDE_COMPILER=DE_COMPILER_CLANG -DDE_CPU=DE_CPU_X86_64 -DDE_FENV_ACCESS_ON="_Pragma(\"STDC FENV_ACCESS ON\")" -DDE_MINGW=0 -DDE_OS=DE_OS_ANDROID -DDE_PTR_SIZE=8 -I/tmp/VK-GL-CTS/external/amber/../vulkancts/framework/vulkan/generated/vulkan -I/tmp/VK-GL-CTS/framework/delibs/debase -I/tmp/VK-GL-CTS/framework/delibs/decpp -I/tmp/VK-GL-CTS/framework/delibs/depool -I/tmp/VK-GL-CTS/framework/delibs/dethread -I/tmp/VK-GL-CTS/framework/delibs/deutil -I/tmp/VK-GL-CTS/framework/delibs/destream -I/tmp/VK-GL-CTS/execserver -I/tmp/VK-GL-CTS/external/spirv-headers/src/include -I/tmp/VK-GL-CTS/external/amber/src/include -I/tmp/VK-GL-CTS/external/amber/src -I/tmp/deqp-vk -I/tmp/deqp-vk/external/vulkancts/framework/vulkan/generated/vulkan -Wall -Wextra -Wno-long-long -Wshadow -Wundef -Wconversion -Wno-sign-conversion -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -frtti -fexceptions -fwrapv -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -std=gnu++17 -fPIC -w -fno-exceptions -fvisibility=hidden -Wall -Wextra -Wno-cast-function-type-strict -Wno-padded -Wno-switch-enum -Wno-unknown-pragmas -Wno-unsafe-buffer-usage -pedantic-errors -Werror -Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-format-pedantic -Wno-unknown-warning-option -Weverything -Wno-zero-as-null-pointer-constant -MD -MT external/amber/src/src/vulkan/CMakeFiles/libamberenginevulkan.dir/blas.cc.o -MF external/amber/src/src/vulkan/CMakeFiles/libamberenginevulkan.dir/blas.cc.o.d -o external/amber/src/src/vulkan/CMakeFiles/libamberenginevulkan.dir/blas.cc.o -c /tmp/VK-GL-CTS/external/amber/src/src/vulkan/blas.cc
In file included from /tmp/VK-GL-CTS/external/amber/src/src/vulkan/blas.cc:16:
In file included from /tmp/VK-GL-CTS/external/amber/src/src/vulkan/blas.h:19:
In file included from /tmp/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/vector:274:
In file included from /tmp/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/__bit_reference:14:
In file included from /tmp/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/bit:57:
In file included from /tmp/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/limits:105:
In file included from /tmp/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/type_traits:417:
In file included from /tmp/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/cstddef:37:
/tmp/deqp-vk/version:1:1: error: unknown type name 'dEQP'
dEQP base version vulkan-cts-1.3.10.0
^
-----------------------------------------------------------------------
This seems to be caused by `cstddef` from the Android NDK which includes
a file named `version`, i.e.:
-----------------------------------------------------------------------
#include <version>
-----------------------------------------------------------------------
To avoid the problem also add the deqp module name to the file name,
like: `/deqp-$deqp_api/deqp-$deqp_api-version`.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
Collect the mustpass lists also for the Android target, in particular
this makes vk-main.txt to be found again when running vulkan tests on
Android.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
The main branch of dEQP build fine for android but the GL and GLES
branches fail with errors like the following:
-----------------------------------------------------------------------
.../external/openglcts/modules/common/glcFramebufferBlitTests.cpp:715:26: error: no member named 'floor' in namespace 'std'
return (GLubyte)std::floor(f == 1.0f ? 255 : f * 255.0);
~~~~~^
.../external/openglcts/modules/common/glcFramebufferBlitTests.cpp:1491:14: error: no member named 'fabs' in namespace 'std'
if (std::fabs(actual - expected) > eps)
~~~~~^
2 errors generated.
-----------------------------------------------------------------------
Cherry-pick a couple of commit from main that fix the errors.
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
When adding patches wit git-am create the date format for
GIT_COMMITTER_DATE in a locale-agnostic format, this avoids potential
issues when running the scripts locally with a non-English locale, e.g.:
-----------------------------------------------------------------------
++ GIT_COMMITTER_DATE='gio 1 gen 1970, 01:00:00, CET'
++ git am
fatal: invalid date format: gio 1 gen 1970, 01:00:00, CET
-----------------------------------------------------------------------
Reviewed-by: Eric Engestrom <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30895>
