Compare commits

..

175 Commits

Author SHA1 Message Date
Eric Engestrom
1fbdd37d4c VERSION: bump for 23.3.0 2023-11-29 19:30:57 +00:00
Rhys Perry
421a8aaff1 ac/nir: fix partial mesh shader output writes on GFX11
Fixes dEQP-VK.mesh_shader.ext.smoke.monolithic.mesh_shader_triangle with
nir_opt_combine_stores disabled.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: 240e16fc8e ("ac/nir/ngg: Use attribute ring for mesh shader params.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25530>
(cherry picked from commit 2d98236dd5)
2023-11-28 13:04:20 +00:00
Mary Guillemard
dced0dba02 venus: Do not submit batch manually when no feedback is required
This fixes hangs with Zink on piglit spec@arb_sparse_buffer tests caused by the double submission.

Fixes: a55d26b566 ("venus: add back sparse binding support")

Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26375>
(cherry picked from commit f59665bb62)
2023-11-28 13:04:19 +00:00
Jesse Natalie
3ff4768692 d3d12: Fix multidimensional array ordering
Apparently my C multimensional array syntax was rusty.

Fixes: a6740ee7 ("d3d12: Fix indexing of local_reference_state")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26362>
(cherry picked from commit 1924cdc289)
2023-11-28 13:04:17 +00:00
Eric Engestrom
54f93b1788 intel/ci: fix .hasvk-manual-rules
Fixes: 570acf5655 ("ci: Add a manual full and 1/10th hasvk CTS runs.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26259>
(cherry picked from commit cf510e38a5)
2023-11-28 13:03:57 +00:00
Eric Engestrom
2525258917 intel/perf: fix regex escaping
`\$` is interpreted before being passed to `re.search()`, but luckily
for us the escape is also invalid and because of that, python 3.12+
warns us about it.

Use a raw string instead, so that the `\` is passed untouched to
`re.search()`.

Fixes: aa04b47c6e ("intel/perf: add support for GtSlice/GtSliceXDualsubsliceY variables")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26355>
(cherry picked from commit 1942073112)
2023-11-28 13:03:56 +00:00
Marek Olšák
94ad18d1e7 nir: fix gathering TESS_LEVEL_INNER/OUTER usage with lowered IO
Those varyings shouldn't flag patch_inputs_read/patch_outputs_written.

Fixes: 10be706778 - nir: gather indirect info from lowered IO intrinsics

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26275>
(cherry picked from commit 7a9b73fcb8)
2023-11-28 13:03:52 +00:00
Pierre-Eric Pelloux-Prayer
3ed7b95a55 radeonsi: check sctx->tess_rings is valid before using it
Fixes: c89ca3b47f ("radeonsi: change si_emit_derived_tess_state into a state atom")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10015

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26190>
(cherry picked from commit 945288ffae)
2023-11-28 13:03:48 +00:00
Eric Engestrom
dd56f591e5 .pick_status.json: Update to f843b14c17 2023-11-28 11:22:43 +00:00
Eric Engestrom
c03abd2d9f VERSION: bump for 23.3.0-rc5 2023-11-24 19:40:33 +00:00
Pierre-Eric Pelloux-Prayer
53bafbe4fd mesa: restore call to _mesa_set_varying_vp_inputs from set_vertex_processing_mode
Otherwise ctx->VertexProgram._VaryingInputs might not be up to date.

We can't do this in update_program because this breaks vbo_save_playback_vertex_list_gallium:

  const GLbitfield enabled = node->enabled_attribs[mode];
  _mesa_set_varying_vp_inputs(ctx, enabled);              <-- update _VaryingInputs

  if (ctx->NewState)
     _mesa_update_state(ctx);                             <-- calls update_program, reverting the
                                                              change made above

Fixes: c97961a855 ("mesa: fix 38% decrease in display list performance of Viewperf2020/NX8_StudioAA")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9441
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25956>
(cherry picked from commit 64352ae8e4)
2023-11-24 17:35:27 +00:00
Robert Mader
b52742c018 v3d/resource: Support offset query for multi-planar planes
This is required in order to return the correct value for
`gbm_dri_bo_get_offset()` for e.g. the second plane of a NV12 image.

Use the newly introduced `util_resource` helper and, while on it, also
add support for `gbm_bo_get_plane_count()`.

Cc: mesa-stable
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26283>
(cherry picked from commit 2404483706)
2023-11-24 17:15:25 +00:00
Robert Mader
090160db3a vc4/resource: Support offset query for multi-planar planes
This is required in order to return the correct value for
`gbm_dri_bo_get_offset()` for e.g. the second plane of a NV12 image.

Use the newly introduced `util_resource` helper and, while on it, also
add support for `gbm_bo_get_plane_count()`.

Cc: mesa-stable
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26283>
(cherry picked from commit cb8cdab928)
2023-11-24 17:15:24 +00:00
Samuel Pitoiset
5e1fe5d0fd radv: do not set OREO_MODE to fix rare corruption on GFX11
Ported from RadeonSI 3f108e7615.
Seems to be a recommendation from the AMD hw team.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26333>
(cherry picked from commit ad7efdea6e)
2023-11-24 17:15:23 +00:00
Samuel Pitoiset
2d4868629f radv: set radv_invariant_geom=true for War Thunder
War Thunder has native Vulkan support.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10186
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26334>
(cherry picked from commit d9f312b86a)
2023-11-24 17:15:22 +00:00
Samuel Pitoiset
c73c7c0b0b radv: disable TC-compatible HTILE on Tonga and Iceland
According to RadeonSI, TC-compat HTILE have issues on Tonga/Iceland
(first GFX8 chips) and a bunch of games seem to have issues.

Let's disable it instead of using a feature that is known broken.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7101
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3894
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26204>
(cherry picked from commit ab34603115)
2023-11-24 17:15:21 +00:00
Iván Briano
ff6ed814f7 anv: use the right vertexOffset on CmdDrawMultiIndexed
Fixes: c70ef757e6 ("anv: Use extended parameters on Gen11+")

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26327>
(cherry picked from commit 43cb4cb6dd)
2023-11-24 17:15:20 +00:00
David Rosca
142c333748 util/rbsp: Fill bits twice if reading more than 16 bits
vl_rbsp_fillbits may fill less than 32 bits if it removes emulation
prevention bytes, but will fill at least 16 bits. We need to call it
twice when reading more than 16 bits.

This fixes parsing H264 SPS packed header in va frontend when emulation
prevention bytes are at position where 32 bit values are read.

Cc: mesa-stable

Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26276>
(cherry picked from commit 73d69ef1e6)
2023-11-24 17:15:18 +00:00
Eric Engestrom
c05aadcf25 .pick_status.json: Update to b07a58157d 2023-11-24 17:15:13 +00:00
Connor Abbott
ac66a49ba6 tu: Fix re-emitting VS param state after it is re-enabled
We need to always re-emit it if it was disabled. Fixes vertex/instance
offset in a direct draw after an indirect draw.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26148>
(cherry picked from commit 6be6b4ca71)
2023-11-22 08:46:26 +00:00
Georg Lehmann
ab01f44513 aco/sched: treat p_dual_src_export_gfx11 like export
This prevents the scheduler from moving the dual source export above mrtz
export, which caused hangs.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10173

Cc: mesa-stable
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26317>
(cherry picked from commit 0a5d3ac8d2)
2023-11-22 08:42:53 +00:00
Tapani Pälli
0f94431922 drirc: Set limit_trig_input_range option for Valheim
Fixes ocean rendering in Valheim.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10174
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26295>
(cherry picked from commit aba00ff391)
2023-11-22 08:42:41 +00:00
Lionel Landwerlin
4e9ef47839 intel/perf: fix querying of configurations
Using the unsized data field is incorrect. The data is located behind
the entire drm_i915_query_perf_config structure.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26285>
(cherry picked from commit f9bab3566b)
2023-11-22 08:42:40 +00:00
Erico Nunes
16361af817 v3dv: Rework to remove drm authentication for wsi
For Wayland wsi allocations, v3dv used the wl_drm protocol, which is now
being phased out in favor of dmabuf feedback.
wl_drm is used to figure out the display device (in v3dv assumed to be
vc4) and then to authenticate with the Wayland compositor in order to
allocate scanout-able buffers (in this case, dumb buffers) directly at
the display device.
Recent commit 88c03ddd34 changed the behavior of the wsi code, and
wl_drm is now passing the render device instead, which broke Wayland
wsi.
It turns out that the authentication code is not really needed and since
we would like to remove wl_drm usage and the master device is assumed to
be vc4 anyway, we can just remove some unneeded device-specific wsi code
and get Vulkan Wayland wsi back to work.

Fixes: 88c03ddd34 ("egl/drm: get compatible render-only device fd for kms-only device")

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26200>
(cherry picked from commit 898700ca64)
2023-11-22 08:42:37 +00:00
Eric Engestrom
1a19815165 .pick_status.json: Update to f39ed0063b 2023-11-22 08:39:48 +00:00
Matt Turner
0b3f2e02fb util: Include stdint.h in libdrm.h
drmGetDevices2()'s first parameter is a uint32_t.

Fixes: e05abb1345 ("util: Add a simple no-op libdrm shim")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26265>
(cherry picked from commit 5022a26b50)
2023-11-18 21:22:33 +00:00
Simon Ser
3db740ff6c egl: ensure a render node is passed to _eglFindDevice()
_eglFindDevice() will fail if it's not provided a render node:
the EGLDevice list only contains one entry per render node, plus
the special software device. Passing a primary node for a
display-only device will not work.

Signed-off-by: Simon Ser <contact@emersion.fr>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10142
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
Tested-by: Iago Toral Quiroga <itoral@igalia.com>
Tested-by: Alejandro Piñeiro <apinheiro@igalia.com>
Fixes: 2be404f557 ("egl: error out if we can't find an EGLDevice in _eglFindDevice()")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26205>
(cherry picked from commit 0f978c3441)
2023-11-18 21:20:39 +00:00
Simon Ser
62455a9a25 egl: move dri2_setup_device() after dri2_setup_extensions()
dri2_setup_device() will depend on the extensions being set up in
the next commit.

None of the code in-between depends on disp->Device AFAIU.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
Tested-by: Iago Toral Quiroga <itoral@igalia.com>
Tested-by: Alejandro Piñeiro <apinheiro@igalia.com>
Backport-to: 23.3
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26205>
(cherry picked from commit 9b70096f5e)
2023-11-18 21:20:38 +00:00
Simon Ser
ef07bc116e egl: extract EGLDevice setup in dedicated function
Extract the logic responsible for populating disp->Device via
_eglFindDevice(). This isn't much for now but will grow in a
following commit.

No functional changes.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
Tested-by: Iago Toral Quiroga <itoral@igalia.com>
Tested-by: Alejandro Piñeiro <apinheiro@igalia.com>
Backport-to: 23.3
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26205>
(cherry picked from commit b4ec0b51ce)
2023-11-18 21:20:37 +00:00
Eric Engestrom
f62a7df6d1 egl/dri2: increase NUM_ATTRIBS to fit all the attributes
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26234>
(cherry picked from commit f011410c1c)
2023-11-18 21:18:50 +00:00
Rhys Perry
a593181819 radv: enable radv_disable_trunc_coord for vkd3d-proton/DXVK
This fixes diagonal SSAO artifacts in some games.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9253
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6395
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25911>
(cherry picked from commit 9e9eb87dd3)
2023-11-18 21:18:45 +00:00
Rhys Perry
0ac2529959 radv: add radv_disable_trunc_coord option
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25911>
(cherry picked from commit 4d2a3b9573)
2023-11-18 21:18:43 +00:00
Lionel Landwerlin
64dee1015c anv: fix dirty state tracking for 3DSTATE_PUSH_CONSTANT_ALLOC
ANV_CMD_DIRTY_PIPELINE also includes reprogramming of
3DSTATE_PUSH_CONSTANT_ALLOC_* instructions.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 50f6903bd9 ("anv: add new low level emission & dirty state tracking")
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26247>
(cherry picked from commit 9689607f10)
2023-11-18 21:18:42 +00:00
Lionel Landwerlin
5f12ed0774 anv: fix CC_VIEWPORT pointer dirty after blorp/simple-shaders
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 50f6903bd9 ("anv: add new low level emission & dirty state tracking")
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26247>
(cherry picked from commit 1257d08bcb)
2023-11-18 21:18:41 +00:00
Boris Brezillon
c972359d41 panfrost: Flag the right shader when updating images
Fixes: fab1fabb4f ("panfrost: Dirty track fragment images")
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26221>
(cherry picked from commit 8ed471a339)
2023-11-18 21:18:23 +00:00
Lionel Landwerlin
317998e30f intel/nir/rt: fix reportIntersection() hitT handling
We're currently updating the hitT value in the traversal result with
the hitT value from reportIntersection(), but this is not correct.

First the hitT value of reportIntersection() should update the
gl_RayTmaxEXT value (maps to brw_nir_rt_mem_ray_defs::t_far).

Second the hitT determined by traversal should only be updated if the
reportIntersection() hitT value has updated the gl_RayTmaxEXT and that
the new gl_RayTmaxEXT is smaller than the determined hitT value from
traversal.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Fixes: 303378e1dd ("intel/rt: Add lowering for combined intersection/any-hit shaders")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25146>
(cherry picked from commit 4eb4197d27)
2023-11-18 21:18:21 +00:00
Lionel Landwerlin
f80e96eb26 intel/fs: rerun divergence analysis prior to convert_from_ssa
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9964
Cc: mesa-stable
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26235>
(cherry picked from commit 6dbb5f1e07)
2023-11-18 21:18:20 +00:00
José Expósito
402553152f zink: allow software rendering only if selected
In environments where 3D acceleration is not available, like in a VM,
the behavior before commit 8cd44b8843 ("egl: add automatic zink
fallback loading between hw and sw drivers") was to fallback to swrast.

This was the output of `eglinfo` in that situation:

    $ eglinfo
    [...]
    Wayland platform:
    EGL driver name: swrast
    OpenGL core profile renderer: llvmpipe (LLVM 17.0.4, 256 bits)

However, after commit 8cd44b8843 ("egl: add automatic zink fallback
loading between hw and sw drivers") Zink support is tested before
falling back to swrast.
Since the system doesn't support 3D acceleration, Zink + software
rendering is used instead of swrast causing issues like the ones
described in #10146.

In this case, `eglinfo` prints:

    $ eglinfo
    [...]
    Wayland platform:
    EGL driver name: zink
    OpenGL core profile renderer: zink Vulkan 1.3(llvmpipe (LLVM 17.0.4,
    256 bits) (MESA_LLVMPIPE))

This patch ensures that Zink + software rendering is used only when the
user opts-in by setting `LIBGL_ALWAYS_SOFTWARE` or `D3D_ALWAYS_SOFTWARE`
and swrast is used otherwise.

After the patch, the output of `eglinfo` is identical to the one before
the regression:

    $ eglinfo
    [...]
    Wayland platform:
    EGL driver name: swrast
    OpenGL core profile renderer: llvmpipe (LLVM 17.0.4, 256 bits)

Resolves: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10146
Fixes: 8cd44b8843 ("egl: add automatic zink fallback loading between
hw and sw drivers")

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Signed-off-by: José Expósito <jexposit@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26220>
(cherry picked from commit 2a71f06f29)
2023-11-18 21:18:19 +00:00
Robert Mader
6e218edda5 panfrost: Support parameter queries for main planes
In order to return correct offsets, strides and possibly other
parameters. This is relevant for formats like NV12 and P010 where
the second plane, when produced by the V4L2 decoder, uses the
same FD like the first one, but with an offset.

Analogous to 7a7e577d.

Also use the new helper to get the number of planes.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10094
Cc: mesa-stable
Reviewed-by: Italo Nicola <italonicola@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26109>
(cherry picked from commit 5f4253e096)
2023-11-18 21:18:18 +00:00
Robert Mader
5542b17f72 util: Add new helpers for pipe resources
They will be handy for drivers supporting multi-planar formats.

Cc: mesa-stable
Reviewed-by: Italo Nicola <italonicola@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26109>
(cherry picked from commit d679154dc0)
2023-11-18 21:18:01 +00:00
Tatsuyuki Ishi
1a238bece9 zink: Fix waiting for texture commit semaphores.
The commit was always being treated as failed...

Fixes: d1456a6b0a ("zink: add semaphore handling for sparse binds")
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26171>
(cherry picked from commit 94e867e4a6)
2023-11-18 21:15:16 +00:00
Tatsuyuki Ishi
c573f2437d zink: Fix missing sparse buffer bind synchronization.
goto oopsies.

Fixes: d1456a6b0a ("zink: add semaphore handling for sparse binds")
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26171>
(cherry picked from commit 3ee283e455)
2023-11-18 21:15:16 +00:00
José Expósito
61bd4b332b zink: fix dereference before NULL check
The `sv->image_view` pointer is dereference before checking whether it's
NULL or not.
Check for NULL before dereferencing it to avoid a possible crash.

Fixes: 9de455bc43 ("zink: check for sampler view existence during
zink_rebind_all_images()")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: José Expósito <jexposit@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26140>
(cherry picked from commit 83ed419cd0)
2023-11-18 21:15:15 +00:00
Samuel Pitoiset
e8c975a642 radv: add a missing async compute workaround for Tonga/Iceland
After digging into PAL code again, I figured that Tonga/Iceland are
both affected by a hw bug related to async compute dispatches.

The solution is to change the "threadgroup" dimension mode to the
"thread" dimension mode unconditionally.

This should fix a bunch of issues related to RADV_DEBUG=nocompute on
these GPUs.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7551
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6334
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4679
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26207>
(cherry picked from commit ec82b42944)
2023-11-18 21:15:13 +00:00
Samuel Pitoiset
dd46e0b6d3 radv: fix a descriptor leak with debug names and host base descriptor set
vk_object_base_finish() needs to be called on the descriptor set in
order to free the debug names.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10098
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26088>
(cherry picked from commit 4239e13ff6)
2023-11-18 21:15:12 +00:00
Chia-I Wu
5952462ecf radv: fix image view extent override for astc
When doing ASTC decoding, the image has format VK_FORMAT_ASTC_*, the
internal plane 1 has format VK_FORMAT_R8G8B8A8_UNORM, and the view has
format VK_FORMAT_R8G8B8A8_UINT.  It does not need the override for
compressed formats.

Fixes: f97b449e9e ("radv: integrate meta astc compute decoder to radv")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26106>
(cherry picked from commit 2533d0a0e2)
2023-11-18 21:15:10 +00:00
Tapani Pälli
0ecb0ec428 anv/drirc: add option to disable FCV optimization
There are rendering issues with FCV on DG2 and Unreal engine 5.1,
patch adds option to disable fcv in drirc.

Cc: mesa-stable
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26169>
(cherry picked from commit 01046cd6ad)
2023-11-18 21:12:05 +00:00
Eric Engestrom
4c0233497b .pick_status.json: Update to f05688aa32 2023-11-18 21:10:57 +00:00
Eric Engestrom
4611aef8a8 VERSION: bump for 23.3.0-rc4 2023-11-15 21:36:49 +00:00
Konstantin Seurer
313457b66e radv/sqtt: Fix tracing acceleration structure commands
ApiRayTracingSeparateCompiled can only used by trace commands.

Fixes: bfb55d0 ("ac/sqtt,radv/sqtt: Add and use marker for separate RT compilation")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26158>
(cherry picked from commit b6e09dd521)
2023-11-15 21:21:24 +00:00
David Heidelberg
f12636cecf ci/traces: drop the freedoom-phase2-gl-high.trace
See https://gitlab.freedesktop.org/mesa/mesa/-/issues/8080#note_2154467

Fixes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8080
Cc: mesa-stable
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26199>
(cherry picked from commit 1eff68dd2a)
2023-11-15 21:21:24 +00:00
Georg Lehmann
178c046b9e aco: add missing scc def for SALU quad broadcast
Cc: mesa-stable

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26163>
(cherry picked from commit 509ce19643)
2023-11-15 21:21:24 +00:00
Samuel Pitoiset
a600a58f89 radv: set radv_zero_vram=true for Unreal Engine 4/5
Unreal Engine seems to rely on uninitialized memory and
RADV_DEBUG=zerovram fixes a bunch of issues.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9025
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9380
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9026
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26188>
(cherry picked from commit bb92c34c28)
2023-11-15 21:21:24 +00:00
Samuel Pitoiset
0960a0c5dd radv: fix registering queues for RGP with compute only
This crashes if the graphics queue isn't created.

Fixes: 930e77e903 ("radv/sqtt: add support for queue info")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10136
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26183>
(cherry picked from commit 627d593443)
2023-11-15 21:21:24 +00:00
Matt Turner
dbb2f6ed66 r600: Add missing dep on git_sha1.h
Bug: https://bugs.gentoo.org/917116
Fixes: 3ab51c7ebd ("r600: Add callbacks for get_driver_uuid and get_device_uuid")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26195>
(cherry picked from commit b66b299eda)
2023-11-15 21:21:24 +00:00
David Rosca
0680995b18 gallium/auxiliary/vl: Scale dst_rect x0/y0 when rendering chroma plane
This fixes incorrect chroma plane position when x0/y0 is not zero.

Fixes: 001358a97c ("vl/compositor: add a new function for YUV deint")

Acked-by: Thong Thai <thong.thai@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26123>
(cherry picked from commit 35b0ccd855)
2023-11-15 21:21:24 +00:00
David Rosca
b1833834dd gallium/auxiliary: Fix coordinates clamp in util_compute_blit
Fixes: 7c8e1596d6 ("gallium/auxiliary: Fix util_compute_blit half texel offset with scaling")

Acked-by: Thong Thai <thong.thai@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26123>
(cherry picked from commit e9091b1f5c)
2023-11-15 21:21:24 +00:00
David Rosca
73b6b7497e gallium/auxiliary/vl: Fix coordinates clamp in compute shaders
Fixes: a6a43963ed ("gallium/auxiliary/vl: Clamp coordinates in compute shaders")

Acked-by: Thong Thai <thong.thai@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26123>
(cherry picked from commit ef0546152f)
2023-11-15 21:21:24 +00:00
Jesse Natalie
f56bb3ec4b nir_lower_mem_access_bit_sizes: Fix write-mask-constrained 3-byte stores as atomics
The code here handled stores of actual 3-byte values (8-bit, 3-component), but didn't
correctly handle stores of larger 8-bit vectors that were constrained by write mask to
just 3 bytes. In that case, the pad-to-vec4 step was unnecessary and problematic.

Seen in CL CTS test_basic vector_swizzle test group for char3 with CLOn12.

Fixes: c70d94a8 ("nir_lower_mem_access_bit_sizes: Support unaligned stores via a pair of atomics")
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26034>
(cherry picked from commit cd0cff951a)
2023-11-15 21:21:24 +00:00
Rhys Perry
85c0b6281c docs: fix RADV_THREAD_TRACE_CACHE_COUNTERS default
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: dbab98d9d6 ("radv: enable RADV_THREAD_TRACE_CACHE_COUNTERS by default")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26193>
(cherry picked from commit c4ee639ca0)
2023-11-15 21:21:24 +00:00
Sagar Ghuge
2b38d1451e anv: Flush data cache while clearing depth using HIZ_CCS_WT
Patch fxes ESO shadow pass ground corruption on Arc A750. In the colour
pass where the rendering corruption first appears, the depth resource
was used as a "PS - Texture".  Immediately afterwards there's a Barrier
where it goes from

	VK_IMAGE_LAYOUT_DEPTH_STENCIL_READ_ONLY_OPTIMAL =>
	VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL

immediately following that there's a Clear from vkCmdBeginRendering
which appears to be a HiZ clear.  Things work when using AUX_USAGE_HIZ
but AUX_USAGE_HIZ_CCS_WT (XXX: and AUX_USAGE_HIZ_CCS?) doesn't work.

current thinking is this is related to 14015264727 where we had to add
HDC and DC flushes to CCS and MCS fast clears.  Maybe HiZ clears with
CCS also have similar problems?  The docs don't appear to indicate that
but the docs were also wrong for color clears until recently...

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9277
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9444
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22717>
(cherry picked from commit 648c0c159d)
2023-11-15 16:37:16 +00:00
Tapani Pälli
e194c6b9ba iris: add data cache flush for pre hiz op
This fixes various failing Piglit tests on DG2.

Cc: mesa-stable
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22717>
(cherry picked from commit e4db19afa7)
2023-11-15 16:37:15 +00:00
Sagar Ghuge
bbf099223e anv: Write timestamp using MI_FLUSH_DW on blitter
On Blitter engine, we don't support PIPE_CONTROL, we have to update
memory locations using the MI_FLUSH_DW command.

v2:
- Handle video queue (Lionel)

Fixes: 056b0cb87f ("anv: add video engine support in various places")
Fixes: 5112b42146 ("anv: Handle end of pipe with MI_FLUSH_DW on transfer queue")

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26121>
(cherry picked from commit 8c9a7f7730)
2023-11-15 16:36:24 +00:00
Eric Engestrom
828b3966fa .pick_status.json: Mark fcfa68a632 as denominated 2023-11-15 16:34:18 +00:00
Friedrich Vock
7cf359fb57 vulkan: Don't use set_foreach_remove when destroying pipeline caches
set_foreach_remove assumes no entries have been removed. That assumption
only holds if no errors occur, since pipeline cache objects can get
removed if an error occurs during deserialization.

This fixes
dEQP-VK.api.device_init.create_instance_device_intentional_alloc_fail.basic
crashing on RADV.

Cc: mesa-stable
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26164>
(cherry picked from commit 8ffdad731c)
2023-11-15 16:33:12 +00:00
Eric Engestrom
16ed2f180c .pick_status.json: Update to 03a7cb2618 2023-11-15 16:32:14 +00:00
Mary Guillemard
eb9f7db980 zink: Check for VK_EXT_extended_dynamic_state3 before setting A2C
Fix crashes for piglit alpha-to-coverage tests when
VK_EXT_extended_dynamic_state3 is not supported (like on Venus)

Fixes: 736577871b ("zink: check for cbuf0 writes before setting A2C")

Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26003>
(cherry picked from commit fe159c85de)
2023-11-13 20:05:05 +00:00
Rob Clark
3902e9302d freedreno: Always attach bo to submit
Even if app triggers undefined behaviour by using a rsc on multiple
contexts without a flush, we still should attach the bo to the batch.

Fixes: b43e5aec0d ("freedreno/batch: Move submit bo tracking to batch")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26162>
(cherry picked from commit 4807c1e173)
2023-11-13 20:05:01 +00:00
Marek Olšák
12ab5dfd52 radeonsi/gfx11: don't set OREO_MODE to fix rare corruption
This is recommended by hw people.

Fixes: 19a59f05a4 - radeonsi/gfx11: program db render control register

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25941>
(cherry picked from commit 3f108e7615)
2023-11-13 20:05:00 +00:00
antonino
a28ea69d0a vulkan: use instance allocator for object_name in some objects
The allocator passed to VkDevice won't be available once it is destroyed
and thefore it cannot be used to allocate `object_name` for instance
level objects such as `VkInstance` or `VkPhysicalDevice` or else there
would be no way of deallocating it when those objects are destroyed.

Cc: mesa-stable
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Mark Collins <mark@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26085>
(cherry picked from commit 2d49f834b2)
2023-11-13 20:04:59 +00:00
Eric Engestrom
c6a9afc57a .pick_status.json: Update to 08f851f436 2023-11-13 20:04:53 +00:00
Roman Stratiienko
81ab4a3a01 v3d: Don't implicitly clear the content of the imported buffer
v3d driver will implicitly clear the buffer's content on the first write
operation. This clearing operation is helpful for allocated buffers,
initializing them with zeros instead of having memory garbage.

Also, this avoids reading the buffer from the RAM to the GPU cache
before rendering, making the first write operation slightly faster.

The clearing operation should not happen for imported buffers where
the buffer may already contain valuable data and the user may want
to render into the buffer only partially.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Signed-off-by: Roman Stratiienko <r.stratiienko@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26136>
(cherry picked from commit 56451ce773)
2023-11-10 14:46:36 +00:00
Connor Abbott
30135c2a89 ir3/ra: Don't swap killed sources for early-clobber destination
We have an optimization to try to swap regular live intervals with
killed sources when evicting them fails in order to make a contiguous
space for the destination to fit in, but this doesn't work when the
destination is early-clobber.

Fixes
dEQP-GLES31.functional.synchronization.inter_invocation.image_atomic_read_write
on a650+.

Fixes: d4b5d2a ("ir3/ra: Use killed sources in register eviction")
Closes: #8886
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26004>
(cherry picked from commit 04ffef15da)
2023-11-10 14:46:20 +00:00
Lionel Landwerlin
193256f9e7 isl: disable MCS compression on R9G9B9E5
Not supported according to the docs and will trigger an assert
isl_get_render_compression_format().

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26112>
(cherry picked from commit d4499c4cb2)
2023-11-10 14:46:19 +00:00
Paulo Zanoni
8d9fa8ef99 vulkan: fix potential memory leak in create_rect_list_pipeline()
I was playing around with possible improvements to STACK_ARRAY(), and
one of my experiments made gcc point us that we were not freeing
'stages'.

Fixes: 514c10344e ("vulkan/meta: Add a concept of rect pipelines")
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26041>
(cherry picked from commit 17e135d3d4)
2023-11-10 14:45:05 +00:00
Eric Engestrom
e59c24f6b8 .pick_status.json: Mark fa7ec4226b as denominated 2023-11-10 14:45:03 +00:00
Eric Engestrom
928e9db008 .pick_status.json: Update to 56451ce773 2023-11-10 14:42:58 +00:00
Eric Engestrom
65109bc8ac VERSION: bump for 23.3.0-rc3 2023-11-08 18:50:58 +00:00
Eric Engestrom
41b7bd398c .pick_status.json: Mark 227300345e as denominated 2023-11-08 18:20:43 +00:00
José Roberto de Souza
9d8e3aac70 anv: Add missing ANV_BO_ALLOC_EXTERNAL flags when calling anv_device_import_bo()
This flag is required to properly calculate the PAT index of the
imported BO.

Cc: mesa-stable
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
(cherry picked from commit 72ba0677f8)
2023-11-08 18:19:30 +00:00
Corentin Noël
b8112ed6db frontends/va: Remove wrong use of ProfileToPipe
The `context->templat.profile` variable is already of enum pipe_video_profile.

Fixes: 0996ec3fc6 ("frontends/va: Add profile param when querying PIPE_VIDEO_CAP_ENC_QUALITY_LEVEL")

Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26067>
(cherry picked from commit f4ed92a823)
2023-11-08 18:19:28 +00:00
Rhys Perry
f30bd0074e radv: disable gs_fast_launch=2 by default
This currently causes severe performance problems in Remnant 2 and
Alan Wake 2.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: 28ebe62af2 ("radv: enable mesh shader gs_fast_launch=2 and multi-row export")
Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10071
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26090>
(cherry picked from commit 2011b678c5)
2023-11-08 18:18:59 +00:00
Yiwei Zhang
765cb46e2d zink: apply can_do_invalid_linear_modifier to Venus
This unblocks Xwayland with zink-on-venus + sommelier wayland proxy.
- For glamor, zink uses linear modifier.
- For Virgl clients, classic 3d resource is used and sommelier fixes
  the modifier and stride infos no matter wl-drm or dma-buf protocol.
- For Venus clients:
  - via the legacy wl-drm protocol, invalid modifier is passed via
    sommelier, and host recovers the tiling in the way dealing with
    modifier unaware clients (e.g. I915_GEM_GET_TILING). For hosts
    unable to recover, they assume linear and venus forces linear on
    legacy path.
  - via the new zwp_linux_dmabuf_feedback_v1 (version 3/4) protocol,
    explicit modifier is used, and zink handles that without issues.

This doesn't deserve a driconfig as zink-on-venus to support xserver
itself already requires special enough integration beyond a config.

Reported-by: Igor Torrente <igor.torrente@collabora.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10066
Fixes: 1c3db3e39a ("zink: blow up broken xservers more reliably")
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26082>
(cherry picked from commit 551f61bdb3)
2023-11-08 11:56:17 +00:00
Eric Engestrom
4d887245b9 .pick_status.json: Update to a77ea9555a 2023-11-08 11:56:14 +00:00
Sviatoslav Peleshko
b419916e7f nir/loop_analyze: Fix inverted condition handling in iterations calculation
In the tagged commit, we stopped actually inverting the condition, and
instead relied on the "invert_cond" flag. But we missed a few places
where this flag should've been handled too.

Also, add a few more tests to make sure this won't regress in the future.

Fixes: 99a7a664 ("nir/loop_analyze: Change invert_cond instead of changing the condition")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10012
Signed-off-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26024>
(cherry picked from commit aa33ca0a52)
2023-11-07 13:30:20 +00:00
Erik Faye-Lund
43540839e7 panfrost: use perf_debug instead of open-coding
We don't need to open-code perf_debug here...

Fixes: bc55d150a9 ("panfrost: Add support for AFBC packing")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26063>
(cherry picked from commit 385b81c8c2)
2023-11-07 13:30:19 +00:00
Alyssa Rosenzweig
ae337176ef agx: Fix fragment side effects scheduling
We can't move discards across side effects, or the side effect might not happen.

Fixes KHR-GLES31.core.shader_image_load_store.basic-allFormats-load-fs
regression. Sigh.

CI is up next.

Fixes: 119e5b9719 ("agx: Schedule for register pressure")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
(cherry picked from commit 7b92c63105)
2023-11-07 13:29:45 +00:00
Mary Guillemard
c531ae3c80 venus: skip bind sparse info when checking for feedback query
Fix an assertion when using vkQueueBindSparse.

Fixes: 7fbf608f2d ("venus: append query feedback at submission time")
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26027>
(cherry picked from commit f71f5cf856)
2023-11-07 13:29:42 +00:00
Vlad Schiller
1357b74292 pvr: Fix VK_EXT_texel_buffer_alignment
In the commit that enabled the extension, I forgot to add the required
properties, which made some tests to fail.

Fixes: 649ebbb0fb ("pvr: Implement VK_EXT_texel_buffer_alignment")
Signed-off-by: Vlad Schiller <vlad-radu.schiller@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26002>
(cherry picked from commit 1dd1c9d610)
2023-11-07 13:28:06 +00:00
Samuel Pitoiset
cf3bd8bedc nir: fix inserting the break instruction for partial loop unrolling
If the break in the original loop isn't in the first top-level if,
this would have re-inserted it in the wrong block.

Fixes this by re-inserting the break block to the corresponding break
block in the new loop by using the remap hashtable.

fossils-db (NAVI21):
Totals from 88 (0.11% of 79330) affected shaders:
Instrs: 109602 -> 109929 (+0.30%); split: -0.10%, +0.40%
CodeSize: 570968 -> 573332 (+0.41%); split: -0.08%, +0.49%
Latency: 1682510 -> 1682505 (-0.00%); split: -0.01%, +0.01%
Copies: 12832 -> 12746 (-0.67%); split: -1.54%, +0.87%
Branches: 2879 -> 2930 (+1.77%)

Deathloop and F1 2023 are affected but I'm not aware of any issues
for these two games.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10001
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26009>
(cherry picked from commit abfd208cb0)
2023-11-07 13:28:05 +00:00
Eric Engestrom
45368572c8 .pick_status.json: Update to aa33ca0a52 2023-11-07 13:27:57 +00:00
Karol Herbst
b328f01942 rusticl/queue: fix implicit flushing of queue dependencies
Needed by Davinci Resolve.

There are two parts to this fix:
1. flush dependencies also on flush, not just finish
2. move the dependency checking logic into Queue::flush as otherwise we
   miss required implicit flushes.

Fixes: 8616c0a52c ("rusticl/event: flush queues from dependencies")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: @LingMan <18294-LingMan@users.noreply.gitlab.freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26053>
(cherry picked from commit 8cbb84dc42)
2023-11-06 09:37:49 +00:00
Karol Herbst
984d8051ef rusticl/queue: do not send empty lists of event to worker queue
This saves us a few CPU cycles and makes properly fixing implicit flushes
less expensive.

Fixes: 8616c0a52c ("rusticl/event: flush queues from dependencies")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: @LingMan <18294-LingMan@users.noreply.gitlab.freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26053>
(cherry picked from commit 52e41d4c97)
2023-11-06 09:37:48 +00:00
David Rosca
f103f773b6 radeonsi: Fix offset for linear surfaces on GFX < 9
Fixes: 86262b6eac ("radeonsi,radv: fix usages of surf_pitch")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9949
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10073

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25970>
(cherry picked from commit bf364cbdb4)
2023-11-06 09:37:47 +00:00
Mauro Rossi
3294d74e20 Android.mk: disable android-libbacktrace to build with Android 14
Android libbacktrace is not available in Android 14

Fixes the following build error:

FAILED: src/util/libmesa_util.a.p/u_debug_stack_android.cpp.o
...
../src/util/u_debug_stack_android.cpp:28:10: fatal error: 'backtrace/Backtrace.h' file not found
         ^~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Cc: mesa-stable
Reviewed-by: Roman Stratiienko <r.stratiienko@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25963>
(cherry picked from commit 95ad0c750c)
2023-11-06 09:37:46 +00:00
Mauro Rossi
1371823a17 Android.mk: filter out cflags to build with Android 14 bundled clang
Android 14 uses prebuild clang version 17.0.2
By filtering these cflags there are no building errors on previous Android releases.

Fixes the following building errors:

../src/c11/time.h:54:8: error: redefinition of 'timespec'
struct timespec
       ^
/media/bigblissdrive/u-x86/out/soong/.intermediates/bionic/libc/libc/android_vendor.34_x86_x86_64_shared/gen/include/bits/timespec.h:46:8: note: previous definition is here
struct timespec {
       ^
1 error generated.

In file included from ../src/util/disk_cache.c:50:
../src/util/disk_cache.h:86:4: error: use of undeclared identifier 'Dl_info'
   Dl_info info;
   ^
...
./src/util/disk_cache.h:114:30: error: incompatible integer to pointer conversion passing 'int' to parameter of type 'const void *' [-Wint-conversion]
      _mesa_sha1_update(ctx, build_id_data(note), build_id_length(note));
                             ^~~~~~~~~~~~~~~~~~~

10 errors generated.

../src/intel/perf/intel_perf.c:91:10: error: call to undeclared function 'major'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
   maj = major(sb.st_rdev);
         ^
../src/intel/perf/intel_perf.c:92:10: error: call to undeclared function 'minor'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
   min = minor(sb.st_rdev);
         ^
2 errors generated.

../src/intel/vulkan/anv_allocator.c:295:13: error: call to undeclared function 'futex_wake'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
            futex_wake(&table->state.end, INT_MAX);
            ^
...
../src/intel/vulkan/anv_allocator.c:711:7: error: call to undeclared function 'futex_wait'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
      futex_wait(&pool->block.end, block.end, NULL);
      ^
6 errors generated.

Cc: mesa-stable
Reviewed-by: Roman Stratiienko <r.stratiienko@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25963>
(cherry picked from commit 520e79a3b5)
2023-11-06 09:37:45 +00:00
Eric Engestrom
6d1b7d90d1 .pick_status.json: Update to 7d6f9ccfbe 2023-11-06 09:37:38 +00:00
Daniel Stone
1c1382b304 ci: Try really hard to print final result string
a630 has been completing jobs, and then corrupting the very last line of
UART output - the one where we pass the overall result back from the DUT
to the job. The bare-metal monitor will wait for this line to appear,
never see it, and then the job times out.

Since this line is the most critical one of all to get out, just spam
the prints to try to make sure they get through.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26032>
(cherry picked from commit 80b87c18d1)
2023-11-06 09:36:38 +00:00
Felix DeGrood
2872c824bc anv: remove CS_FLUSH from query regression
Fixes performance regression introduced by prior refactoring of
pipe control code that unnecessarily added CS_FLUSH to query start
and end. Issue was diagnosed by Ben L (thank you!)

Confirmed this restores performance on:
* Borderlands3 +2%
* Payday +3%
* Factorio +3%
* HogwartsLegacy +4%
* Ghostrunner +7%

Fixes: 6dc95685 (convert genX_query pipe controls to use pc helper)
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25983>
(cherry picked from commit aa23120e4f)
2023-11-06 09:36:38 +00:00
José Expósito
a317466914 zink: Fix crash on zink_create_screen error path
The `zink_internal_create_screen()` function can fail before
`screen->loader_lib` and/or `screen->instance` are initialized.
The `zink_destroy_screen()` doesn't check those cases and crashes.

The error was found by Fedora's CI. The back trace is available at [1].

[1] https://bodhi.fedoraproject.org/updates/FEDORA-2023-c39f82c465

Fixes: 0c2045553f ("zink: use screen destructor for creation fails")
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Signed-off-by: José Expósito <jexposit@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26029>
(cherry picked from commit 8a635e516e)
2023-11-06 09:36:38 +00:00
Yonggang Luo
b03733fc6b nvk: Should use alignment instead of align
align is a function, don't know why align > 0 is working

Fixes: 87686a2220 ("nvk: Rework side-band data upload")

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26035>
(cherry picked from commit 5e9dcffffd)
2023-11-06 09:36:38 +00:00
Eric Engestrom
d0d8b3508d .pick_status.json: Mark 8dda860f83 as denominated 2023-11-06 09:36:38 +00:00
David Rosca
ebbecb9d52 frontends/va: Map decoder and postproc surfaces for reading
If application requests to map surface that was most recently used
as decoder or postproc target and also doesn't explicitly set the
map flags (vaMapBuffer2) it's very likely the intent is to read from
this surface, so we need to map it as such.

This fixes regression on radeonsi where mapping NV12 surfaces for
reading would fail with applications using vaDeriveImage. The reason
for this is that the VA frontend doesn't allow vaDeriveImage for
interlaced surfaces so the applications would use vaGetImage fallback,
but radeonsi doesn't allocate NV12 surfaces as interlaced anymore.

This also fixes mapping other formats surfaces (P010, RGBx, ...)
for reading, which never worked before.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9935
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10048

Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26008>
(cherry picked from commit c638e61ef5)
2023-11-04 14:16:05 +00:00
Rob Clark
7afb7e9a42 freedreno/drm: Fix race in zombie import
The check for the zombie case (racing with final unref of the bo vs
removal from table) must be atomic.

Fixes spec@ext_image_dma_buf_import@ext_image_dma_buf_import-refcount-multithread

Fixes: a192923f99 ("freedreno/drm: Restart import on zombie race")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25995>
(cherry picked from commit 6ac133c646)
2023-11-04 14:16:04 +00:00
antonino
28e36118af nir: don't take the derivative of the array index in nir_lower_tex
Previosuly when lowering to txd for sampler array the index would be
derived as well, therefore the resulting derivative would have been a
vec with one more component than what the txd instruction expects.

This patch truncates the coordinate vector in this case to make sure the
index is not derived.

Fixes: b154a4154b ("nir/lower_tex: rewrite tex/txb -> txd/txl before saturating srcs")
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26012>
(cherry picked from commit 4a627af0e3)
2023-11-04 14:16:02 +00:00
Gert Wollny
35fea70555 r600/sfn: Fixup component count only if intrinsic has it
Fixes: 33d878e
   r600/sfn: Handle load_global in 64 to vec2 lowering

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24229>
(cherry picked from commit 28b79b2ea5)
2023-11-04 14:16:01 +00:00
Gert Wollny
eb41ceec14 r600: Link with libgalliumvl, when enabling rusticl this is needed
Fixes: 33673bcc2a
    rusticl: stop linking with libgalliumvl

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24229>
(cherry picked from commit 1662897294)
2023-11-04 14:15:59 +00:00
Karol Herbst
0afd2930fd nvc0: implement PIPE_CAP_TIMER_RESOLUTION
This allows rusticl to create profiling queues.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10042
Fixes: 660f2eabe1 ("gallium: add PIPE_CAP_TIMER_RESOLUTION")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26016>
(cherry picked from commit 7b41921504)
2023-11-04 14:15:59 +00:00
Connor Abbott
4cbcc74d8f v3d/ci: Remove minetest trace
For the same reason as the previous commit. For examples of commits
causing spurious changes see 8019a1b9 and d89ca14e.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25864>
(cherry picked from commit 2313a517d2)
2023-11-04 14:15:58 +00:00
Connor Abbott
ab2a7f32e6 freedreno/ci: Remove minetest trace
I've repeatedly seen minor pixel changes due to changes that only affect
RA decisions, most recently in !22072. We changed the trace to hopefully
remove a use of texture() in control flow, but it seems there are more,
or the problem is something slightly different like reading
uninitialized values. On the other hand minetest has never actually
caught an issue for me that some other trace hasn't also caught. Just
remove it.

Cc: mesa-stable
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25864>
(cherry picked from commit 16214710ca)
2023-11-04 14:15:57 +00:00
Faith Ekstrand
2cfc7776bd nir: Handle wildcards with casts in copy_prop_vars
If we're propagating a copy from a cast where the copy copies an entire
array, we end up with something like &((S *)ssa_N)->f[*] in the source
where a wildcard has a cast in its parent chain.  If we then try to
propagate the read into a non-wildcard array load, we have to specialize
the wildcard.  This breaks because nir_build_deref_follower() doesn't
handle casts.  Since we know a priori that, because wildcards are only
generated by copy_deref on arrays, we cannot have a cast with a wildcard
parent so simply chasing the source deref to the first wildcard will
ensure that any casts in the deref are handled properly.

Fixes: ba2bd20f87 ("nir: Rework opt_copy_prop_vars to use deref instructions")
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
(cherry picked from commit 15ab4d397f)
2023-11-04 14:15:56 +00:00
Yiwei Zhang
844e330299 venus: fix query feedback batch leak and race upon submission
Summary:
- fixed the combined query batches leak
- fixed the race condition of accessing feedback cmd pool
- very scoped code refactor

Cc: 23.3 <mesa-stable>
Fixes: 5b24ab91e4 ("venus: switch to unconditionally deferred query feedback")
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25988>
(cherry picked from commit ae3b022fa0)
2023-11-04 14:15:56 +00:00
Rob Clark
6f99c62086 tu/virtio: Fix timeline semaphore support
Fixes: f17c5297d7 ("tu: Add virtgpu support")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25981>
(cherry picked from commit ceeab44fd1)
2023-11-04 14:15:55 +00:00
Rob Clark
2bd7e293bf tu/msm: Fix timeline semaphore support
Fixes: daefc6e2a4 ("turnip: prep work for timeline semaphore support")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25981>
(cherry picked from commit 79b907f941)
2023-11-04 14:15:54 +00:00
Rhys Perry
a8e266b7b3 radv: skip radv_remove_varyings for mesh shaders
Fixes compilation of a Talos Principle 2 shader.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: 9fa9782c17 ("radv: stop compiling a noop FS when the application doesn't provide a FS")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25659>
(cherry picked from commit 0a418561da)
2023-11-04 14:15:53 +00:00
Rhys Perry
9e45151243 radv: call lower_array_deref_of_vec before lower_io_arrays_to_elements
nir_lower_io_arrays_to_elements does not support array derefs of vectors,
even when nir_deref_instr_is_known_out_of_bounds is fixed.

They can occur with mesh shaders.

Found by inspection.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25659>
(cherry picked from commit ed12be533e)
2023-11-04 14:15:52 +00:00
Faith Ekstrand
8081cb909b nir: add deref follower builder for casts.
This fixes intel_clc builds with llvm 17 on gfx125_bvh_build_DFS_DFS
where it dies in the lower indirect derefs pass.

Co-authored-by: Dave Airlie <airlied@redhat.com>
Fixes: 4a4e175738 ("nir: Support deref instructions in lower_var_copies")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25536>
(cherry picked from commit 6388896985)
2023-11-04 14:12:29 +00:00
Gert Wollny
11398ee00f r600: Add callbacks for get_driver_uuid and get_device_uuid
v2: Evaluate driver ID dynamically (Adam Jackson)
v3: Align the stars (Triang3l)
v4: include "r600" in driver ID for UUID evaluation (Triang3l)
v5: remove unused local variable

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10040
CC: mesa-stable

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25895>
(cherry picked from commit 3ab51c7ebd)
2023-11-04 14:12:27 +00:00
LingMan
5443e66190 rusticl: Show an error message if the version of bindgen can't be detected
bindgen 0.69.0 broke the `--version` switch, resulting in misleading errors about requiring at
least bindgen 0.62 or about unexpected arguments.

Ideally the build system would fetch the correct bindgen version automatically like cargo does.
Until then, provide a hopefully more helpful error message to the user.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26046>
(cherry picked from commit 1f1ec1c6bc)
2023-11-04 14:11:01 +00:00
LingMan
bfbd6f22c3 rusticl: Show an error message if the build is attempted with an outdated bindgen version
Ideally the build system would fetch the correct bindgen version automatically like cargo does.
Until then, provide an error message that is hopefully more helpful than whatever cryptic error the
build runs into otherwise.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9457
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10029
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25859>
(cherry picked from commit 1cc26e8b66)
2023-11-04 14:10:57 +00:00
Eric Engestrom
364214f1c7 .pick_status.json: Update to 1f1ec1c6bc 2023-11-04 14:09:43 +00:00
Eric Engestrom
c97af690c9 VERSION: bump for 23.3.0-rc2 2023-11-01 17:27:10 +00:00
Samuel Pitoiset
2b66b23045 radv: emit COMPUTE_PIPELINESTAT_ENABLE for CS invocations on ACE
This register seems needed to enable compute shader shader invocations
on GFX7. On GFX8+ it's working fine without emitting this register but
I think it doesn't hurt.

This fixes dEQP-VK.query_pool.statistics_query.*_cq on GFX7.

Fixes: a9945216ba ("radv: fix COMPUTE_SHADER_INVOCATIONS query on compute queue")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25957>
(cherry picked from commit 17daa08dff)
2023-11-01 14:52:55 +00:00
Samuel Pitoiset
18397e51ad radv: fix compute shader invocations query on compute queue on GFX6
Looks like GFX6 always writes the number of compute shader invocations
at offset 0 when used on compute queue.

This fixes dEQP-VK.query_pool.statistics_query.*_cq on GFX6.

Fixes: a9945216ba ("radv: fix COMPUTE_SHADER_INVOCATIONS query on compute queue")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25957>
(cherry picked from commit 9a0a77cb53)
2023-11-01 14:52:54 +00:00
Eric Engestrom
4ac0a87868 .pick_status.json: Update to faed5d647f 2023-11-01 14:52:52 +00:00
Samuel Pitoiset
3cb8275141 radv: bind the non-dynamic graphics state from the pipeline unconditionally
The following sequence is valid (although weird) but many other drivers
(including RADV) were broken:
    - bind pipeline with some static state
    - set state command for that static state (to a bad value)
    - bind the same pipeline again
    - draw

Fixes new dEQP-VK.dynamic_state.*.double_static_bind.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25954>
(cherry picked from commit e4a1bc70dd)
2023-11-01 12:48:16 +00:00
Karol Herbst
14fac5200d rusticl/mesa/screen: do not derefence the entire pipe_screen struct
Apparently the dereferences made tsan report data races on the pipe_screen
object.

Cc: mesa-stable
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25944>
(cherry picked from commit f7830240ac)
2023-11-01 12:48:16 +00:00
Eric Engestrom
4e4cd7065b .pick_status.json: Update to e4a1bc70dd 2023-11-01 12:48:13 +00:00
Juston Li
69e6b0307b venus: switch to unconditionally deferred query feedback
All commands that make queries available have feedback cmds batched
and stored during recording. At submission time, for each batch
(SubmitInfo) these feedback cmds are recorded in a cmd buffer that is
appended after the last original cmd buffer (but before
semaphore/fence feedback).

Query reset cmds are deferred as well and also remove any prior feedback
cmds for queries its resetting within the batch.

Cc: 23.3 <mesa-stable>
Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25413>
(cherry picked from commit 5b24ab91e4)
2023-10-31 14:33:28 +00:00
Juston Li
a33f38d2bf venus: append query feedback at submission time
Cc: 23.3 <mesa-stable>
Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25413>
(cherry picked from commit 7fbf608f2d)
2023-10-31 14:33:16 +00:00
Juston Li
62ea10c0e0 venus: track/recycle appended query feedback cmds
Link the query feedback cmd lifecycle to a cmd in the batch so that when
that last cmd gets reset/freed, we assert its safe to reset the query
feedback cmd. The cmd is then placed on the free list for reuse.

Some edge cases if the the last cmd is simultaneous or gets resubmitted.

Cc: 23.3 <mesa-stable>
Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25413>
(cherry picked from commit d2a626787e)
2023-10-31 14:33:16 +00:00
Juston Li
49aa21e327 venus: support deferred query feedback recording
Add function to alloc a cmd buffer and record all the deferred query
feedback cmds into it at submission time.

Cc: 23.3 <mesa-stable>
Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25413>
(cherry picked from commit 6dcf033dc3)
2023-10-31 14:33:15 +00:00
Juston Li
0bf24452f3 venus: refactor out common cmd feedback functions
defered query feedback cmds have similaries with timeline semaphore
feedback so refactor out some common functions for reuse

Cc: 23.3 <mesa-stable>
Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25413>
(cherry picked from commit 99807b3db6)
2023-10-31 14:33:15 +00:00
Juston Li
aab94295ea venus: add helper function to get cmd handle
rename previous vn_get_cmd_handle as that was getting cmd
handles from the temp storage cmd_handle buffer.

Cc: 23.3 <mesa-stable>
Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25413>
(cherry picked from commit 2ea6f935ce)
2023-10-31 14:33:14 +00:00
Lionel Landwerlin
d282666f96 anv: fix corner case of mutable descriptor pool creation
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 63e91148b7 ("anv: Enable VK_VALVE_mutable_descriptor_type")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10065
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25958>
(cherry picked from commit cdca0b2ce4)
2023-10-31 14:33:13 +00:00
Eric Engestrom
7b81db9f37 .pick_status.json: Update to 4cdd094ae1 2023-10-31 14:33:11 +00:00
Danylo Piliaiev
21798650de tu: Fix reading of stale (V)PC_PRIMITIVE_CNTL_0
PC_PRIMITIVE_CNTL_0 is not set in a draw state and may be changed
by 3d blits, so we have to re-emit it a the start of a renderpass.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9941

Fixes: 7be8d0f7f4
("tu: Use common dirty tracking for PC_PRIMITIVE_CNTL_0")

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25918>
(cherry picked from commit cdbb8b5412)
2023-10-30 15:49:45 +00:00
Samuel Pitoiset
c03932b6e6 radv: fix a synchronization issue with primitives generated query on RDNA1-2
Only RDNA1-2 are affected because RADV needs to handle the legacy vs
NGG path for this query, and the NGG results are stored with 2 extra
64-bit values.

Fixes flakes with
dEQP-VK.transform_feedback.primitives_generated_query.* since VKCTS
1.3.7.0.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25862>
(cherry picked from commit 7d96fe853e)
2023-10-30 15:49:44 +00:00
Iago Toral Quiroga
8a50c841cf v3d,v3dv: fix MMU error from hardware prefetch after ldunifa
ldunifa works exactly the same as ldunif: the hw will prefetch the
next 4 bytes after a read, so if a buffer is exactly a multiple of
a page size and a shader uses ldunifa to read exactly the last 4 bytes
the prefetch will read out of bounds and spam the error on the kernel
log. Avoid that by allocating extra bytes in this scenario.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25752>
(cherry picked from commit 82bef62c17)
2023-10-30 15:48:35 +00:00
Karol Herbst
8986b7f9ae zink: deallocate global_bindings array
Fixes: a6e9e0f0d7 ("zink: add set_global_binding")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25937>
(cherry picked from commit 2f34a1db58)
2023-10-30 15:48:33 +00:00
Karol Herbst
5e690f4097 rusticl/mesa: pass PIPE_BIND_LINEAR in resource_create_texture_from_user
Host pointer allocations are all linear laid out, so just tell the drivers
in case they don't assume this implicitly.

Fixes: 71a9af4910 ("rusticl/mem: support read/write/copy ops for images")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25937>
(cherry picked from commit 7f08036abc)
2023-10-30 15:48:32 +00:00
Marek Olšák
62c4a2273e radeonsi: initialize perfetto in the right place
Compute contexts don't execute the second half of the function.

Fixes: a164e147e9 - radeonsi: Add perfetto support in radeonsi
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10043

Tested-by: Mike Lothian <mike@fireburn.co.uk>
Tested-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25873>
(cherry picked from commit 276b9b13cf)
2023-10-30 15:48:31 +00:00
Caio Oliveira
39f9dc50f5 anv: Fix leak when compiling internal kernels
Cc: mesa-stable
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Tested-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25928>
(cherry picked from commit 9d73bfc9cd)
2023-10-30 15:48:31 +00:00
Mike Blumenkrantz
ede1cdbcaf zink: check for cbuf0 writes before setting A2C
VUID-vkCmdDrawMultiIndexedEXT-alphaToCoverageEnable-08919 requires
a cbuf0 write for A2C to be active

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25938>
(cherry picked from commit 736577871b)
2023-10-30 15:48:30 +00:00
Mike Blumenkrantz
2ee4ef998a radv: correctly return oom from the device when failing to create a cs
cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25927>
(cherry picked from commit c4283e32e3)
2023-10-30 15:48:29 +00:00
Mike Blumenkrantz
e8e6ad5692 zink: add more locking for compute pipelines
if multiple contexts are accessing this all at once then this needs
more locking to avoid unsynchronized cache access

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25929>
(cherry picked from commit f8909e7d55)
2023-10-30 15:48:29 +00:00
Karol Herbst
423202cae4 rusticl/queue: Only take a weak ref to the last Event
This resolves a memory leak when the application drops its last reference
to the queue, but never waits explicitly.

The problem was, that the queue was refed by QueueState::last and that ref
only gets dropped on a blocking wait. This is problematic as non user
Event objects also hold a ref on the Queue they are created on, therefore
causing a cyclic ref relation.

In order to resolve it, just use a weak reference. A failure of upgrading
the Weak ref is not an issue as in this case we'd only wait on an already
destroyed or processed event. The worker thread already makes sure
everything stays in sync.

Fixes: 5b3ff7e3f3 ("rusticl/queue: overhaul of the queue+event handling")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: @LingMan <18294-LingMan@users.noreply.gitlab.freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25926>
(cherry picked from commit 9a3af6e1d8)
2023-10-30 15:48:28 +00:00
Karol Herbst
77eb71a612 radv: fix buffers in vkGetDescriptorEXT with size not aligned to 4
The range alignment didn't happen through GetDescriptorEXT as it called
write_buffer_descriptor directly. So simply move the align
from write_buffer_descriptor_impl into write_buffer_descriptor.

Fixes: 46e0c77582 ("radv: implement VK_EXT_descriptor_buffer")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25837>
(cherry picked from commit 1c619b668d)
2023-10-30 15:48:27 +00:00
Karol Herbst
f9337e1c78 zink: lower fisnormal as it requires the Kernel Cap
I didn't check if it's a valid vulkan SPIR-V opcode and turns out it isn't

Fixes: 82eed326f4 ("zink: support more nir opcodes")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25837>
(cherry picked from commit 01b6ccccc6)
2023-10-30 15:47:33 +00:00
Karol Herbst
65bd87bd35 rusticl/mem: properly set pipe_image_view::access
Cc: mesa-stable
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25837>
(cherry picked from commit abd8ef84ff)
2023-10-30 15:47:32 +00:00
Karol Herbst
bcfdd7476f rusticl/device: restrict param_max_size further
It's kinda pointless to have it too big, it also causes weird shaders to
be generated and causes stack overflows in `nir_opt_gcm`.

Nothing needs big values here anyway.

Cc: mesa-stable
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25837>
(cherry picked from commit 694001eef7)
2023-10-30 15:47:32 +00:00
Karol Herbst
336e6df3ef rusticl/device: restrict image_buffer_size
It's pointless to advertise more than CL_DEVICE_MAX_MEM_ALLOC_SIZE and
also the CTS tests against this.

Cc: mesa-stable
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25837>
(cherry picked from commit 9b6ac56d72)
2023-10-30 15:47:31 +00:00
Mike Blumenkrantz
7b731ab6e8 mesa/st/texture: match width+height for texture downloads of cube textures
some drivers require this

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25914>
(cherry picked from commit ee1039877c)
2023-10-30 15:47:31 +00:00
Mike Blumenkrantz
199f9783a0 zink: unset explicit_xfb_buffer for non-xfb shaders
this catches duplicated xfb when generated geometry shaders are used

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25914>
(cherry picked from commit df74ea7717)
2023-10-30 15:47:30 +00:00
Mike Blumenkrantz
08ad7e3e0f aux/u_transfer_helper: set rendertarget bind for msaa staging resource
this matches other resources created with staging blit-like mechanics

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25914>
(cherry picked from commit 87e3720b66)
2023-10-30 15:47:29 +00:00
Mike Blumenkrantz
1e7a25df6b zink: only emit xfb execution mode for last vertex stage
this is otherwise illegal

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25914>
(cherry picked from commit 694ebe8c72)
2023-10-30 15:47:29 +00:00
Mike Blumenkrantz
c7822e2b6f zink: clamp resolve extents to src/dst geometry
exceeding src/dst extents is illegal

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25914>
(cherry picked from commit e8b2680045)
2023-10-30 15:47:25 +00:00
Mike Blumenkrantz
dea50199a1 zink: always set VK_EXTERNAL_MEMORY_HANDLE_TYPE_HOST_ALLOCATION_BIT_EXT for usermem
required by spec

backport-to: 23.3

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25914>
(cherry picked from commit 009d4a5fda)
2023-10-30 15:47:24 +00:00
Mike Blumenkrantz
da8fcbaef5 zink: emit SpvCapabilitySampleRateShading with SampleId
required by spec

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25914>
(cherry picked from commit 7035b5a8e8)
2023-10-30 15:47:23 +00:00
Faith Ekstrand
8a7498e13f nir/lower_bit_size: Fix subgroup lowering for floats
Using u2u is always correct for integers, including signed integers,
because we're doing a down-cast.  It's wrong for floats, though.

Fixes: f95665cfeb ("nir/lower_bit_size: Add support for lowering subgroup ops")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894>
(cherry picked from commit 5979e74177)
2023-10-30 15:47:22 +00:00
Mike Blumenkrantz
0144e4d0b3 zink: add copy box locking
this can technically be accessed by multiple threads, so ensure
access is serialized

backport-to: 23.3

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25924>
(cherry picked from commit 782481c429)
2023-10-30 15:47:21 +00:00
Alyssa Rosenzweig
a5bcd8b84f meson: Add gallium-drivers=all option
Again, useful to lint common code changes and for our macbook-wielding rel
manager <3

Omits i915g due to dependency hell, everything else builds on fedora asahi.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Cc: mesa-stable
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25882>
(cherry picked from commit ef4aa24a15)
2023-10-30 15:47:14 +00:00
Alyssa Rosenzweig
048a358bf5 meson: Add vulkan-drivers=all option
To build-test everything, helpful to check common code changes before pounding
CI.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Cc: mesa-stable
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25882>
(cherry picked from commit b70e948886)
2023-10-30 15:47:14 +00:00
Alyssa Rosenzweig
b42d824822 crocus: Support building on non-Intel
Ditto.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Cc: mesa-stable
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25882>
(cherry picked from commit 2552ac360d)
2023-10-30 15:47:13 +00:00
Alyssa Rosenzweig
eb6e3a2a89 hasvk: Support builiding on non-Intel
Should help Eric build test releases on their MacBook :-)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Cc: mesa-stable
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25882>
(cherry picked from commit c8192c1c93)
2023-10-30 15:47:13 +00:00
Sil Vilerino
4a30434a0c d3d12: d3d12_video_buffer_create_impl - Fix resource importing
Only align resource dimensions on creation, not when importing existing D3D resource object.
Otherwise importing the resource fails since the resource descriptor does not match the aligned
dimensions passed in the template.

Fixes: 62fded5e4f ("d3d12: Allocate d3d12_video_buffer with higher alignment for compatibility")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25913>
(cherry picked from commit dfb9516026)
2023-10-30 15:47:12 +00:00
Ian Romanick
c23ba4e83a nir/split_vars: Don't split arrays of cooperative matrix types
glsl_type_is_vector_or_scalar would more accruately be called "can be an
r-value that isn't an array, structure, or matrix. This optimization
pass really shouldn't do anything to cooperative matrices. These
matrices will eventually be lowered to something else (dependent on the
backend), and that thing may (or may not) be handled by this or another
pass.

Fixes: 2d0f4f2c17 ("compiler/types: Add support for Cooperative Matrix types")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25871>
(cherry picked from commit 18d8a96a00)
2023-10-30 15:47:12 +00:00
Corentin Noël
c9040f482e mesa/bufferobj: ensure that very large width+offset are always rejected
In the case width+offset is triggering an integer overflow, the checks in place
are not working as the comparison will fail.

Cc: mesa-stable

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25909>
(cherry picked from commit e95c9b0515)
2023-10-30 15:47:10 +00:00
Samuel Pitoiset
e3bef4dfa3 ac/gpu_info: remove bogus assertion about number of COMPUTE/SDMA queues
For example, my polaris10 GPU now returns 3 compute queues.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25876>
(cherry picked from commit 4f8a225387)
2023-10-30 15:47:09 +00:00
Lionel Landwerlin
e0b436252f anv/meson: add missing dependency on the interface header
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: db335d9b73 ("anv: factor out host/gpu internal shaders interfaces")
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25905>
(cherry picked from commit ce5472137f)
2023-10-30 15:47:08 +00:00
Mike Blumenkrantz
aac2d7f100 zink: don't block large vram allocations
I think this was masking some other problem that has long since been fixed

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25893>
(cherry picked from commit ac6139ad73)
2023-10-30 15:47:04 +00:00
Neha Bhende
e886ef760c ntt: lower indirect tesslevels in ntt
Tessellation shader which are using indirect
addressing for tesslevels e.g
  gl_TessLevelOuter[gl_InvocationID] = tessLevelOuter;
are crashing because gl_TessLevelOuter is now a
compact array variable and nir expects a constant
array index into the compact array variable.

This patch handles such cases.

This fixes MR 21940
Fixes: 84006587d7 ("glsl: Delete the lower_tess_level pass.")

Tested with glretrace

Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25773>
(cherry picked from commit 8cfb46e27d)
2023-10-30 15:47:03 +00:00
Lionel Landwerlin
a0690566bd intel/fs: fix dynamic interpolation mode selection
We can end up in situation where we are dispatched with a multisample
framebuffer but not at per-sample. In this case we would request the
at_sample value with the wrong message configuration.

Relying on the BRW_WM_MSAA_FLAG_MULTISAMPLE_FBO flag superseeds
BRW_WM_MSAA_FLAG_PERSAMPLE_DISPATCH.

Fixes piglit tests :

spec@arb_gpu_shader5@arb_gpu_shader5-interpolateatsample*

With Zink on Anv

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 68027bd38e ("intel/fs: implement dynamic interpolation mode for dynamic persample shaders")
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25854>
(cherry picked from commit 439b0e8688)
2023-10-30 15:47:02 +00:00
Eric Engestrom
30a1ebe034 .pick_status.json: Update to e64a97694a 2023-10-30 15:46:58 +00:00
Eric Engestrom
c8c90eb456 VERSION: bump for rc1 2023-10-25 21:29:33 +01:00
5091 changed files with 193996 additions and 549077 deletions

View File

View File

@@ -1,9 +1,9 @@
# The following files are opted into `ninja clang-format` and
# enforcement in the CI.
src/**/asahi/**/*
src/**/panfrost/**/*
src/gallium/drivers/i915
src/gallium/targets/teflon/**/*
src/amd/vulkan/**/*
src/amd/compiler/**/*
src/egl/**/*
src/etnaviv/isa/**/*

1
.gitattributes vendored
View File

@@ -4,4 +4,3 @@
*.png binary
*.gif binary
*.ico binary
*.cl gitlab-language=c

1
.gitignore vendored
View File

@@ -3,4 +3,3 @@
*.pyo
*.out
/build
.venv/

View File

@@ -1,67 +1,24 @@
# Types of CI pipelines:
# | pipeline name | context | description |
# |----------------------|-----------|-------------------------------------------------------------|
# | merge pipeline | mesa/mesa | pipeline running for an MR; if it passes the MR gets merged |
# | pre-merge pipeline | mesa/mesa | same as above, except its status doesn't affect the MR |
# | post-merge pipeline | mesa/mesa | pipeline immediately after merging |
# | fork pipeline | fork | pipeline running in a user fork |
# | scheduled pipeline | mesa/mesa | nightly pipelines, running every morning at 4am UTC |
# | direct-push pipeline | mesa/mesa | when commits are pushed directly to mesa/mesa, bypassing Marge and its gating pipeline |
#
# Note that the release branches maintained by the release manager fall under
# the "direct push" category.
#
# "context" indicates the permissions that the jobs get; notably, any
# container created in mesa/mesa gets pushed immediately for everyone to use
# as soon as the image tag change is merged.
#
# Merge pipelines contain all jobs that must pass before the MR can be merged.
# Pre-merge pipelines contain the exact same jobs as merge pipelines.
# Post-merge pipelines contain *only* the `pages` job that deploys the new
# version of the website.
# Fork pipelines contain everything.
# Scheduled pipelines only contain the container+build jobs, and some extra
# test jobs (typically "full" variants of pre-merge jobs that only run 1/X
# test cases), but not a repeat of the merge pipeline jobs.
# Direct-push pipelines contain the same jobs as merge pipelines.
workflow:
rules:
# do not duplicate pipelines on merge pipelines
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"
when: never
# merge pipeline
- if: &is-merge-attempt $GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $GITLAB_USER_LOGIN == "marge-bot" && $CI_COMMIT_BRANCH == null
variables:
KERNEL_IMAGE_BASE: https://${S3_HOST}/${S3_KERNEL_BUCKET}/${KERNEL_REPO}/${KERNEL_TAG}
MESA_CI_PERFORMANCE_ENABLED: 1
VALVE_INFRA_VANGOGH_JOB_PRIORITY: "" # Empty tags are ignored by gitlab
# post-merge pipeline
- if: &is-post-merge $GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "push"
# nightly pipeline
- if: &is-scheduled-pipeline $CI_PIPELINE_SOURCE == "schedule"
- if: $GITLAB_USER_LOGIN == "marge-bot" && $CI_COMMIT_BRANCH
variables:
KERNEL_IMAGE_BASE: https://${S3_HOST}/${S3_KERNEL_BUCKET}/${KERNEL_REPO}/${KERNEL_TAG}
JOB_PRIORITY: 50
VALVE_INFRA_VANGOGH_JOB_PRIORITY: priority:low
# pipeline for direct pushes that bypassed the CI
- if: &is-direct-push $CI_PROJECT_NAMESPACE == "mesa" && $CI_PIPELINE_SOURCE == "push" && $GITLAB_USER_LOGIN != "marge-bot"
variables:
KERNEL_IMAGE_BASE: https://${S3_HOST}/${S3_KERNEL_BUCKET}/${KERNEL_REPO}/${KERNEL_TAG}
JOB_PRIORITY: 40
VALVE_INFRA_VANGOGH_JOB_PRIORITY: priority:low
# pre-merge or fork pipeline
- if: $FORCE_KERNEL_TAG != null
# any other pipeline
- if: $GITLAB_USER_LOGIN != "marge-bot"
variables:
KERNEL_IMAGE_BASE: https://${S3_HOST}/${S3_KERNEL_BUCKET}/${KERNEL_REPO}/${FORCE_KERNEL_TAG}
JOB_PRIORITY: 50
VALVE_INFRA_VANGOGH_JOB_PRIORITY: priority:low
- if: $FORCE_KERNEL_TAG == null
variables:
KERNEL_IMAGE_BASE: https://${S3_HOST}/${S3_KERNEL_BUCKET}/${KERNEL_REPO}/${KERNEL_TAG}
JOB_PRIORITY: 50
VALVE_INFRA_VANGOGH_JOB_PRIORITY: priority:low
- when: always
variables:
FDO_UPSTREAM_REPO: mesa/mesa
@@ -72,47 +29,28 @@ variables:
bash download-git-cache.sh
rm download-git-cache.sh
set +o xtrace
S3_JWT_FILE: /s3_jwt
CI_JOB_JWT_FILE: /minio_jwt
S3_HOST: s3.freedesktop.org
# This bucket is used to fetch the kernel image
S3_KERNEL_BUCKET: mesa-rootfs
# Bucket for git cache
S3_GITCACHE_BUCKET: git-cache
# Bucket for the pipeline artifacts pushed to S3
S3_ARTIFACTS_BUCKET: artifacts
# Buckets for traces
S3_TRACIE_RESULTS_BUCKET: mesa-tracie-results
S3_TRACIE_PUBLIC_BUCKET: mesa-tracie-public
S3_TRACIE_PRIVATE_BUCKET: mesa-tracie-private
# per-pipeline artifact storage on MinIO
PIPELINE_ARTIFACTS_BASE: ${S3_HOST}/${S3_ARTIFACTS_BUCKET}/${CI_PROJECT_PATH}/${CI_PIPELINE_ID}
PIPELINE_ARTIFACTS_BASE: ${S3_HOST}/artifacts/${CI_PROJECT_PATH}/${CI_PIPELINE_ID}
# per-job artifact storage on MinIO
JOB_ARTIFACTS_BASE: ${PIPELINE_ARTIFACTS_BASE}/${CI_JOB_ID}
KERNEL_IMAGE_BASE: https://${S3_HOST}/mesa-lava/gfx-ci/linux/${KERNEL_TAG}
# reference images stored for traces
PIGLIT_REPLAY_REFERENCE_IMAGES_BASE: "${S3_HOST}/${S3_TRACIE_RESULTS_BUCKET}/$FDO_UPSTREAM_REPO"
PIGLIT_REPLAY_REFERENCE_IMAGES_BASE: "${S3_HOST}/mesa-tracie-results/$FDO_UPSTREAM_REPO"
# For individual CI farm status see .ci-farms folder
# Disable farm with `git mv .ci-farms{,-disabled}/$farm_name`
# Re-enable farm with `git mv .ci-farms{-disabled,}/$farm_name`
# NEVER MIX FARM MAINTENANCE WITH ANY OTHER CHANGE IN THE SAME MERGE REQUEST!
ARTIFACTS_BASE_URL: https://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_NAME}/-/jobs/${CI_JOB_ID}/artifacts
# Python scripts for structured logger
PYTHONPATH: "$PYTHONPATH:$CI_PROJECT_DIR/install"
# Drop once deqp-runner is upgraded to > 0.18.0
MESA_VK_ABORT_ON_DEVICE_LOSS: 1
# Avoid the wall of "Unsupported SPIR-V capability" warnings in CI job log, hiding away useful output
MESA_SPIRV_LOG_LEVEL: error
default:
id_tokens:
S3_JWT:
aud: https://s3.freedesktop.org
before_script:
- >
export SCRIPTS_DIR=$(mktemp -d) &&
curl -L -s --retry 4 -f --retry-all-errors --retry-delay 60 -O --output-dir "${SCRIPTS_DIR}" "${CI_PROJECT_URL}/-/raw/${CI_COMMIT_SHA}/.gitlab-ci/setup-test-env.sh" &&
. ${SCRIPTS_DIR}/setup-test-env.sh &&
echo -n "${S3_JWT}" > "${S3_JWT_FILE}" &&
unset CI_JOB_JWT S3_JWT # Unsetting vulnerable env variables
echo -n "${CI_JOB_JWT}" > "${CI_JOB_JWT_FILE}" &&
unset CI_JOB_JWT # Unsetting vulnerable env variables
after_script:
# Work around https://gitlab.com/gitlab-org/gitlab/-/issues/20338
@@ -121,9 +59,9 @@ default:
- >
set +x
test -e "${S3_JWT_FILE}" &&
export S3_JWT="$(<${S3_JWT_FILE})" &&
rm "${S3_JWT_FILE}"
test -e "${CI_JOB_JWT_FILE}" &&
export CI_JOB_JWT="$(<${CI_JOB_JWT_FILE})" &&
rm "${CI_JOB_JWT_FILE}"
# Retry when job fails. Failed jobs can be found in the Mesa CI Daily Reports:
# https://gitlab.freedesktop.org/mesa/mesa/-/issues/?sort=created_date&state=opened&label_name%5B%5D=CI%20daily
@@ -146,7 +84,7 @@ stages:
- git-archive
- build-x86_64
- build-misc
- code-validation
- lint
- amd
- intel
- nouveau
@@ -177,31 +115,51 @@ include:
- local: '.gitlab-ci/farm-rules.yml'
- local: '.gitlab-ci/test-source-dep.yml'
- local: 'docs/gitlab-ci.yml'
- local: 'src/**/ci/gitlab-ci.yml'
- local: 'src/amd/ci/gitlab-ci.yml'
- local: 'src/broadcom/ci/gitlab-ci.yml'
- local: 'src/etnaviv/ci/gitlab-ci.yml'
- local: 'src/freedreno/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/crocus/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/d3d12/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/i915/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/lima/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/llvmpipe/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/nouveau/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/softpipe/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/virgl/ci/gitlab-ci.yml'
- local: 'src/gallium/drivers/zink/ci/gitlab-ci.yml'
- local: 'src/gallium/frontends/lavapipe/ci/gitlab-ci.yml'
- local: 'src/intel/ci/gitlab-ci.yml'
- local: 'src/microsoft/ci/gitlab-ci.yml'
- local: 'src/panfrost/ci/gitlab-ci.yml'
- local: 'src/virtio/ci/gitlab-ci.yml'
# YAML anchors for rule conditions
# --------------------------------
.rules-anchors:
# Pre-merge pipeline
- &is-pre-merge '$CI_PIPELINE_SOURCE == "merge_request_event"'
.never-post-merge-rules:
rules:
- if: *is-post-merge
when: never
# Post-merge pipeline
- if: &is-post-merge '$CI_PROJECT_NAMESPACE == "mesa" && $CI_COMMIT_BRANCH'
when: on_success
# Post-merge pipeline, not for Marge Bot
- if: &is-post-merge-not-for-marge '$CI_PROJECT_NAMESPACE == "mesa" && $GITLAB_USER_LOGIN != "marge-bot" && $CI_COMMIT_BRANCH'
when: on_success
# Pre-merge pipeline
- if: &is-pre-merge '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: on_success
# Pre-merge pipeline for Marge Bot
- if: &is-pre-merge-for-marge '$GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event"'
when: on_success
.container+build-rules:
rules:
# Run when re-enabling a disabled farm, but not when disabling it
- !reference [.disable-farm-mr-rules, rules]
# Never run immediately after merging, as we just ran everything
- !reference [.never-post-merge-rules, rules]
# Build everything in merge pipelines, if any files affecting the pipeline
# were changed
- if: *is-merge-attempt
# Run pipeline by default in the main project if any CI pipeline
# configuration files were changed, to ensure docker images are up to date
- if: *is-post-merge
changes: &all_paths
- VERSION
- bin/git_sha1_gen.py
@@ -210,38 +168,33 @@ include:
# GitLab CI
- .gitlab-ci.yml
- .gitlab-ci/**/*
- .ci-farms/*
# Meson
- meson*
- build-support/**/*
- subprojects/**/*
# clang format
- .clang-format
- .clang-format-include
- .clang-format-ignore
# Source code
- include/**/*
- src/**/*
when: on_success
# Same as above, but for pre-merge pipelines
- if: *is-pre-merge
# Run pipeline by default if it was triggered by Marge Bot, is for a
# merge request, and any files affecting the pipeline were changed
- if: *is-pre-merge-for-marge
changes:
*all_paths
when: manual
# Skip everything for pre-merge and merge pipelines which don't change
# anything in the build
- if: *is-merge-attempt
when: never
- if: *is-pre-merge
when: never
# Build everything after someone bypassed the CI
- if: *is-direct-push
when: on_success
# Build everything in scheduled pipelines
- if: *is-scheduled-pipeline
# Run pipeline by default in the main project if it was not triggered by
# Marge Bot, and any files affecting the pipeline were changed
- if: *is-post-merge-not-for-marge
changes:
*all_paths
when: on_success
# Allow building everything in fork pipelines, but build nothing unless
# manually triggered
# Just skip everything for MRs which don't actually change anything in the
# build - the same rules as above, but without the file-change rules
- if: *is-pre-merge-for-marge
when: never
- if: *is-post-merge
when: never
# Always allow user branches etc to trigger jobs manually
- when: manual
@@ -276,7 +229,8 @@ make git archive:
# compress the current folder
- tar -cvzf ../$CI_PROJECT_NAME.tar.gz .
- ci-fairy s3cp --token-file "${S3_JWT_FILE}" ../$CI_PROJECT_NAME.tar.gz https://$S3_HOST/git-cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/$CI_PROJECT_NAME.tar.gz
- ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" ../$CI_PROJECT_NAME.tar.gz https://$S3_HOST/git-cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/$CI_PROJECT_NAME.tar.gz
# Sanity checks of MR settings and commit logs
sanity:
@@ -286,62 +240,18 @@ sanity:
rules:
- if: *is-pre-merge
when: on_success
- when: never
# Other cases default to never
variables:
GIT_STRATEGY: none
script:
# ci-fairy check-commits --junit-xml=check-commits.xml
- ci-fairy check-merge-request --require-allow-collaboration --junit-xml=check-merge-request.xml
- |
set -eu
image_tags=(
DEBIAN_BASE_TAG
DEBIAN_BUILD_TAG
DEBIAN_X86_64_TEST_ANDROID_TAG
DEBIAN_X86_64_TEST_GL_TAG
DEBIAN_X86_64_TEST_VK_TAG
ALPINE_X86_64_BUILD_TAG
ALPINE_X86_64_LAVA_SSH_TAG
FEDORA_X86_64_BUILD_TAG
KERNEL_ROOTFS_TAG
KERNEL_TAG
PKG_REPO_REV
WINDOWS_X64_MSVC_TAG
WINDOWS_X64_BUILD_TAG
WINDOWS_X64_TEST_TAG
)
for var in "${image_tags[@]}"
do
if [ "$(echo -n "${!var}" | wc -c)" -gt 20 ]
then
echo "$var is too long; please make sure it is at most 20 chars."
exit 1
fi
done
artifacts:
when: on_failure
reports:
junit: check-*.xml
tags:
- placeholder-job
mr-label-maker-test:
extends:
- .fdo.ci-fairy
stage: sanity
rules:
- !reference [.mr-label-maker-rules, rules]
variables:
GIT_STRATEGY: fetch
timeout: 10m
script:
- set -eu
- python3 -m venv .venv
- source .venv/bin/activate
- pip install git+https://gitlab.freedesktop.org/freedesktop/mr-label-maker
- mr-label-maker --dry-run --mr $CI_MERGE_REQUEST_IID
# Jobs that need to pass before spending hardware resources on further testing
.required-for-hardware-jobs:
needs:

View File

@@ -9,9 +9,6 @@
# submission, so skip it in the regular CI.
dEQP-VK.api.driver_properties.conformance_version
# Exclude this test which might fail when a new extension is implemented.
dEQP-VK.info.device_extensions
# These are tremendously slow (pushing toward a minute), and aren't
# reliable to be run in parallel with other tests due to CPU-side timing.
dEQP-GLES[0-9]*.functional.flush_finish.*
@@ -39,44 +36,3 @@ spec@!opengl 1.1@windowoverlap
# Note that "glx-" tests don't appear in x11-skips.txt because they can be
# run even if PIGLIT_PLATFORM=gbm (for example)
glx@glx-copy-sub-buffer.*
# A majority of the tests introduced in CTS 1.3.7.0 are experiencing failures and flakes.
# Disable these tests until someone with a more deeper understanding of EGL examines them.
#
# Note: on sc8280xp/a690 I get identical results (same passes and fails)
# between freedreno, zink, and llvmpipe, so I believe this is either a
# deqp bug or egl/wayland bug, rather than driver issue.
#
# With llvmpipe, the failing tests have the error message:
#
# "Illegal sampler view creation without bind flag"
#
# which might be a hint. (But some passing tests also have the same
# error message.)
#
# more context from David Heidelberg on IRC: the deqp commit where these
# started failing is: https://github.com/KhronosGroup/VK-GL-CTS/commit/79b25659bcbced0cfc2c3fe318951c585f682abe
# prior to that they were skipping.
wayland-dEQP-EGL.functional.color_clears.single_context.gles1.other
wayland-dEQP-EGL.functional.color_clears.single_context.gles2.other
wayland-dEQP-EGL.functional.color_clears.single_context.gles3.other
wayland-dEQP-EGL.functional.color_clears.multi_context.gles1.other
wayland-dEQP-EGL.functional.color_clears.multi_context.gles2.other
wayland-dEQP-EGL.functional.color_clears.multi_context.gles3.other
wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2.other
wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2_gles3.other
wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1.other
wayland-dEQP-EGL.functional.color_clears.multi_thread.gles2.other
wayland-dEQP-EGL.functional.color_clears.multi_thread.gles3.other
wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2.other
wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2_gles3.other
# Seems to be the same is as wayland-dEQP-EGL.functional.color_clears.*
wayland-dEQP-EGL.functional.render.single_context.gles2.other
wayland-dEQP-EGL.functional.render.single_context.gles3.other
wayland-dEQP-EGL.functional.render.multi_context.gles2.other
wayland-dEQP-EGL.functional.render.multi_context.gles3.other
wayland-dEQP-EGL.functional.render.multi_context.gles2_gles3.other
wayland-dEQP-EGL.functional.render.multi_thread.gles2.other
wayland-dEQP-EGL.functional.render.multi_thread.gles3.other
wayland-dEQP-EGL.functional.render.multi_thread.gles2_gles3.other

View File

@@ -2,7 +2,12 @@ version: 1
# Rules to match for a machine to qualify
target:
id: '{{ ci_runner_id }}'
{% if tags %}
tags:
{% for tag in tags %}
- '{{ tag | trim }}'
{% endfor %}
{% endif %}
timeouts:
first_console_activity: # This limits the time it can take to receive the first console log
@@ -54,14 +59,10 @@ deployment:
b2c.volume={{ volume }}
{% endfor %}
b2c.container="-v {{ '{{' }} job_bucket }}-results:{{ working_dir }} -w {{ working_dir }} {% for mount_volume in mount_volumes %} -v {{ mount_volume }}{% endfor %} --tls-verify=false docker://{{ local_container }} {{ container_cmd }}"
{% if kernel_cmdline_extras is defined %}
{{ kernel_cmdline_extras }}
{% if cmdline_extras is defined %}
{{ cmdline_extras }}
{% endif %}
initramfs:
url: '{{ initramfs_url }}'
{% if dtb_url is defined %}
dtb:
url: '{{ dtb_url }}'
{% endif %}

View File

@@ -22,34 +22,86 @@
# IN THE SOFTWARE.
from jinja2 import Environment, FileSystemLoader
from argparse import ArgumentParser
from os import environ, path
import json
# Pass all the environment variables prefixed by B2C_
values = {
key.removeprefix("B2C_").lower(): environ[key]
for key in environ if key.startswith("B2C_")
}
parser = ArgumentParser()
parser.add_argument('--ci-job-id')
parser.add_argument('--container-cmd')
parser.add_argument('--initramfs-url')
parser.add_argument('--job-success-regex')
parser.add_argument('--job-warn-regex')
parser.add_argument('--kernel-url')
parser.add_argument('--log-level', type=int)
parser.add_argument('--poweroff-delay', type=int)
parser.add_argument('--session-end-regex')
parser.add_argument('--session-reboot-regex')
parser.add_argument('--tags', nargs='?', default='')
parser.add_argument('--template', default='b2c.yml.jinja2.jinja2')
parser.add_argument('--timeout-boot-minutes', type=int)
parser.add_argument('--timeout-boot-retries', type=int)
parser.add_argument('--timeout-first-minutes', type=int)
parser.add_argument('--timeout-first-retries', type=int)
parser.add_argument('--timeout-minutes', type=int)
parser.add_argument('--timeout-overall-minutes', type=int)
parser.add_argument('--timeout-retries', type=int)
parser.add_argument('--job-volume-exclusions', nargs='?', default='')
parser.add_argument('--volume', action='append')
parser.add_argument('--mount-volume', action='append')
parser.add_argument('--local-container', default=environ.get('B2C_LOCAL_CONTAINER', 'alpine:latest'))
parser.add_argument('--working-dir')
args = parser.parse_args()
env = Environment(loader=FileSystemLoader(path.dirname(values['job_template'])),
env = Environment(loader=FileSystemLoader(path.dirname(args.template)),
trim_blocks=True, lstrip_blocks=True)
template = env.get_template(path.basename(values['job_template']))
template = env.get_template(path.basename(args.template))
values['ci_job_id'] = environ['CI_JOB_ID']
values['ci_runner_id'] = environ['CI_RUNNER_ID']
values['job_volume_exclusions'] = [excl for excl in values['job_volume_exclusions'].split(",") if excl]
values['working_dir'] = environ['CI_PROJECT_DIR']
values = {}
values['ci_job_id'] = args.ci_job_id
values['container_cmd'] = args.container_cmd
values['initramfs_url'] = args.initramfs_url
values['job_success_regex'] = args.job_success_regex
values['job_warn_regex'] = args.job_warn_regex
values['kernel_url'] = args.kernel_url
values['log_level'] = args.log_level
values['poweroff_delay'] = args.poweroff_delay
values['session_end_regex'] = args.session_end_regex
values['session_reboot_regex'] = args.session_reboot_regex
try:
values['tags'] = json.loads(args.tags)
except json.decoder.JSONDecodeError:
values['tags'] = args.tags.split(",")
values['template'] = args.template
values['timeout_boot_minutes'] = args.timeout_boot_minutes
values['timeout_boot_retries'] = args.timeout_boot_retries
values['timeout_first_minutes'] = args.timeout_first_minutes
values['timeout_first_retries'] = args.timeout_first_retries
values['timeout_minutes'] = args.timeout_minutes
values['timeout_overall_minutes'] = args.timeout_overall_minutes
values['timeout_retries'] = args.timeout_retries
if len(args.job_volume_exclusions) > 0:
exclusions = args.job_volume_exclusions.split(",")
values['job_volume_exclusions'] = [excl for excl in exclusions if len(excl) > 0]
if args.volume is not None:
values['volumes'] = args.volume
if args.mount_volume is not None:
values['mount_volumes'] = args.mount_volume
values['working_dir'] = args.working_dir
assert(len(args.local_container) > 0)
# Use the gateway's pull-through registry caches to reduce load on fd.o.
values['local_container'] = environ['IMAGE_UNDER_TEST']
values['local_container'] = values['local_container'].replace(
'registry.freedesktop.org',
'{{ fdo_proxy_registry }}'
)
values['local_container'] = args.local_container
for url, replacement in [('registry.freedesktop.org', '{{ fdo_proxy_registry }}'),
('harbor.freedesktop.org', '{{ harbor_fdo_registry }}')]:
values['local_container'] = values['local_container'].replace(url, replacement)
if 'kernel_cmdline_extras' not in values:
values['kernel_cmdline_extras'] = ''
if 'B2C_KERNEL_CMDLINE_EXTRAS' in environ:
values['cmdline_extras'] = environ['B2C_KERNEL_CMDLINE_EXTRAS']
with open(path.splitext(path.basename(values['job_template']))[0], "w") as f:
f.write(template.render(values))
f = open(path.splitext(path.basename(args.template))[0], "w")
f.write(template.render(values))
f.close()

View File

@@ -9,7 +9,6 @@
# We're run from the root of the repo, make a helper var for our paths
BM=$CI_PROJECT_DIR/install/bare-metal
CI_COMMON=$CI_PROJECT_DIR/install/common
CI_INSTALL=$CI_PROJECT_DIR/install
# Runner config checks
if [ -z "$BM_SERIAL" ]; then
@@ -86,39 +85,21 @@ rm -rf /tftp/*
if echo "$BM_KERNEL" | grep -q http; then
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
$BM_KERNEL -o /tftp/vmlinuz
elif [ -n "${FORCE_KERNEL_TAG}" ]; then
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
"${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${BM_KERNEL}" -o /tftp/vmlinuz
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
"${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/modules.tar.zst" -o modules.tar.zst
tar --keep-directory-symlink --zstd -xf modules.tar.zst -C "/nfs/"
rm modules.tar.zst &
else
cp /baremetal-files/"$BM_KERNEL" /tftp/vmlinuz
fi
echo "$BM_CMDLINE" > /tftp/cmdline
set +e
STRUCTURED_LOG_FILE=job_detail.json
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update dut_job_type "${DEVICE_TYPE}"
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update farm "${FARM}"
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --create-dut-job dut_name "${CI_RUNNER_DESCRIPTION}"
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update-dut-time submit "${CI_JOB_STARTED_AT}"
python3 $BM/cros_servo_run.py \
--cpu $BM_SERIAL \
--ec $BM_SERIAL_EC \
--test-timeout ${TEST_PHASE_TIMEOUT:-20}
ret=$?
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --close-dut-job
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --close
set -e
# Bring artifacts back from the NFS dir to the build dir where gitlab-runner
# will look for them.
cp -Rp /nfs/results/. results/
if [ -f "${STRUCTURED_LOG_FILE}" ]; then
cp -p ${STRUCTURED_LOG_FILE} results/
echo "Structured log file is available at https://${CI_PROJECT_ROOT_NAMESPACE}.pages.freedesktop.org/-/${CI_PROJECT_NAME}/-/jobs/${CI_JOB_ID}/artifacts/results/${STRUCTURED_LOG_FILE}"
fi
exit $ret

View File

@@ -7,12 +7,11 @@ import argparse
import re
import sys
from custom_logger import CustomLogger
from serial_buffer import SerialBuffer
class CrosServoRun:
def __init__(self, cpu, ec, test_timeout, logger):
def __init__(self, cpu, ec, test_timeout):
self.cpu_ser = SerialBuffer(
cpu, "results/serial.txt", "R SERIAL-CPU> ")
# Merge the EC serial into the cpu_ser's line stream so that we can
@@ -20,7 +19,6 @@ class CrosServoRun:
self.ec_ser = SerialBuffer(
ec, "results/serial-ec.txt", "R SERIAL-EC> ", line_queue=self.cpu_ser.line_queue)
self.test_timeout = test_timeout
self.logger = logger
def close(self):
self.ec_ser.close()
@@ -38,7 +36,6 @@ class CrosServoRun:
RED = '\033[0;31m'
NO_COLOR = '\033[0m'
print(RED + message + NO_COLOR)
self.logger.update_status_fail(message)
def run(self):
# Flush any partial commands in the EC's prompt, then ask for a reboot.
@@ -46,7 +43,6 @@ class CrosServoRun:
self.ec_write("reboot\n")
bootloader_done = False
self.logger.create_job_phase("boot")
tftp_failures = 0
# This is emitted right when the bootloader pauses to check for input.
# Emit a ^N character to request network boot, because we don't have a
@@ -83,10 +79,9 @@ class CrosServoRun:
return 1
if not bootloader_done:
self.print_error("Failed to make it through bootloader, abandoning run.")
print("Failed to make it through bootloader, abandoning run.")
return 1
self.logger.create_job_phase("test")
for line in self.cpu_ser.lines(timeout=self.test_timeout, phase="test"):
if re.search("---. end Kernel panic", line):
return 1
@@ -130,10 +125,8 @@ class CrosServoRun:
result = re.search("hwci: mesa: (\S*)", line)
if result:
if result.group(1) == "pass":
self.logger.update_dut_job("status", "pass")
return 0
else:
self.logger.update_status_fail("test fail")
return 1
self.print_error(
@@ -151,14 +144,11 @@ def main():
'--test-timeout', type=int, help='Test phase timeout (minutes)', required=True)
args = parser.parse_args()
logger = CustomLogger("job_detail.json")
logger.update_dut_time("start", None)
servo = CrosServoRun(args.cpu, args.ec, args.test_timeout * 60, logger)
servo = CrosServoRun(args.cpu, args.ec, args.test_timeout * 60)
retval = servo.run()
# power down the CPU on the device
servo.ec_write("power off\n")
logger.update_dut_time("end", None)
servo.close()
sys.exit(retval)

View File

@@ -96,30 +96,22 @@ else
popd
fi
# Make the combined kernel image and dtb for passing to fastboot. For normal
# Mesa development, we build the kernel and store it in the docker container
# that this script is running in.
#
# However, container builds are expensive, so when you're hacking on the
# kernel, it's nice to be able to skip the half hour container build and plus
# moving that container to the runner. So, if BM_KERNEL+BM_DTB are URLs,
# fetch them instead of looking in the container.
if echo "$BM_KERNEL $BM_DTB" | grep -q http; then
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
"$BM_KERNEL" -o kernel
# FIXME: modules should be supplied too
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
"$BM_DTB" -o dtb
cat kernel dtb > Image.gz-dtb
elif [ -n "${FORCE_KERNEL_TAG}" ]; then
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
"${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${BM_KERNEL}" -o kernel
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
"${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/modules.tar.zst" -o modules.tar.zst
if [ -n "$BM_DTB" ]; then
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
"${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${BM_DTB}.dtb" -o dtb
fi
cat kernel dtb > Image.gz-dtb || echo "No DTB available, using pure kernel."
rm kernel
tar --keep-directory-symlink --zstd -xf modules.tar.zst -C "$BM_ROOTFS/"
rm modules.tar.zst &
else
cat /baremetal-files/"$BM_KERNEL" /baremetal-files/"$BM_DTB".dtb > Image.gz-dtb
cp /baremetal-files/"$BM_DTB".dtb dtb

View File

@@ -12,7 +12,6 @@
# We're run from the root of the repo, make a helper var for our paths
BM=$CI_PROJECT_DIR/install/bare-metal
CI_COMMON=$CI_PROJECT_DIR/install/common
CI_INSTALL=$CI_PROJECT_DIR/install
# Runner config checks
if [ -z "$BM_SERIAL" ]; then
@@ -61,8 +60,8 @@ if [ -z "$BM_ROOTFS" ]; then
exit 1
fi
if [ -z "$BM_BOOTFS" ] && { [ -z "$BM_KERNEL" ] || [ -z "$BM_DTB" ]; } ; then
echo "Must set /boot files for the TFTP boot in the job's variables or set kernel and dtb"
if [ -z "$BM_BOOTFS" ]; then
echo "Must set /boot files for the TFTP boot in the job's variables"
exit 1
fi
@@ -71,6 +70,11 @@ if [ -z "$BM_CMDLINE" ]; then
exit 1
fi
if [ -z "$BM_BOOTCONFIG" ]; then
echo "Must set BM_BOOTCONFIG to your board's required boot configuration arguments"
exit 1
fi
set -ex
date +'%F %T'
@@ -95,50 +99,23 @@ fi
date +'%F %T'
# If BM_BOOTFS is a file, assume it is a tarball and uncompress it
if [ -f "${BM_BOOTFS}" ]; then
if [ -f $BM_BOOTFS ]; then
mkdir -p /tmp/bootfs
tar xf $BM_BOOTFS -C /tmp/bootfs
BM_BOOTFS=/tmp/bootfs
fi
# If BM_KERNEL and BM_DTS is present
if [ -n "${FORCE_KERNEL_TAG}" ]; then
if [ -z "${BM_KERNEL}" ] || [ -z "${BM_DTB}" ]; then
echo "This machine cannot be tested with external kernel since BM_KERNEL or BM_DTB missing!"
exit 1
fi
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
"${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${BM_KERNEL}" -o "${BM_KERNEL}"
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
"${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${BM_DTB}.dtb" -o "${BM_DTB}.dtb"
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
"${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/modules.tar.zst" -o modules.tar.zst
fi
date +'%F %T'
# Install kernel modules (it could be either in /lib/modules or
# /usr/lib/modules, but we want to install in the latter)
if [ -n "${FORCE_KERNEL_TAG}" ]; then
tar --keep-directory-symlink --zstd -xf modules.tar.zst -C /nfs/
rm modules.tar.zst &
elif [ -n "${BM_BOOTFS}" ]; then
[ -d $BM_BOOTFS/usr/lib/modules ] && rsync -a $BM_BOOTFS/usr/lib/modules/ /nfs/usr/lib/modules/
[ -d $BM_BOOTFS/lib/modules ] && rsync -a $BM_BOOTFS/lib/modules/ /nfs/lib/modules/
else
echo "No modules!"
fi
[ -d $BM_BOOTFS/usr/lib/modules ] && rsync -a $BM_BOOTFS/usr/lib/modules/ /nfs/usr/lib/modules/
[ -d $BM_BOOTFS/lib/modules ] && rsync -a $BM_BOOTFS/lib/modules/ /nfs/lib/modules/
date +'%F %T'
# Install kernel image + bootloader files
if [ -n "${FORCE_KERNEL_TAG}" ] || [ -z "$BM_BOOTFS" ]; then
mv "${BM_KERNEL}" "${BM_DTB}.dtb" /tftp/
else # BM_BOOTFS
rsync -aL --delete $BM_BOOTFS/boot/ /tftp/
fi
rsync -aL --delete $BM_BOOTFS/boot/ /tftp/
date +'%F %T'
@@ -170,31 +147,19 @@ LABEL primary
EOF
# Create the rootfs in the NFS directory
mkdir -p /nfs/results
. $BM/rootfs-setup.sh /nfs
date +'%F %T'
echo "$BM_CMDLINE" > /tftp/cmdline.txt
# Add some options in config.txt, if defined
if [ -n "$BM_BOOTCONFIG" ]; then
printf "$BM_BOOTCONFIG" >> /tftp/config.txt
fi
# Add some required options in config.txt
printf "$BM_BOOTCONFIG" >> /tftp/config.txt
set +e
STRUCTURED_LOG_FILE=job_detail.json
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update dut_job_type "${DEVICE_TYPE}"
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update farm "${FARM}"
ATTEMPTS=3
first_attempt=True
while [ $((ATTEMPTS--)) -gt 0 ]; do
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --create-dut-job dut_name "${CI_RUNNER_DESCRIPTION}"
# Update subtime time to CI_JOB_STARTED_AT only for the first run
if [ "$first_attempt" = "True" ]; then
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update-dut-time submit "${CI_JOB_STARTED_AT}"
else
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update-dut-time submit
fi
python3 $BM/poe_run.py \
--dev="$BM_SERIAL" \
--powerup="$BM_POWERUP" \
@@ -204,14 +169,10 @@ while [ $((ATTEMPTS--)) -gt 0 ]; do
if [ $ret -eq 2 ]; then
echo "Did not detect boot sequence, retrying..."
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --close-dut-job
first_attempt=False
else
ATTEMPTS=0
fi
done
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --close-dut-job
python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --close
set -e
date +'%F %T'
@@ -219,10 +180,6 @@ date +'%F %T'
# Bring artifacts back from the NFS dir to the build dir where gitlab-runner
# will look for them.
cp -Rp /nfs/results/. results/
if [ -f "${STRUCTURED_LOG_FILE}" ]; then
cp -p ${STRUCTURED_LOG_FILE} results/
echo "Structured log file is available at ${ARTIFACTS_BASE_URL}/results/${STRUCTURED_LOG_FILE}"
fi
date +'%F %T'

View File

@@ -24,26 +24,23 @@
import argparse
import os
import re
from serial_buffer import SerialBuffer
import sys
import threading
from custom_logger import CustomLogger
from serial_buffer import SerialBuffer
class PoERun:
def __init__(self, args, test_timeout, logger):
def __init__(self, args, test_timeout):
self.powerup = args.powerup
self.powerdown = args.powerdown
self.ser = SerialBuffer(
args.dev, "results/serial-output.txt", "")
self.test_timeout = test_timeout
self.logger = logger
def print_error(self, message):
RED = '\033[0;31m'
NO_COLOR = '\033[0m'
print(RED + message + NO_COLOR)
self.logger.update_status_fail(message)
def logged_system(self, cmd):
print("Running '{}'".format(cmd))
@@ -51,11 +48,9 @@ class PoERun:
def run(self):
if self.logged_system(self.powerup) != 0:
self.logger.update_status_fail("powerup failed")
return 1
boot_detected = False
self.logger.create_job_phase("boot")
for line in self.ser.lines(timeout=5 * 60, phase="bootloader"):
if re.search("Booting Linux", line):
boot_detected = True
@@ -66,10 +61,8 @@ class PoERun:
"Something wrong; couldn't detect the boot start up sequence")
return 1
self.logger.create_job_phase("test")
for line in self.ser.lines(timeout=self.test_timeout, phase="test"):
if re.search("---. end Kernel panic", line):
self.logger.update_status_fail("kernel panic")
return 1
# Binning memory problems
@@ -89,10 +82,8 @@ class PoERun:
result = re.search("hwci: mesa: (\S*)", line)
if result:
if result.group(1) == "pass":
self.logger.update_dut_job("status", "pass")
return 0
else:
self.logger.update_status_fail("test fail")
return 1
self.print_error(
@@ -112,13 +103,10 @@ def main():
'--test-timeout', type=int, help='Test phase timeout (minutes)', required=True)
args = parser.parse_args()
logger = CustomLogger("job_detail.json")
logger.update_dut_time("start", None)
poe = PoERun(args, args.test_timeout * 60, logger)
poe = PoERun(args, args.test_timeout * 60)
retval = poe.run()
poe.logged_system(args.powerdown)
logger.update_dut_time("end", None)
sys.exit(retval)

View File

@@ -13,7 +13,7 @@ date +'%F %T'
# Make JWT token available as file in the bare-metal storage to enable access
# to MinIO
cp "${S3_JWT_FILE}" "${rootfs_dst}${S3_JWT_FILE}"
cp "${CI_JOB_JWT_FILE}" "${rootfs_dst}${CI_JOB_JWT_FILE}"
date +'%F %T'

View File

@@ -17,6 +17,7 @@
paths:
- _build/meson-logs/*.txt
- _build/meson-logs/strace
- shader-db
- artifacts
# Just Linux
@@ -60,6 +61,15 @@
script:
- .gitlab-ci/meson/build.sh
.meson-build_mingw:
extends:
- .build-linux
- .use-debian/x86_64_build_mingw
- .use-wine
stage: build-x86_64
script:
- .gitlab-ci/meson/build.sh
debian-testing:
extends:
- .meson-build
@@ -70,15 +80,12 @@ debian-testing:
-D glx=dri
-D gbm=enabled
-D egl=enabled
-D glvnd=disabled
-D platforms=x11,wayland
GALLIUM_ST: >
-D dri3=enabled
-D gallium-nine=true
-D gallium-va=enabled
-D gallium-rusticl=true
GALLIUM_DRIVERS: "swrast,virgl,radeonsi,zink,crocus,iris,i915,r300,svga"
VULKAN_DRIVERS: "swrast,amd,intel,intel_hasvk,virtio,nouveau"
GALLIUM_DRIVERS: "swrast,virgl,radeonsi,zink,crocus,iris,i915"
VULKAN_DRIVERS: "swrast,amd,intel,intel_hasvk,virtio,nouveau-experimental"
BUILDTYPE: "debugoptimized"
EXTRA_OPTION: >
-D spirv-to-dxil=true
@@ -104,22 +111,8 @@ debian-testing-asan:
-D b_sanitize=address
-D valgrind=disabled
-D tools=dlclose-skip
-D intel-clc=system
S3_ARTIFACT_NAME: ""
ARTIFACTS_DEBUG_SYMBOLS: 1
# Do a host build for intel-clc (asan complains not being loaded
# as the first library)
HOST_BUILD_OPTIONS: >
-D build-tests=false
-D enable-glcpp-tests=false
-D gallium-opencl=disabled
-D gallium-drivers=
-D vulkan-drivers=
-D video-codecs=
-D glx=disabled
-D platforms=
-D intel-clc=enabled
-D install-intel-clc=true
debian-testing-msan:
# https://github.com/google/sanitizers/wiki/MemorySanitizerLibcxxHowTo
@@ -131,7 +124,6 @@ debian-testing-msan:
EXTRA_OPTION:
-D b_sanitize=memory
-D b_lundef=false
-D intel-clc=system
S3_ARTIFACT_NAME: ""
ARTIFACTS_DEBUG_SYMBOLS: 1
# Don't run all the tests yet:
@@ -140,19 +132,33 @@ debian-testing-msan:
MESON_TEST_ARGS: "--suite glcpp --suite format"
GALLIUM_DRIVERS: "freedreno,iris,nouveau,kmsro,r300,r600,swrast,svga,v3d,vc4,virgl,etnaviv,panfrost,lima,zink,radeonsi,tegra,d3d12,crocus"
VULKAN_DRIVERS: intel,amd,broadcom,virtio
# Do a host build for intel-clc (msan complains about
# uninitialized values in the LLVM libs)
HOST_BUILD_OPTIONS: >
-D build-tests=false
-D enable-glcpp-tests=false
-D gallium-opencl=disabled
-D gallium-drivers=
-D vulkan-drivers=
-D video-codecs=
.debian-cl-testing:
extends:
- .meson-build
- .ci-deqp-artifacts
variables:
LLVM_VERSION: 15
UNWIND: "enabled"
DRI_LOADERS: >
-D glx=disabled
-D platforms=
-D intel-clc=enabled
-D install-intel-clc=true
-D egl=disabled
-D gbm=disabled
GALLIUM_DRIVERS: "swrast"
BUILDTYPE: "debugoptimized"
EXTRA_OPTION: >
-D valgrind=disabled
script:
- .gitlab-ci/meson/build.sh
- .gitlab-ci/prepare-artifacts.sh
debian-rusticl-testing:
extends:
- .debian-cl-testing
variables:
GALLIUM_ST: >
-D gallium-rusticl=true
-D opencl-spirv=true
debian-build-testing:
extends: .meson-build
@@ -163,7 +169,6 @@ debian-build-testing:
-D glx=dri
-D gbm=enabled
-D egl=enabled
-D glvnd=disabled
-D platforms=x11,wayland
GALLIUM_ST: >
-D dri3=enabled
@@ -173,6 +178,7 @@ debian-build-testing:
-D gallium-va=enabled
-D gallium-xa=enabled
-D gallium-nine=true
-D gallium-opencl=disabled
-D gallium-rusticl=false
GALLIUM_DRIVERS: "iris,nouveau,kmsro,r300,r600,freedreno,swrast,svga,v3d,vc4,virgl,etnaviv,panfrost,lima,zink,d3d12,asahi,crocus"
VULKAN_DRIVERS: swrast
@@ -182,7 +188,6 @@ debian-build-testing:
-D tools=drm-shim,etnaviv,freedreno,glsl,intel,intel-ui,nir,nouveau,lima,panfrost,asahi
-D b_lto=true
LLVM_VERSION: 15
S3_ARTIFACT_NAME: debian-build-testing
script: |
section_start lava-pytest "lava-pytest"
.gitlab-ci/lava/lava-pytest.sh
@@ -192,27 +197,9 @@ debian-build-testing:
.gitlab-ci/run-yamllint.sh
section_switch meson "meson"
.gitlab-ci/meson/build.sh
.gitlab-ci/prepare-artifacts.sh
timeout: 15m
shader-db:
stage: code-validation
extends:
- .use-debian/x86_64_build
- .container+build-rules
needs:
- debian-build-testing
variables:
S3_ARTIFACT_NAME: debian-build-testing
before_script:
- !reference [.download_s3, before_script]
script: |
section_switch shader-db "shader-db"
.gitlab-ci/run-shader-db.sh
artifacts:
paths:
- shader-db
timeout: 15m
timeout: 30m
# Test a release build with -Werror so new warnings don't sneak in.
debian-release:
@@ -226,7 +213,6 @@ debian-release:
-D glx=dri
-D gbm=enabled
-D egl=enabled
-D glvnd=disabled
-D platforms=x11,wayland
GALLIUM_ST: >
-D dri3=enabled
@@ -236,6 +222,7 @@ debian-release:
-D gallium-va=enabled
-D gallium-xa=enabled
-D gallium-nine=false
-D gallium-opencl=disabled
-D gallium-rusticl=false
-D llvm=enabled
GALLIUM_DRIVERS: "i915,iris,nouveau,kmsro,freedreno,r300,svga,swrast,v3d,vc4,virgl,etnaviv,panfrost,lima,zink,d3d12,crocus"
@@ -245,7 +232,6 @@ debian-release:
-D osmesa=true
-D tools=all
-D intel-clc=enabled
-D intel-rt=enabled
-D imagination-srv=true
BUILDTYPE: "release"
S3_ARTIFACT_NAME: "mesa-x86_64-default-${BUILDTYPE}"
@@ -268,9 +254,9 @@ alpine-build-testing:
-D glx=disabled
-D gbm=enabled
-D egl=enabled
-D glvnd=disabled
-D glvnd=false
-D platforms=wayland
LLVM_VERSION: "16"
LLVM_VERSION: ""
GALLIUM_DRIVERS: "crocus,etnaviv,freedreno,iris,kmsro,lima,nouveau,panfrost,r300,r600,radeonsi,svga,swrast,tegra,v3d,vc4,virgl,zink"
GALLIUM_ST: >
-D dri3=enabled
@@ -288,6 +274,8 @@ alpine-build-testing:
-D shared-llvm=enabled
UNWIND: "disabled"
VULKAN_DRIVERS: "amd,broadcom,freedreno,intel,imagination-experimental"
script:
- .gitlab-ci/meson/build.sh
fedora-release:
extends:
@@ -308,7 +296,7 @@ fedora-release:
-D glx=dri
-D gbm=enabled
-D egl=enabled
-D glvnd=enabled
-D glvnd=true
-D platforms=x11,wayland
EXTRA_OPTION: >
-D b_lto=true
@@ -316,9 +304,8 @@ fedora-release:
-D selinux=true
-D tools=drm-shim,etnaviv,freedreno,glsl,intel,nir,nouveau,lima,panfrost,imagination
-D vulkan-layers=device-select,overlay
-D intel-rt=enabled
-D intel-clc=enabled
-D imagination-srv=true
-D teflon=true
GALLIUM_DRIVERS: "crocus,etnaviv,freedreno,i915,iris,kmsro,lima,nouveau,panfrost,r300,r600,radeonsi,svga,swrast,tegra,v3d,vc4,virgl,zink"
GALLIUM_ST: >
-D dri3=enabled
@@ -328,6 +315,7 @@ fedora-release:
-D gallium-va=enabled
-D gallium-xa=enabled
-D gallium-nine=false
-D gallium-opencl=icd
-D gallium-rusticl=true
-D gles1=disabled
-D gles2=enabled
@@ -337,6 +325,8 @@ fedora-release:
LLVM_VERSION: ""
UNWIND: "disabled"
VULKAN_DRIVERS: "amd,broadcom,freedreno,imagination-experimental,intel,intel_hasvk"
script:
- .gitlab-ci/meson/build.sh
debian-android:
extends:
@@ -361,7 +351,6 @@ debian-android:
-D glx=disabled
-D gbm=disabled
-D egl=enabled
-D glvnd=disabled
-D platforms=android
EXTRA_OPTION: >
-D android-stub=true
@@ -369,7 +358,6 @@ debian-android:
-D platform-sdk-version=33
-D valgrind=disabled
-D android-libbacktrace=disabled
-D intel-clc=system
GALLIUM_ST: >
-D dri3=disabled
-D gallium-vdpau=disabled
@@ -377,20 +365,10 @@ debian-android:
-D gallium-va=disabled
-D gallium-xa=disabled
-D gallium-nine=false
-D gallium-rusticl=false
LLVM_VERSION: "15"
PKG_CONFIG_LIBDIR: "/disable/non/android/system/pc/files"
HOST_BUILD_OPTIONS: >
-D build-tests=false
-D enable-glcpp-tests=false
-D gallium-opencl=disabled
-D gallium-drivers=
-D vulkan-drivers=
-D video-codecs=
-D glx=disabled
-D platforms=
-D intel-clc=enabled
-D install-intel-clc=true
-D gallium-rusticl=false
LLVM_VERSION: ""
PKG_CONFIG_LIBDIR: "/disable/non/android/system/pc/files"
ARTIFACTS_DEBUG_SYMBOLS: 1
S3_ARTIFACT_NAME: mesa-x86_64-android-${BUILDTYPE}
script:
@@ -442,8 +420,6 @@ debian-arm32:
- .ci-deqp-artifacts
variables:
CROSS: armhf
DRI_LOADERS:
-D glvnd=disabled
EXTRA_OPTION: >
-D llvm=disabled
-D valgrind=disabled
@@ -459,8 +435,6 @@ debian-arm32-asan:
extends:
- debian-arm32
variables:
DRI_LOADERS:
-D glvnd=disabled
EXTRA_OPTION: >
-D llvm=disabled
-D b_sanitize=address
@@ -479,15 +453,12 @@ debian-arm64:
-Wno-error=array-bounds
-Wno-error=stringop-truncation
VULKAN_DRIVERS: "freedreno,broadcom,panfrost,imagination-experimental"
DRI_LOADERS:
-D glvnd=disabled
EXTRA_OPTION: >
-D llvm=disabled
-D valgrind=disabled
-D imagination-srv=true
-D perfetto=true
-D freedreno-kmds=msm,virtio
-D teflon=true
S3_ARTIFACT_NAME: mesa-arm64-default-${BUILDTYPE}
script:
- .gitlab-ci/meson/build.sh
@@ -497,8 +468,6 @@ debian-arm64-asan:
extends:
- debian-arm64
variables:
DRI_LOADERS:
-D glvnd=disabled
EXTRA_OPTION: >
-D llvm=disabled
-D b_sanitize=address
@@ -514,10 +483,10 @@ debian-arm64-build-test:
- .ci-deqp-artifacts
variables:
VULKAN_DRIVERS: "amd"
DRI_LOADERS:
-D glvnd=disabled
EXTRA_OPTION: >
-Dtools=panfrost,imagination
script:
- .gitlab-ci/meson/build.sh
debian-arm64-release:
extends:
@@ -539,12 +508,12 @@ debian-clang:
BUILDTYPE: debug
LLVM_VERSION: 15
UNWIND: "enabled"
GALLIUM_DUMP_CPU: "true"
C_ARGS: >
-Wno-error=constant-conversion
-Wno-error=enum-conversion
-Wno-error=initializer-overrides
-Wno-error=sometimes-uninitialized
-Werror=misleading-indentation
CPP_ARGS: >
-Wno-error=c99-designator
-Wno-error=overloaded-virtual
@@ -554,7 +523,7 @@ debian-clang:
-D glx=dri
-D gbm=enabled
-D egl=enabled
-D glvnd=enabled
-D glvnd=true
-D platforms=x11,wayland
GALLIUM_ST: >
-D dri3=enabled
@@ -564,6 +533,7 @@ debian-clang:
-D gallium-va=enabled
-D gallium-xa=enabled
-D gallium-nine=true
-D gallium-opencl=icd
-D gles1=enabled
-D gles2=enabled
-D llvm=enabled
@@ -572,7 +542,7 @@ debian-clang:
-D opencl-spirv=true
-D shared-glapi=enabled
GALLIUM_DRIVERS: "iris,nouveau,kmsro,r300,r600,freedreno,swrast,svga,v3d,vc4,virgl,etnaviv,panfrost,lima,zink,radeonsi,tegra,d3d12,crocus,i915,asahi"
VULKAN_DRIVERS: intel,amd,freedreno,broadcom,virtio,swrast,panfrost,imagination-experimental,microsoft-experimental,nouveau
VULKAN_DRIVERS: intel,amd,freedreno,broadcom,virtio,swrast,panfrost,imagination-experimental,microsoft-experimental
EXTRA_OPTION:
-D spirv-to-dxil=true
-D osmesa=true
@@ -581,9 +551,7 @@ debian-clang:
-D vulkan-layers=device-select,overlay
-D build-aco-tests=true
-D intel-clc=enabled
-D intel-rt=enabled
-D imagination-srv=true
-D teflon=true
CC: clang-${LLVM_VERSION}
CXX: clang++-${LLVM_VERSION}
@@ -602,6 +570,7 @@ debian-clang-release:
-D gallium-va=enabled
-D gallium-xa=enabled
-D gallium-nine=true
-D gallium-opencl=icd
-D gles1=disabled
-D gles2=disabled
-D llvm=enabled
@@ -610,10 +579,10 @@ debian-clang-release:
-D opencl-spirv=true
-D shared-glapi=disabled
windows-msvc:
windows-vs2019:
extends:
- .build-windows
- .use-windows_build_msvc
- .use-windows_build_vs2019
- .windows-build-rules
stage: build-misc
script:
@@ -623,6 +592,34 @@ windows-msvc:
- _build/meson-logs/*.txt
- _install/
.debian-cl:
extends: .meson-build
variables:
LLVM_VERSION: 15
UNWIND: "enabled"
DRI_LOADERS: >
-D glx=disabled
-D egl=disabled
-D gbm=disabled
EXTRA_OPTION: >
-D valgrind=disabled
debian-rusticl:
extends: .debian-cl
variables:
BUILDTYPE: debug
GALLIUM_DRIVERS: "iris,swrast"
GALLIUM_ST: >
-D dri3=disabled
-D gallium-vdpau=disabled
-D gallium-omx=disabled
-D gallium-va=disabled
-D gallium-xa=disabled
-D gallium-nine=false
-D gallium-opencl=disabled
-D gallium-rusticl=true
RUSTC: clippy-driver
debian-vulkan:
extends: .meson-build
variables:
@@ -633,10 +630,6 @@ debian-vulkan:
-D glx=disabled
-D gbm=disabled
-D egl=disabled
-D opengl=false
-D gles1=disabled
-D gles2=disabled
-D glvnd=disabled
-D platforms=x11,wayland
-D osmesa=false
GALLIUM_ST: >
@@ -646,16 +639,17 @@ debian-vulkan:
-D gallium-va=disabled
-D gallium-xa=disabled
-D gallium-nine=false
-D gallium-opencl=disabled
-D gallium-rusticl=false
-D b_sanitize=undefined
-D c_args=-fno-sanitize-recover=all
-D cpp_args=-fno-sanitize-recover=all
UBSAN_OPTIONS: "print_stacktrace=1"
VULKAN_DRIVERS: amd,broadcom,freedreno,intel,intel_hasvk,panfrost,virtio,imagination-experimental,microsoft-experimental,nouveau
VULKAN_DRIVERS: amd,broadcom,freedreno,intel,intel_hasvk,virtio,imagination-experimental,microsoft-experimental
EXTRA_OPTION: >
-D vulkan-layers=device-select,overlay
-D build-aco-tests=true
-D intel-rt=disabled
-D intel-clc=disabled
-D imagination-srv=true
debian-x86_32:
@@ -666,24 +660,10 @@ debian-x86_32:
BUILDTYPE: debug
CROSS: i386
VULKAN_DRIVERS: intel,amd,swrast,virtio
GALLIUM_DRIVERS: "iris,nouveau,r300,r600,radeonsi,swrast,virgl,zink,crocus,d3d12"
GALLIUM_DRIVERS: "iris,nouveau,r300,r600,radeonsi,swrast,virgl,zink,crocus"
LLVM_VERSION: 15
DRI_LOADERS:
-D glvnd=disabled
EXTRA_OPTION: >
-D vulkan-layers=device-select,overlay
-D intel-clc=system
HOST_BUILD_OPTIONS: >
-D build-tests=false
-D enable-glcpp-tests=false
-D gallium-opencl=disabled
-D gallium-drivers=
-D vulkan-drivers=
-D video-codecs=
-D glx=disabled
-D platforms=
-D intel-clc=enabled
-D install-intel-clc=true
debian-s390x:
extends:
@@ -697,8 +677,6 @@ debian-s390x:
GALLIUM_DRIVERS: "swrast,zink"
LLVM_VERSION: 15
VULKAN_DRIVERS: "swrast"
DRI_LOADERS:
-D glvnd=disabled
debian-ppc64el:
extends:
@@ -710,5 +688,41 @@ debian-ppc64el:
CROSS: ppc64el
GALLIUM_DRIVERS: "nouveau,radeonsi,swrast,virgl,zink"
VULKAN_DRIVERS: "amd,swrast"
DRI_LOADERS:
-D glvnd=disabled
# Disabled as it hangs with winedbg on shared runners
.debian-mingw32-x86_64:
extends: .meson-build_mingw
stage: build-misc
variables:
UNWIND: "disabled"
C_ARGS: >
-Wno-error=format
-Wno-error=unused-but-set-variable
CPP_ARGS: >
-Wno-error=format
-Wno-error=unused-function
-Wno-error=unused-variable
-Wno-error=sign-compare
-Wno-error=narrowing
GALLIUM_DRIVERS: "swrast,d3d12,zink"
VULKAN_DRIVERS: "swrast,amd,microsoft-experimental"
GALLIUM_ST: >
-D gallium-opencl=icd
-D gallium-rusticl=false
-D opencl-spirv=true
-D microsoft-clc=enabled
-D static-libclc=all
-D opencl-external-clang-headers=disabled
-D llvm=enabled
-D gallium-va=enabled
-D video-codecs=h264dec,h264enc,h265dec,h265enc,vc1dec
EXTRA_OPTION: >
-D min-windows-version=7
-D spirv-to-dxil=true
-D gles1=enabled
-D gles2=enabled
-D osmesa=true
-D cpp_rtti=true
-D shared-glapi=enabled
-D zlib=enabled
--cross-file=.gitlab-ci/x86_64-w64-mingw32

View File

@@ -1,136 +1,127 @@
#!/bin/bash
VARS=(
ACO_DEBUG
ARTIFACTS_BASE_URL
ASAN_OPTIONS
BASE_SYSTEM_FORK_HOST_PREFIX
BASE_SYSTEM_MAINLINE_HOST_PREFIX
CI_COMMIT_BRANCH
CI_COMMIT_REF_NAME
CI_COMMIT_TITLE
CI_JOB_ID
S3_JWT_FILE
CI_JOB_STARTED_AT
CI_JOB_NAME
CI_JOB_URL
CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
CI_MERGE_REQUEST_TITLE
CI_NODE_INDEX
CI_NODE_TOTAL
CI_PAGES_DOMAIN
CI_PIPELINE_ID
CI_PIPELINE_URL
CI_PROJECT_DIR
CI_PROJECT_NAME
CI_PROJECT_PATH
CI_PROJECT_ROOT_NAMESPACE
CI_RUNNER_DESCRIPTION
CI_SERVER_URL
CROSVM_GALLIUM_DRIVER
CROSVM_GPU_ARGS
CURRENT_SECTION
DEQP_BIN_DIR
DEQP_CONFIG
DEQP_EXPECTED_RENDERER
DEQP_FRACTION
DEQP_HEIGHT
DEQP_RESULTS_DIR
DEQP_RUNNER_OPTIONS
DEQP_SUITE
DEQP_TEMP_DIR
DEQP_VER
DEQP_WIDTH
DEVICE_NAME
DRIVER_NAME
EGL_PLATFORM
ETNA_MESA_DEBUG
FDO_CI_CONCURRENT
FDO_UPSTREAM_REPO
FD_MESA_DEBUG
FLAKES_CHANNEL
FREEDRENO_HANGCHECK_MS
GALLIUM_DRIVER
GALLIVM_PERF
GPU_VERSION
GTEST
GTEST_FAILS
GTEST_FRACTION
GTEST_RESULTS_DIR
GTEST_RUNNER_OPTIONS
GTEST_SKIPS
HWCI_FREQ_MAX
HWCI_KERNEL_MODULES
HWCI_KVM
HWCI_START_WESTON
HWCI_START_XORG
HWCI_TEST_SCRIPT
IR3_SHADER_DEBUG
JOB_ARTIFACTS_BASE
JOB_RESULTS_PATH
JOB_ROOTFS_OVERLAY_PATH
KERNEL_IMAGE_BASE
KERNEL_IMAGE_NAME
LD_LIBRARY_PATH
LIBGL_ALWAYS_SOFTWARE
LP_NUM_THREADS
MESA_BASE_TAG
MESA_BUILD_PATH
MESA_DEBUG
MESA_GLES_VERSION_OVERRIDE
MESA_GLSL_VERSION_OVERRIDE
MESA_GL_VERSION_OVERRIDE
MESA_IMAGE
MESA_IMAGE_PATH
MESA_IMAGE_TAG
MESA_LOADER_DRIVER_OVERRIDE
MESA_TEMPLATES_COMMIT
MESA_VK_ABORT_ON_DEVICE_LOSS
MESA_VK_IGNORE_CONFORMANCE_WARNING
S3_HOST
S3_RESULTS_UPLOAD
NIR_DEBUG
PAN_I_WANT_A_BROKEN_VULKAN_DRIVER
PAN_MESA_DEBUG
PANVK_DEBUG
PIGLIT_FRACTION
PIGLIT_NO_WINDOW
PIGLIT_OPTIONS
PIGLIT_PLATFORM
PIGLIT_PROFILES
PIGLIT_REPLAY_ANGLE_TAG
PIGLIT_REPLAY_ARTIFACTS_BASE_URL
PIGLIT_REPLAY_DEVICE_NAME
PIGLIT_REPLAY_EXTRA_ARGS
PIGLIT_REPLAY_LOOP_TIMES
PIGLIT_REPLAY_REFERENCE_IMAGES_BASE
PIGLIT_REPLAY_SUBCOMMAND
PIGLIT_RESULTS
PIGLIT_TESTS
PIGLIT_TRACES_FILE
PIPELINE_ARTIFACTS_BASE
RADEON_DEBUG
RADV_DEBUG
RADV_PERFTEST
SKQP_ASSETS_DIR
SKQP_BACKENDS
TU_DEBUG
USE_ANGLE
VIRGL_HOST_API
WAFFLE_PLATFORM
VK_CPU
VK_DRIVER
# required by virglrender CI
VK_DRIVER_FILES
VKD3D_PROTON_RESULTS
VKD3D_CONFIG
VKD3D_TEST_EXCLUDE
ZINK_DESCRIPTORS
ZINK_DEBUG
LVP_POISON_MEMORY
)
for var in "${VARS[@]}"; do
for var in \
ACO_DEBUG \
ASAN_OPTIONS \
BASE_SYSTEM_FORK_HOST_PREFIX \
BASE_SYSTEM_MAINLINE_HOST_PREFIX \
CI_COMMIT_BRANCH \
CI_COMMIT_REF_NAME \
CI_COMMIT_TITLE \
CI_JOB_ID \
CI_JOB_JWT_FILE \
CI_JOB_STARTED_AT \
CI_JOB_NAME \
CI_JOB_URL \
CI_MERGE_REQUEST_SOURCE_BRANCH_NAME \
CI_MERGE_REQUEST_TITLE \
CI_NODE_INDEX \
CI_NODE_TOTAL \
CI_PAGES_DOMAIN \
CI_PIPELINE_ID \
CI_PIPELINE_URL \
CI_PROJECT_DIR \
CI_PROJECT_NAME \
CI_PROJECT_PATH \
CI_PROJECT_ROOT_NAMESPACE \
CI_RUNNER_DESCRIPTION \
CI_SERVER_URL \
CROSVM_GALLIUM_DRIVER \
CROSVM_GPU_ARGS \
CURRENT_SECTION \
DEQP_BIN_DIR \
DEQP_CONFIG \
DEQP_EXPECTED_RENDERER \
DEQP_FRACTION \
DEQP_HEIGHT \
DEQP_RESULTS_DIR \
DEQP_RUNNER_OPTIONS \
DEQP_SUITE \
DEQP_TEMP_DIR \
DEQP_VARIANT \
DEQP_VER \
DEQP_WIDTH \
DEVICE_NAME \
DRIVER_NAME \
EGL_PLATFORM \
ETNA_MESA_DEBUG \
FDO_CI_CONCURRENT \
FDO_UPSTREAM_REPO \
FD_MESA_DEBUG \
FLAKES_CHANNEL \
FREEDRENO_HANGCHECK_MS \
GALLIUM_DRIVER \
GALLIVM_PERF \
GPU_VERSION \
GTEST \
GTEST_FAILS \
GTEST_FRACTION \
GTEST_RESULTS_DIR \
GTEST_RUNNER_OPTIONS \
GTEST_SKIPS \
HWCI_FREQ_MAX \
HWCI_KERNEL_MODULES \
HWCI_KVM \
HWCI_START_WESTON \
HWCI_START_XORG \
HWCI_TEST_SCRIPT \
IR3_SHADER_DEBUG \
JOB_ARTIFACTS_BASE \
JOB_RESULTS_PATH \
JOB_ROOTFS_OVERLAY_PATH \
KERNEL_IMAGE_BASE \
KERNEL_IMAGE_NAME \
LD_LIBRARY_PATH \
LP_NUM_THREADS \
MESA_BASE_TAG \
MESA_BUILD_PATH \
MESA_DEBUG \
MESA_GLES_VERSION_OVERRIDE \
MESA_GLSL_VERSION_OVERRIDE \
MESA_GL_VERSION_OVERRIDE \
MESA_IMAGE \
MESA_IMAGE_PATH \
MESA_IMAGE_TAG \
MESA_LOADER_DRIVER_OVERRIDE \
MESA_TEMPLATES_COMMIT \
MESA_VK_IGNORE_CONFORMANCE_WARNING \
S3_HOST \
S3_RESULTS_UPLOAD \
NIR_DEBUG \
PAN_I_WANT_A_BROKEN_VULKAN_DRIVER \
PAN_MESA_DEBUG \
PIGLIT_FRACTION \
PIGLIT_NO_WINDOW \
PIGLIT_OPTIONS \
PIGLIT_PLATFORM \
PIGLIT_PROFILES \
PIGLIT_REPLAY_ARTIFACTS_BASE_URL \
PIGLIT_REPLAY_DESCRIPTION_FILE \
PIGLIT_REPLAY_DEVICE_NAME \
PIGLIT_REPLAY_EXTRA_ARGS \
PIGLIT_REPLAY_LOOP_TIMES \
PIGLIT_REPLAY_REFERENCE_IMAGES_BASE \
PIGLIT_REPLAY_SUBCOMMAND \
PIGLIT_RESULTS \
PIGLIT_TESTS \
PIPELINE_ARTIFACTS_BASE \
RADV_DEBUG \
RADV_PERFTEST \
SKQP_ASSETS_DIR \
SKQP_BACKENDS \
TU_DEBUG \
USE_ANGLE \
VIRGL_HOST_API \
WAFFLE_PLATFORM \
VK_CPU \
VK_DRIVER \
VK_ICD_FILENAMES \
VKD3D_PROTON_RESULTS \
VKD3D_CONFIG \
ZINK_DESCRIPTORS \
ZINK_DEBUG \
LVP_POISON_MEMORY \
; do
if [ -n "${!var+x}" ]; then
echo "export $var=${!var@Q}"
fi

View File

@@ -7,10 +7,10 @@ set -ex
cd /
findmnt --mountpoint /proc || mount -t proc none /proc
findmnt --mountpoint /sys || mount -t sysfs none /sys
mount -t proc none /proc
mount -t sysfs none /sys
mount -t debugfs none /sys/kernel/debug
findmnt --mountpoint /dev || mount -t devtmpfs none /dev
mount -t devtmpfs none /dev || echo possibly already mounted
mkdir -p /dev/pts
mount -t devpts devpts /dev/pts
mkdir /dev/shm

View File

@@ -7,8 +7,6 @@
# Second-stage init, used to set up devices and our job environment before
# running tests.
shopt -s extglob
# Make sure to kill itself and all the children process from this script on
# exiting, since any console output may interfere with LAVA signals handling,
# which based on the log console.
@@ -108,13 +106,6 @@ export XDG_CACHE_HOME=/tmp
# Make sure Python can find all our imports
export PYTHONPATH=$(python3 -c "import sys;print(\":\".join(sys.path))")
# If we need to specify a driver, it means several drivers could pick up this gpu;
# ensure that the other driver can't accidentally be used
if [ -n "$MESA_LOADER_DRIVER_OVERRIDE" ]; then
rm /install/lib/dri/!($MESA_LOADER_DRIVER_OVERRIDE)_dri.so
fi
ls -1 /install/lib/dri/*_dri.so || true
if [ "$HWCI_FREQ_MAX" = "true" ]; then
# Ensure initialization of the DRM device (needed by MSM)
head -0 /dev/dri/renderD128
@@ -165,7 +156,7 @@ fi
if [ -n "$HWCI_START_XORG" ]; then
echo "touch /xorg-started; sleep 100000" > /xorg-script
env \
VK_DRIVER_FILES="/install/share/vulkan/icd.d/${VK_DRIVER}_icd.$(uname -m).json" \
VK_ICD_FILENAMES="/install/share/vulkan/icd.d/${VK_DRIVER}_icd.$(uname -m).json" \
xinit /bin/sh /xorg-script -- /usr/bin/Xorg -noreset -s 0 -dpms -logfile /Xorg.0.log &
BACKGROUND_PIDS="$! $BACKGROUND_PIDS"
@@ -192,7 +183,7 @@ if [ -n "$HWCI_START_WESTON" ]; then
mkdir -p /tmp/.X11-unix
env \
VK_DRIVER_FILES="/install/share/vulkan/icd.d/${VK_DRIVER}_icd.$(uname -m).json" \
VK_ICD_FILENAMES="/install/share/vulkan/icd.d/${VK_DRIVER}_icd.$(uname -m).json" \
weston -Bheadless-backend.so --use-gl -Swayland-0 --xwayland --idle-time=0 &
BACKGROUND_PIDS="$! $BACKGROUND_PIDS"
@@ -217,7 +208,7 @@ cleanup
# upload artifacts
if [ -n "$S3_RESULTS_UPLOAD" ]; then
tar --zstd -cf results.tar.zst results/;
ci-fairy s3cp --token-file "${S3_JWT_FILE}" results.tar.zst https://"$S3_RESULTS_UPLOAD"/results.tar.zst;
ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" results.tar.zst https://"$S3_RESULTS_UPLOAD"/results.tar.zst;
fi
# We still need to echo the hwci: mesa message, as some scripts rely on it, such

View File

@@ -8,8 +8,6 @@
set -e
set -o xtrace
export LLVM_VERSION="${LLVM_VERSION:=16}"
EPHEMERAL=(
)
@@ -18,7 +16,6 @@ DEPS=(
bash
bison
ccache
clang16-dev
cmake
clang-dev
coreutils
@@ -30,7 +27,6 @@ DEPS=(
gettext
glslang
linux-headers
llvm16-static
llvm16-dev
meson
expat-dev
@@ -41,7 +37,6 @@ DEPS=(
libpciaccess-dev
zlib-dev
python3-dev
py3-cparser
py3-mako
py3-ply
vulkan-headers
@@ -51,11 +46,7 @@ DEPS=(
wayland-protocols
)
apk --no-cache add "${DEPS[@]}" "${EPHEMERAL[@]}"
. .gitlab-ci/container/build-llvm-spirv.sh
. .gitlab-ci/container/build-libclc.sh
apk add "${DEPS[@]}" "${EPHEMERAL[@]}"
. .gitlab-ci/container/container_pre_build.sh

View File

@@ -18,7 +18,7 @@ DEPS=(
)
apk --no-cache add "${DEPS[@]}" "${EPHEMERAL[@]}"
apk add "${DEPS[@]}" "${EPHEMERAL[@]}"
. .gitlab-ci/container/container_pre_build.sh

View File

@@ -6,13 +6,13 @@ set -ex
git config --global user.email "mesa@example.com"
git config --global user.name "Mesa CI"
CROSVM_VERSION=1641c55bcc922588e24de73e9cca7b5e4005bd6d
CROSVM_VERSION=e3815e62d675ef436956a992e0ed58b7309c759d
git clone --single-branch -b main --no-checkout https://chromium.googlesource.com/crosvm/crosvm /platform/crosvm
pushd /platform/crosvm
git checkout "$CROSVM_VERSION"
git submodule update --init
VIRGLRENDERER_VERSION=d9c002fac153b834a2c17731f2b85c36e333e102
VIRGLRENDERER_VERSION=10120c0d9ebdc00eae1b5c9f7c98fc0d198ba602
rm -rf third_party/virglrenderer
git clone --single-branch -b main --no-checkout https://gitlab.freedesktop.org/virgl/virglrenderer.git third_party/virglrenderer
pushd third_party/virglrenderer
@@ -31,10 +31,10 @@ RUSTFLAGS='-L native=/usr/local/lib' cargo install \
--version 0.65.1 \
$EXTRA_CARGO_ARGS
CROSVM_USE_SYSTEM_MINIGBM=1 CROSVM_USE_SYSTEM_VIRGLRENDERER=1 RUSTFLAGS='-L native=/usr/local/lib' cargo install \
CROSVM_USE_SYSTEM_VIRGLRENDERER=1 RUSTFLAGS='-L native=/usr/local/lib' cargo install \
-j ${FDO_CI_CONCURRENT:-4} \
--locked \
--features 'default-no-sandbox gpu x virgl_renderer' \
--features 'default-no-sandbox gpu x virgl_renderer virgl_renderer_next' \
--path . \
--root /usr/local \
$EXTRA_CARGO_ARGS

View File

@@ -1,39 +1,25 @@
#!/usr/bin/env bash
# shellcheck disable=SC2086 # we want word splitting
# When changing this file, you need to bump the following
# .gitlab-ci/image-tags.yml tags:
# DEBIAN_X86_64_TEST_ANDROID_TAG
# DEBIAN_X86_64_TEST_GL_TAG
# DEBIAN_X86_64_TEST_VK_TAG
# KERNEL_ROOTFS_TAG
set -ex
DEQP_RUNNER_VERSION=0.18.0
DEQP_RUNNER_GIT_URL="${DEQP_RUNNER_GIT_URL:-https://gitlab.freedesktop.org/mesa/deqp-runner.git}"
if [ -n "${DEQP_RUNNER_GIT_TAG}${DEQP_RUNNER_GIT_REV}" ]; then
# Build and install from source
DEQP_RUNNER_CARGO_ARGS="--git $DEQP_RUNNER_GIT_URL"
DEQP_RUNNER_CARGO_ARGS="--git ${DEQP_RUNNER_GIT_URL:-https://gitlab.freedesktop.org/anholt/deqp-runner.git}"
if [ -n "${DEQP_RUNNER_GIT_TAG}" ]; then
DEQP_RUNNER_CARGO_ARGS="--tag ${DEQP_RUNNER_GIT_TAG} ${DEQP_RUNNER_CARGO_ARGS}"
DEQP_RUNNER_GIT_CHECKOUT="$DEQP_RUNNER_GIT_TAG"
else
DEQP_RUNNER_CARGO_ARGS="--rev ${DEQP_RUNNER_GIT_REV} ${DEQP_RUNNER_CARGO_ARGS}"
DEQP_RUNNER_GIT_CHECKOUT="$DEQP_RUNNER_GIT_REV"
fi
DEQP_RUNNER_CARGO_ARGS="${DEQP_RUNNER_CARGO_ARGS} ${EXTRA_CARGO_ARGS}"
else
# Install from package registry
DEQP_RUNNER_CARGO_ARGS="--version ${DEQP_RUNNER_VERSION} ${EXTRA_CARGO_ARGS} -- deqp-runner"
DEQP_RUNNER_GIT_CHECKOUT="v$DEQP_RUNNER_VERSION"
DEQP_RUNNER_CARGO_ARGS="--version 0.16.0 ${EXTRA_CARGO_ARGS} -- deqp-runner"
fi
if [[ "$RUST_TARGET" != *-android ]]; then
if [ -z "$ANDROID_NDK_HOME" ]; then
cargo install --locked \
-j ${FDO_CI_CONCURRENT:-4} \
--root /usr/local \
@@ -41,7 +27,7 @@ if [[ "$RUST_TARGET" != *-android ]]; then
else
mkdir -p /deqp-runner
pushd /deqp-runner
git clone --branch "$DEQP_RUNNER_GIT_CHECKOUT" --depth 1 "$DEQP_RUNNER_GIT_URL" deqp-runner-git
git clone --branch v0.16.1 --depth 1 https://gitlab.freedesktop.org/anholt/deqp-runner.git deqp-runner-git
pushd deqp-runner-git
cargo install --locked \
@@ -49,10 +35,10 @@ else
--root /usr/local --version 2.10.0 \
cargo-ndk
rustup target add $RUST_TARGET
RUSTFLAGS='-C target-feature=+crt-static' cargo ndk --target $RUST_TARGET build --release
rustup target add x86_64-linux-android
RUSTFLAGS='-C target-feature=+crt-static' cargo ndk --target x86_64-linux-android build
mv target/$RUST_TARGET/release/deqp-runner /deqp-runner
mv target/x86_64-linux-android/debug/deqp-runner /deqp-runner
cargo uninstall --locked \
--root /usr/local \

View File

@@ -8,17 +8,16 @@
# DEBIAN_X86_64_TEST_VK_TAG
# KERNEL_ROOTFS_TAG
set -ex -o pipefail
set -ex
# See `deqp_build_targets` below for which release is used to produce which
# binary. Unless this comment has bitrotten:
# - the VK release produces `deqp-vk`,
# - the GL release produces `glcts`, and
# - the GLES release produces `deqp-gles*` and `deqp-egl`
DEQP_VK_VERSION=1.3.8.2
DEQP_GL_VERSION=4.6.4.0
DEQP_GLES_VERSION=3.2.10.0
git config --global user.email "mesa@example.com"
git config --global user.name "Mesa CI"
git clone \
https://github.com/KhronosGroup/VK-GL-CTS.git \
-b vulkan-cts-1.3.7.0 \
--depth 1 \
/VK-GL-CTS
pushd /VK-GL-CTS
# Patches to VulkanCTS may come from commits in their repo (listed in
# cts_commits_to_backport) or patch files stored in our repo (in the patch
@@ -26,211 +25,95 @@ DEQP_GLES_VERSION=3.2.10.0
# Both list variables would have comments explaining the reasons behind the
# patches.
# shellcheck disable=SC2034
vk_cts_commits_to_backport=(
# Fix more ASAN errors due to missing virtual destructors
dd40bcfef1b4035ea55480b6fd4d884447120768
# Remove "unused shader stages" tests
7dac86c6bbd15dec91d7d9a98cd6dd57c11092a7
cts_commits_to_backport=(
)
# shellcheck disable=SC2034
vk_cts_patch_files=(
)
if [ "${DEQP_TARGET}" = 'android' ]; then
vk_cts_patch_files+=(
build-deqp-vk_Allow-running-on-Android-from-the-command-line.patch
build-deqp-vk_Android-prints-to-stdout-instead-of-logcat.patch
)
fi
# shellcheck disable=SC2034
gl_cts_commits_to_backport=(
)
# shellcheck disable=SC2034
gl_cts_patch_files=(
)
if [ "${DEQP_TARGET}" = 'android' ]; then
gl_cts_patch_files+=(
build-deqp-gl_Allow-running-on-Android-from-the-command-line.patch
build-deqp-gl_Android-prints-to-stdout-instead-of-logcat.patch
)
fi
# shellcheck disable=SC2034
# GLES builds also EGL
gles_cts_commits_to_backport=(
# Implement support for the EGL_EXT_config_select_group extension
88ba9ac270db5be600b1ecacbc6d9db0c55d5be4
)
# shellcheck disable=SC2034
gles_cts_patch_files=(
# Correct detection mechanism for EGL_EXT_config_select_group extension
build-deqp-egl_Correct-EGL_EXT_config_select_group-extension-query.patch
)
if [ "${DEQP_TARGET}" = 'android' ]; then
gles_cts_patch_files+=(
build-deqp-gles_Allow-running-on-Android-from-the-command-line.patch
build-deqp-gles_Android-prints-to-stdout-instead-of-logcat.patch
)
fi
### Careful editing anything below this line
git config --global user.email "mesa@example.com"
git config --global user.name "Mesa CI"
# shellcheck disable=SC2153
case "${DEQP_API}" in
VK) DEQP_VERSION="vulkan-cts-$DEQP_VK_VERSION";;
GL) DEQP_VERSION="opengl-cts-$DEQP_GL_VERSION";;
GLES) DEQP_VERSION="opengl-es-cts-$DEQP_GLES_VERSION";;
esac
git clone \
https://github.com/KhronosGroup/VK-GL-CTS.git \
-b $DEQP_VERSION \
--depth 1 \
/VK-GL-CTS
pushd /VK-GL-CTS
mkdir -p /deqp
# shellcheck disable=SC2153
deqp_api=${DEQP_API,,}
cts_commits_to_backport="${deqp_api}_cts_commits_to_backport[@]"
for commit in "${!cts_commits_to_backport}"
for commit in "${cts_commits_to_backport[@]}"
do
PATCH_URL="https://github.com/KhronosGroup/VK-GL-CTS/commit/$commit.patch"
echo "Apply patch to ${DEQP_API} CTS from $PATCH_URL"
echo "Apply patch to VK-GL-CTS from $PATCH_URL"
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 $PATCH_URL | \
git am -
done
cts_patch_files="${deqp_api}_cts_patch_files[@]"
for patch in "${!cts_patch_files}"
cts_patch_files=(
# Android specific patches.
build-deqp_Allow-running-on-Android-from-the-command-line.patch
build-deqp_Android-prints-to-stdout-instead-of-logcat.patch
)
for patch in "${cts_patch_files[@]}"
do
echo "Apply patch to ${DEQP_API} CTS from $patch"
echo "Apply patch to VK-GL-CTS from $patch"
git am < $OLDPWD/.gitlab-ci/container/patches/$patch
done
{
echo "dEQP base version $DEQP_VERSION"
echo "The following local patches are applied on top:"
git log --reverse --oneline $DEQP_VERSION.. --format=%s | sed 's/^/- /'
} > /deqp/version-$deqp_api
# --insecure is due to SSL cert failures hitting sourceforge for zlib and
# libpng (sigh). The archives get their checksums checked anyway, and git
# always goes through ssh or https.
python3 external/fetch_sources.py --insecure
mkdir -p /deqp
# Save the testlog stylesheets:
cp doc/testlog-stylesheet/testlog.{css,xsl} /deqp
popd
pushd /deqp
if [ "${DEQP_API}" = 'GLES' ]; then
if [ "${DEQP_TARGET}" = 'android' ]; then
cmake -S /VK-GL-CTS -B . -G Ninja \
-DDEQP_TARGET=android \
-DCMAKE_BUILD_TYPE=Release \
$EXTRA_CMAKE_ARGS
mold --run ninja modules/egl/deqp-egl
mv /deqp/modules/egl/deqp-egl /deqp/modules/egl/deqp-egl-android
else
if [ "${DEQP_TARGET}" != 'android' ]; then
# When including EGL/X11 testing, do that build first and save off its
# deqp-egl binary.
cmake -S /VK-GL-CTS -B . -G Ninja \
-DDEQP_TARGET=x11_egl_glx \
-DCMAKE_BUILD_TYPE=Release \
$EXTRA_CMAKE_ARGS
mold --run ninja modules/egl/deqp-egl
ninja modules/egl/deqp-egl
mv /deqp/modules/egl/deqp-egl /deqp/modules/egl/deqp-egl-x11
cmake -S /VK-GL-CTS -B . -G Ninja \
-DDEQP_TARGET=wayland \
-DCMAKE_BUILD_TYPE=Release \
$EXTRA_CMAKE_ARGS
mold --run ninja modules/egl/deqp-egl
ninja modules/egl/deqp-egl
mv /deqp/modules/egl/deqp-egl /deqp/modules/egl/deqp-egl-wayland
fi
fi
cmake -S /VK-GL-CTS -B . -G Ninja \
-DDEQP_TARGET=${DEQP_TARGET} \
-DDEQP_TARGET=${DEQP_TARGET:-x11_glx} \
-DCMAKE_BUILD_TYPE=Release \
$EXTRA_CMAKE_ARGS
mold --run ninja
# Make sure `default` doesn't silently stop detecting one of the platforms we care about
if [ "${DEQP_TARGET}" = 'default' ]; then
grep -q DEQP_SUPPORT_WAYLAND=1 build.ninja
grep -q DEQP_SUPPORT_X11=1 build.ninja
grep -q DEQP_SUPPORT_XCB=1 build.ninja
if [ "${DEQP_TARGET}" = 'android' ]; then
mv /deqp/modules/egl/deqp-egl /deqp/modules/egl/deqp-egl-android
fi
deqp_build_targets=()
case "${DEQP_API}" in
VK)
deqp_build_targets+=(deqp-vk)
;;
GL)
deqp_build_targets+=(glcts)
;;
GLES)
deqp_build_targets+=(deqp-gles{2,3,31})
# deqp-egl also comes from this build, but it is handled separately above.
;;
esac
if [ "${DEQP_TARGET}" != 'android' ]; then
deqp_build_targets+=(testlog-to-xml)
deqp_build_targets+=(testlog-to-csv)
deqp_build_targets+=(testlog-to-junit)
fi
git -C /VK-GL-CTS describe --long > /deqp/version
mold --run ninja "${deqp_build_targets[@]}"
# Copy out the mustpass lists we want.
mkdir /deqp/mustpass
for mustpass in $(< /VK-GL-CTS/external/vulkancts/mustpass/main/vk-default.txt) ; do
cat /VK-GL-CTS/external/vulkancts/mustpass/main/$mustpass \
>> /deqp/mustpass/vk-master.txt
done
if [ "${DEQP_TARGET}" != 'android' ]; then
# Copy out the mustpass lists we want.
mkdir -p /deqp/mustpass
if [ "${DEQP_API}" = 'VK' ]; then
for mustpass in $(< /VK-GL-CTS/external/vulkancts/mustpass/main/vk-default.txt) ; do
cat /VK-GL-CTS/external/vulkancts/mustpass/main/$mustpass \
>> /deqp/mustpass/vk-main.txt
done
fi
if [ "${DEQP_API}" = 'GL' ]; then
cp \
/VK-GL-CTS/external/openglcts/data/mustpass/gl/khronos_mustpass/4.6.1.x/*-main.txt \
/deqp/mustpass/
cp \
/VK-GL-CTS/external/openglcts/data/mustpass/gl/khronos_mustpass_single/4.6.1.x/*-single.txt \
/deqp/mustpass/
fi
if [ "${DEQP_API}" = 'GLES' ]; then
cp \
/VK-GL-CTS/external/openglcts/data/mustpass/gles/aosp_mustpass/3.2.6.x/*.txt \
/deqp/mustpass/
cp \
/VK-GL-CTS/external/openglcts/data/mustpass/egl/aosp_mustpass/3.2.6.x/egl-main.txt \
/deqp/mustpass/
cp \
/VK-GL-CTS/external/openglcts/data/mustpass/gles/khronos_mustpass/3.2.6.x/*-main.txt \
/deqp/mustpass/
fi
cp \
/deqp/external/openglcts/modules/gl_cts/data/mustpass/gles/aosp_mustpass/3.2.6.x/*.txt \
/deqp/mustpass/.
cp \
/deqp/external/openglcts/modules/gl_cts/data/mustpass/egl/aosp_mustpass/3.2.6.x/egl-master.txt \
/deqp/mustpass/.
cp \
/deqp/external/openglcts/modules/gl_cts/data/mustpass/gles/khronos_mustpass/3.2.6.x/*-master.txt \
/deqp/mustpass/.
cp \
/deqp/external/openglcts/modules/gl_cts/data/mustpass/gl/khronos_mustpass/4.6.1.x/*-master.txt \
/deqp/mustpass/.
cp \
/deqp/external/openglcts/modules/gl_cts/data/mustpass/gl/khronos_mustpass_single/4.6.1.x/*-single.txt \
/deqp/mustpass/.
# Save *some* executor utils, but otherwise strip things down
# to reduct deqp build size:
@@ -241,24 +124,19 @@ if [ "${DEQP_TARGET}" != 'android' ]; then
fi
# Remove other mustpass files, since we saved off the ones we wanted to conventient locations above.
rm -rf /deqp/external/**/mustpass/
rm -rf /deqp/external/vulkancts/modules/vulkan/vk-main*
rm -rf /deqp/external/openglcts/modules/gl_cts/data/mustpass
rm -rf /deqp/external/vulkancts/modules/vulkan/vk-master*
rm -rf /deqp/external/vulkancts/modules/vulkan/vk-default
rm -rf /deqp/external/openglcts/modules/cts-runner
rm -rf /deqp/modules/internal
rm -rf /deqp/execserver
rm -rf /deqp/framework
find . -depth \( -iname '*cmake*' -o -name '*ninja*' -o -name '*.o' -o -name '*.a' \) -exec rm -rf {} \;
if [ "${DEQP_API}" = 'VK' ]; then
${STRIP_CMD:-strip} external/vulkancts/modules/vulkan/deqp-vk
fi
if [ "${DEQP_API}" = 'GL' ]; then
${STRIP_CMD:-strip} external/openglcts/modules/glcts
fi
if [ "${DEQP_API}" = 'GLES' ]; then
${STRIP_CMD:-strip} modules/*/deqp-*
fi
# shellcheck disable=SC2038,SC2185 # TODO: rewrite find
find -iname '*cmake*' -o -name '*ninja*' -o -name '*.o' -o -name '*.a' | xargs rm -rf
${STRIP_CMD:-strip} external/vulkancts/modules/vulkan/deqp-vk
${STRIP_CMD:-strip} external/openglcts/modules/glcts
${STRIP_CMD:-strip} modules/*/deqp-*
du -sh ./*
rm -rf /VK-GL-CTS
popd

View File

@@ -1,15 +0,0 @@
#!/usr/bin/env bash
# shellcheck disable=SC2086 # we want word splitting
# When changing this file, you need to bump the following
# .gitlab-ci/image-tags.yml tags:
# DEBIAN_BUILD_TAG
set -ex
git clone https://github.com/microsoft/DirectX-Headers -b v1.613.1 --depth 1
pushd DirectX-Headers
meson setup build --backend=ninja --buildtype=release -Dbuild-test=false $EXTRA_MESON_ARGS
meson install -C build
popd
rm -rf DirectX-Headers

View File

@@ -5,12 +5,12 @@ set -ex
KDL_REVISION="5056f71b100a68b72b285c6fc845a66a2ed25985"
mkdir ci-kdl.git
git clone \
https://gitlab.freedesktop.org/gfx-ci/ci-kdl.git \
--depth 1 \
ci-kdl.git
pushd ci-kdl.git
git init
git remote add origin https://gitlab.freedesktop.org/gfx-ci/ci-kdl.git
git fetch --depth 1 origin ${KDL_REVISION}
git checkout FETCH_HEAD
git checkout ${KDL_REVISION}
popd
python3 -m venv ci-kdl.venv

View File

@@ -1,10 +1,9 @@
#!/usr/bin/env bash
# Script used for Android and Fedora builds
# shellcheck disable=SC2086 # we want word splitting
set -ex
export LIBDRM_VERSION=libdrm-2.4.119
export LIBDRM_VERSION=libdrm-2.4.114
curl -L -O --retry 4 -f --retry-all-errors --retry-delay 60 \
https://dri.freedesktop.org/libdrm/"$LIBDRM_VERSION".tar.xz

View File

@@ -2,14 +2,7 @@
set -ex
# When changing this file, you need to bump the following
# .gitlab-ci/image-tags.yml tags:
# DEBIAN_BASE_TAG
# DEBIAN_BUILD_TAG
# FEDORA_X86_64_BUILD_TAG
# KERNEL_ROOTFS_TAG
MOLD_VERSION="2.4.1"
MOLD_VERSION="1.11.0"
git clone -b v"$MOLD_VERSION" --single-branch --depth 1 https://github.com/rui314/mold.git
pushd mold

View File

@@ -1,25 +0,0 @@
#!/usr/bin/env bash
# When changing this file, you need to bump the following
# .gitlab-ci/image-tags.yml tags:
# DEBIAN_X86_64_TEST_GL_TAG
set -ex -o pipefail
### Careful editing anything below this line
git config --global user.email "mesa@example.com"
git config --global user.name "Mesa CI"
git clone https://github.com/axeldavy/Xnine.git /Xnine
mkdir /Xnine/build
pushd /Xnine/build
git checkout c64753d224c08006bcdcfa7880ada826f27164b1
cmake .. -DBUILD_TESTS=1 -DWITH_DRI3=1 -DD3DADAPTER9_LOCATION=/install/lib/d3d/d3dadapter9.so
make
mkdir -p /NineTests/
mv NineTests/NineTests /NineTests/
popd
rm -rf /Xnine

View File

@@ -8,7 +8,7 @@ set -ex
# DEBIAN_X86_64_TEST_VK_TAG
# KERNEL_ROOTFS_TAG
REV="f7ece74a107a2f99b2f494d978c84f8d51faa703"
REV="f7db20b03de6896d013826c0a731bc4417c1a5a0"
git clone https://gitlab.freedesktop.org/mesa/piglit.git --single-branch --no-checkout /piglit
pushd /piglit
@@ -16,18 +16,18 @@ git checkout "$REV"
patch -p1 <$OLDPWD/.gitlab-ci/piglit/disable-vs_in.diff
cmake -S . -B . -G Ninja -DCMAKE_BUILD_TYPE=Release $PIGLIT_OPTS $EXTRA_CMAKE_ARGS
ninja $PIGLIT_BUILD_TARGETS
find . -depth \( -name .git -o -name '*ninja*' -o -iname '*cmake*' -o -name '*.[chao]' \) -exec rm -rf {} \;
# shellcheck disable=SC2038,SC2185 # TODO: rewrite find
find -name .git -o -name '*ninja*' -o -iname '*cmake*' -o -name '*.[chao]' | xargs rm -rf
rm -rf target_api
if [ "$PIGLIT_BUILD_TARGETS" = "piglit_replayer" ]; then
find . -depth \
! -regex "^\.$" \
# shellcheck disable=SC2038,SC2185 # TODO: rewrite find
find ! -regex "^\.$" \
! -regex "^\.\/piglit.*" \
! -regex "^\.\/framework.*" \
! -regex "^\.\/bin$" \
! -regex "^\.\/bin\/replayer\.py" \
! -regex "^\.\/templates.*" \
! -regex "^\.\/tests$" \
! -regex "^\.\/tests\/replay\.py" \
-exec rm -rf {} \; 2>/dev/null
! -regex "^\.\/tests\/replay\.py" 2>/dev/null | xargs rm -rf
fi
popd

View File

@@ -11,11 +11,11 @@ set -ex
mkdir -p "$HOME"/.cargo
ln -s /usr/local/bin "$HOME"/.cargo/bin
# Rusticl requires at least Rust 1.66.0 and NAK requires 1.73.0
# Rusticl requires at least Rust 1.66.0
#
# Also, pick a specific snapshot from rustup so the compiler doesn't drift on
# us.
RUST_VERSION=1.73.0-2023-10-05
RUST_VERSION=1.66.1-2023-01-10
# For rust in Mesa, we use rustup to install. This lets us pick an arbitrary
# version of the compiler, rather than whatever the container's Debian comes

View File

@@ -6,7 +6,7 @@
# KERNEL_ROOTFS_TAG
set -ex
VKD3D_PROTON_COMMIT="c3b385606a93baed42482d822805e0d9c2f3f603"
VKD3D_PROTON_COMMIT="2ad6cfdeaacdf47e2689e30a8fb5ac8193725f0d"
VKD3D_PROTON_DST_DIR="/vkd3d-proton-tests"
VKD3D_PROTON_SRC_DIR="/vkd3d-proton-src"

View File

@@ -7,7 +7,7 @@
set -ex
VALIDATION_TAG="v1.3.281"
VALIDATION_TAG="v1.3.263"
git clone -b "$VALIDATION_TAG" --single-branch --depth 1 https://github.com/KhronosGroup/Vulkan-ValidationLayers.git
pushd Vulkan-ValidationLayers

View File

@@ -3,17 +3,8 @@
set -ex
# When changing this file, you need to bump the following
# .gitlab-ci/image-tags.yml tags:
# DEBIAN_BUILD_TAG
# DEBIAN_X86_64_TEST_ANDROID_TAG
# DEBIAN_X86_64_TEST_GL_TAG
# DEBIAN_X86_64_TEST_VK_TAG
# FEDORA_X86_64_BUILD_TAG
# KERNEL_ROOTFS_TAG
export LIBWAYLAND_VERSION="1.21.0"
export WAYLAND_PROTOCOLS_VERSION="1.34"
export WAYLAND_PROTOCOLS_VERSION="1.31"
git clone https://gitlab.freedesktop.org/wayland/wayland
cd wayland

View File

@@ -1,12 +1,10 @@
#!/usr/bin/env bash
#!/bin/sh
if test -f /etc/debian_version; then
apt-get autoremove -y --purge
fi
# Clean up any build cache
rm -rf /root/.cache
rm -rf /root/.cargo
# Clean up any build cache for rust.
rm -rf /.cargo
if test -x /usr/bin/ccache; then

View File

@@ -25,7 +25,7 @@ fi
# When not using the mold linker (e.g. unsupported architecture), force
# linkers to gold, since it's so much faster for building. We can't use
# lld because we're on old debian and it's buggy. mingw fails meson builds
# lld because we're on old debian and it's buggy. ming fails meson builds
# with it with "meson.build:21:0: ERROR: Unable to determine dynamic linker"
find /usr/bin -name \*-ld -o -name ld | \
grep -v mingw | \

View File

@@ -8,62 +8,56 @@ export DEBIAN_FRONTEND=noninteractive
export LLVM_VERSION="${LLVM_VERSION:=15}"
# Ephemeral packages (installed for this script and removed again at the end)
EPHEMERAL=(
)
DEPS=(
"crossbuild-essential-$arch"
"pkgconf:$arch"
"libasan8:$arch"
"libdrm-dev:$arch"
"libelf-dev:$arch"
"libexpat1-dev:$arch"
"libffi-dev:$arch"
"libpciaccess-dev:$arch"
"libstdc++6:$arch"
"libvulkan-dev:$arch"
"libx11-dev:$arch"
"libx11-xcb-dev:$arch"
"libxcb-dri2-0-dev:$arch"
"libxcb-dri3-dev:$arch"
"libxcb-glx0-dev:$arch"
"libxcb-present-dev:$arch"
"libxcb-randr0-dev:$arch"
"libxcb-shm0-dev:$arch"
"libxcb-xfixes0-dev:$arch"
"libxdamage-dev:$arch"
"libxext-dev:$arch"
"libxrandr-dev:$arch"
"libxshmfence-dev:$arch"
"libxxf86vm-dev:$arch"
"libwayland-dev:$arch"
)
STABLE_EPHEMERAL=" \
"
dpkg --add-architecture $arch
echo "deb [trusted=yes] https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/raw/${PKG_REPO_REV}/ ${FDO_DISTRIBUTION_VERSION%-*} main" | tee /etc/apt/sources.list.d/gfx-ci_.list
apt-get update
apt-get install -y --no-remove "${DEPS[@]}" "${EPHEMERAL[@]}" \
$EXTRA_LOCAL_PACKAGES
apt-get install -y --no-remove \
$EXTRA_LOCAL_PACKAGES \
$STABLE_EPHEMERAL \
crossbuild-essential-$arch \
pkgconf:$arch \
libasan8:$arch \
libdrm-dev:$arch \
libelf-dev:$arch \
libexpat1-dev:$arch \
libffi-dev:$arch \
libpciaccess-dev:$arch \
libstdc++6:$arch \
libvulkan-dev:$arch \
libx11-dev:$arch \
libx11-xcb-dev:$arch \
libxcb-dri2-0-dev:$arch \
libxcb-dri3-dev:$arch \
libxcb-glx0-dev:$arch \
libxcb-present-dev:$arch \
libxcb-randr0-dev:$arch \
libxcb-shm0-dev:$arch \
libxcb-xfixes0-dev:$arch \
libxdamage-dev:$arch \
libxext-dev:$arch \
libxrandr-dev:$arch \
libxshmfence-dev:$arch \
libxxf86vm-dev:$arch \
libwayland-dev:$arch
if [[ $arch != "armhf" ]]; then
# We don't need clang-format for the crossbuilds, but the installed amd64
# package will conflict with libclang. Uninstall clang-format (and its
# problematic dependency) to fix.
apt-get remove -y "clang-format-${LLVM_VERSION}" "libclang-cpp${LLVM_VERSION}" \
"llvm-${LLVM_VERSION}-runtime" "llvm-${LLVM_VERSION}-linker-tools"
apt-get remove -y clang-format-${LLVM_VERSION} libclang-cpp${LLVM_VERSION}
# llvm-*-tools:$arch conflicts with python3:amd64. Install dependencies only
# with apt-get, then force-install llvm-*-{dev,tools}:$arch with dpkg to get
# around this.
apt-get install -y --no-remove --no-install-recommends \
"libclang-cpp${LLVM_VERSION}:$arch" \
"libgcc-s1:$arch" \
"libtinfo-dev:$arch" \
"libz3-dev:$arch" \
"llvm-${LLVM_VERSION}:$arch" \
libclang-cpp${LLVM_VERSION}:$arch \
libgcc-s1:$arch \
libtinfo-dev:$arch \
libz3-dev:$arch \
llvm-${LLVM_VERSION}:$arch \
zlib1g
fi
@@ -74,19 +68,19 @@ fi
# dependencies where we want a specific version
MULTIARCH_PATH=$(dpkg-architecture -A $arch -qDEB_TARGET_MULTIARCH)
export EXTRA_MESON_ARGS="--cross-file=/cross_file-${arch}.txt -D libdir=lib/${MULTIARCH_PATH}"
EXTRA_MESON_ARGS="--cross-file=/cross_file-${arch}.txt -D libdir=lib/$(dpkg-architecture -A $arch -qDEB_TARGET_MULTIARCH)"
. .gitlab-ci/container/build-libdrm.sh
. .gitlab-ci/container/build-wayland.sh
. .gitlab-ci/container/build-directx-headers.sh
apt-get purge -y "${EPHEMERAL[@]}"
apt-get purge -y \
$STABLE_EPHEMERAL
. .gitlab-ci/container/container_post_build.sh
# This needs to be done after container_post_build.sh, or apt-get breaks in there
if [[ $arch != "armhf" ]]; then
apt-get download llvm-"${LLVM_VERSION}"-{dev,tools}:"$arch"
dpkg -i --force-depends llvm-"${LLVM_VERSION}"-*_"${arch}".deb
rm llvm-"${LLVM_VERSION}"-*_"${arch}".deb
apt-get download llvm-${LLVM_VERSION}-{dev,tools}:$arch
dpkg -i --force-depends llvm-${LLVM_VERSION}-*_${arch}.deb
rm llvm-${LLVM_VERSION}-*_${arch}.deb
fi

View File

@@ -1,19 +1,15 @@
#!/usr/bin/env bash
# shellcheck disable=SC2086 # we want word splitting
# When changing this file, you need to bump the following
# .gitlab-ci/image-tags.yml tags:
# DEBIAN_BUILD_TAG
set -ex
EPHEMERAL=(
autoconf
rdfind
unzip
)
EPHEMERAL="\
autoconf \
rdfind \
unzip \
"
apt-get install -y --no-remove "${EPHEMERAL[@]}"
apt-get install -y --no-remove $EPHEMERAL
# Fetch the NDK and extract just the toolchain we want.
ndk=$ANDROID_NDK
@@ -25,7 +21,7 @@ rm $ndk.zip
# duplicate files. Turn them into hardlinks to save on container space.
rdfind -makehardlinks true -makeresultsfile false /${ndk}/
# Drop some large tools we won't use in this build.
find /${ndk}/ -type f \( -iname '*clang-check*' -o -iname '*clang-tidy*' -o -iname '*lldb*' \) -exec rm -f {} \;
find /${ndk}/ -type f | grep -E -i "clang-check|clang-tidy|lldb" | xargs rm -f
sh .gitlab-ci/container/create-android-ndk-pc.sh /$ndk zlib.pc "" "-lz" "1.2.3" $ANDROID_SDK_VERSION
@@ -34,13 +30,33 @@ sh .gitlab-ci/container/create-android-cross-file.sh /$ndk i686-linux-android x8
sh .gitlab-ci/container/create-android-cross-file.sh /$ndk aarch64-linux-android aarch64 armv8 $ANDROID_SDK_VERSION
sh .gitlab-ci/container/create-android-cross-file.sh /$ndk arm-linux-androideabi arm armv7hl $ANDROID_SDK_VERSION armv7a-linux-androideabi
# Not using build-libdrm.sh because we don't want its cleanup after building
# each arch. Fetch and extract now.
export LIBDRM_VERSION=libdrm-2.4.114
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-O https://dri.freedesktop.org/libdrm/$LIBDRM_VERSION.tar.xz
tar -xf $LIBDRM_VERSION.tar.xz && rm $LIBDRM_VERSION.tar.xz
for arch in \
x86_64-linux-android \
i686-linux-android \
aarch64-linux-android \
arm-linux-androideabi ; do
EXTRA_MESON_ARGS="--cross-file=/cross_file-$arch.txt --libdir=lib/$arch -Dnouveau=disabled -Dintel=disabled" \
. .gitlab-ci/container/build-libdrm.sh
cd $LIBDRM_VERSION
rm -rf build-$arch
meson setup build-$arch \
--cross-file=/cross_file-$arch.txt \
--libdir=lib/$arch \
-Dnouveau=disabled \
-Dvc4=disabled \
-Detnaviv=disabled \
-Dfreedreno=disabled \
-Dintel=disabled \
-Dcairo-tests=disabled \
-Dvalgrind=disabled
meson install -C build-$arch
cd ..
done
rm -rf $LIBDRM_VERSION
@@ -90,4 +106,4 @@ done
cd ..
rm -rf $LIBELF_VERSION
apt-get purge -y "${EPHEMERAL[@]}"
apt-get purge -y $EPHEMERAL

View File

@@ -8,72 +8,67 @@ export LLVM_VERSION="${LLVM_VERSION:=15}"
apt-get -y install ca-certificates
sed -i -e 's/http:\/\/deb/https:\/\/deb/g' /etc/apt/sources.list.d/*
echo "deb [trusted=yes] https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/raw/${PKG_REPO_REV}/ ${FDO_DISTRIBUTION_VERSION%-*} main" | tee /etc/apt/sources.list.d/gfx-ci_.list
apt-get update
# Ephemeral packages (installed for this script and removed again at the end)
EPHEMERAL=(
libssl-dev
)
# Ephemeral packages (installed for this script and removed again at
# the end)
STABLE_EPHEMERAL=" \
libssl-dev \
"
DEPS=(
apt-utils
android-libext4-utils
autoconf
automake
bc
bison
ccache
cmake
curl
fastboot
flatbuffers-compiler
flex
g++
git
glslang-tools
kmod
libasan8
libdrm-dev
libelf-dev
libexpat1-dev
libflatbuffers-dev
libvulkan-dev
libx11-dev
libx11-xcb-dev
libxcb-dri2-0-dev
libxcb-dri3-dev
libxcb-glx0-dev
libxcb-present-dev
libxcb-randr0-dev
libxcb-shm0-dev
libxcb-xfixes0-dev
libxdamage-dev
libxext-dev
libxrandr-dev
libxshmfence-dev
libxtensor-dev
libxxf86vm-dev
libwayland-dev
libwayland-egl-backend-dev
"llvm-${LLVM_VERSION}-dev"
ninja-build
meson
openssh-server
pkgconf
python3-mako
python3-pil
python3-pip
python3-pycparser
python3-requests
python3-setuptools
u-boot-tools
xz-utils
zlib1g-dev
zstd
)
apt-get -y install "${DEPS[@]}" "${EPHEMERAL[@]}"
apt-get -y install \
${STABLE_EPHEMERAL} \
apt-utils \
android-libext4-utils \
autoconf \
automake \
bc \
bison \
ccache \
cmake \
curl \
debootstrap \
fastboot \
flex \
g++ \
git \
glslang-tools \
kmod \
libasan8 \
libdrm-dev \
libelf-dev \
libexpat1-dev \
libvulkan-dev \
libx11-dev \
libx11-xcb-dev \
libxcb-dri2-0-dev \
libxcb-dri3-dev \
libxcb-glx0-dev \
libxcb-present-dev \
libxcb-randr0-dev \
libxcb-shm0-dev \
libxcb-xfixes0-dev \
libxdamage-dev \
libxext-dev \
libxrandr-dev \
libxshmfence-dev \
libxxf86vm-dev \
libwayland-dev \
libwayland-egl-backend-dev \
llvm-${LLVM_VERSION}-dev \
ninja-build \
meson \
openssh-server \
pkgconf \
python3-mako \
python3-pil \
python3-pip \
python3-requests \
python3-setuptools \
u-boot-tools \
xz-utils \
zlib1g-dev \
zstd
pip3 install --break-system-packages git+http://gitlab.freedesktop.org/freedesktop/ci-templates@ffe4d1b10aab7534489f0c4bbc4c5899df17d3f2
@@ -86,6 +81,6 @@ arch=armhf
. .gitlab-ci/container/build-wayland.sh
apt-get purge -y "${EPHEMERAL[@]}"
apt-get purge -y $STABLE_EPHEMERAL
. .gitlab-ci/container/container_post_build.sh

View File

@@ -9,29 +9,23 @@ set -e
set -o xtrace
############### Install packages for baremetal testing
DEPS=(
cpio
curl
fastboot
netcat-openbsd
openssh-server
procps
python3-distutils
python3-filelock
python3-fire
python3-minimal
python3-serial
rsync
snmp
zstd
)
apt-get install -y ca-certificates
sed -i -e 's/http:\/\/deb/https:\/\/deb/g' /etc/apt/sources.list.d/*
echo "deb [trusted=yes] https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/raw/${PKG_REPO_REV}/ ${FDO_DISTRIBUTION_VERSION%-*} main" | tee /etc/apt/sources.list.d/gfx-ci_.list
apt-get update
apt-get install -y --no-remove "${DEPS[@]}"
apt-get install -y --no-remove \
cpio \
curl \
fastboot \
netcat-openbsd \
openssh-server \
procps \
python3-distutils \
python3-minimal \
python3-serial \
rsync \
snmp \
zstd
# setup SNMPv2 SMI MIB
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \

View File

@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
arch=ppc64el

View File

@@ -1,18 +1,16 @@
#!/usr/bin/env bash
#!/bin/bash
set -e
arch=s390x
# Ephemeral packages (installed for this script and removed again at the end)
EPHEMERAL=(
libssl-dev
)
STABLE_EPHEMERAL="libssl-dev"
apt-get -y install "${EPHEMERAL[@]}"
apt-get -y install "$STABLE_EPHEMERAL"
. .gitlab-ci/container/build-mold.sh
apt-get purge -y "${EPHEMERAL[@]}"
apt-get purge -y "$STABLE_EPHEMERAL"
. .gitlab-ci/container/cross_build.sh

View File

@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
arch=i386

View File

@@ -0,0 +1,15 @@
#!/usr/bin/env bash
set -e
set -o xtrace
# Installing wine, need this for testing mingw or nine
apt-get update
apt-get install -y --no-remove \
wine \
wine64 \
xvfb
# Used to initialize the Wine environment to reduce build time
wine wineboot.exe --init

View File

@@ -1,10 +1,6 @@
#!/usr/bin/env bash
#!/bin/bash
# shellcheck disable=SC2086 # we want word splitting
# When changing this file, you need to bump the following
# .gitlab-ci/image-tags.yml tags:
# DEBIAN_BUILD_TAG
set -e
set -o xtrace
@@ -13,95 +9,85 @@ export LLVM_VERSION="${LLVM_VERSION:=15}"
apt-get install -y ca-certificates
sed -i -e 's/http:\/\/deb/https:\/\/deb/g' /etc/apt/sources.list.d/*
echo "deb [trusted=yes] https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/raw/${PKG_REPO_REV}/ ${FDO_DISTRIBUTION_VERSION%-*} main" | tee /etc/apt/sources.list.d/gfx-ci_.list
# Ephemeral packages (installed for this script and removed again at
# the end)
EPHEMERAL=(
)
DEPS=(
apt-utils
bison
ccache
curl
"clang-${LLVM_VERSION}"
"clang-format-${LLVM_VERSION}"
dpkg-cross
findutils
flex
flatbuffers-compiler
g++
cmake
gcc
git
glslang-tools
kmod
"libclang-${LLVM_VERSION}-dev"
"libclang-cpp${LLVM_VERSION}-dev"
"libclang-common-${LLVM_VERSION}-dev"
libelf-dev
libepoxy-dev
libexpat1-dev
libflatbuffers-dev
libgtk-3-dev
"libllvm${LLVM_VERSION}"
libomxil-bellagio-dev
libpciaccess-dev
libunwind-dev
libva-dev
libvdpau-dev
libvulkan-dev
libx11-dev
libx11-xcb-dev
libxext-dev
libxml2-utils
libxrandr-dev
libxrender-dev
libxshmfence-dev
libxtensor-dev
libxxf86vm-dev
libwayland-egl-backend-dev
make
ninja-build
openssh-server
pkgconf
python3-mako
python3-pil
python3-pip
python3-ply
python3-pycparser
python3-requests
python3-setuptools
qemu-user
valgrind
x11proto-dri2-dev
x11proto-gl-dev
x11proto-randr-dev
xz-utils
zlib1g-dev
zstd
)
STABLE_EPHEMERAL=" \
"
apt-get update
apt-get install -y --no-remove "${DEPS[@]}" "${EPHEMERAL[@]}" \
$EXTRA_LOCAL_PACKAGES
. .gitlab-ci/container/build-llvm-spirv.sh
. .gitlab-ci/container/build-libclc.sh
apt-get install -y --no-remove \
$EXTRA_LOCAL_PACKAGES \
$STABLE_EPHEMERAL \
apt-utils \
bison \
ccache \
curl \
clang-format-${LLVM_VERSION} \
dpkg-cross \
findutils \
flex \
g++ \
cmake \
gcc \
git \
glslang-tools \
kmod \
libclang-${LLVM_VERSION}-dev \
libclang-cpp${LLVM_VERSION}-dev \
libclang-common-${LLVM_VERSION}-dev \
libelf-dev \
libepoxy-dev \
libexpat1-dev \
libgtk-3-dev \
libllvm${LLVM_VERSION} \
libomxil-bellagio-dev \
libpciaccess-dev \
libunwind-dev \
libva-dev \
libvdpau-dev \
libvulkan-dev \
libx11-dev \
libx11-xcb-dev \
libxext-dev \
libxml2-utils \
libxrandr-dev \
libxrender-dev \
libxshmfence-dev \
libxxf86vm-dev \
libwayland-egl-backend-dev \
make \
ninja-build \
openssh-server \
pkgconf \
python3-mako \
python3-pil \
python3-pip \
python3-ply \
python3-requests \
python3-setuptools \
qemu-user \
valgrind \
x11proto-dri2-dev \
x11proto-gl-dev \
x11proto-randr-dev \
xz-utils \
zlib1g-dev \
zstd
# Needed for ci-fairy, this revision is able to upload files to S3
pip3 install --break-system-packages git+http://gitlab.freedesktop.org/freedesktop/ci-templates@ffe4d1b10aab7534489f0c4bbc4c5899df17d3f2
# We need at least 1.3.1 for rusticl
pip3 install --break-system-packages 'meson==1.3.1'
# We need at least 1.2 for Rust's `debug_assertions`
pip3 install --break-system-packages meson==1.2.0
. .gitlab-ci/container/build-rust.sh
. .gitlab-ci/container/debian/x86_64_build-base-wine.sh
############### Uninstall ephemeral packages
apt-get purge -y "${EPHEMERAL[@]}"
apt-get purge -y $STABLE_EPHEMERAL
. .gitlab-ci/container/container_post_build.sh

View File

@@ -0,0 +1,78 @@
#!/bin/bash
# shellcheck disable=SC2086 # we want word splitting
set -e
# Pull packages from msys2 repository that can be directly used.
# We can use https://packages.msys2.org/ to retrieve the newest package
mkdir ~/tmp
pushd ~/tmp
MINGW_PACKET_LIST="
mingw-w64-x86_64-headers-git-10.0.0.r14.ga08c638f8-1-any.pkg.tar.zst
mingw-w64-x86_64-vulkan-loader-1.3.211-1-any.pkg.tar.zst
mingw-w64-x86_64-libelf-0.8.13-6-any.pkg.tar.zst
mingw-w64-x86_64-zlib-1.2.12-1-any.pkg.tar.zst
mingw-w64-x86_64-zstd-1.5.2-2-any.pkg.tar.zst
"
for i in $MINGW_PACKET_LIST
do
curl -L -s --retry 4 -f --retry-all-errors --retry-delay 60 \
-O "https://mirror.msys2.org/mingw/mingw64/$i"
tar xf $i --strip-components=1 -C /usr/x86_64-w64-mingw32/
done
popd
rm -rf ~/tmp
mkdir -p /usr/x86_64-w64-mingw32/bin
# The output of `wine64 llvm-config --system-libs --cxxflags mcdisassembler`
# containes absolute path like '-IZ:'
# The sed is used to replace `-IZ:/usr/x86_64-w64-mingw32/include`
# to `-I/usr/x86_64-w64-mingw32/include`
# Debian's pkg-config wrapers for mingw are broken, and there's no sign that
# they're going to be fixed, so we'll just have to fix it ourselves
# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=930492
cat >/usr/x86_64-w64-mingw32/bin/pkg-config <<EOF
#!/bin/sh
PKG_CONFIG_LIBDIR=/usr/x86_64-w64-mingw32/lib/pkgconfig:/usr/x86_64-w64-mingw32/share/pkgconfig pkg-config \$@
EOF
chmod +x /usr/x86_64-w64-mingw32/bin/pkg-config
cat >/usr/x86_64-w64-mingw32/bin/llvm-config <<EOF
#!/bin/sh
wine64 llvm-config \$@ | sed -e "s,Z:/,/,gi"
EOF
chmod +x /usr/x86_64-w64-mingw32/bin/llvm-config
cat >/usr/x86_64-w64-mingw32/bin/clang <<EOF
#!/bin/sh
wine64 clang \$@
EOF
chmod +x /usr/x86_64-w64-mingw32/bin/clang
cat >/usr/x86_64-w64-mingw32/bin/llvm-as <<EOF
#!/bin/sh
wine64 llvm-as \$@
EOF
chmod +x /usr/x86_64-w64-mingw32/bin/llvm-as
cat >/usr/x86_64-w64-mingw32/bin/llvm-link <<EOF
#!/bin/sh
wine64 llvm-link \$@
EOF
chmod +x /usr/x86_64-w64-mingw32/bin/llvm-link
cat >/usr/x86_64-w64-mingw32/bin/opt <<EOF
#!/bin/sh
wine64 opt \$@
EOF
chmod +x /usr/x86_64-w64-mingw32/bin/opt
cat >/usr/x86_64-w64-mingw32/bin/llvm-spirv <<EOF
#!/bin/sh
wine64 llvm-spirv \$@
EOF
chmod +x /usr/x86_64-w64-mingw32/bin/llvm-spirv

View File

@@ -0,0 +1,125 @@
#!/bin/bash
# shellcheck disable=SC2086 # we want word splitting
set -e
# Building libdrm (libva dependency)
. .gitlab-ci/container/build-libdrm.sh
wd=$PWD
CMAKE_TOOLCHAIN_MINGW_PATH=$wd/.gitlab-ci/container/debian/x86_mingw-toolchain.cmake
mkdir -p ~/tmp
pushd ~/tmp
# Building DirectX-Headers
git clone https://github.com/microsoft/DirectX-Headers -b v1.606.4 --depth 1
mkdir -p DirectX-Headers/build
pushd DirectX-Headers/build
meson .. \
--backend=ninja \
--buildtype=release -Dbuild-test=false \
-Dprefix=/usr/x86_64-w64-mingw32/ \
--cross-file=$wd/.gitlab-ci/x86_64-w64-mingw32
ninja install
popd
# Building libva
git clone https://github.com/intel/libva
pushd libva/
# libva-win32 is released with libva version 2.17 (see https://github.com/intel/libva/releases/tag/2.17.0)
git checkout 2.17.0
popd
# libva already has a build dir in their repo, use builddir instead
mkdir -p libva/builddir
pushd libva/builddir
meson .. \
--backend=ninja \
--buildtype=release \
-Dprefix=/usr/x86_64-w64-mingw32/ \
--cross-file=$wd/.gitlab-ci/x86_64-w64-mingw32
ninja install
popd
export VULKAN_SDK_VERSION=1.3.211.0
# Building SPIRV Tools
git clone -b sdk-$VULKAN_SDK_VERSION --depth=1 \
https://github.com/KhronosGroup/SPIRV-Tools SPIRV-Tools
git clone -b sdk-$VULKAN_SDK_VERSION --depth=1 \
https://github.com/KhronosGroup/SPIRV-Headers SPIRV-Tools/external/SPIRV-Headers
mkdir -p SPIRV-Tools/build
pushd SPIRV-Tools/build
cmake .. \
-DCMAKE_TOOLCHAIN_FILE=$CMAKE_TOOLCHAIN_MINGW_PATH \
-DCMAKE_INSTALL_PREFIX=/usr/x86_64-w64-mingw32/ \
-GNinja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CROSSCOMPILING=1 \
-DCMAKE_POLICY_DEFAULT_CMP0091=NEW
ninja install
popd
# Building LLVM
git clone -b release/15.x --depth=1 \
https://github.com/llvm/llvm-project llvm-project
git clone -b v15.0.0 --depth=1 \
https://github.com/KhronosGroup/SPIRV-LLVM-Translator llvm-project/llvm/projects/SPIRV-LLVM-Translator
mkdir llvm-project/build
pushd llvm-project/build
cmake ../llvm \
-DCMAKE_TOOLCHAIN_FILE=$CMAKE_TOOLCHAIN_MINGW_PATH \
-DCMAKE_INSTALL_PREFIX=/usr/x86_64-w64-mingw32/ \
-GNinja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CROSSCOMPILING=1 \
-DLLVM_ENABLE_RTTI=ON \
-DCROSS_TOOLCHAIN_FLAGS_NATIVE=-DLLVM_EXTERNAL_SPIRV_HEADERS_SOURCE_DIR=$PWD/../../SPIRV-Tools/external/SPIRV-Headers \
-DLLVM_EXTERNAL_SPIRV_HEADERS_SOURCE_DIR=$PWD/../../SPIRV-Tools/external/SPIRV-Headers \
-DLLVM_ENABLE_PROJECTS="clang" \
-DLLVM_TARGETS_TO_BUILD="AMDGPU;X86" \
-DLLVM_OPTIMIZED_TABLEGEN=TRUE \
-DLLVM_ENABLE_ASSERTIONS=TRUE \
-DLLVM_INCLUDE_UTILS=OFF \
-DLLVM_INCLUDE_RUNTIMES=OFF \
-DLLVM_INCLUDE_TESTS=OFF \
-DLLVM_INCLUDE_EXAMPLES=OFF \
-DLLVM_INCLUDE_GO_TESTS=OFF \
-DLLVM_INCLUDE_BENCHMARKS=OFF \
-DLLVM_BUILD_LLVM_C_DYLIB=OFF \
-DLLVM_ENABLE_DIA_SDK=OFF \
-DCLANG_BUILD_TOOLS=ON \
-DLLVM_SPIRV_INCLUDE_TESTS=OFF
ninja install
popd
# Building libclc
mkdir llvm-project/build-libclc
pushd llvm-project/build-libclc
cmake ../libclc \
-DCMAKE_TOOLCHAIN_FILE=$CMAKE_TOOLCHAIN_MINGW_PATH \
-DCMAKE_INSTALL_PREFIX=/usr/x86_64-w64-mingw32/ \
-GNinja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CROSSCOMPILING=1 \
-DCMAKE_POLICY_DEFAULT_CMP0091=NEW \
-DCMAKE_CXX_FLAGS="-m64" \
-DLLVM_CONFIG="/usr/x86_64-w64-mingw32/bin/llvm-config" \
-DLLVM_CLANG="/usr/x86_64-w64-mingw32/bin/clang" \
-DLLVM_AS="/usr/x86_64-w64-mingw32/bin/llvm-as" \
-DLLVM_LINK="/usr/x86_64-w64-mingw32/bin/llvm-link" \
-DLLVM_OPT="/usr/x86_64-w64-mingw32/bin/opt" \
-DLLVM_SPIRV="/usr/x86_64-w64-mingw32/bin/llvm-spirv" \
-DLIBCLC_TARGETS_TO_BUILD="spirv-mesa3d-;spirv64-mesa3d-"
ninja install
popd
popd # ~/tmp
# Cleanup ~/tmp
rm -rf ~/tmp

View File

@@ -0,0 +1,13 @@
#!/usr/bin/env bash
set -e
set -o xtrace
apt-get update
apt-get install -y --no-remove \
zstd \
g++-mingw-w64-i686 \
g++-mingw-w64-x86-64
. .gitlab-ci/container/debian/x86_64_build-mingw-patch.sh
. .gitlab-ci/container/debian/x86_64_build-mingw-source-deps.sh

View File

@@ -12,52 +12,49 @@ export DEBIAN_FRONTEND=noninteractive
export LLVM_VERSION="${LLVM_VERSION:=15}"
# Ephemeral packages (installed for this script and removed again at the end)
EPHEMERAL=(
autoconf
automake
autotools-dev
bzip2
libtool
libssl-dev
)
DEPS=(
check
"clang-${LLVM_VERSION}"
libasan8
libarchive-dev
libdrm-dev
"libclang-cpp${LLVM_VERSION}-dev"
libgbm-dev
libglvnd-dev
liblua5.3-dev
libxcb-dri2-0-dev
libxcb-dri3-dev
libxcb-glx0-dev
libxcb-present-dev
libxcb-randr0-dev
libxcb-shm0-dev
libxcb-sync-dev
libxcb-xfixes0-dev
libxcb1-dev
libxml2-dev
"llvm-${LLVM_VERSION}-dev"
ocl-icd-opencl-dev
python3-pip
python3-venv
procps
spirv-tools
shellcheck
strace
time
yamllint
zstd
)
STABLE_EPHEMERAL=" \
autoconf \
automake \
autotools-dev \
bzip2 \
libtool \
libssl-dev \
"
apt-get update
apt-get install -y --no-remove \
"${DEPS[@]}" "${EPHEMERAL[@]}"
$STABLE_EPHEMERAL \
check \
clang-${LLVM_VERSION} \
libasan8 \
libarchive-dev \
libdrm-dev \
libclang-cpp${LLVM_VERSION}-dev \
libgbm-dev \
libglvnd-dev \
liblua5.3-dev \
libxcb-dri2-0-dev \
libxcb-dri3-dev \
libxcb-glx0-dev \
libxcb-present-dev \
libxcb-randr0-dev \
libxcb-shm0-dev \
libxcb-sync-dev \
libxcb-xfixes0-dev \
libxcb1-dev \
libxml2-dev \
llvm-${LLVM_VERSION}-dev \
ocl-icd-opencl-dev \
python3-pip \
python3-venv \
procps \
spirv-tools \
shellcheck \
strace \
time \
yamllint \
zstd
. .gitlab-ci/container/container_pre_build.sh
@@ -75,11 +72,20 @@ tar -xvf $XORGMACROS_VERSION.tar.bz2 && rm $XORGMACROS_VERSION.tar.bz2
cd $XORGMACROS_VERSION; ./configure; make install; cd ..
rm -rf $XORGMACROS_VERSION
. .gitlab-ci/container/build-llvm-spirv.sh
. .gitlab-ci/container/build-libclc.sh
. .gitlab-ci/container/build-wayland.sh
. .gitlab-ci/container/build-shader-db.sh
. .gitlab-ci/container/build-directx-headers.sh
git clone https://github.com/microsoft/DirectX-Headers -b v1.711.3-preview --depth 1
pushd DirectX-Headers
meson setup build --backend=ninja --buildtype=release -Dbuild-test=false
meson install -C build
popd
rm -rf DirectX-Headers
python3 -m pip install --break-system-packages -r .gitlab-ci/lava/requirements.txt
@@ -90,15 +96,9 @@ RUSTFLAGS='-L native=/usr/local/lib' cargo install \
-j ${FDO_CI_CONCURRENT:-4} \
--root /usr/local
# install cbindgen
RUSTFLAGS='-L native=/usr/local/lib' cargo install \
cbindgen --version 0.26.0 \
--locked \
-j ${FDO_CI_CONCURRENT:-4} \
--root /usr/local
############### Uninstall the build software
apt-get purge -y "${EPHEMERAL[@]}"
apt-get purge -y \
$STABLE_EPHEMERAL
. .gitlab-ci/container/container_post_build.sh

View File

@@ -0,0 +1,8 @@
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR x86_64)
set(CMAKE_SYSROOT /usr/x86_64-w64-mingw32/)
set(ENV{PKG_CONFIG} /usr/x86_64-w64-mingw32/bin/pkgconf)
set(CMAKE_C_COMPILER x86_64-w64-mingw32-gcc-posix)
set(CMAKE_CXX_COMPILER x86_64-w64-mingw32-g++-posix)

View File

@@ -1,6 +1,7 @@
#!/usr/bin/env bash
# The relative paths in this file only become valid at runtime.
# shellcheck disable=SC1091
# shellcheck disable=SC2086 # we want word splitting
set -e
set -o xtrace
@@ -8,22 +9,20 @@ set -o xtrace
export DEBIAN_FRONTEND=noninteractive
# Ephemeral packages (installed for this script and removed again at the end)
EPHEMERAL=(
build-essential:native
ccache
cmake
config-package-dev
debhelper-compat
dpkg-dev
ninja-build
unzip
)
STABLE_EPHEMERAL=" \
ccache \
unzip \
dpkg-dev \
build-essential:native \
config-package-dev \
debhelper-compat \
cmake \
ninja-build \
"
DEPS=(
iproute2
)
apt-get install -y --no-remove --no-install-recommends \
"${DEPS[@]}" "${EPHEMERAL[@]}"
$STABLE_EPHEMERAL \
iproute2
############### Building ...
@@ -34,14 +33,13 @@ apt-get install -y --no-remove --no-install-recommends \
# Fetch the NDK and extract just the toolchain we want.
ndk=$ANDROID_NDK
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-o "$ndk.zip" "https://dl.google.com/android/repository/$ndk-linux.zip"
unzip -d / "$ndk.zip"
rm "$ndk.zip"
-o $ndk.zip https://dl.google.com/android/repository/$ndk-linux.zip
unzip -d / $ndk.zip
rm $ndk.zip
############### Build dEQP runner
export ANDROID_NDK_HOME=/$ndk
export RUST_TARGET=x86_64-linux-android
. .gitlab-ci/container/build-rust.sh
. .gitlab-ci/container/build-deqp-runner.sh
@@ -50,12 +48,6 @@ rm -rf /root/.rustup
############### Build dEQP GL
DEQP_API=GL \
DEQP_TARGET="android" \
EXTRA_CMAKE_ARGS="-DDEQP_TARGET_TOOLCHAIN=ndk-modern -DANDROID_NDK_PATH=/$ndk -DANDROID_ABI=x86_64 -DDE_ANDROID_API=28" \
. .gitlab-ci/container/build-deqp.sh
DEQP_API=GLES \
DEQP_TARGET="android" \
EXTRA_CMAKE_ARGS="-DDEQP_TARGET_TOOLCHAIN=ndk-modern -DANDROID_NDK_PATH=/$ndk -DANDROID_ABI=x86_64 -DDE_ANDROID_API=28" \
. .gitlab-ci/container/build-deqp.sh
@@ -80,14 +72,8 @@ popd
############### Building and installing Debian package ...
ANDROID_CUTTLEFISH_VERSION=f6494d9fbeaa9974b56923e3029909e5d5f440dd
mkdir android-cuttlefish
git clone --depth 1 https://github.com/google/android-cuttlefish.git
pushd android-cuttlefish
git init
git remote add origin https://github.com/google/android-cuttlefish.git
git fetch --depth 1 origin "$ANDROID_CUTTLEFISH_VERSION"
git checkout FETCH_HEAD
pushd base
dpkg-buildpackage -uc -us
@@ -105,6 +91,9 @@ usermod -a -G kvm,cvdnetwork root
rm -rf "/${ndk:?}"
apt-get purge -y "${EPHEMERAL[@]}"
ccache --show-stats
. .gitlab-ci/container/container_post_build.sh
apt-get purge -y \
$STABLE_EPHEMERAL
apt-get autoremove -y --purge

View File

@@ -14,92 +14,51 @@ apt-get install -y ca-certificates gnupg2 software-properties-common
sed -i -e 's/http:\/\/deb/https:\/\/deb/g' /etc/apt/sources.list.d/*
echo "deb [trusted=yes] https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/raw/${PKG_REPO_REV}/ ${FDO_DISTRIBUTION_VERSION%-*} main" | tee /etc/apt/sources.list.d/gfx-ci_.list
export LLVM_VERSION="${LLVM_VERSION:=15}"
# Ephemeral packages (installed for this script and removed again at the end)
EPHEMERAL=(
autoconf
automake
bc
bison
bzip2
ccache
cmake
"clang-${LLVM_VERSION}"
flex
glslang-tools
g++
libasound2-dev
libcap-dev
"libclang-cpp${LLVM_VERSION}-dev"
libdrm-dev
libegl-dev
libelf-dev
libepoxy-dev
libgbm-dev
libpciaccess-dev
libssl-dev
libvulkan-dev
libwayland-dev
libx11-xcb-dev
libxext-dev
"llvm-${LLVM_VERSION}-dev"
make
meson
openssh-server
patch
pkgconf
protobuf-compiler
python3-dev
python3-pip
python3-setuptools
python3-wheel
spirv-tools
wayland-protocols
xz-utils
)
DEPS=(
apt-utils
curl
git
git-lfs
inetutils-syslogd
iptables
jq
libasan8
libdrm2
libexpat1
"libllvm${LLVM_VERSION}"
liblz4-1
libpng16-16
libpython3.11
libvulkan1
libwayland-client0
libwayland-server0
libxcb-ewmh2
libxcb-randr0
libxcb-xfixes0
libxkbcommon0
libxrandr2
libxrender1
python3-mako
python3-numpy
python3-packaging
python3-pil
python3-requests
python3-six
python3-yaml
socat
vulkan-tools
waffle-utils
xauth
xvfb
zlib1g
zstd
)
# Ephemeral packages (installed for this script and removed again at
# the end)
STABLE_EPHEMERAL=" \
autoconf \
automake \
bc \
bison \
bzip2 \
ccache \
cmake \
clang-${LLVM_VERSION} \
flex \
glslang-tools \
g++ \
libasound2-dev \
libcap-dev \
libclang-cpp${LLVM_VERSION}-dev \
libdrm-dev \
libegl-dev \
libelf-dev \
libepoxy-dev \
libgbm-dev \
libpciaccess-dev \
libssl-dev
libvulkan-dev \
libwayland-dev \
libx11-xcb-dev \
libxext-dev \
llvm-${LLVM_VERSION}-dev \
make \
meson \
openssh-server \
patch \
pkgconf \
protobuf-compiler \
python3-dev \
python3-pip \
python3-setuptools \
python3-wheel \
spirv-tools \
wayland-protocols \
xz-utils \
"
apt-get update
apt-get dist-upgrade -y
@@ -107,9 +66,48 @@ apt-get dist-upgrade -y
apt-get install --purge -y \
sysvinit-core libelogind0
apt-get install -y --no-remove "${DEPS[@]}"
apt-get install -y --no-remove \
apt-utils \
curl \
git \
git-lfs \
inetutils-syslogd \
iptables \
jq \
libasan8 \
libdrm2 \
libexpat1 \
libllvm${LLVM_VERSION} \
liblz4-1 \
libpng16-16 \
libpython3.11 \
libvulkan1 \
libwayland-client0 \
libwayland-server0 \
libxcb-ewmh2 \
libxcb-randr0 \
libxcb-xfixes0 \
libxkbcommon0 \
libxrandr2 \
libxrender1 \
python3-mako \
python3-numpy \
python3-packaging \
python3-pil \
python3-requests \
python3-six \
python3-yaml \
socat \
vulkan-tools \
waffle-utils \
xauth \
xvfb \
zlib1g \
zstd
apt-get install -y --no-install-recommends \
$STABLE_EPHEMERAL
apt-get install -y --no-install-recommends "${EPHEMERAL[@]}"
. .gitlab-ci/container/container_pre_build.sh
@@ -152,9 +150,11 @@ pip3 install --break-system-packages yq
############### Build dEQP runner
. .gitlab-ci/container/build-deqp-runner.sh
apt-get purge -y "${EPHEMERAL[@]}"
rm -rf /root/.cargo
rm -rf /root/.rustup
. .gitlab-ci/container/container_post_build.sh
ccache --show-stats
apt-get purge -y $STABLE_EPHEMERAL
apt-get autoremove -y --purge

View File

@@ -10,68 +10,60 @@ export LLVM_VERSION="${LLVM_VERSION:=15}"
apt-get install -y libelogind0 # this interfere with systemd deps, install separately
# Ephemeral packages (installed for this script and removed again at the end)
EPHEMERAL=(
bzip2
ccache
"clang-${LLVM_VERSION}"
cmake
g++
glslang-tools
libasound2-dev
libcap-dev
"libclang-cpp${LLVM_VERSION}-dev"
libdrm-dev
libgles2-mesa-dev
libgtest-dev
libpciaccess-dev
libpng-dev
libudev-dev
libvulkan-dev
libwaffle-dev
libwayland-dev
libx11-xcb-dev
libxcb-dri2-0-dev
libxcb-dri3-dev
libxcb-present-dev
libxfixes-dev
libxkbcommon-dev
libxrandr-dev
libxrender-dev
"llvm-${LLVM_VERSION}-dev"
make
meson
ocl-icd-opencl-dev
patch
pkgconf
python3-distutils
xz-utils
)
DEPS=(
clinfo
iptables
kmod
"libclang-common-${LLVM_VERSION}-dev"
"libclang-cpp${LLVM_VERSION}"
libcap2
libegl1
libepoxy0
libfdt1
libxcb-shm0
ocl-icd-libopencl1
python3-lxml
python3-renderdoc
python3-simplejson
spirv-tools
sysvinit-core
weston
xwayland
)
STABLE_EPHEMERAL=" \
bzip2 \
ccache \
clang-${LLVM_VERSION} \
cmake \
g++ \
glslang-tools \
libasound2-dev \
libcap-dev \
libclang-cpp${LLVM_VERSION}-dev \
libdrm-dev \
libgles2-mesa-dev \
libpciaccess-dev \
libpng-dev \
libudev-dev \
libvulkan-dev \
libwaffle-dev \
libwayland-dev \
libx11-xcb-dev \
libxcb-dri2-0-dev \
libxkbcommon-dev \
libxrandr-dev \
libxrender-dev \
llvm-${LLVM_VERSION}-dev \
make \
meson \
ocl-icd-opencl-dev \
patch \
pkgconf \
python3-distutils \
xz-utils \
"
apt-get update
apt-get install -y --no-remove "${DEPS[@]}" "${EPHEMERAL[@]}" \
$EXTRA_LOCAL_PACKAGES
apt-get install -y --no-remove \
$EXTRA_LOCAL_PACKAGES \
$STABLE_EPHEMERAL \
clinfo \
iptables \
libclang-common-${LLVM_VERSION}-dev \
libclang-cpp${LLVM_VERSION} \
libcap2 \
libegl1 \
libepoxy0 \
libfdt1 \
libxcb-shm0 \
ocl-icd-libopencl1 \
python3-lxml \
python3-renderdoc \
python3-simplejson \
spirv-tools \
sysvinit-core \
weston
. .gitlab-ci/container/container_pre_build.sh
@@ -82,13 +74,7 @@ PIGLIT_OPTS="-DPIGLIT_BUILD_GLX_TESTS=ON -DPIGLIT_BUILD_CL_TESTS=ON -DPIGLIT_BUI
############### Build dEQP GL
DEQP_API=GL \
DEQP_TARGET=surfaceless \
. .gitlab-ci/container/build-deqp.sh
DEQP_API=GLES \
DEQP_TARGET=surfaceless \
. .gitlab-ci/container/build-deqp.sh
DEQP_TARGET=surfaceless . .gitlab-ci/container/build-deqp.sh
############### Build apitrace
@@ -98,12 +84,11 @@ DEQP_TARGET=surfaceless \
. .gitlab-ci/container/build-vulkan-validation.sh
############### Build nine tests
. .gitlab-ci/container/build-ninetests.sh
############### Uninstall the build software
apt-get purge -y "${EPHEMERAL[@]}"
ccache --show-stats
. .gitlab-ci/container/container_post_build.sh
apt-get purge -y \
$STABLE_EPHEMERAL
apt-get autoremove -y --purge

View File

@@ -11,64 +11,65 @@ export DEBIAN_FRONTEND=noninteractive
apt-get install -y libelogind0 # this interfere with systemd deps, install separately
# Ephemeral packages (installed for this script and removed again at the end)
EPHEMERAL=(
ccache
cmake
g++
glslang-tools
libexpat1-dev
gnupg2
libdrm-dev
libgbm-dev
libgles2-mesa-dev
liblz4-dev
libpciaccess-dev
libudev-dev
libvulkan-dev
libwaffle-dev
libx11-xcb-dev
libxcb-dri2-0-dev
libxcb-ewmh-dev
libxcb-keysyms1-dev
libxkbcommon-dev
libxrandr-dev
libxrender-dev
libzstd-dev
meson
p7zip
patch
pkgconf
python3-dev
python3-distutils
python3-pip
python3-setuptools
python3-wheel
software-properties-common
wine64-tools
xz-utils
)
DEPS=(
curl
libepoxy0
libxcb-shm0
pciutils
python3-lxml
python3-simplejson
sysvinit-core
weston
xwayland
wine
wine64
xinit
xserver-xorg-video-amdgpu
xserver-xorg-video-ati
)
apt-get update
STABLE_EPHEMERAL=" \
ccache \
cmake \
g++ \
g++-mingw-w64-i686-posix \
g++-mingw-w64-x86-64-posix \
glslang-tools \
libexpat1-dev \
gnupg2 \
libdrm-dev \
libgbm-dev \
libgles2-mesa-dev \
liblz4-dev \
libpciaccess-dev \
libudev-dev \
libvulkan-dev \
libwaffle-dev \
libx11-xcb-dev \
libxcb-ewmh-dev \
libxcb-keysyms1-dev \
libxkbcommon-dev \
libxrandr-dev \
libxrender-dev \
libzstd-dev \
meson \
mingw-w64-i686-dev \
mingw-w64-tools \
mingw-w64-x86-64-dev \
p7zip \
patch \
pkgconf \
python3-dev \
python3-distutils \
python3-pip \
python3-setuptools \
python3-wheel \
software-properties-common \
wine64-tools \
xz-utils \
"
apt-get install -y --no-remove --no-install-recommends \
"${DEPS[@]}" "${EPHEMERAL[@]}"
$STABLE_EPHEMERAL \
curl \
libepoxy0 \
libxcb-shm0 \
pciutils \
python3-lxml \
python3-simplejson \
sysvinit-core \
weston \
xwayland \
wine \
wine64 \
xinit \
xserver-xorg-video-amdgpu \
xserver-xorg-video-ati
apt-get update -q
############### Install DXVK
@@ -104,8 +105,6 @@ PIGLIT_BUILD_TARGETS="piglit_replayer" . .gitlab-ci/container/build-piglit.sh
############### Build dEQP VK
DEQP_API=VK \
DEQP_TARGET=default \
. .gitlab-ci/container/build-deqp.sh
############### Build apitrace
@@ -124,6 +123,11 @@ DEQP_TARGET=default \
############### Uninstall the build software
apt-get purge -y "${EPHEMERAL[@]}"
ccache --show-stats
. .gitlab-ci/container/container_post_build.sh
apt-get purge -y \
$STABLE_EPHEMERAL
apt-get autoremove -y --purge
#dpkg -r --force-depends "mesa-vulkan-drivers" "mesa-vdpau-drivers" "mesa-va-drivers" "libgl1-mesa-dri" "libglx-mesa0" "vdpau-driver-all" "va-driver-all" "libglx0" "libgl1" "libvdpau-va-gl1" "libglu1-mesa" "libegl-mesa0" "libgl1-mesa-dri" "libglapi-mesa" "libosmesa6"

View File

@@ -27,11 +27,9 @@ EPHEMERAL=(
DEPS=(
bindgen
bison
cbindgen
ccache
clang-devel
flex
flatbuffers-compiler
gcc
gcc-c++
gettext
@@ -43,7 +41,6 @@ DEPS=(
"pkgconfig(SPIRV-Tools)"
"pkgconfig(dri2proto)"
"pkgconfig(expat)"
"pkgconfig(flatbuffers)"
"pkgconfig(glproto)"
"pkgconfig(libclc)"
"pkgconfig(libelf)"
@@ -69,7 +66,6 @@ DEPS=(
"pkgconfig(xfixes)"
"pkgconfig(xrandr)"
"pkgconfig(xshmfence)"
"pkgconfig(xtensor)"
"pkgconfig(xxf86vm)"
"pkgconfig(zlib)"
procps-ng
@@ -77,7 +73,6 @@ DEPS=(
python3-devel
python3-mako
python3-ply
python3-pycparser
rust-packaging
vulkan-headers
spirv-tools-devel
@@ -101,8 +96,8 @@ tar -xvf $XORGMACROS_VERSION.tar.bz2 && rm $XORGMACROS_VERSION.tar.bz2
cd $XORGMACROS_VERSION; ./configure; make install; cd ..
rm -rf $XORGMACROS_VERSION
# We need at least 1.3.1 for rusticl
pip install meson==1.3.1
# We need at least 1.2 for Rust's `debug_assertions`
pip install meson==1.2.0
. .gitlab-ci/container/build-mold.sh

View File

@@ -23,6 +23,10 @@
variables:
MESA_IMAGE: "$CI_REGISTRY_IMAGE/${MESA_IMAGE_PATH}:${FDO_DISTRIBUTION_TAG}"
.use-wine:
variables:
WINEPATH: "/usr/x86_64-w64-mingw32/bin;/usr/x86_64-w64-mingw32/lib;/usr/lib/gcc/x86_64-w64-mingw32/10-posix;c:/windows;c:/windows/system32"
# Build the CI docker images.
#
# MESA_IMAGE_TAG is the tag of the docker image used by later stage jobs. If the
@@ -40,16 +44,14 @@
# repository's container registry, so that the image from the main
# repository's registry will be used there as well.
.debian-container:
variables:
FDO_DISTRIBUTION_VERSION: bookworm-slim
.container:
stage: container
extends:
- .container+build-rules
- .incorporate-templates-commit
- .use-wine
variables:
FDO_DISTRIBUTION_VERSION: bookworm-slim
FDO_REPO_SUFFIX: $CI_JOB_NAME
FDO_DISTRIBUTION_EXEC: 'bash .gitlab-ci/container/${CI_JOB_NAME}.sh'
# no need to pull the whole repo to build the container image
@@ -65,18 +67,12 @@ debian/x86_64_build-base:
extends:
- .fdo.container-build@debian
- .container
- .debian-container
variables:
MESA_IMAGE_TAG: &debian-x86_64_build-base "${DEBIAN_BASE_TAG}--${PKG_REPO_REV}"
rules:
# python-test requires debian/x86_64_build, which requires this job
- !reference [python-test, rules]
- !reference [.container, rules]
MESA_IMAGE_TAG: &debian-x86_64_build-base ${DEBIAN_BASE_TAG}
.use-debian/x86_64_build-base:
extends:
- .fdo.container-build@debian
- .debian-container
- .use-base-image
variables:
MESA_BASE_IMAGE: ${DEBIAN_X86_64_BUILD_BASE_IMAGE}
@@ -91,10 +87,6 @@ debian/x86_64_build:
- .use-debian/x86_64_build-base
variables:
MESA_IMAGE_TAG: &debian-x86_64_build ${DEBIAN_BUILD_TAG}
rules:
# python-test requires this job
- !reference [python-test, rules]
- !reference [.use-debian/x86_64_build-base, rules]
.use-debian/x86_64_build:
extends:
@@ -123,6 +115,24 @@ debian/x86_32_build:
needs:
- debian/x86_32_build
# Debian based x86_64-mingw cross main build image
# FIXME: Until gets fixed on Debian 12, disabled.
.debian/x86_64_build-mingw:
extends:
- .use-debian/x86_64_build-base
variables:
MESA_IMAGE_TAG: &debian-x86_64_build_mingw ${DEBIAN_BUILD_MINGW_TAG}
.use-debian/x86_64_build_mingw:
extends:
- .set-image-base-tag
variables:
MESA_BASE_TAG: *debian-x86_64_build-base
MESA_IMAGE_PATH: ${DEBIAN_X86_64_BUILD_MINGW_IMAGE_PATH}
MESA_IMAGE_TAG: *debian-x86_64_build_mingw
needs:
- .debian/x86_64_build-mingw
# Debian based ppc64el cross-build image
debian/ppc64el_build:
extends:
@@ -180,12 +190,11 @@ debian/android_build:
debian/x86_64_test-base:
extends: debian/x86_64_build-base
variables:
MESA_IMAGE_TAG: &debian-x86_64_test-base "${DEBIAN_BASE_TAG}--${PKG_REPO_REV}--${KERNEL_TAG}"
MESA_IMAGE_TAG: &debian-x86_64_test-base "${DEBIAN_BASE_TAG}--${KERNEL_TAG}"
.use-debian/x86_64_test-base:
extends:
- .fdo.container-build@debian
- .debian-container
- .use-base-image
variables:
MESA_BASE_IMAGE: ${DEBIAN_X86_64_TEST_BASE_IMAGE}
@@ -226,7 +235,7 @@ debian/x86_64_test-vk:
- debian/x86_64_test-vk
# Debian based x86_64 test image for Android
.debian/x86_64_test-android:
debian/x86_64_test-android:
extends: .use-debian/x86_64_test-base
variables:
MESA_IMAGE_TAG: &debian-x86_64_test-android ${DEBIAN_X86_64_TEST_ANDROID_TAG}
@@ -247,11 +256,10 @@ debian/arm64_build:
extends:
- .fdo.container-build@debian
- .container
- .debian-container
tags:
- aarch64
variables:
MESA_IMAGE_TAG: &debian-arm64_build "${DEBIAN_BASE_TAG}--${PKG_REPO_REV}"
MESA_IMAGE_TAG: &debian-arm64_build ${DEBIAN_BASE_TAG}
.use-debian/arm64_build:
extends:
@@ -270,8 +278,7 @@ debian/arm64_build:
- .fdo.container-build@alpine
- .container
variables:
FDO_DISTRIBUTION_VERSION: "edge" # switch to 3.20 when gets released
FDO_BASE_IMAGE: alpine:$FDO_DISTRIBUTION_VERSION # since cbuild ignores it
FDO_DISTRIBUTION_VERSION: "3.18"
# Alpine based x86_64 build image
alpine/x86_64_build:
@@ -318,7 +325,6 @@ fedora/x86_64_build:
.kernel+rootfs:
extends:
- .container+build-rules
- .debian-container
stage: container
variables:
GIT_STRATEGY: fetch
@@ -367,12 +373,11 @@ kernel+rootfs_arm32:
extends:
- .fdo.container-build@debian
- .container
- .debian-container
# Don't want the .container rules
- .container+build-rules
variables:
FDO_DISTRIBUTION_TAG: "${MESA_IMAGE_TAG}--${MESA_ROOTFS_TAG}--${KERNEL_TAG}--${MESA_TEMPLATES_COMMIT}"
ARTIFACTS_PREFIX: "https://${S3_HOST}/${S3_KERNEL_BUCKET}"
ARTIFACTS_PREFIX: "https://${S3_HOST}/mesa-lava"
ARTIFACTS_SUFFIX: "${MESA_ROOTFS_TAG}--${KERNEL_TAG}--${MESA_ARTIFACTS_TAG}--${MESA_TEMPLATES_COMMIT}"
MESA_ARTIFACTS_TAG: *debian-arm64_build
MESA_ROOTFS_TAG: *kernel-rootfs
@@ -383,7 +388,7 @@ debian/arm32_test:
needs:
- kernel+rootfs_arm32
variables:
MESA_IMAGE_TAG: &debian-arm32_test "${DEBIAN_BASE_TAG}--${PKG_REPO_REV}"
MESA_IMAGE_TAG: &debian-arm32_test ${DEBIAN_BASE_TAG}
debian/arm64_test:
extends:
@@ -391,7 +396,7 @@ debian/arm64_test:
needs:
- kernel+rootfs_arm64
variables:
MESA_IMAGE_TAG: &debian-arm64_test "${DEBIAN_BASE_TAG}--${PKG_REPO_REV}"
MESA_IMAGE_TAG: &debian-arm64_test ${DEBIAN_BASE_TAG}
.use-debian/arm_test:
variables:
@@ -429,7 +434,7 @@ debian/arm64_test:
# (host PowerShell) in the container stage to build and push the image,
# then in the build stage by executing inside Docker.
.windows-docker-msvc:
.windows-docker-vs2019:
variables:
MESA_IMAGE: "$CI_REGISTRY_IMAGE/${MESA_IMAGE_PATH}:${MESA_IMAGE_TAG}"
MESA_UPSTREAM_IMAGE: "$CI_REGISTRY/$FDO_UPSTREAM_REPO/$MESA_IMAGE_PATH:${MESA_IMAGE_TAG}"
@@ -441,7 +446,7 @@ debian/arm64_test:
default: [retry]
extends:
- .container
- .windows-docker-msvc
- .windows-docker-vs2019
- .windows-shell-tags
rules:
- !reference [.microsoft-farm-container-rules, rules]
@@ -452,18 +457,18 @@ debian/arm64_test:
script:
- .\.gitlab-ci\windows\mesa_container.ps1 $CI_REGISTRY $CI_REGISTRY_USER $CI_REGISTRY_PASSWORD $MESA_IMAGE $MESA_UPSTREAM_IMAGE ${DOCKERFILE} ${MESA_BASE_IMAGE}
windows_msvc:
windows_vs2019:
inherit:
default: [retry]
extends:
- .windows_container_build
variables:
MESA_IMAGE_PATH: &windows_msvc_image_path ${WINDOWS_X64_MSVC_PATH}
MESA_IMAGE_TAG: ${WINDOWS_X64_MSVC_TAG}
DOCKERFILE: Dockerfile_msvc
MESA_IMAGE_PATH: &windows_vs_image_path ${WINDOWS_X64_VS_PATH}
MESA_IMAGE_TAG: &windows_vs_image_tag ${WINDOWS_X64_VS_TAG}
DOCKERFILE: Dockerfile_vs
MESA_BASE_IMAGE: "mcr.microsoft.com/windows/server:ltsc2022"
windows_build_msvc:
windows_build_vs2019:
inherit:
default: [retry]
extends:
@@ -473,15 +478,16 @@ windows_build_msvc:
- !reference [.container+build-rules, rules]
variables:
MESA_IMAGE_PATH: &windows_build_image_path ${WINDOWS_X64_BUILD_PATH}
MESA_IMAGE_TAG: &windows_build_image_tag ${WINDOWS_X64_MSVC_TAG}--${WINDOWS_X64_BUILD_TAG}
MESA_IMAGE_TAG: &windows_build_image_tag ${MESA_BASE_IMAGE_TAG}--${WINDOWS_X64_BUILD_TAG}
DOCKERFILE: Dockerfile_build
MESA_BASE_IMAGE_PATH: *windows_msvc_image_path
MESA_BASE_IMAGE: "$CI_REGISTRY_IMAGE/${MESA_BASE_IMAGE_PATH}:${WINDOWS_X64_MSVC_TAG}"
MESA_BASE_IMAGE_PATH: *windows_vs_image_path
MESA_BASE_IMAGE_TAG: *windows_vs_image_tag
MESA_BASE_IMAGE: "$CI_REGISTRY_IMAGE/${MESA_BASE_IMAGE_PATH}:${MESA_BASE_IMAGE_TAG}"
timeout: 2h 30m # LLVM takes ages
needs:
- windows_msvc
- windows_vs2019
windows_test_msvc:
windows_test_vs2019:
inherit:
default: [retry]
extends:
@@ -491,30 +497,33 @@ windows_test_msvc:
- !reference [.container+build-rules, rules]
variables:
MESA_IMAGE_PATH: &windows_test_image_path ${WINDOWS_X64_TEST_PATH}
MESA_IMAGE_TAG: &windows_test_image_tag ${WINDOWS_X64_MSVC_TAG}--${WINDOWS_X64_TEST_TAG}
MESA_IMAGE_TAG: &windows_test_image_tag ${MESA_BASE_IMAGE_TAG}--${WINDOWS_X64_TEST_TAG}
DOCKERFILE: Dockerfile_test
MESA_BASE_IMAGE_PATH: *windows_msvc_image_path
MESA_BASE_IMAGE: "$CI_REGISTRY_IMAGE/${MESA_BASE_IMAGE_PATH}:${WINDOWS_X64_MSVC_TAG}"
MESA_BASE_IMAGE_PATH: *windows_vs_image_path
MESA_BASE_IMAGE_TAG: *windows_vs_image_tag
MESA_BASE_IMAGE: "$CI_REGISTRY_IMAGE/${MESA_BASE_IMAGE_PATH}:${MESA_BASE_IMAGE_TAG}"
timeout: 2h 30m
needs:
- windows_msvc
- windows_vs2019
.use-windows_build_msvc:
.use-windows_build_vs2019:
inherit:
default: [retry]
extends: .windows-docker-msvc
extends: .windows-docker-vs2019
image: "$MESA_IMAGE"
variables:
MESA_IMAGE_PATH: *windows_build_image_path
MESA_IMAGE_TAG: *windows_build_image_tag
MESA_BASE_IMAGE_TAG: *windows_vs_image_tag
needs:
- windows_build_msvc
- windows_build_vs2019
.use-windows_test_msvc:
.use-windows_test_vs2019:
inherit:
default: [retry]
extends: .windows-docker-msvc
extends: .windows-docker-vs2019
image: "$MESA_IMAGE"
variables:
MESA_IMAGE_PATH: *windows_test_image_path
MESA_IMAGE_TAG: *windows_test_image_tag
MESA_BASE_IMAGE_TAG: *windows_vs_image_tag

View File

@@ -14,7 +14,7 @@ export LLVM_VERSION="${LLVM_VERSION:=15}"
check_minio()
{
S3_PATH="${S3_HOST}/${S3_KERNEL_BUCKET}/$1/${DISTRIBUTION_TAG}/${DEBIAN_ARCH}"
S3_PATH="${S3_HOST}/mesa-lava/$1/${DISTRIBUTION_TAG}/${DEBIAN_ARCH}"
if curl -L --retry 4 -f --retry-delay 60 -s -X HEAD \
"https://${S3_PATH}/done"; then
echo "Remote files are up-to-date, skip rebuilding them."
@@ -62,19 +62,6 @@ elif [[ "$DEBIAN_ARCH" = "armhf" ]]; then
DEVICE_TREES+=" tegra124-jetson-tk1.dtb"
KERNEL_IMAGE_NAME="zImage"
. .gitlab-ci/container/create-cross-file.sh armhf
CONTAINER_ARCH_PACKAGES=(
libegl1-mesa-dev:armhf
libelf-dev:armhf
libgbm-dev:armhf
libgles2-mesa-dev:armhf
libpng-dev:armhf
libudev-dev:armhf
libvulkan-dev:armhf
libwaffle-dev:armhf
libwayland-dev:armhf
libx11-xcb-dev:armhf
libxkbcommon-dev:armhf
)
else
GCC_ARCH="x86_64-linux-gnu"
KERNEL_ARCH="x86_64"
@@ -82,9 +69,7 @@ else
DEFCONFIG="arch/x86/configs/x86_64_defconfig"
DEVICE_TREES=""
KERNEL_IMAGE_NAME="bzImage"
CONTAINER_ARCH_PACKAGES=(
libasound2-dev libcap-dev libfdt-dev libva-dev wayland-protocols p7zip wine
)
ARCH_PACKAGES="libasound2-dev libcap-dev libfdt-dev libva-dev wayland-protocols p7zip"
fi
# Determine if we're in a cross build.
@@ -104,57 +89,66 @@ if [[ -e /cross_file-$DEBIAN_ARCH.txt ]]; then
export CROSS_COMPILE="${GCC_ARCH}-"
fi
# no need to remove these at end, image isn't saved at the end
CONTAINER_EPHEMERAL=(
automake
bc
"clang-${LLVM_VERSION}"
cmake
curl
mmdebstrap
git
glslang-tools
libdrm-dev
libegl1-mesa-dev
libxext-dev
libfontconfig-dev
libgbm-dev
libgl-dev
libgles2-mesa-dev
libglu1-mesa-dev
libglx-dev
libpng-dev
libssl-dev
libudev-dev
libvulkan-dev
libwaffle-dev
libwayland-dev
libx11-xcb-dev
libxcb-dri2-0-dev
libxkbcommon-dev
libwayland-dev
ninja-build
openssh-server
patch
protobuf-compiler
python-is-python3
python3-distutils
python3-mako
python3-numpy
python3-serial
python3-venv
unzip
zstd
)
echo "deb [trusted=yes] https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/raw/${PKG_REPO_REV}/ ${FDO_DISTRIBUTION_VERSION%-*} main" | tee /etc/apt/sources.list.d/gfx-ci_.list
apt-get update
apt-get install -y --no-remove \
-o Dpkg::Options::='--force-confdef' -o Dpkg::Options::='--force-confold' \
"${CONTAINER_EPHEMERAL[@]}" \
"${CONTAINER_ARCH_PACKAGES[@]}" \
${EXTRA_LOCAL_PACKAGES}
${EXTRA_LOCAL_PACKAGES} \
${ARCH_PACKAGES} \
automake \
bc \
clang-${LLVM_VERSION} \
cmake \
curl \
mmdebstrap \
git \
glslang-tools \
libdrm-dev \
libegl1-mesa-dev \
libxext-dev \
libfontconfig-dev \
libgbm-dev \
libgl-dev \
libgles2-mesa-dev \
libglu1-mesa-dev \
libglx-dev \
libpng-dev \
libssl-dev \
libudev-dev \
libvulkan-dev \
libwaffle-dev \
libwayland-dev \
libx11-xcb-dev \
libxcb-dri2-0-dev \
libxkbcommon-dev \
libwayland-dev \
ninja-build \
openssh-server \
patch \
protobuf-compiler \
python-is-python3 \
python3-distutils \
python3-mako \
python3-numpy \
python3-serial \
python3-venv \
unzip \
zstd
if [[ "$DEBIAN_ARCH" = "armhf" ]]; then
apt-get install -y --no-remove \
libegl1-mesa-dev:armhf \
libelf-dev:armhf \
libgbm-dev:armhf \
libgles2-mesa-dev:armhf \
libpng-dev:armhf \
libudev-dev:armhf \
libvulkan-dev:armhf \
libwaffle-dev:armhf \
libwayland-dev:armhf \
libx11-xcb-dev:armhf \
libxkbcommon-dev:armhf
fi
ROOTFS=/lava-files/rootfs-${DEBIAN_ARCH}
mkdir -p "$ROOTFS"
@@ -196,7 +190,6 @@ PKG_DEP=(
)
[ "$DEBIAN_ARCH" = "amd64" ] && PKG_ARCH=(
firmware-amd-graphics
firmware-misc-nonfree
libgl1 libglu1-mesa
inetutils-syslogd iptables libcap2
libfontconfig1
@@ -219,8 +212,7 @@ mmdebstrap \
--include "${PKG_BASE[*]} ${PKG_CI[*]} ${PKG_DEP[*]} ${PKG_MESA_DEP[*]} ${PKG_ARCH[*]}" \
bookworm \
"$ROOTFS/" \
"http://deb.debian.org/debian" \
"deb [trusted=yes] https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/raw/${PKG_REPO_REV}/ ${FDO_DISTRIBUTION_VERSION%-*} main"
"http://deb.debian.org/debian"
############### Install mold
. .gitlab-ci/container/build-mold.sh
@@ -272,17 +264,7 @@ mv /usr/local/bin/*-runner $ROOTFS/usr/bin/.
############### Build dEQP
DEQP_API=GL \
DEQP_TARGET=surfaceless \
. .gitlab-ci/container/build-deqp.sh
DEQP_API=GLES \
DEQP_TARGET=surfaceless \
. .gitlab-ci/container/build-deqp.sh
DEQP_API=VK \
DEQP_TARGET=default \
. .gitlab-ci/container/build-deqp.sh
DEQP_TARGET=surfaceless . .gitlab-ci/container/build-deqp.sh
mv /deqp $ROOTFS/.
@@ -334,11 +316,6 @@ fi
rm -rf /root/.cargo
rm -rf /root/.rustup
############### Delete firmware files we don't need
if [ "$DEBIAN_ARCH" = "amd64" ]; then
dpkg -L firmware-misc-nonfree | grep -v "i915" | xargs rm || true
fi
############### Fill rootfs
cp .gitlab-ci/container/setup-rootfs.sh $ROOTFS/.
cp .gitlab-ci/container/strip-rootfs.sh $ROOTFS/.
@@ -365,8 +342,8 @@ popd
. .gitlab-ci/container/container_post_build.sh
ci-fairy s3cp --token-file "${S3_JWT_FILE}" /lava-files/"${ROOTFSTAR}" \
ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" /lava-files/"${ROOTFSTAR}" \
https://${S3_PATH}/"${ROOTFSTAR}"
touch /lava-files/done
ci-fairy s3cp --token-file "${S3_JWT_FILE}" /lava-files/done https://${S3_PATH}/done
ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" /lava-files/done https://${S3_PATH}/done

View File

@@ -1,45 +0,0 @@
From cab41ed387c66a5e7f3454c547fc9ea53587ec1e Mon Sep 17 00:00:00 2001
From: David Heidelberg <david.heidelberg@collabora.com>
Date: Thu, 9 May 2024 14:08:59 -0700
Subject: [PATCH] Correct EGL_EXT_config_select_group extension query
EGL_EXT_config_select_group is a display extension,
not a client extension.
Affects:
dEQP-EGL.functional.choose_config.simple.selection_and_sort.*
Ref: https://github.com/KhronosGroup/EGL-Registry/pull/199
Fixes: 88ba9ac270db ("Implement support for the EGL_EXT_config_select_group extension")
Change-Id: I38956511bdcb8e99d585ea9b99aeab53da0457e2
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
---
framework/egl/egluConfigInfo.cpp | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/framework/egl/egluConfigInfo.cpp b/framework/egl/egluConfigInfo.cpp
index 88c30fd65..10936055a 100644
--- a/framework/egl/egluConfigInfo.cpp
+++ b/framework/egl/egluConfigInfo.cpp
@@ -129,7 +129,6 @@ void queryCoreConfigInfo (const Library& egl, EGLDisplay display, EGLConfig conf
void queryExtConfigInfo (const eglw::Library& egl, eglw::EGLDisplay display, eglw::EGLConfig config, ConfigInfo* dst)
{
const std::vector<std::string> extensions = getDisplayExtensions(egl, display);
- const std::vector<std::string> clientExtensions = getClientExtensions(egl);
if (de::contains(extensions.begin(), extensions.end(), "EGL_EXT_yuv_surface"))
{
@@ -159,7 +158,7 @@ void queryExtConfigInfo (const eglw::Library& egl, eglw::EGLDisplay display, egl
else
dst->colorComponentType = EGL_COLOR_COMPONENT_TYPE_FIXED_EXT;
- if (de::contains(clientExtensions.begin(), clientExtensions.end(), "EGL_EXT_config_select_group"))
+ if (hasExtension(egl, display, "EGL_EXT_config_select_group"))
{
egl.getConfigAttrib(display, config, EGL_CONFIG_SELECT_GROUP_EXT, (EGLint*)&dst->groupId);
--
2.43.0

View File

@@ -1,182 +0,0 @@
From dc97ee83a813f6b170079ddf2a04bbb06221a5a7 Mon Sep 17 00:00:00 2001
From: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Date: Fri, 26 Aug 2022 18:24:27 +0200
Subject: [PATCH 1/2] Allow running on Android from the command line
For testing the Android EGL platform without having to go via the
Android activity manager, build deqp-egl.
Tests that render to native windows are unsupported, as command line
programs cannot create windows on Android.
$ cmake -S . -B build/ -DDEQP_TARGET=android -DDEQP_TARGET_TOOLCHAIN=ndk-modern -DCMAKE_C_FLAGS=-Werror -DCMAKE_CXX_FLAGS=-Werror -DANDROID_NDK_PATH=./android-ndk-r21d -DANDROID_ABI=x86_64 -DDE_ANDROID_API=28 -DGLCTS_GTF_TARGET=gles32 -G Ninja
$ ninja -C build modules/egl/deqp-egl
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
---
CMakeLists.txt | 38 ++-----------------
.../android/tcuAndroidNativeActivity.cpp | 36 +++++++++---------
.../platform/android/tcuAndroidPlatform.cpp | 12 +++++-
3 files changed, 34 insertions(+), 52 deletions(-)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index eb58cc7ba..98b8fc6cc 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -272,7 +272,7 @@ include_directories(
external/vulkancts/framework/vulkan
)
-if (DE_OS_IS_ANDROID OR DE_OS_IS_IOS)
+if (DE_OS_IS_IOS)
# On Android deqp modules are compiled as libraries and linked into final .so
set(DEQP_MODULE_LIBRARIES )
set(DEQP_MODULE_ENTRY_POINTS )
@@ -316,7 +316,7 @@ macro (add_deqp_module MODULE_NAME SRCS LIBS EXECLIBS ENTRY)
set(DEQP_MODULE_LIBRARIES ${DEQP_MODULE_LIBRARIES} PARENT_SCOPE)
set(DEQP_MODULE_ENTRY_POINTS ${DEQP_MODULE_ENTRY_POINTS} PARENT_SCOPE)
- if (NOT DE_OS_IS_ANDROID AND NOT DE_OS_IS_IOS)
+ if (NOT DE_OS_IS_IOS)
# Executable target
add_executable(${MODULE_NAME} ${PROJECT_SOURCE_DIR}/framework/platform/tcuMain.cpp ${ENTRY})
target_link_libraries(${MODULE_NAME} PUBLIC "${EXECLIBS}" "${MODULE_NAME}${MODULE_LIB_TARGET_POSTFIX}")
@@ -338,7 +338,7 @@ macro (add_deqp_module_skip_android MODULE_NAME SRCS LIBS EXECLIBS ENTRY)
add_library("${MODULE_NAME}${MODULE_LIB_TARGET_POSTFIX}" STATIC ${SRCS})
target_link_libraries("${MODULE_NAME}${MODULE_LIB_TARGET_POSTFIX}" ${LIBS})
- if (NOT DE_OS_IS_ANDROID AND NOT DE_OS_IS_IOS)
+ if (NOT DE_OS_IS_IOS)
# Executable target
add_executable(${MODULE_NAME} ${PROJECT_SOURCE_DIR}/framework/platform/tcuMain.cpp ${ENTRY})
target_link_libraries(${MODULE_NAME} PUBLIC "${EXECLIBS}" "${MODULE_NAME}${MODULE_LIB_TARGET_POSTFIX}")
@@ -390,37 +390,7 @@ add_subdirectory(external/vulkancts/vkscpc ${MAYBE_EXCLUDE_FROM_ALL})
add_subdirectory(external/openglcts ${MAYBE_EXCLUDE_FROM_ALL})
# Single-binary targets
-if (DE_OS_IS_ANDROID)
- include_directories(executor)
- include_directories(${PROJECT_BINARY_DIR}/external/vulkancts/framework/vulkan)
-
- set(DEQP_SRCS
- framework/platform/android/tcuAndroidMain.cpp
- framework/platform/android/tcuAndroidJNI.cpp
- framework/platform/android/tcuAndroidPlatformCapabilityQueryJNI.cpp
- framework/platform/android/tcuTestLogParserJNI.cpp
- ${DEQP_MODULE_ENTRY_POINTS}
- )
-
- set(DEQP_LIBS
- tcutil-platform
- xecore
- ${DEQP_MODULE_LIBRARIES}
- )
-
- add_library(deqp SHARED ${DEQP_SRCS})
- target_link_libraries(deqp ${DEQP_LIBS})
-
- # Separate out the debug information because it's enormous
- add_custom_command(TARGET deqp POST_BUILD
- COMMAND ${CMAKE_STRIP} --only-keep-debug -o $<TARGET_FILE:deqp>.debug $<TARGET_FILE:deqp>
- COMMAND ${CMAKE_STRIP} -g $<TARGET_FILE:deqp>)
-
- # Needed by OpenGL CTS that defines its own activity but depends on
- # common Android support code.
- target_include_directories(deqp PRIVATE framework/platform/android)
-
-elseif (DE_OS_IS_IOS)
+if (DE_OS_IS_IOS)
# Code sign identity
set(DEQP_IOS_CODE_SIGN_IDENTITY "drawElements" CACHE STRING "Code sign identity for iOS build")
diff --git a/framework/platform/android/tcuAndroidNativeActivity.cpp b/framework/platform/android/tcuAndroidNativeActivity.cpp
index 6f8cd8fc5..b83e30f41 100644
--- a/framework/platform/android/tcuAndroidNativeActivity.cpp
+++ b/framework/platform/android/tcuAndroidNativeActivity.cpp
@@ -116,23 +116,25 @@ namespace Android
NativeActivity::NativeActivity (ANativeActivity* activity)
: m_activity(activity)
{
- activity->instance = (void*)this;
- activity->callbacks->onStart = onStartCallback;
- activity->callbacks->onResume = onResumeCallback;
- activity->callbacks->onSaveInstanceState = onSaveInstanceStateCallback;
- activity->callbacks->onPause = onPauseCallback;
- activity->callbacks->onStop = onStopCallback;
- activity->callbacks->onDestroy = onDestroyCallback;
- activity->callbacks->onWindowFocusChanged = onWindowFocusChangedCallback;
- activity->callbacks->onNativeWindowCreated = onNativeWindowCreatedCallback;
- activity->callbacks->onNativeWindowResized = onNativeWindowResizedCallback;
- activity->callbacks->onNativeWindowRedrawNeeded = onNativeWindowRedrawNeededCallback;
- activity->callbacks->onNativeWindowDestroyed = onNativeWindowDestroyedCallback;
- activity->callbacks->onInputQueueCreated = onInputQueueCreatedCallback;
- activity->callbacks->onInputQueueDestroyed = onInputQueueDestroyedCallback;
- activity->callbacks->onContentRectChanged = onContentRectChangedCallback;
- activity->callbacks->onConfigurationChanged = onConfigurationChangedCallback;
- activity->callbacks->onLowMemory = onLowMemoryCallback;
+ if (activity) {
+ activity->instance = (void*)this;
+ activity->callbacks->onStart = onStartCallback;
+ activity->callbacks->onResume = onResumeCallback;
+ activity->callbacks->onSaveInstanceState = onSaveInstanceStateCallback;
+ activity->callbacks->onPause = onPauseCallback;
+ activity->callbacks->onStop = onStopCallback;
+ activity->callbacks->onDestroy = onDestroyCallback;
+ activity->callbacks->onWindowFocusChanged = onWindowFocusChangedCallback;
+ activity->callbacks->onNativeWindowCreated = onNativeWindowCreatedCallback;
+ activity->callbacks->onNativeWindowResized = onNativeWindowResizedCallback;
+ activity->callbacks->onNativeWindowRedrawNeeded = onNativeWindowRedrawNeededCallback;
+ activity->callbacks->onNativeWindowDestroyed = onNativeWindowDestroyedCallback;
+ activity->callbacks->onInputQueueCreated = onInputQueueCreatedCallback;
+ activity->callbacks->onInputQueueDestroyed = onInputQueueDestroyedCallback;
+ activity->callbacks->onContentRectChanged = onContentRectChangedCallback;
+ activity->callbacks->onConfigurationChanged = onConfigurationChangedCallback;
+ activity->callbacks->onLowMemory = onLowMemoryCallback;
+ }
}
NativeActivity::~NativeActivity (void)
diff --git a/framework/platform/android/tcuAndroidPlatform.cpp b/framework/platform/android/tcuAndroidPlatform.cpp
index b8a35898c..cf02e6b70 100644
--- a/framework/platform/android/tcuAndroidPlatform.cpp
+++ b/framework/platform/android/tcuAndroidPlatform.cpp
@@ -22,6 +22,7 @@
*//*--------------------------------------------------------------------*/
#include "tcuAndroidPlatform.hpp"
+#include "tcuAndroidNativeActivity.hpp"
#include "tcuAndroidUtil.hpp"
#include "gluRenderContext.hpp"
#include "egluNativeDisplay.hpp"
@@ -170,7 +171,7 @@ eglu::NativeWindow* NativeWindowFactory::createWindow (const eglu::WindowParams&
Window* window = m_windowRegistry.tryAcquireWindow();
if (!window)
- throw ResourceError("Native window is not available", DE_NULL, __FILE__, __LINE__);
+ throw NotSupportedError("Native window is not available", DE_NULL, __FILE__, __LINE__);
return new NativeWindow(window, params.width, params.height, format);
}
@@ -292,6 +293,9 @@ static size_t getTotalSystemMemory (ANativeActivity* activity)
try
{
+ if (!activity)
+ throw tcu::InternalError("No activity (running from command line?");
+
const size_t totalMemory = getTotalAndroidSystemMemory(activity);
print("Device has %.2f MiB of system memory\n", static_cast<double>(totalMemory) / static_cast<double>(MiB));
return totalMemory;
@@ -388,3 +392,9 @@ bool Platform::hasDisplay (vk::wsi::Type wsiType) const
} // Android
} // tcu
+
+tcu::Platform* createPlatform (void)
+{
+ tcu::Android::NativeActivity activity(NULL);
+ return new tcu::Android::Platform(activity);
+}
--
2.42.0

View File

@@ -1,182 +0,0 @@
From dc97ee83a813f6b170079ddf2a04bbb06221a5a7 Mon Sep 17 00:00:00 2001
From: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Date: Fri, 26 Aug 2022 18:24:27 +0200
Subject: [PATCH 1/2] Allow running on Android from the command line
For testing the Android EGL platform without having to go via the
Android activity manager, build deqp-egl.
Tests that render to native windows are unsupported, as command line
programs cannot create windows on Android.
$ cmake -S . -B build/ -DDEQP_TARGET=android -DDEQP_TARGET_TOOLCHAIN=ndk-modern -DCMAKE_C_FLAGS=-Werror -DCMAKE_CXX_FLAGS=-Werror -DANDROID_NDK_PATH=./android-ndk-r21d -DANDROID_ABI=x86_64 -DDE_ANDROID_API=28 -DGLCTS_GTF_TARGET=gles32 -G Ninja
$ ninja -C build modules/egl/deqp-egl
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
---
CMakeLists.txt | 38 ++-----------------
.../android/tcuAndroidNativeActivity.cpp | 36 +++++++++---------
.../platform/android/tcuAndroidPlatform.cpp | 12 +++++-
3 files changed, 34 insertions(+), 52 deletions(-)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index eb58cc7ba..98b8fc6cc 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -272,7 +272,7 @@ include_directories(
external/vulkancts/framework/vulkan
)
-if (DE_OS_IS_ANDROID OR DE_OS_IS_IOS)
+if (DE_OS_IS_IOS)
# On Android deqp modules are compiled as libraries and linked into final .so
set(DEQP_MODULE_LIBRARIES )
set(DEQP_MODULE_ENTRY_POINTS )
@@ -316,7 +316,7 @@ macro (add_deqp_module MODULE_NAME SRCS LIBS EXECLIBS ENTRY)
set(DEQP_MODULE_LIBRARIES ${DEQP_MODULE_LIBRARIES} PARENT_SCOPE)
set(DEQP_MODULE_ENTRY_POINTS ${DEQP_MODULE_ENTRY_POINTS} PARENT_SCOPE)
- if (NOT DE_OS_IS_ANDROID AND NOT DE_OS_IS_IOS)
+ if (NOT DE_OS_IS_IOS)
# Executable target
add_executable(${MODULE_NAME} ${PROJECT_SOURCE_DIR}/framework/platform/tcuMain.cpp ${ENTRY})
target_link_libraries(${MODULE_NAME} PUBLIC "${EXECLIBS}" "${MODULE_NAME}${MODULE_LIB_TARGET_POSTFIX}")
@@ -338,7 +338,7 @@ macro (add_deqp_module_skip_android MODULE_NAME SRCS LIBS EXECLIBS ENTRY)
add_library("${MODULE_NAME}${MODULE_LIB_TARGET_POSTFIX}" STATIC ${SRCS})
target_link_libraries("${MODULE_NAME}${MODULE_LIB_TARGET_POSTFIX}" ${LIBS})
- if (NOT DE_OS_IS_ANDROID AND NOT DE_OS_IS_IOS)
+ if (NOT DE_OS_IS_IOS)
# Executable target
add_executable(${MODULE_NAME} ${PROJECT_SOURCE_DIR}/framework/platform/tcuMain.cpp ${ENTRY})
target_link_libraries(${MODULE_NAME} PUBLIC "${EXECLIBS}" "${MODULE_NAME}${MODULE_LIB_TARGET_POSTFIX}")
@@ -390,37 +390,7 @@ add_subdirectory(external/vulkancts/vkscpc ${MAYBE_EXCLUDE_FROM_ALL})
add_subdirectory(external/openglcts ${MAYBE_EXCLUDE_FROM_ALL})
# Single-binary targets
-if (DE_OS_IS_ANDROID)
- include_directories(executor)
- include_directories(${PROJECT_BINARY_DIR}/external/vulkancts/framework/vulkan)
-
- set(DEQP_SRCS
- framework/platform/android/tcuAndroidMain.cpp
- framework/platform/android/tcuAndroidJNI.cpp
- framework/platform/android/tcuAndroidPlatformCapabilityQueryJNI.cpp
- framework/platform/android/tcuTestLogParserJNI.cpp
- ${DEQP_MODULE_ENTRY_POINTS}
- )
-
- set(DEQP_LIBS
- tcutil-platform
- xecore
- ${DEQP_MODULE_LIBRARIES}
- )
-
- add_library(deqp SHARED ${DEQP_SRCS})
- target_link_libraries(deqp ${DEQP_LIBS})
-
- # Separate out the debug information because it's enormous
- add_custom_command(TARGET deqp POST_BUILD
- COMMAND ${CMAKE_STRIP} --only-keep-debug -o $<TARGET_FILE:deqp>.debug $<TARGET_FILE:deqp>
- COMMAND ${CMAKE_STRIP} -g $<TARGET_FILE:deqp>)
-
- # Needed by OpenGL CTS that defines its own activity but depends on
- # common Android support code.
- target_include_directories(deqp PRIVATE framework/platform/android)
-
-elseif (DE_OS_IS_IOS)
+if (DE_OS_IS_IOS)
# Code sign identity
set(DEQP_IOS_CODE_SIGN_IDENTITY "drawElements" CACHE STRING "Code sign identity for iOS build")
diff --git a/framework/platform/android/tcuAndroidNativeActivity.cpp b/framework/platform/android/tcuAndroidNativeActivity.cpp
index 6f8cd8fc5..b83e30f41 100644
--- a/framework/platform/android/tcuAndroidNativeActivity.cpp
+++ b/framework/platform/android/tcuAndroidNativeActivity.cpp
@@ -116,23 +116,25 @@ namespace Android
NativeActivity::NativeActivity (ANativeActivity* activity)
: m_activity(activity)
{
- activity->instance = (void*)this;
- activity->callbacks->onStart = onStartCallback;
- activity->callbacks->onResume = onResumeCallback;
- activity->callbacks->onSaveInstanceState = onSaveInstanceStateCallback;
- activity->callbacks->onPause = onPauseCallback;
- activity->callbacks->onStop = onStopCallback;
- activity->callbacks->onDestroy = onDestroyCallback;
- activity->callbacks->onWindowFocusChanged = onWindowFocusChangedCallback;
- activity->callbacks->onNativeWindowCreated = onNativeWindowCreatedCallback;
- activity->callbacks->onNativeWindowResized = onNativeWindowResizedCallback;
- activity->callbacks->onNativeWindowRedrawNeeded = onNativeWindowRedrawNeededCallback;
- activity->callbacks->onNativeWindowDestroyed = onNativeWindowDestroyedCallback;
- activity->callbacks->onInputQueueCreated = onInputQueueCreatedCallback;
- activity->callbacks->onInputQueueDestroyed = onInputQueueDestroyedCallback;
- activity->callbacks->onContentRectChanged = onContentRectChangedCallback;
- activity->callbacks->onConfigurationChanged = onConfigurationChangedCallback;
- activity->callbacks->onLowMemory = onLowMemoryCallback;
+ if (activity) {
+ activity->instance = (void*)this;
+ activity->callbacks->onStart = onStartCallback;
+ activity->callbacks->onResume = onResumeCallback;
+ activity->callbacks->onSaveInstanceState = onSaveInstanceStateCallback;
+ activity->callbacks->onPause = onPauseCallback;
+ activity->callbacks->onStop = onStopCallback;
+ activity->callbacks->onDestroy = onDestroyCallback;
+ activity->callbacks->onWindowFocusChanged = onWindowFocusChangedCallback;
+ activity->callbacks->onNativeWindowCreated = onNativeWindowCreatedCallback;
+ activity->callbacks->onNativeWindowResized = onNativeWindowResizedCallback;
+ activity->callbacks->onNativeWindowRedrawNeeded = onNativeWindowRedrawNeededCallback;
+ activity->callbacks->onNativeWindowDestroyed = onNativeWindowDestroyedCallback;
+ activity->callbacks->onInputQueueCreated = onInputQueueCreatedCallback;
+ activity->callbacks->onInputQueueDestroyed = onInputQueueDestroyedCallback;
+ activity->callbacks->onContentRectChanged = onContentRectChangedCallback;
+ activity->callbacks->onConfigurationChanged = onConfigurationChangedCallback;
+ activity->callbacks->onLowMemory = onLowMemoryCallback;
+ }
}
NativeActivity::~NativeActivity (void)
diff --git a/framework/platform/android/tcuAndroidPlatform.cpp b/framework/platform/android/tcuAndroidPlatform.cpp
index b8a35898c..cf02e6b70 100644
--- a/framework/platform/android/tcuAndroidPlatform.cpp
+++ b/framework/platform/android/tcuAndroidPlatform.cpp
@@ -22,6 +22,7 @@
*//*--------------------------------------------------------------------*/
#include "tcuAndroidPlatform.hpp"
+#include "tcuAndroidNativeActivity.hpp"
#include "tcuAndroidUtil.hpp"
#include "gluRenderContext.hpp"
#include "egluNativeDisplay.hpp"
@@ -170,7 +171,7 @@ eglu::NativeWindow* NativeWindowFactory::createWindow (const eglu::WindowParams&
Window* window = m_windowRegistry.tryAcquireWindow();
if (!window)
- throw ResourceError("Native window is not available", DE_NULL, __FILE__, __LINE__);
+ throw NotSupportedError("Native window is not available", DE_NULL, __FILE__, __LINE__);
return new NativeWindow(window, params.width, params.height, format);
}
@@ -292,6 +293,9 @@ static size_t getTotalSystemMemory (ANativeActivity* activity)
try
{
+ if (!activity)
+ throw tcu::InternalError("No activity (running from command line?");
+
const size_t totalMemory = getTotalAndroidSystemMemory(activity);
print("Device has %.2f MiB of system memory\n", static_cast<double>(totalMemory) / static_cast<double>(MiB));
return totalMemory;
@@ -388,3 +392,9 @@ bool Platform::hasDisplay (vk::wsi::Type wsiType) const
} // Android
} // tcu
+
+tcu::Platform* createPlatform (void)
+{
+ tcu::Android::NativeActivity activity(NULL);
+ return new tcu::Android::Platform(activity);
+}
--
2.42.0

View File

@@ -1,26 +0,0 @@
From a602822c53e22e985f942f843ccadbfb64613212 Mon Sep 17 00:00:00 2001
From: Helen Koike <helen.koike@collabora.com>
Date: Tue, 27 Sep 2022 12:35:22 -0300
Subject: [PATCH 2/2] Android prints to stdout instead of logcat
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
framework/qphelper/qpDebugOut.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/framework/qphelper/qpDebugOut.c b/framework/qphelper/qpDebugOut.c
index 6579e9f48..c200c6f6b 100644
--- a/framework/qphelper/qpDebugOut.c
+++ b/framework/qphelper/qpDebugOut.c
@@ -98,7 +98,7 @@ void qpDiev (const char* format, va_list args)
}
/* print() implementation. */
-#if (DE_OS == DE_OS_ANDROID)
+#if (0)
#include <android/log.h>
--
2.42.0

View File

@@ -1,26 +0,0 @@
From a602822c53e22e985f942f843ccadbfb64613212 Mon Sep 17 00:00:00 2001
From: Helen Koike <helen.koike@collabora.com>
Date: Tue, 27 Sep 2022 12:35:22 -0300
Subject: [PATCH 2/2] Android prints to stdout instead of logcat
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
framework/qphelper/qpDebugOut.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/framework/qphelper/qpDebugOut.c b/framework/qphelper/qpDebugOut.c
index 6579e9f48..c200c6f6b 100644
--- a/framework/qphelper/qpDebugOut.c
+++ b/framework/qphelper/qpDebugOut.c
@@ -98,7 +98,7 @@ void qpDiev (const char* format, va_list args)
}
/* print() implementation. */
-#if (DE_OS == DE_OS_ANDROID)
+#if (0)
#include <android/log.h>
--
2.42.0

View File

@@ -96,7 +96,7 @@ set +e -x
NIR_DEBUG="novalidate" \
LIBGL_ALWAYS_SOFTWARE=${CROSVM_LIBGL_ALWAYS_SOFTWARE} \
GALLIUM_DRIVER=${CROSVM_GALLIUM_DRIVER} \
VK_DRIVER_FILES=$CI_PROJECT_DIR/install/share/vulkan/icd.d/${CROSVM_VK_DRIVER}_icd.x86_64.json \
VK_ICD_FILENAMES=$CI_PROJECT_DIR/install/share/vulkan/icd.d/${CROSVM_VK_DRIVER}_icd.x86_64.json \
crosvm --no-syslog run \
--gpu "${CROSVM_GPU_ARGS}" --gpu-render-server "path=/usr/local/libexec/virgl_render_server" \
-m "${CROSVM_MEMORY:-4096}" -c "${CROSVM_CPU:-2}" --disable-sandbox \

View File

@@ -48,7 +48,7 @@ $ADB shell setenforce 0
# deqp
$ADB push /deqp/modules/egl/deqp-egl-android /data/.
$ADB push /deqp/assets/gl_cts/data/mustpass/egl/aosp_mustpass/3.2.6.x/egl-main.txt /data/
$ADB push /deqp/assets/gl_cts/data/mustpass/egl/aosp_mustpass/3.2.6.x/egl-master.txt /data/.
$ADB push /deqp-runner/deqp-runner /data/.
# download Android Mesa from S3

View File

@@ -18,7 +18,7 @@ INSTALL=$(realpath -s "$PWD"/install)
# Set up the driver environment.
export LD_LIBRARY_PATH="$INSTALL"/lib/:$LD_LIBRARY_PATH
export EGL_PLATFORM=surfaceless
export VK_DRIVER_FILES="$PWD"/install/share/vulkan/icd.d/"$VK_DRIVER"_icd.${VK_CPU:-$(uname -m)}.json
export VK_ICD_FILENAMES="$PWD"/install/share/vulkan/icd.d/"$VK_DRIVER"_icd.${VK_CPU:-$(uname -m)}.json
export OCL_ICD_VENDORS="$PWD"/install/etc/OpenCL/vendors/
if [ -n "$USE_ANGLE" ]; then
@@ -46,6 +46,7 @@ if [ -z "$DEQP_SUITE" ]; then
DEQP_WIDTH=${DEQP_WIDTH:-256}
DEQP_HEIGHT=${DEQP_HEIGHT:-256}
DEQP_CONFIG=${DEQP_CONFIG:-rgba8888d24s8ms0}
DEQP_VARIANT=${DEQP_VARIANT:-master}
DEQP_OPTIONS="$DEQP_OPTIONS --deqp-surface-width=$DEQP_WIDTH --deqp-surface-height=$DEQP_HEIGHT"
DEQP_OPTIONS="$DEQP_OPTIONS --deqp-surface-type=${DEQP_SURFACE_TYPE:-pbuffer}"
@@ -59,16 +60,16 @@ if [ -z "$DEQP_SUITE" ]; then
# Generate test case list file.
if [ "$DEQP_VER" = "vk" ]; then
MUSTPASS=/deqp/mustpass/vk-main.txt
MUSTPASS=/deqp/mustpass/vk-$DEQP_VARIANT.txt
DEQP=/deqp/external/vulkancts/modules/vulkan/deqp-vk
elif [ "$DEQP_VER" = "gles2" ] || [ "$DEQP_VER" = "gles3" ] || [ "$DEQP_VER" = "gles31" ] || [ "$DEQP_VER" = "egl" ]; then
MUSTPASS=/deqp/mustpass/$DEQP_VER-main.txt
MUSTPASS=/deqp/mustpass/$DEQP_VER-$DEQP_VARIANT.txt
DEQP=/deqp/modules/$DEQP_VER/deqp-$DEQP_VER
elif [ "$DEQP_VER" = "gles2-khr" ] || [ "$DEQP_VER" = "gles3-khr" ] || [ "$DEQP_VER" = "gles31-khr" ] || [ "$DEQP_VER" = "gles32-khr" ]; then
MUSTPASS=/deqp/mustpass/$DEQP_VER-main.txt
MUSTPASS=/deqp/mustpass/$DEQP_VER-$DEQP_VARIANT.txt
DEQP=/deqp/external/openglcts/modules/glcts
else
MUSTPASS=/deqp/mustpass/$DEQP_VER-main.txt
MUSTPASS=/deqp/mustpass/$DEQP_VER-$DEQP_VARIANT.txt
DEQP=/deqp/external/openglcts/modules/glcts
fi
@@ -131,9 +132,6 @@ if [ -n "$VK_DRIVER" ] && [ -z "$DEQP_SUITE" ]; then
fi
# Set the path to VK validation layer settings (in case it ends up getting loaded)
# Note: If you change the format of this filename, look through the rest of the
# tree for other places that need to be kept in sync (e.g.
# src/gallium/drivers/zink/ci/gitlab-ci-inc.yml)
export VK_LAYER_SETTINGS_PATH=$INSTALL/$GPU_VERSION-validation-settings.txt
report_load() {
@@ -168,13 +166,7 @@ fi
uncollapsed_section_switch deqp "deqp: deqp-runner"
# Print the detailed version with the list of backports and local patches
for api in vk gl gles; do
deqp_version_log=/deqp/version-$api
if [ -r "$deqp_version_log" ]; then
cat "$deqp_version_log"
fi
done
echo "deqp $(cat /deqp/version)"
set +e
if [ -z "$DEQP_SUITE" ]; then
@@ -191,10 +183,6 @@ if [ -z "$DEQP_SUITE" ]; then
-- \
$DEQP_OPTIONS
else
# If you change the format of the suite toml filenames or the
# $GPU_VERSION-{fails,flakes,skips}.txt filenames, look through the rest
# of the tree for other places that need to be kept in sync (e.g.
# src/**/ci/gitlab-ci*.yml)
deqp-runner \
suite \
--suite $INSTALL/deqp-$DEQP_SUITE.toml \
@@ -235,7 +223,7 @@ deqp-runner junit \
--results $RESULTS/failures.csv \
--output $RESULTS/junit.xml \
--limit 50 \
--template "See $ARTIFACTS_BASE_URL/results/{{testcase}}.xml"
--template "See https://$CI_PROJECT_ROOT_NAMESPACE.pages.freedesktop.org/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/results/{{testcase}}.xml"
# Report the flakes to the IRC channel for monitoring (if configured):
if [ -n "$FLAKES_CHANNEL" ]; then

View File

@@ -18,7 +18,7 @@ TMP_DIR=$(mktemp -d)
echo "$(date +"%F %T") Downloading archived master..."
if ! /usr/bin/wget \
-O "$TMP_DIR/$CI_PROJECT_NAME.tar.gz" \
"https://${S3_HOST}/${S3_GITCACHE_BUCKET}/${FDO_UPSTREAM_REPO}/$CI_PROJECT_NAME.tar.gz";
"https://${S3_HOST}/git-cache/${FDO_UPSTREAM_REPO}/$CI_PROJECT_NAME.tar.gz";
then
echo "Repository cache not available"
exit

View File

@@ -237,46 +237,6 @@
when: never
- !reference [.freedreno-farm-rules, rules]
.vmware-farm-rules:
rules:
- exists: [ .ci-farms-disabled/vmware ]
when: never
- changes: [ .ci-farms-disabled/vmware ]
if: '$CI_PIPELINE_SOURCE != "schedule"'
when: on_success
- changes: [ .ci-farms-disabled/* ]
if: '$CI_PIPELINE_SOURCE != "schedule"'
when: never
.vmware-farm-manual-rules:
rules:
- exists: [ .ci-farms-disabled/vmware ]
when: never
- changes: [ .ci-farms-disabled/vmware ]
if: '$CI_PIPELINE_SOURCE != "schedule"'
when: never
- !reference [.vmware-farm-rules, rules]
.ondracka-farm-rules:
rules:
- exists: [ .ci-farms-disabled/ondracka ]
when: never
- changes: [ .ci-farms-disabled/ondracka ]
if: '$CI_PIPELINE_SOURCE != "schedule"'
when: on_success
- changes: [ .ci-farms-disabled/* ]
if: '$CI_PIPELINE_SOURCE != "schedule"'
when: never
.ondracka-farm-manual-rules:
rules:
- exists: [ .ci-farms-disabled/ondracka ]
when: never
- changes: [ .ci-farms-disabled/ondracka ]
if: '$CI_PIPELINE_SOURCE != "schedule"'
when: never
- !reference [.ondracka-farm-rules, rules]
# Skip container & build jobs when disabling any farm, and run them if any
# farm gets re-enabled.
@@ -326,14 +286,6 @@
changes: [ .ci-farms-disabled/freedreno ]
exists: [ .ci-farms-disabled/freedreno ]
when: never
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes: [ .ci-farms-disabled/ondracka ]
exists: [ .ci-farms-disabled/ondracka ]
when: never
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes: [ .ci-farms-disabled/vmware ]
exists: [ .ci-farms-disabled/vmware ]
when: never
# Any other change to ci-farms/* means some farm is getting re-enabled.
# Run jobs in Marge pipelines (and let it fallback to manual otherwise)
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $GITLAB_USER_LOGIN == "marge-bot"'

View File

@@ -11,7 +11,7 @@ INSTALL=$PWD/install
# Set up the driver environment.
export LD_LIBRARY_PATH="$INSTALL/lib/"
export VK_DRIVER_FILES="$INSTALL/share/vulkan/icd.d/${VK_DRIVER}_icd.x86_64.json"
export VK_ICD_FILENAMES="$INSTALL/share/vulkan/icd.d/${VK_DRIVER}_icd.x86_64.json"
# To store Fossilize logs on failure.
RESULTS="$PWD/results"

View File

@@ -55,7 +55,7 @@ deqp-runner junit \
--results $RESULTS/failures.csv \
--output $RESULTS/junit.xml \
--limit 50 \
--template "See $ARTIFACTS_BASE_URL/results/{{testcase}}.xml"
--template "See https://$CI_PROJECT_ROOT_NAMESPACE.pages.freedesktop.org/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/results/{{testcase}}.xml"
# Report the flakes to the IRC channel for monitoring (if configured):
if [ -n "$FLAKES_CHANNEL" ]; then

View File

@@ -1,22 +1,19 @@
# Keep the tags below under 20 chars each, as they end up combined into
# Keep the tags below under 25-30 chars each, as they end up combined into
# docker image tags, and docker has a length limit of 128 chars total in tags.
#
# The word after the date doesn't need to be overly descriptive, it exists
# purely to avoid the risk of two people updating an image tag on the same day
# while working on different changes, so it doesn't matter whether that word
# will make sense to someone else. Aim for "nobody else will pick that word
# for their change".
#
# If you update a tag and you get an error like this:
# cannot parse input: "$image:$tag": invalid reference format
# check the length of $tag; if it's > 128 chars you need to shorten your tag.
variables:
DEBIAN_X86_64_BUILD_BASE_IMAGE: "debian/x86_64_build-base"
DEBIAN_BASE_TAG: "20240412-pycparser"
DEBIAN_BASE_TAG: "2023-10-13-rust-1.66"
DEBIAN_X86_64_BUILD_IMAGE_PATH: "debian/x86_64_build"
DEBIAN_BUILD_TAG: "20240408-cbindgen"
DEBIAN_BUILD_TAG: "2023-09-30-shader-db"
DEBIAN_X86_64_BUILD_MINGW_IMAGE_PATH: "debian/x86_64_build-mingw"
DEBIAN_BUILD_MINGW_TAG: "2023-05-25-bookworm"
DEBIAN_X86_64_TEST_BASE_IMAGE: "debian/x86_64_test-base"
@@ -24,23 +21,21 @@ variables:
DEBIAN_X86_64_TEST_IMAGE_VK_PATH: "debian/x86_64_test-vk"
DEBIAN_X86_64_TEST_ANDROID_IMAGE_PATH: "debian/x86_64_test-android"
DEBIAN_X86_64_TEST_ANDROID_TAG: "20240423-deqp"
DEBIAN_X86_64_TEST_GL_TAG: "20240514-egltrans241"
DEBIAN_X86_64_TEST_VK_TAG: "20240423-deqp"
KERNEL_ROOTFS_TAG: "20240507-kernel241"
DEBIAN_X86_64_TEST_ANDROID_TAG: "2023-10-15-deqp"
DEBIAN_X86_64_TEST_GL_TAG: "2023-10-15-deqp"
DEBIAN_X86_64_TEST_VK_TAG: "2023-10-15-deqp"
ALPINE_X86_64_BUILD_TAG: "20240412-pycparser"
ALPINE_X86_64_LAVA_SSH_TAG: "20240401-wlproto"
FEDORA_X86_64_BUILD_TAG: "20240412-pycparser"
KERNEL_TAG: "v6.6.21-mesa-f8ea"
KERNEL_REPO: "gfx-ci/linux"
PKG_REPO_REV: "3cc12a2a"
ALPINE_X86_64_BUILD_TAG: "2023-10-04-ephemeral"
ALPINE_X86_64_LAVA_SSH_TAG: "2023-06-26-first-version"
FEDORA_X86_64_BUILD_TAG: "2023-08-04-shader-db"
KERNEL_ROOTFS_TAG: "2023-10-13-deqp"
KERNEL_TAG: "v6.4.12-for-mesa-ci-f6b4ad45f48d"
WINDOWS_X64_MSVC_PATH: "windows/x86_64_msvc"
WINDOWS_X64_MSVC_TAG: "20231222-msvc"
WINDOWS_X64_VS_PATH: "windows/x64_vs"
WINDOWS_X64_VS_TAG: "2022-10-20-upgrade-zlib"
WINDOWS_X64_BUILD_PATH: "windows/x86_64_build"
WINDOWS_X64_BUILD_TAG: "20240405-vainfo-ci-1"
WINDOWS_X64_BUILD_PATH: "windows/x64_build"
WINDOWS_X64_BUILD_TAG: "2023-06-24-agility-711"
WINDOWS_X64_TEST_PATH: "windows/x86_64_test"
WINDOWS_X64_TEST_TAG: "20240405-vainfo-ci-1"
WINDOWS_X64_TEST_PATH: "windows/x64_test"
WINDOWS_X64_TEST_TAG: "2023-05-30-warp-1.0.7.1"

View File

@@ -5,36 +5,24 @@ class MesaCIException(Exception):
pass
class MesaCIRetriableException(MesaCIException):
pass
class MesaCITimeoutError(MesaCIRetriableException):
class MesaCITimeoutError(MesaCIException):
def __init__(self, *args, timeout_duration: timedelta) -> None:
super().__init__(*args)
self.timeout_duration = timeout_duration
class MesaCIRetryError(MesaCIRetriableException):
class MesaCIRetryError(MesaCIException):
def __init__(self, *args, retry_count: int, last_job: None) -> None:
super().__init__(*args)
self.retry_count = retry_count
self.last_job = last_job
class MesaCIFatalException(MesaCIException):
"""Exception raised when the Mesa CI script encounters a fatal error that
prevents the script from continuing."""
def __init__(self, *args) -> None:
super().__init__(*args)
class MesaCIParseException(MesaCIRetriableException):
class MesaCIParseException(MesaCIException):
pass
class MesaCIKnownIssueException(MesaCIRetriableException):
class MesaCIKnownIssueException(MesaCIException):
"""Exception raised when the Mesa CI script finds something in the logs that
is known to cause the LAVA job to eventually fail"""

View File

@@ -11,7 +11,7 @@ variables:
# proxy used to cache data locally
FDO_HTTP_CACHE_URI: "http://caching-proxy/cache/?uri="
# base system generated by the container build job, shared between many pipelines
BASE_SYSTEM_HOST_PREFIX: "${S3_HOST}/${S3_KERNEL_BUCKET}"
BASE_SYSTEM_HOST_PREFIX: "${S3_HOST}/mesa-lava"
BASE_SYSTEM_MAINLINE_HOST_PATH: "${BASE_SYSTEM_HOST_PREFIX}/${FDO_UPSTREAM_REPO}/${DISTRIBUTION_TAG}/${DEBIAN_ARCH}"
BASE_SYSTEM_FORK_HOST_PATH: "${BASE_SYSTEM_HOST_PREFIX}/${CI_PROJECT_PATH}/${DISTRIBUTION_TAG}/${DEBIAN_ARCH}"
# per-job build artifacts

View File

@@ -30,7 +30,7 @@ artifacts/ci-common/generate-env.sh | tee results/job-rootfs-overlay/set-job-env
section_end variables
tar zcf job-rootfs-overlay.tar.gz -C results/job-rootfs-overlay/ .
ci-fairy s3cp --token-file "${S3_JWT_FILE}" job-rootfs-overlay.tar.gz "https://${JOB_ROOTFS_OVERLAY_PATH}"
ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" job-rootfs-overlay.tar.gz "https://${JOB_ROOTFS_OVERLAY_PATH}"
ARTIFACT_URL="${FDO_HTTP_CACHE_URI:-}https://${PIPELINE_ARTIFACTS_BASE}/${S3_ARTIFACT_NAME:?}.tar.zst"
@@ -42,7 +42,6 @@ PYTHONPATH=artifacts/ artifacts/lava/lava_job_submitter.py \
--pipeline-info "$CI_JOB_NAME: $CI_PIPELINE_URL on $CI_COMMIT_REF_NAME ${CI_NODE_INDEX}/${CI_NODE_TOTAL}" \
--rootfs-url-prefix "https://${BASE_SYSTEM_HOST_PATH}" \
--kernel-url-prefix "${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}" \
--kernel-external "${FORCE_KERNEL_TAG}" \
--build-url "${ARTIFACT_URL}" \
--job-rootfs-overlay-url "${FDO_HTTP_CACHE_URI:-}https://${JOB_ROOTFS_OVERLAY_PATH}" \
--job-timeout-min ${JOB_TIMEOUT:-30} \
@@ -50,7 +49,7 @@ PYTHONPATH=artifacts/ artifacts/lava/lava_job_submitter.py \
--ci-project-dir "${CI_PROJECT_DIR}" \
--device-type "${DEVICE_TYPE}" \
--dtb-filename "${DTB}" \
--jwt-file "${S3_JWT_FILE}" \
--jwt-file "${CI_JOB_JWT_FILE}" \
--kernel-image-name "${KERNEL_IMAGE_NAME}" \
--kernel-image-type "${KERNEL_IMAGE_TYPE}" \
--boot-method "${BOOT_METHOD}" \
@@ -59,5 +58,4 @@ PYTHONPATH=artifacts/ artifacts/lava/lava_job_submitter.py \
--mesa-job-name "$CI_JOB_NAME" \
--structured-log-file "results/lava_job_detail.json" \
--ssh-client-image "${LAVA_SSH_CLIENT_IMAGE}" \
--project-name "${CI_PROJECT_NAME}" \
>> results/lava.log

View File

@@ -16,35 +16,33 @@ import sys
import time
from collections import defaultdict
from dataclasses import dataclass, fields
from datetime import datetime, timedelta, timezone
from datetime import datetime, timedelta
from io import StringIO
from os import environ, getenv, path
from typing import Any, Optional
import fire
from lavacli.utils import flow_yaml as lava_yaml
from lava.exceptions import (
MesaCIException,
MesaCIFatalException,
MesaCIRetriableException,
MesaCIParseException,
MesaCIRetryError,
MesaCITimeoutError,
)
from lava.utils import CONSOLE_LOG
from lava.utils import DEFAULT_GITLAB_SECTION_TIMEOUTS as GL_SECTION_TIMEOUTS
from lava.utils import (
CONSOLE_LOG,
GitlabSection,
LAVAJob,
LAVAJobDefinition,
LogFollower,
LogSectionType,
call_proxy,
fatal_err,
generate_lava_job_definition,
hide_sensitive_data,
print_log,
setup_lava_proxy,
)
from lava.utils import DEFAULT_GITLAB_SECTION_TIMEOUTS as GL_SECTION_TIMEOUTS
from lavacli.utils import flow_yaml as lava_yaml
# Initialize structural logging with a defaultdict, it can be changed for more
# sophisticated dict-like data abstractions.
@@ -60,7 +58,7 @@ except ImportError as e:
# Timeout in seconds to decide if the device from the dispatched LAVA job has
# hung or not due to the lack of new log output.
DEVICE_HANGING_TIMEOUT_SEC = int(getenv("DEVICE_HANGING_TIMEOUT_SEC", 5 * 60))
DEVICE_HANGING_TIMEOUT_SEC = int(getenv("DEVICE_HANGING_TIMEOUT_SEC", 5*60))
# How many seconds the script should wait before try a new polling iteration to
# check if the dispatched LAVA job is running or waiting in the job queue.
@@ -83,29 +81,18 @@ NUMBER_OF_RETRIES_TIMEOUT_DETECTION = int(
getenv("LAVA_NUMBER_OF_RETRIES_TIMEOUT_DETECTION", 2)
)
CI_JOB_TIMEOUT_SEC = int(getenv("CI_JOB_TIMEOUT", 3600))
# How many seconds the script will wait to let LAVA run the job and give the final details.
EXPECTED_JOB_DURATION_SEC = int(getenv("EXPECTED_JOB_DURATION_SEC", 60 * 10))
# CI_JOB_STARTED is given by GitLab CI/CD in UTC timezone by default.
CI_JOB_STARTED_AT_RAW = getenv("CI_JOB_STARTED_AT", "")
CI_JOB_STARTED_AT: datetime = (
datetime.fromisoformat(CI_JOB_STARTED_AT_RAW)
if CI_JOB_STARTED_AT_RAW
else datetime.now(timezone.utc)
)
def raise_exception_from_metadata(metadata: dict, job_id: int) -> None:
"""
Investigate infrastructure errors from the job metadata.
If it finds an error, raise it as MesaCIRetriableException.
If it finds an error, raise it as MesaCIException.
"""
if "result" not in metadata or metadata["result"] != "fail":
return
if "error_type" in metadata:
error_type = metadata["error_type"]
if error_type == "Infrastructure":
raise MesaCIRetriableException(
raise MesaCIException(
f"LAVA job {job_id} failed with Infrastructure Error. Retry."
)
if error_type == "Job":
@@ -113,12 +100,12 @@ def raise_exception_from_metadata(metadata: dict, job_id: int) -> None:
# with mal-formed job definitions. As we are always validating the
# jobs, only the former is probable to happen. E.g.: When some LAVA
# action timed out more times than expected in job definition.
raise MesaCIRetriableException(
raise MesaCIException(
f"LAVA job {job_id} failed with JobError "
"(possible LAVA timeout misconfiguration/bug). Retry."
)
if "case" in metadata and metadata["case"] == "validate":
raise MesaCIRetriableException(
raise MesaCIException(
f"LAVA job {job_id} failed validation (possible download error). Retry."
)
@@ -195,6 +182,7 @@ def is_job_hanging(job, max_idle_time):
def parse_log_lines(job, log_follower, new_log_lines):
if log_follower.feed(new_log_lines):
# If we had non-empty log data, we can assure that the device is alive.
job.heartbeat()
@@ -212,6 +200,7 @@ def parse_log_lines(job, log_follower, new_log_lines):
def fetch_new_log_lines(job):
# The XMLRPC binary packet may be corrupted, causing a YAML scanner error.
# Retry the log fetching several times before exposing the error.
for _ in range(5):
@@ -227,28 +216,14 @@ def submit_job(job):
try:
job.submit()
except Exception as mesa_ci_err:
raise MesaCIRetriableException(
raise MesaCIException(
f"Could not submit LAVA job. Reason: {mesa_ci_err}"
) from mesa_ci_err
def wait_for_job_get_started(job, attempt_no):
def wait_for_job_get_started(job):
print_log(f"Waiting for job {job.job_id} to start.")
while not job.is_started():
current_job_duration_sec: int = int(
(datetime.now(timezone.utc) - CI_JOB_STARTED_AT).total_seconds()
)
remaining_time_sec: int = max(0, CI_JOB_TIMEOUT_SEC - current_job_duration_sec)
if remaining_time_sec < EXPECTED_JOB_DURATION_SEC:
job.cancel()
raise MesaCIFatalException(
f"{CONSOLE_LOG['BOLD']}"
f"{CONSOLE_LOG['FG_YELLOW']}"
f"Job {job.job_id} only has {remaining_time_sec} seconds "
"remaining to run, but it is expected to take at least "
f"{EXPECTED_JOB_DURATION_SEC} seconds."
f"{CONSOLE_LOG['RESET']}",
)
time.sleep(WAIT_FOR_DEVICE_POLLING_TIME_SEC)
job.refresh_log()
print_log(f"Job {job.job_id} started.")
@@ -324,7 +299,7 @@ def execute_job_with_retries(
try:
job_log["submitter_start_time"] = datetime.now().isoformat()
submit_job(job)
wait_for_job_get_started(job, attempt_no)
wait_for_job_get_started(job)
log_follower: LogFollower = bootstrap_log_follower()
follow_job_execution(job, log_follower)
return job
@@ -343,8 +318,6 @@ def execute_job_with_retries(
f"Finished executing LAVA job in the attempt #{attempt_no}"
f"{CONSOLE_LOG['RESET']}"
)
if job.exception and not isinstance(job.exception, MesaCIRetriableException):
break
return last_failed_job
@@ -397,7 +370,6 @@ class LAVAJobSubmitter(PathResolver):
kernel_image_name: str = None
kernel_image_type: str = ""
kernel_url_prefix: str = None
kernel_external: str = None
lava_tags: str = "" # Comma-separated LAVA tags for the job
mesa_job_name: str = "mesa_ci_job"
pipeline_info: str = ""
@@ -407,7 +379,6 @@ class LAVAJobSubmitter(PathResolver):
job_rootfs_overlay_url: str = None
structured_log_file: pathlib.Path = None # Log file path with structured LAVA log
ssh_client_image: str = None # x86_64 SSH client image to follow the job's output
project_name: str = None # Project name to be used in the job name
__structured_log_context = contextlib.nullcontext() # Structured Logger context
def __post_init__(self) -> None:
@@ -431,7 +402,7 @@ class LAVAJobSubmitter(PathResolver):
minutes=self.job_timeout_min
)
job_definition = LAVAJobDefinition(self).generate_lava_job_definition()
job_definition = generate_lava_job_definition(self)
if self.dump_yaml:
self.dump_job_definition(job_definition)
@@ -485,8 +456,10 @@ class LAVAJobSubmitter(PathResolver):
self.finish_script(last_attempt_job)
def print_log_artifact_url(self):
base_url = "https://$CI_PROJECT_ROOT_NAMESPACE.pages.freedesktop.org/"
artifacts_path = "-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/"
relative_log_path = self.structured_log_file.relative_to(pathlib.Path.cwd())
full_path = f"$ARTIFACTS_BASE_URL/{relative_log_path}"
full_path = f"{base_url}{artifacts_path}{relative_log_path}"
artifact_url = path.expandvars(full_path)
print_log(f"Structural Logging data available at: {artifact_url}")
@@ -498,9 +471,8 @@ class LAVAJobSubmitter(PathResolver):
if not last_attempt_job:
# No job was run, something bad happened
STRUCTURAL_LOG["job_combined_status"] = "script_crash"
current_exception = str(sys.exc_info()[1])
current_exception = str(sys.exc_info()[0])
STRUCTURAL_LOG["job_combined_fail_reason"] = current_exception
print(f"Interrupting the script. Reason: {current_exception}")
raise SystemExit(1)
STRUCTURAL_LOG["job_combined_status"] = last_attempt_job.status
@@ -537,6 +509,7 @@ class StructuredLoggerWrapper:
def logger_context(self):
context = contextlib.nullcontext()
try:
global STRUCTURAL_LOG
STRUCTURAL_LOG = StructuredLogger(
self.__submitter.structured_log_file, truncate=True

View File

@@ -1,7 +1,7 @@
from .console_format import CONSOLE_LOG
from .gitlab_section import GitlabSection
from .lava_job import LAVAJob
from .lava_job_definition import LAVAJobDefinition
from .lava_job_definition import generate_lava_job_definition
from .lava_proxy import call_proxy, setup_lava_proxy
from .log_follower import (
LogFollower,

View File

@@ -1,25 +0,0 @@
from os import getenv
# How many attempts should be made when a timeout happen during LAVA device boot.
NUMBER_OF_ATTEMPTS_LAVA_BOOT = int(getenv("LAVA_NUMBER_OF_ATTEMPTS_LAVA_BOOT", 3))
# Supports any integers in [0, 100].
# The scheduler considers the job priority when ordering the queue
# to consider which job should run next.
JOB_PRIORITY = int(getenv("JOB_PRIORITY", 75))
# Use UART over the default SSH mechanism to follow logs.
# Caution: this can lead to device silence in some devices in Mesa CI.
FORCE_UART = bool(getenv("LAVA_FORCE_UART", False))
# How many times the r8152 error may happen to consider it a known issue.
KNOWN_ISSUE_R8152_MAX_CONSECUTIVE_COUNTER: int = 10
KNOWN_ISSUE_R8152_PATTERNS: tuple[str, ...] = (
r"r8152 \S+ eth0: Tx status -71",
r"nfs: server \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3} not responding, still trying",
)
# This is considered noise, since LAVA produces this log after receiving a package of feedback
# messages.
LOG_DEBUG_FEEDBACK_NOISE = "Listened to connection for namespace 'dut' done"

View File

@@ -6,7 +6,6 @@ from typing import Any, Optional
from lava.exceptions import (
MesaCIException,
MesaCIRetriableException,
MesaCIKnownIssueException,
MesaCIParseException,
MesaCITimeoutError,
@@ -35,7 +34,7 @@ class LAVAJob:
self._is_finished = False
self.log: dict[str, Any] = log
self.status = "not_submitted"
self.__exception: Optional[Exception] = None
self.__exception: Optional[str] = None
def heartbeat(self) -> None:
self.last_log_time: datetime = datetime.now()
@@ -64,13 +63,13 @@ class LAVAJob:
return self._is_finished
@property
def exception(self) -> Optional[Exception]:
def exception(self) -> str:
return self.__exception
@exception.setter
def exception(self, exception: Exception) -> None:
self.__exception = exception
self.log["dut_job_fail_reason"] = repr(self.__exception)
self.__exception = repr(exception)
self.log["dut_job_fail_reason"] = self.__exception
def validate(self) -> Optional[dict]:
"""Returns a dict with errors, if the validation fails.
@@ -177,15 +176,11 @@ class LAVAJob:
self.status = "canceled"
elif isinstance(exception, MesaCITimeoutError):
self.status = "hung"
elif isinstance(exception, MesaCIRetriableException):
elif isinstance(exception, MesaCIException):
self.status = "failed"
elif isinstance(exception, KeyboardInterrupt):
self.status = "interrupted"
print_log("LAVA job submitter was interrupted. Cancelling the job.")
raise
elif isinstance(exception, MesaCIException):
self.status = "interrupted"
print_log("LAVA job submitter was interrupted. Cancelling the job.")
raise
else:
self.status = "job_submitter_error"

View File

@@ -1,231 +1,150 @@
from io import StringIO
from typing import TYPE_CHECKING, Any
from ruamel.yaml import YAML
import re
from lava.utils.lava_farm import LavaFarm, get_lava_farm
from lava.utils.ssh_job_definition import (
generate_docker_test,
generate_dut_test,
wrap_boot_action,
wrap_final_deploy_action,
)
from lava.utils.uart_job_definition import (
fastboot_boot_action,
fastboot_deploy_actions,
tftp_boot_action,
tftp_deploy_actions,
qemu_boot_action,
qemu_deploy_actions,
uart_test_actions,
)
from ruamel.yaml.scalarstring import LiteralScalarString
from ruamel.yaml import YAML
from os import getenv
if TYPE_CHECKING:
from lava.lava_job_submitter import LAVAJobSubmitter
from .constants import FORCE_UART, JOB_PRIORITY, NUMBER_OF_ATTEMPTS_LAVA_BOOT
# How many attempts should be made when a timeout happen during LAVA device boot.
NUMBER_OF_ATTEMPTS_LAVA_BOOT = int(getenv("LAVA_NUMBER_OF_ATTEMPTS_LAVA_BOOT", 3))
# Supports any integers in [0, 100].
# The scheduler considers the job priority when ordering the queue
# to consider which job should run next.
JOB_PRIORITY = int(getenv("JOB_PRIORITY", 75))
class LAVAJobDefinition:
def has_ssh_support(job_submitter: "LAVAJobSubmitter") -> bool:
force_uart = bool(getenv("LAVA_FORCE_UART", False))
if force_uart:
return False
# Only Collabora's farm supports to run docker container as a LAVA actions,
# which is required to follow the job in a SSH section
current_farm = get_lava_farm()
# SSH job definition still needs to add support for fastboot.
job_uses_fastboot: bool = job_submitter.boot_method == "fastboot"
return current_farm == LavaFarm.COLLABORA and not job_uses_fastboot
def generate_lava_yaml_payload(job_submitter: "LAVAJobSubmitter") -> dict[str, Any]:
"""
This class is responsible for generating the YAML payload to submit a LAVA
job.
Bridge function to use the supported job definition depending on some Mesa
CI job characteristics.
The strategy here, is to use LAVA with a containerized SSH session to follow
the job output, escaping from dumping data to the UART, which proves to be
error prone in some devices.
"""
from lava.utils.ssh_job_definition import (
generate_lava_yaml_payload as ssh_lava_yaml,
)
from lava.utils.uart_job_definition import (
generate_lava_yaml_payload as uart_lava_yaml,
)
def __init__(self, job_submitter: "LAVAJobSubmitter") -> None:
self.job_submitter: "LAVAJobSubmitter" = job_submitter
if has_ssh_support(job_submitter):
return ssh_lava_yaml(job_submitter)
def has_ssh_support(self) -> bool:
if FORCE_UART:
return False
return uart_lava_yaml(job_submitter)
# Only Collabora's farm supports to run docker container as a LAVA actions,
# which is required to follow the job in a SSH section
current_farm = get_lava_farm()
return current_farm == LavaFarm.COLLABORA
def generate_lava_job_definition(job_submitter: "LAVAJobSubmitter") -> str:
job_stream = StringIO()
yaml = YAML()
yaml.width = 4096
yaml.dump(generate_lava_yaml_payload(job_submitter), job_stream)
return job_stream.getvalue()
def generate_lava_yaml_payload(self) -> dict[str, Any]:
"""
Generates a YAML payload for submitting a LAVA job, based on the provided arguments.
Args:
None
def to_yaml_block(steps_array: list[str], escape_vars=[]) -> LiteralScalarString:
def escape_envvar(match):
return "\\" + match.group(0)
Returns:
a dictionary containing the values generated by the `generate_metadata` function and the
actions for the LAVA job submission.
"""
args = self.job_submitter
values = self.generate_metadata()
nfsrootfs = {
"url": f"{args.rootfs_url_prefix}/lava-rootfs.tar.zst",
"compression": "zstd",
}
filtered_array = [s for s in steps_array if s.strip() and not s.startswith("#")]
final_str = "\n".join(filtered_array)
init_stage1_steps = self.init_stage1_steps()
artifact_download_steps = self.artifact_download_steps()
for escape_var in escape_vars:
# Find env vars and add '\\' before them
final_str = re.sub(rf"\${escape_var}*", escape_envvar, final_str)
return LiteralScalarString(final_str)
deploy_actions = []
boot_action = []
test_actions = uart_test_actions(args, init_stage1_steps, artifact_download_steps)
if args.boot_method == "fastboot":
deploy_actions = fastboot_deploy_actions(self, nfsrootfs)
boot_action = fastboot_boot_action(args)
elif args.boot_method == "qemu-nfs":
deploy_actions = qemu_deploy_actions(self, nfsrootfs)
boot_action = qemu_boot_action(args)
else: # tftp
deploy_actions = tftp_deploy_actions(self, nfsrootfs)
boot_action = tftp_boot_action(args)
if self.has_ssh_support():
wrap_final_deploy_action(deploy_actions[-1])
# SSH jobs use namespaces to differentiate between the DUT and the
# docker container. Every LAVA action needs an explicit namespace, when we are not using
# the default one.
for deploy_action in deploy_actions:
deploy_action["namespace"] = "dut"
wrap_boot_action(boot_action)
test_actions = (
generate_dut_test(args, init_stage1_steps),
generate_docker_test(args, artifact_download_steps),
)
values["actions"] = [
*[{"deploy": d} for d in deploy_actions],
{"boot": boot_action},
*[{"test": t} for t in test_actions],
]
return values
def generate_lava_job_definition(self) -> str:
"""
Generates a LAVA job definition in YAML format and returns it as a string.
Returns:
a string representation of the job definition generated by analysing job submitter
arguments and environment variables
"""
job_stream = StringIO()
yaml = YAML()
yaml.width = 4096
yaml.dump(self.generate_lava_yaml_payload(), job_stream)
return job_stream.getvalue()
def generate_metadata(self) -> dict[str, Any]:
# General metadata and permissions
values = {
"job_name": f"{self.job_submitter.project_name}: {self.job_submitter.pipeline_info}",
"device_type": self.job_submitter.device_type,
"visibility": {"group": [self.job_submitter.visibility_group]},
"priority": JOB_PRIORITY,
"context": {"extra_nfsroot_args": " init=/init rootwait usbcore.quirks=0bda:8153:k"},
"timeouts": {
"job": {"minutes": self.job_submitter.job_timeout_min},
"actions": {
"depthcharge-retry": {
# Could take between 1 and 1.5 min in slower boots
"minutes": 4
},
"depthcharge-start": {
# Should take less than 1 min.
"minutes": 1,
},
"depthcharge-action": {
# This timeout englobes the entire depthcharge timing,
# including retries
"minutes": 5
* NUMBER_OF_ATTEMPTS_LAVA_BOOT,
},
def generate_metadata(args) -> dict[str, Any]:
# General metadata and permissions
values = {
"job_name": f"mesa: {args.pipeline_info}",
"device_type": args.device_type,
"visibility": {"group": [args.visibility_group]},
"priority": JOB_PRIORITY,
"context": {
"extra_nfsroot_args": " init=/init rootwait usbcore.quirks=0bda:8153:k"
},
"timeouts": {
"job": {"minutes": args.job_timeout_min},
"actions": {
"depthcharge-retry": {
# Could take between 1 and 1.5 min in slower boots
"minutes": 4
},
"depthcharge-start": {
# Should take less than 1 min.
"minutes": 1,
},
"depthcharge-action": {
# This timeout englobes the entire depthcharge timing,
# including retries
"minutes": 5
* NUMBER_OF_ATTEMPTS_LAVA_BOOT,
},
},
}
},
}
if self.job_submitter.lava_tags:
values["tags"] = self.job_submitter.lava_tags.split(",")
if args.lava_tags:
values["tags"] = args.lava_tags.split(",")
# QEMU lava jobs mandate proper arch value in the context
if self.job_submitter.boot_method == "qemu-nfs":
values["context"]["arch"] = self.job_submitter.mesa_job_name.split(":")[1]
return values
return values
def attach_kernel_and_dtb(self, deploy_field):
if self.job_submitter.kernel_image_type:
deploy_field["kernel"]["type"] = self.job_submitter.kernel_image_type
if self.job_submitter.dtb_filename:
deploy_field["dtb"] = {
"url": f"{self.job_submitter.kernel_url_prefix}/"
f"{self.job_submitter.dtb_filename}.dtb"
}
def artifact_download_steps(args):
"""
This function is responsible for setting up the SSH server in the DUT and to
export the first boot environment to a file.
"""
# Putting JWT pre-processing and mesa download, within init-stage1.sh file,
# as we do with non-SSH version.
download_steps = [
"set -ex",
"curl -L --retry 4 -f --retry-all-errors --retry-delay 60 "
f"{args.job_rootfs_overlay_url} | tar -xz -C /",
f"mkdir -p {args.ci_project_dir}",
f"curl -L --retry 4 -f --retry-all-errors --retry-delay 60 {args.build_url} | "
f"tar --zstd -x -C {args.ci_project_dir}",
]
def attach_external_modules(self, deploy_field):
if self.job_submitter.kernel_external:
deploy_field["modules"] = {
"url": f"{self.job_submitter.kernel_url_prefix}/modules.tar.zst",
"compression": "zstd"
}
def artifact_download_steps(self):
"""
This function is responsible for setting up the SSH server in the DUT and to
export the first boot environment to a file.
"""
# Putting JWT pre-processing and mesa download, within init-stage1.sh file,
# as we do with non-SSH version.
download_steps = [
"set -ex",
"curl -L --retry 4 -f --retry-all-errors --retry-delay 60 "
f"{self.job_submitter.job_rootfs_overlay_url} | tar -xz -C /",
f"mkdir -p {self.job_submitter.ci_project_dir}",
f"curl -L --retry 4 -f --retry-all-errors --retry-delay 60 {self.job_submitter.build_url} | "
f"tar --zstd -x -C {self.job_submitter.ci_project_dir}",
# If the JWT file is provided, we will use it to authenticate with the cloud
# storage provider and will hide it from the job output in Gitlab.
if args.jwt_file:
with open(args.jwt_file) as jwt_file:
download_steps += [
"set +x # HIDE_START",
f'echo -n "{jwt_file.read()}" > "{args.jwt_file}"',
"set -x # HIDE_END",
f'echo "export CI_JOB_JWT_FILE={args.jwt_file}" >> /set-job-env-vars.sh',
]
else:
download_steps += [
"echo Could not find jwt file, disabling S3 requests...",
"sed -i '/S3_RESULTS_UPLOAD/d' /set-job-env-vars.sh",
]
# If the JWT file is provided, we will use it to authenticate with the cloud
# storage provider and will hide it from the job output in Gitlab.
if self.job_submitter.jwt_file:
with open(self.job_submitter.jwt_file) as jwt_file:
download_steps += [
"set +x # HIDE_START",
f'echo -n "{jwt_file.read()}" > "{self.job_submitter.jwt_file}"',
"set -x # HIDE_END",
f'echo "export S3_JWT_FILE={self.job_submitter.jwt_file}" >> /set-job-env-vars.sh',
]
else:
download_steps += [
"echo Could not find jwt file, disabling S3 requests...",
"sed -i '/S3_RESULTS_UPLOAD/d' /set-job-env-vars.sh",
]
return download_steps
def init_stage1_steps(self) -> list[str]:
run_steps = []
# job execution script:
# - inline .gitlab-ci/common/init-stage1.sh
# - fetch and unpack per-pipeline build artifacts from build job
# - fetch and unpack per-job environment from lava-submit.sh
# - exec .gitlab-ci/common/init-stage2.sh
with open(self.job_submitter.first_stage_init, "r") as init_sh:
# For vmware farm, patch nameserver as 8.8.8.8 is off limit.
# This is temporary and will be reverted once the farm is moved.
if self.job_submitter.mesa_job_name.startswith("vmware-"):
run_steps += [x.rstrip().replace("nameserver 8.8.8.8", "nameserver 10.25.198.110") for x in init_sh if not x.startswith("#") and x.rstrip()]
else:
run_steps += [x.rstrip() for x in init_sh if not x.startswith("#") and x.rstrip()]
# We cannot distribute the Adreno 660 shader firmware inside rootfs,
# since the license isn't bundled inside the repository
if self.job_submitter.device_type == "sm8350-hdk":
run_steps.append(
"curl -L --retry 4 -f --retry-all-errors --retry-delay 60 "
+ "https://github.com/allahjasif1990/hdk888-firmware/raw/main/a660_zap.mbn "
+ '-o "/lib/firmware/qcom/sm8350/a660_zap.mbn"'
)
return run_steps
return download_steps

View File

@@ -2,78 +2,42 @@ from __future__ import annotations
import re
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any, Sequence
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from lava.utils import LogFollower
from lava.exceptions import MesaCIKnownIssueException
from lava.utils.console_format import CONSOLE_LOG
from lava.utils.constants import (
KNOWN_ISSUE_R8152_MAX_CONSECUTIVE_COUNTER,
LOG_DEBUG_FEEDBACK_NOISE,
KNOWN_ISSUE_R8152_PATTERNS,
)
from lava.utils.log_section import LogSectionType
def search_known_issue_patterns(patterns: Sequence[str], line: str) -> str:
for pattern in patterns:
if re.search(pattern, line):
return pattern
return ""
@dataclass
class LAVALogHints:
log_follower: LogFollower
r8152_issue_consecutive_counter: int = field(default=0, init=False)
reboot_counter: int = field(default=0, init=False)
def raise_known_issue(self, message) -> None:
raise MesaCIKnownIssueException(
"Found known issue: "
f"{CONSOLE_LOG['FG_MAGENTA']}"
f"{message}"
f"{CONSOLE_LOG['RESET']}"
)
has_r8152_issue_history: bool = field(default=False, init=False)
def detect_failure(self, new_lines: list[dict[str, Any]]):
for line in new_lines:
if line["msg"] == LOG_DEBUG_FEEDBACK_NOISE:
continue
self.detect_r8152_issue(line)
self.detect_forced_reboot(line)
def detect_r8152_issue(self, line):
if self.log_follower.phase in (
LogSectionType.LAVA_BOOT,
LogSectionType.TEST_CASE,
) and line["lvl"] in ("feedback", "target"):
if search_known_issue_patterns(KNOWN_ISSUE_R8152_PATTERNS, line["msg"]):
if (
self.r8152_issue_consecutive_counter
< KNOWN_ISSUE_R8152_MAX_CONSECUTIVE_COUNTER
):
self.r8152_issue_consecutive_counter += 1
return
self.raise_known_issue(
"Probable network issue failure encountered, retrying the job"
)
# Reset the status, as the `nfs... still trying` complaint was not detected
self.r8152_issue_consecutive_counter = 0
def detect_forced_reboot(self, line: dict[str, Any]) -> None:
if (
self.log_follower.phase == LogSectionType.TEST_CASE
and line["lvl"] == "feedback"
and line["lvl"] == "target"
):
if re.search(r"^Reboot requested", line["msg"]):
self.reboot_counter += 1
if re.search(r"r8152 \S+ eth0: Tx status -71", line["msg"]):
self.has_r8152_issue_history = True
return
if self.reboot_counter > 0:
self.raise_known_issue(
"Forced reboot detected during test phase, failing the job..."
)
if self.has_r8152_issue_history and re.search(
r"nfs: server \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3} not responding, still trying",
line["msg"],
):
raise MesaCIKnownIssueException(
f"{CONSOLE_LOG['FG_MAGENTA']}"
"Probable network issue failure encountered, retrying the job"
f"{CONSOLE_LOG['RESET']}"
)
self.has_r8152_issue_history = False

View File

@@ -187,25 +187,6 @@ class LogFollower:
return False
def ignore_dut_feedback_lines(self, line: dict[str, str]) -> bool:
"""
Ignores feedback lines from LAVA.
If we only receive this level of message for some time, it means that the job is
misbehaving. E.g Rebooting.
Args:
line: A dictionary representing a single log line.
Returns:
A boolean indicating whether the current line is a feedback line.
"""
if line["lvl"] == "feedback" and line["ns"] == "dut":
return True
if line["lvl"] == "debug":
# This message happens after LAVA end receiving the feedback from the DUT
if line["msg"] == "Listened to connection for namespace 'dut' done":
return True
return False
def feed(self, new_lines: list[dict[str, str]]) -> bool:
"""Input data to be processed by LogFollower instance
@@ -226,9 +207,6 @@ class LogFollower:
if self.merge_carriage_return_lines(line):
continue
if self.ignore_dut_feedback_lines(line):
continue
# At least we are fed with a non-kernel dump log, it seems that the
# job is progressing
is_job_healthy = True

View File

@@ -28,15 +28,15 @@ script after sourcing "dut-env-vars.sh" again for the second SSH test case.
"""
import re
from typing import TYPE_CHECKING, Any, Iterable
from pathlib import Path
from typing import Any
from ruamel.yaml.scalarstring import LiteralScalarString
from .constants import NUMBER_OF_ATTEMPTS_LAVA_BOOT
if TYPE_CHECKING:
from ..lava_job_submitter import LAVAJobSubmitter
from .lava_job_definition import (
NUMBER_OF_ATTEMPTS_LAVA_BOOT,
artifact_download_steps,
generate_metadata,
to_yaml_block,
)
# Very early SSH server setup. Uses /dut_ready file to flag it is done.
SSH_SERVER_COMMANDS = {
@@ -79,23 +79,12 @@ lava_ssh_test_case() {
]
def to_yaml_block(steps_array: Iterable[str], escape_vars=[]) -> LiteralScalarString:
def escape_envvar(match):
return "\\" + match.group(0)
filtered_array = [s for s in steps_array if s.strip() and not s.startswith("#")]
final_str = "\n".join(filtered_array)
for escape_var in escape_vars:
# Find env vars and add '\\' before them
final_str = re.sub(rf"\${escape_var}*", escape_envvar, final_str)
return LiteralScalarString(final_str)
def generate_dut_test(args: "LAVAJobSubmitter", first_stage_steps: list[str]) -> dict[str, Any]:
def generate_dut_test(args):
# Commands executed on DUT.
# Trying to execute the minimal number of commands, because the console data is
# retrieved via UART, which is hang-prone in some devices.
first_stage_steps: list[str] = Path(args.first_stage_init).read_text().splitlines()
return {
"namespace": "dut",
"definitions": [
@@ -120,9 +109,7 @@ def generate_dut_test(args: "LAVAJobSubmitter", first_stage_steps: list[str]) ->
}
def generate_docker_test(
args: "LAVAJobSubmitter", artifact_download_steps: list[str]
) -> dict[str, Any]:
def generate_docker_test(args):
# This is a growing list of commands that will be executed by the docker
# guest, which will be the SSH client.
docker_commands = []
@@ -161,14 +148,14 @@ def generate_docker_test(
(
"lava_ssh_test_case 'artifact_download' 'bash --' << EOF",
"source /dut-env-vars.sh",
*artifact_download_steps,
*artifact_download_steps(args),
"EOF",
)
),
"export SSH_PTY_ARGS=-tt",
# Putting CI_JOB name as the testcase name, it may help LAVA farm
# maintainers with monitoring
f"lava_ssh_test_case '{args.project_name}_{args.mesa_job_name}' "
f"lava_ssh_test_case 'mesa-ci_{args.mesa_job_name}' "
# Changing directory to /, as the HWCI_SCRIPT expects that
"'\"cd / && /init-stage2.sh\"'",
]
@@ -176,21 +163,46 @@ def generate_docker_test(
return init_stages_test
def wrap_final_deploy_action(final_deploy_action: dict):
wrap = {
def generate_lava_yaml_payload(args) -> dict[str, Any]:
values = generate_metadata(args)
# URLs to our kernel rootfs to boot from, both generated by the base
# container build
deploy = {
"namespace": "dut",
"failure_retry": NUMBER_OF_ATTEMPTS_LAVA_BOOT,
"timeout": {"minutes": 10},
"timeouts": {"http-download": {"minutes": 2}},
"to": "tftp",
"os": "oe",
"kernel": {"url": f"{args.kernel_url_prefix}/{args.kernel_image_name}"},
"nfsrootfs": {
"url": f"{args.rootfs_url_prefix}/lava-rootfs.tar.zst",
"compression": "zstd",
},
}
if args.kernel_image_type:
deploy["kernel"]["type"] = args.kernel_image_type
if args.dtb_filename:
deploy["dtb"] = {"url": f"{args.kernel_url_prefix}/{args.dtb_filename}.dtb"}
final_deploy_action.update(wrap)
def wrap_boot_action(boot_action: dict):
wrap = {
# always boot over NFS
boot = {
"namespace": "dut",
"failure_retry": NUMBER_OF_ATTEMPTS_LAVA_BOOT,
"method": args.boot_method,
"commands": "nfs",
"prompts": ["lava-shell:"],
**SSH_SERVER_COMMANDS,
}
boot_action.update(wrap)
# only declaring each job as a single 'test' since LAVA's test parsing is
# not useful to us
values["actions"] = [
{"deploy": deploy},
{"boot": boot},
{"test": generate_dut_test(args)},
{"test": generate_docker_test(args)},
]
return values

View File

@@ -1,23 +1,22 @@
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from ..lava_job_submitter import LAVAJobSubmitter
from .lava_job_definition import LAVAJobDefinition
from .constants import NUMBER_OF_ATTEMPTS_LAVA_BOOT
# Use the same image that is being used for the hardware enablement and health-checks.
# They are pretty small (<100MB) and have all the tools we need to run LAVA, so it is a safe choice.
# You can find the Dockerfile here:
# https://gitlab.collabora.com/lava/health-check-docker/-/blob/main/Dockerfile
# And the registry here: https://gitlab.collabora.com/lava/health-check-docker/container_registry/
DOCKER_IMAGE = "registry.gitlab.collabora.com/lava/health-check-docker"
from typing import Any
from .lava_job_definition import (
generate_metadata,
NUMBER_OF_ATTEMPTS_LAVA_BOOT,
artifact_download_steps,
)
def fastboot_deploy_actions(
job_definition: "LAVAJobDefinition", nfsrootfs
) -> tuple[dict[str, Any], ...]:
args = job_definition.job_submitter
def generate_lava_yaml_payload(args) -> dict[str, Any]:
values = generate_metadata(args)
# URLs to our kernel rootfs to boot from, both generated by the base
# container build
nfsrootfs = {
"url": f"{args.rootfs_url_prefix}/lava-rootfs.tar.zst",
"compression": "zstd",
}
fastboot_deploy_nfs = {
"timeout": {"minutes": 10},
"to": "nfs",
@@ -35,7 +34,7 @@ def fastboot_deploy_actions(
},
"postprocess": {
"docker": {
"image": DOCKER_IMAGE,
"image": "registry.gitlab.collabora.com/lava/health-check-docker",
"steps": [
f"cat Image.gz {args.dtb_filename}.dtb > Image.gz+dtb",
"mkbootimg --kernel Image.gz+dtb"
@@ -45,28 +44,13 @@ def fastboot_deploy_actions(
}
},
}
if args.kernel_image_type:
fastboot_deploy_prepare["images"]["kernel"]["type"] = args.kernel_image_type
if args.dtb_filename:
fastboot_deploy_prepare["images"]["dtb"] = {
"url": f"{args.kernel_url_prefix}/{args.dtb_filename}.dtb"
}
fastboot_deploy = {
"timeout": {"minutes": 2},
"to": "fastboot",
"docker": {
"image": DOCKER_IMAGE,
},
"images": {
"boot": {"url": "downloads://boot.img"},
},
}
# URLs to our kernel rootfs to boot from, both generated by the base
# container build
job_definition.attach_kernel_and_dtb(fastboot_deploy_prepare["images"])
job_definition.attach_external_modules(fastboot_deploy_nfs)
return (fastboot_deploy_nfs, fastboot_deploy_prepare, fastboot_deploy)
def tftp_deploy_actions(job_definition: "LAVAJobDefinition", nfsrootfs) -> tuple[dict[str, Any]]:
args = job_definition.job_submitter
tftp_deploy = {
"timeout": {"minutes": 5},
"to": "tftp",
@@ -76,33 +60,40 @@ def tftp_deploy_actions(job_definition: "LAVAJobDefinition", nfsrootfs) -> tuple
},
"nfsrootfs": nfsrootfs,
}
job_definition.attach_kernel_and_dtb(tftp_deploy)
job_definition.attach_external_modules(tftp_deploy)
if args.kernel_image_type:
tftp_deploy["kernel"]["type"] = args.kernel_image_type
if args.dtb_filename:
tftp_deploy["dtb"] = {
"url": f"{args.kernel_url_prefix}/{args.dtb_filename}.dtb"
}
return (tftp_deploy,)
def qemu_deploy_actions(job_definition: "LAVAJobDefinition", nfsrootfs) -> tuple[dict[str, Any]]:
args = job_definition.job_submitter
qemu_deploy = {
"timeout": {"minutes": 5},
"to": "nfs",
fastboot_deploy = {
"timeout": {"minutes": 2},
"to": "fastboot",
"docker": {
"image": "registry.gitlab.collabora.com/lava/health-check-docker",
},
"images": {
"kernel": {
"image_arg": "-kernel {kernel}",
"url": f"{args.kernel_url_prefix}/{args.kernel_image_name}",
},
"nfsrootfs": nfsrootfs,
"boot": {"url": "downloads://boot.img"},
},
}
job_definition.attach_external_modules(qemu_deploy)
return (qemu_deploy,)
fastboot_boot = {
"timeout": {"minutes": 2},
"docker": {"image": "registry.gitlab.collabora.com/lava/health-check-docker"},
"failure_retry": NUMBER_OF_ATTEMPTS_LAVA_BOOT,
"method": args.boot_method,
"prompts": ["lava-shell:"],
"commands": ["set_active a"],
}
tftp_boot = {
"failure_retry": NUMBER_OF_ATTEMPTS_LAVA_BOOT,
"method": args.boot_method,
"prompts": ["lava-shell:"],
"commands": "nfs",
}
def uart_test_actions(
args: "LAVAJobSubmitter", init_stage1_steps: list[str], artifact_download_steps: list[str]
) -> tuple[dict[str, Any]]:
# skeleton test definition: only declaring each job as a single 'test'
# since LAVA's test parsing is not useful to us
run_steps = []
@@ -129,8 +120,26 @@ def uart_test_actions(
],
}
run_steps += init_stage1_steps
run_steps += artifact_download_steps
# job execution script:
# - inline .gitlab-ci/common/init-stage1.sh
# - fetch and unpack per-pipeline build artifacts from build job
# - fetch and unpack per-job environment from lava-submit.sh
# - exec .gitlab-ci/common/init-stage2.sh
with open(args.first_stage_init, "r") as init_sh:
run_steps += [
x.rstrip() for x in init_sh if not x.startswith("#") and x.rstrip()
]
# We cannot distribute the Adreno 660 shader firmware inside rootfs,
# since the license isn't bundled inside the repository
if args.device_type == "sm8350-hdk":
run_steps.append(
"curl -L --retry 4 -f --retry-all-errors --retry-delay 60 "
+ "https://github.com/allahjasif1990/hdk888-firmware/raw/main/a660_zap.mbn "
+ '-o "/lib/firmware/qcom/sm8350/a660_zap.mbn"'
)
run_steps += artifact_download_steps(args)
run_steps += [
f"mkdir -p {args.ci_project_dir}",
@@ -141,41 +150,22 @@ def uart_test_actions(
"sleep 1",
# Putting CI_JOB name as the testcase name, it may help LAVA farm
# maintainers with monitoring
f"lava-test-case '{args.project_name}_{args.mesa_job_name}' --shell /init-stage2.sh",
f"lava-test-case 'mesa-ci_{args.mesa_job_name}' --shell /init-stage2.sh",
]
return (test,)
if args.boot_method == "fastboot":
values["actions"] = [
{"deploy": fastboot_deploy_nfs},
{"deploy": fastboot_deploy_prepare},
{"deploy": fastboot_deploy},
{"boot": fastboot_boot},
{"test": test},
]
else: # tftp
values["actions"] = [
{"deploy": tftp_deploy},
{"boot": tftp_boot},
{"test": test},
]
def tftp_boot_action(args: "LAVAJobSubmitter") -> dict[str, Any]:
tftp_boot = {
"failure_retry": NUMBER_OF_ATTEMPTS_LAVA_BOOT,
"method": args.boot_method,
"prompts": ["lava-shell:"],
"commands": "nfs",
}
return tftp_boot
def qemu_boot_action(args: "LAVAJobSubmitter") -> dict[str, Any]:
qemu_boot = {
"failure_retry": NUMBER_OF_ATTEMPTS_LAVA_BOOT,
"method": args.boot_method,
"prompts": ["lava-shell:"],
}
return qemu_boot
def fastboot_boot_action(args: "LAVAJobSubmitter") -> dict[str, Any]:
fastboot_boot = {
"timeout": {"minutes": 2},
"docker": {"image": DOCKER_IMAGE},
"failure_retry": NUMBER_OF_ATTEMPTS_LAVA_BOOT,
"method": args.boot_method,
"prompts": ["lava-shell:"],
"commands": ["set_active a"],
}
return fastboot_boot
return values

View File

@@ -2,7 +2,7 @@
# shellcheck disable=SC1003 # works for us now...
# shellcheck disable=SC2086 # we want word splitting
section_switch meson-cross-file "meson: cross file generate"
section_switch meson-configure "meson: configure"
set -e
set -o xtrace
@@ -49,42 +49,11 @@ if [ -n "$CROSS" ]; then
fi
fi
if [ -n "$HOST_BUILD_OPTIONS" ]; then
section_switch meson-host-configure "meson: host configure"
# Stash the PKG_CONFIG_LIBDIR so that we can use the base x86_64 image
# libraries.
tmp_pkg_config_libdir=$PKG_CONFIG_LIBDIR
unset PKG_CONFIG_LIBDIR
# Compile a host version for the few tools we need for a cross build (for
# now just intel-clc)
rm -rf _host_build
meson setup _host_build \
--native-file=native.file \
-D prefix=/usr \
-D libdir=lib \
${HOST_BUILD_OPTIONS}
pushd _host_build
section_switch meson-host-build "meson: host build"
meson configure
ninja
ninja install
popd
# Restore PKG_CONFIG_LIBDIR
if [ -n "$tmp_pkg_config_libdir" ]; then
export PKG_CONFIG_LIBDIR=$tmp_pkg_config_libdir
fi
fi
# Only use GNU time if available, not any shell built-in command
case $CI_JOB_NAME in
# strace and wine don't seem to mix well
# ASAN leak detection is incompatible with strace
*-asan*)
debian-mingw32-x86_64|*-asan*)
if test -f /usr/bin/time; then
MESON_TEST_ARGS+=--wrapper=$PWD/.gitlab-ci/meson/time.sh
fi
@@ -98,13 +67,11 @@ case $CI_JOB_NAME in
;;
esac
section_switch meson-configure "meson: configure"
rm -rf _build
meson setup _build \
--native-file=native.file \
--wrap-mode=nofallback \
--force-fallback-for perfetto,syn,paste \
--force-fallback-for perfetto \
${CROSS+--cross "$CROSS_FILE"} \
-D prefix=$PWD/install \
-D libdir=lib \
@@ -118,10 +85,9 @@ meson setup _build \
-D libunwind=${UNWIND} \
${DRI_LOADERS} \
${GALLIUM_ST} \
-D gallium-opencl=disabled \
-D gallium-drivers=${GALLIUM_DRIVERS:-[]} \
-D vulkan-drivers=${VULKAN_DRIVERS:-[]} \
-D video-codecs=all \
-D video-codecs=h264dec,h264enc,h265dec,h265enc,vc1dec \
-D werror=true \
${EXTRA_OPTION}
cd _build
@@ -138,11 +104,10 @@ fi
uncollapsed_section_switch meson-test "meson: test"
LC_ALL=C.UTF-8 meson test --num-processes "${FDO_CI_CONCURRENT:-4}" --print-errorlogs ${MESON_TEST_ARGS}
section_switch meson-install "meson: install"
if command -V mold &> /dev/null ; then
mold --run ninja install
else
ninja install
fi
cd ..
section_end meson-install
section_end meson-test

View File

@@ -13,7 +13,7 @@ INSTALL="$PWD/install"
# Set up the driver environment.
export LD_LIBRARY_PATH="$INSTALL/lib/"
export EGL_PLATFORM=surfaceless
export VK_DRIVER_FILES="$INSTALL/share/vulkan/icd.d/${VK_DRIVER}_icd.${VK_CPU:-$(uname -m)}.json"
export VK_ICD_FILENAMES="$INSTALL/share/vulkan/icd.d/${VK_DRIVER}_icd.${VK_CPU:-$(uname -m)}.json"
RESULTS=$PWD/${PIGLIT_RESULTS_DIR:-results}
mkdir -p $RESULTS
@@ -106,7 +106,7 @@ deqp-runner junit \
--results $RESULTS/failures.csv \
--output $RESULTS/junit.xml \
--limit 50 \
--template "See $ARTIFACTS_BASE_URL/results/{{testcase}}.xml"
--template "See https://$CI_PROJECT_ROOT_NAMESPACE.pages.freedesktop.org/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/results/{{testcase}}.xml"
# Report the flakes to the IRC channel for monitoring (if configured):
if [ -n "$FLAKES_CHANNEL" ]; then

View File

@@ -8,17 +8,11 @@ set -ex
export PAGER=cat # FIXME: export everywhere
INSTALL=$(realpath -s "$PWD"/install)
S3_ARGS="--token-file ${S3_JWT_FILE}"
S3_ARGS="--token-file ${CI_JOB_JWT_FILE}"
RESULTS=$(realpath -s "$PWD"/results)
mkdir -p "$RESULTS"
export PIGLIT_REPLAY_DESCRIPTION_FILE="$INSTALL/$PIGLIT_TRACES_FILE"
# FIXME: guess why /usr/local/bin is not included in all runners PATH.
# Needed because yq and ci-fairy are installed there.
PATH="/usr/local/bin:$PATH"
if [ "$PIGLIT_REPLAY_SUBCOMMAND" = "profile" ]; then
yq -iY 'del(.traces[][] | select(.label[]? == "no-perf"))' \
"$PIGLIT_REPLAY_DESCRIPTION_FILE"
@@ -54,7 +48,7 @@ if [ -n "${VK_DRIVER}" ]; then
export DXVK_LOG="$RESULTS/dxvk"
[ -d "$DXVK_LOG" ] || mkdir -pv "$DXVK_LOG"
export DXVK_STATE_CACHE=0
export VK_DRIVER_FILES="$INSTALL/share/vulkan/icd.d/${VK_DRIVER}_icd.${VK_CPU:-$(uname -m)}.json"
export VK_ICD_FILENAMES="$INSTALL/share/vulkan/icd.d/${VK_DRIVER}_icd.${VK_CPU:-$(uname -m)}.json"
fi
# Sanity check to ensure that our environment is sufficient to make our tests
@@ -117,7 +111,7 @@ else
mkdir -p /tmp/.X11-unix
env \
VK_DRIVER_FILES="/install/share/vulkan/icd.d/${VK_DRIVER}_icd.$(uname -m).json" \
VK_ICD_FILENAMES="/install/share/vulkan/icd.d/${VK_DRIVER}_icd.$(uname -m).json" \
weston -Bheadless-backend.so --use-gl -Swayland-0 --xwayland --idle-time=0 &
while [ ! -S "$WESTON_X11_SOCK" ]; do sleep 1; done
@@ -189,20 +183,13 @@ RUN_CMD="export LD_LIBRARY_PATH=$__LD_LIBRARY_PATH; $SANITY_MESA_VERSION_CMD &&
# run.
rm -rf replayer-db
# ANGLE: download compiled ANGLE runtime and the compiled restricted traces (all-in-one package)
if [ -n "$PIGLIT_REPLAY_ANGLE_TAG" ]; then
ARCH="amd64"
FILE="angle-bin-${ARCH}-${PIGLIT_REPLAY_ANGLE_TAG}.tar.zst"
ci-fairy s3cp $S3_ARGS "https://s3.freedesktop.org/mesa-tracie-private/${FILE}" "${FILE}"
mkdir -p replayer-db/angle
tar --zstd -xf ${FILE} -C replayer-db/angle/
fi
if ! eval $RUN_CMD;
then
printf "%s\n" "Found $(cat /tmp/version.txt), expected $MESA_VERSION"
fi
ARTIFACTS_BASE_URL="https://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_NAME}/-/jobs/${CI_JOB_ID}/artifacts"
./piglit summary aggregate "$RESULTS" -o junit.xml
PIGLIT_RESULTS="${PIGLIT_RESULTS:-replay}"

View File

@@ -23,7 +23,7 @@ else
STRIP="strip"
fi
if [ -z "$ARTIFACTS_DEBUG_SYMBOLS" ]; then
find install -name \*.so -exec $STRIP --strip-debug {} \;
find install -name \*.so -exec $STRIP {} \;
fi
# Test runs don't pull down the git tree, so put the dEQP helper
@@ -38,11 +38,10 @@ cp -Rp .gitlab-ci/fossilize-runner.sh install/
cp -Rp .gitlab-ci/crosvm-init.sh install/
cp -Rp .gitlab-ci/*.txt install/
cp -Rp .gitlab-ci/report-flakes.py install/
cp -Rp .gitlab-ci/valve install/
cp -Rp .gitlab-ci/vkd3d-proton install/
cp -Rp .gitlab-ci/setup-test-env.sh install/
cp -Rp .gitlab-ci/*-runner.sh install/
cp -Rp .gitlab-ci/bin/structured_logger.py install/
cp -Rp .gitlab-ci/bin/custom_logger.py install/
find . -path \*/ci/\*.txt \
-o -path \*/ci/\*.toml \
-o -path \*/ci/\*traces\*.yml \
@@ -60,7 +59,7 @@ if [ -n "$S3_ARTIFACT_NAME" ]; then
# Pass needed files to the test stage
S3_ARTIFACT_NAME="$S3_ARTIFACT_NAME.tar.zst"
zstd artifacts/install.tar -o ${S3_ARTIFACT_NAME}
ci-fairy s3cp --token-file "${S3_JWT_FILE}" ${S3_ARTIFACT_NAME} https://${PIPELINE_ARTIFACTS_BASE}/${S3_ARTIFACT_NAME}
ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" ${S3_ARTIFACT_NAME} https://${PIPELINE_ARTIFACTS_BASE}/${S3_ARTIFACT_NAME}
fi
section_end prepare-artifacts

View File

@@ -10,7 +10,7 @@ export LD_LIBRARY_PATH=$LIBDIR
cd /usr/local/shader-db
for driver in freedreno intel lima v3d vc4; do
for driver in freedreno intel v3d vc4; do
section_start shader-db-${driver} "Running shader-db for $driver"
env LD_PRELOAD="$LIBDIR/lib${driver}_noop_drm_shim.so" \
./run -j"${FDO_CI_CONCURRENT:-4}" ./shaders \

View File

@@ -14,14 +14,6 @@ function x_off {
# TODO: implement x_on !
export JOB_START_S=$(date -u +"%s" -d "${CI_JOB_STARTED_AT:?}")
function get_current_minsec {
DATE_S=$(date -u +"%s")
CURR_TIME=$((DATE_S-JOB_START_S))
printf "%02d:%02d" $((CURR_TIME/60)) $((CURR_TIME%60))
}
function error {
x_off 2>/dev/null
RED="\e[0;31m"
@@ -29,7 +21,10 @@ function error {
# we force the following to be not in a section
section_end $CURRENT_SECTION
CURR_MINSEC=$(get_current_minsec)
DATE_S=$(date -u +"%s")
JOB_START_S=$(date -u +"%s" -d "${CI_JOB_STARTED_AT:?}")
CURR_TIME=$((DATE_S-JOB_START_S))
CURR_MINSEC="$(printf "%02d" $((CURR_TIME/60))):$(printf "%02d" $((CURR_TIME%60)))"
echo -e "\n${RED}[${CURR_MINSEC}] ERROR: $*${ENDCOLOR}\n"
[ "$state_x" -eq 0 ] || set -x
}
@@ -47,7 +42,10 @@ function build_section_start {
CYAN="\e[0;36m"
ENDCOLOR="\e[0m"
CURR_MINSEC=$(get_current_minsec)
DATE_S=$(date -u +"%s")
JOB_START_S=$(date -u +"%s" -d "${CI_JOB_STARTED_AT:?}")
CURR_TIME=$((DATE_S-JOB_START_S))
CURR_MINSEC="$(printf "%02d" $((CURR_TIME/60))):$(printf "%02d" $((CURR_TIME%60)))"
echo -e "\n\e[0Ksection_start:$(date +%s):$section_name$section_params\r\e[0K${CYAN}[${CURR_MINSEC}] $*${ENDCOLOR}\n"
}
@@ -89,7 +87,6 @@ function uncollapsed_section_switch {
}
export -f x_off
export -f get_current_minsec
export -f error
export -f trap_err
export -f build_section_start

View File

@@ -45,24 +45,21 @@
rules:
- !reference [.no_scheduled_pipelines-rules, rules]
- changes: &core_file_list
- .gitlab-ci.yml
- .gitlab-ci/**/*
- include/**/*
- meson.build
- meson_options.txt
- build-support/**/*
- subprojects/**/*
- .gitattributes
- src/*
- src/compiler/**/*
- src/drm-shim/**/*
- src/gbm/**/*
- src/gtest/**/*
# Some src/util and src/compiler files use headers from mesa/ (e.g.
# mtypes.h). We should clean that up.
- src/mesa/**/*.h
- src/tool/**/*
- src/util/**/*
- .gitlab-ci.yml
- .gitlab-ci/**/*
- include/**/*
- meson.build
- .gitattributes
- src/*
- src/compiler/**/*
- src/drm-shim/**/*
- src/gbm/**/*
- src/gtest/**/*
# Some src/util and src/compiler files use headers from mesa/ (e.g.
# mtypes.h). We should clean that up.
- src/mesa/**/*.h
- src/tool/**/*
- src/util/**/*
when: on_success
# Same core dependencies for doing manual runs.
@@ -140,36 +137,36 @@
rules:
- !reference [.core-rules, rules]
- changes: &gallium_core_file_list
- src/gallium/*
- src/gallium/auxiliary/**/*
- src/gallium/drivers/*
- src/gallium/include/**/*
- src/gallium/frontends/dri/*
- src/gallium/frontends/glx/**/*
- src/gallium/targets/**/*
- src/gallium/tests/**/*
- src/gallium/winsys/*
- src/gallium/*
- src/gallium/auxiliary/**/*
- src/gallium/drivers/*
- src/gallium/include/**/*
- src/gallium/frontends/dri/*
- src/gallium/frontends/glx/**/*
- src/gallium/targets/**/*
- src/gallium/tests/**/*
- src/gallium/winsys/*
when: on_success
.gl-rules:
rules:
- !reference [.core-rules, rules]
- changes: &mesa_core_file_list
- src/egl/**/*
- src/glx/**/*
- src/loader/**/*
- src/mapi/**/*
- src/mesa/*
- src/mesa/main/**/*
- src/mesa/math/**/*
- src/mesa/program/**/*
- src/mesa/sparc/**/*
- src/mesa/state_tracker/**/*
- src/mesa/swrast/**/*
- src/mesa/swrast_setup/**/*
- src/mesa/vbo/**/*
- src/mesa/x86/**/*
- src/mesa/x86-64/**/*
- src/egl/**/*
- src/glx/**/*
- src/loader/**/*
- src/mapi/**/*
- src/mesa/*
- src/mesa/main/**/*
- src/mesa/math/**/*
- src/mesa/program/**/*
- src/mesa/sparc/**/*
- src/mesa/state_tracker/**/*
- src/mesa/swrast/**/*
- src/mesa/swrast_setup/**/*
- src/mesa/vbo/**/*
- src/mesa/x86/**/*
- src/mesa/x86-64/**/*
when: on_success
- !reference [.gallium-core-rules, rules]
@@ -189,7 +186,7 @@
rules:
- !reference [.core-rules, rules]
- changes: &vulkan_file_list
- src/vulkan/**/*
- src/vulkan/**/*
when: on_success
.vulkan-manual-rules:
@@ -202,8 +199,7 @@
# Rules for unusual architectures that only build a subset of drivers
.ppc64el-rules:
rules:
- !reference [.never-post-merge-rules, rules]
rules:
- !reference [.no_scheduled_pipelines-rules, rules]
- !reference [.zink-common-rules, rules]
- !reference [.softpipe-rules, rules]
@@ -215,8 +211,7 @@
- !reference [.nouveau-rules, rules]
.s390x-rules:
rules:
- !reference [.never-post-merge-rules, rules]
rules:
- !reference [.no_scheduled_pipelines-rules, rules]
- !reference [.zink-common-rules, rules]
- !reference [.softpipe-rules, rules]
@@ -225,29 +220,23 @@
# Rules for linters
.lint-rustfmt-rules:
rules:
- !reference [.never-post-merge-rules, rules]
- !reference [.no_scheduled_pipelines-rules, rules]
rules:
- !reference [.core-rules, rules]
- changes:
- .gitlab-ci.yml
- .gitlab-ci/**/*
# in merge pipeline, formatting checks are not allowed to fail
- if: $GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event"
changes: &rust_file_list
- src/**/*.rs
- src/**/*.rs
when: on_success
allow_failure: false
# in other pipelines, formatting checks are allowed to fail
- changes: *rust_file_list
when: on_success
allow_failure: true
# Rules for .mr-label-maker.yml
.mr-label-maker-rules:
rules:
- !reference [.never-post-merge-rules, rules]
- !reference [.no_scheduled_pipelines-rules, rules]
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
- .mr-label-maker.yml
.lint-clang-format-rules:
rules:
- !reference [.core-rules, rules]
- changes:
- .clang-format
- .clang-format-include
- .clang-format-ignore
- src/**/.clang-format
- src/egl/**/*
- src/**/asahi/**/*
- src/**/panfrost/**/*
- src/amd/vulkan/**/*
- src/amd/compiler/**/*
when: on_success

View File

@@ -16,22 +16,23 @@
name: "mesa_${CI_JOB_NAME}"
paths:
- results/
rules:
- !reference [.never-post-merge-rules, rules]
.formatting-check:
# Cancel job if a newer commit is pushed to the same branch
interruptible: true
stage: code-validation
stage: lint
extends:
- .use-debian/x86_64_build
rules:
# in merge pipeline, don't touch the default settings
- if: $GITLAB_USER_LOGIN == "marge-bot" && $CI_COMMIT_BRANCH == null
# in other pipelines, formatting checks are allowed to fail
- allow_failure: true
variables:
GIT_STRATEGY: fetch
timeout: 10m
script:
- git diff --color=always --exit-code # Fails if there are diffs
tags:
- placeholder-job
rustfmt:
extends:
@@ -41,36 +42,20 @@ rustfmt:
- shopt -s globstar
- rustfmt --version
- rustfmt --verbose src/**/lib.rs
- rustfmt --verbose src/**/main.rs
python-test:
# Cancel job if a newer commit is pushed to the same branch
interruptible: true
stage: code-validation
clang-format:
extends:
- .use-debian/x86_64_build
- .formatting-check
- .lint-clang-format-rules
variables:
GIT_STRATEGY: fetch
timeout: 10m
script:
- cd bin/ci
- pip install --break-system-packages -r test/requirements.txt
- PYTHONPATH=. pytest -v
rules:
- !reference [.disable-farm-mr-rules, rules]
- !reference [.never-post-merge-rules, rules]
- if: $CI_PIPELINE_SOURCE == "schedule"
when: on_success
- if: $CI_PIPELINE_SOURCE == "push" && $CI_PROJECT_NAMESPACE == "mesa" && $GITLAB_USER_LOGIN != "marge-bot"
when: on_success
- if: $GITLAB_USER_LOGIN == "marge-bot"
changes: &bin_ci_files
- .gitlab-ci.yml
- .gitlab-ci/**/*
- bin/ci/**/*
when: on_success
- changes: *bin_ci_files
when: manual
LLVM_VERSION: 15
before_script:
- shopt -s globstar
# We need a meson build dir, but its config doesn't actually matter, so
# let's just use the default.
- meson setup build
- clang-format-${LLVM_VERSION} --version
- ninja -C build clang-format
.test-gl:
extends:
@@ -158,7 +143,7 @@ python-test:
exclude:
- results/*.shader_cache
variables:
PIGLIT_REPLAY_EXTRA_ARGS: --db-path ${CI_PROJECT_DIR}/replayer-db/ --minio_bucket=${S3_TRACIE_PUBLIC_BUCKET} --jwt-file=${S3_JWT_FILE}
PIGLIT_REPLAY_EXTRA_ARGS: --db-path ${CI_PROJECT_DIR}/replayer-db/ --minio_bucket=mesa-tracie-public --jwt-file=${CI_JOB_JWT_FILE}
# until we overcome Infrastructure issues, give traces extra 5 min before timeout
DEVICE_HANGING_TIMEOUT_SEC: 600
script:
@@ -177,6 +162,12 @@ python-test:
reports:
junit: results/junit.xml
.deqp-test-vk:
extends:
- .deqp-test
variables:
DEQP_VER: vk
.fossilize-test:
script:
- ./install/fossilize-runner.sh
@@ -186,7 +177,11 @@ python-test:
paths:
- results/
.download_s3:
.baremetal-test:
extends:
- .test
# Cancel job if a newer commit is pushed to the same branch
interruptible: true
before_script:
- !reference [default, before_script]
# Use this instead of gitlab's artifacts download because it hits packet.net
@@ -198,14 +193,6 @@ python-test:
- rm -rf install
- (set -x; curl -L --retry 4 -f --retry-all-errors --retry-delay 60 ${FDO_HTTP_CACHE_URI:-}https://${PIPELINE_ARTIFACTS_BASE}/${S3_ARTIFACT_NAME}.tar.zst | tar --zstd -x)
- section_end artifacts_download
.baremetal-test:
extends:
- .test
# Cancel job if a newer commit is pushed to the same branch
interruptible: true
before_script:
- !reference [.download_s3, before_script]
variables:
BM_ROOTFS: /rootfs-${DEBIAN_ARCH}
artifacts:
@@ -291,11 +278,8 @@ python-test:
# like FDO_DISTRIBUTION_TAG for *the* image, there is no way to
# depend on more than one image per job. So, the job container is
# built as part of the CI in the boot2container project.
image: registry.freedesktop.org/gfx-ci/ci-tron/mesa-trigger:2024-01-05.1
image: registry.freedesktop.org/gfx-ci/ci-tron/mesa-trigger:2023-06-02.1
timeout: 1h 40m
rules:
- if: $FORCE_KERNEL_TAG != null
when: never
variables:
# No need by default to pull the whole repo
GIT_STRATEGY: none
@@ -335,7 +319,7 @@ python-test:
- !reference [default, before_script]
- |
set -eux
set -x
# Useful as a hook point for runner admins. You may edit the
# config.toml for the Gitlab runner and use a bind-mount to
@@ -356,17 +340,44 @@ python-test:
[ -d "$CI_B2C_ARTIFACTS" ] || exit 1
[ -d "$CI_COMMON_SCRIPTS" ] || exit 1
export B2C_CONTAINER_CMD="bash -euc 'tar xf ${INSTALL_TARBALL_NAME}; ./install/common/init-stage2.sh'"
B2C_TEST_SCRIPT="bash -euc 'tar xf ${INSTALL_TARBALL_NAME}; ./install/common/init-stage2.sh'"
# The Valve CI gateway receives jobs in a YAML format. Create a
# job description from the CI environment.
python3 "$CI_B2C_ARTIFACTS"/generate_b2c.py
python3 "$CI_B2C_ARTIFACTS"/generate_b2c.py \
--ci-job-id "${CI_JOB_ID}" \
--container-cmd "${B2C_TEST_SCRIPT}" \
--initramfs-url "${B2C_INITRAMFS_URL}" \
--job-success-regex "${B2C_JOB_SUCCESS_REGEX}" \
--job-warn-regex "${B2C_JOB_WARN_REGEX}" \
--kernel-url "${B2C_KERNEL_URL}" \
--log-level "${B2C_LOG_LEVEL}" \
--poweroff-delay "${B2C_POWEROFF_DELAY}" \
--session-end-regex "${B2C_SESSION_END_REGEX}" \
--session-reboot-regex "${B2C_SESSION_REBOOT_REGEX}" \
--tags "${CI_RUNNER_TAGS}" \
--template "${B2C_JOB_TEMPLATE}" \
--timeout-boot-minutes "${B2C_TIMEOUT_BOOT_MINUTES}" \
--timeout-boot-retries "${B2C_TIMEOUT_BOOT_RETRIES}" \
--timeout-first-minutes "${B2C_TIMEOUT_FIRST_MINUTES}" \
--timeout-first-retries "${B2C_TIMEOUT_FIRST_RETRIES}" \
--timeout-minutes "${B2C_TIMEOUT_MINUTES}" \
--timeout-overall-minutes "${B2C_TIMEOUT_OVERALL_MINUTES}" \
--timeout-retries "${B2C_TIMEOUT_RETRIES}" \
--job-volume-exclusions "${B2C_JOB_VOLUME_EXCLUSIONS}" \
--local-container "${IMAGE_UNDER_TEST}" \
${B2C_EXTRA_VOLUME_ARGS} \
--working-dir "$CI_PROJECT_DIR"
cat b2c.yml.jinja2
rm -rf ${JOB_FOLDER} || true
mkdir -v ${JOB_FOLDER}
# Keep the results path the same as baremetal and LAVA
ln -s "$JOB_FOLDER"/results/ .
# Create a script to regenerate the CI environment when this job
# begins running on the remote DUT.
set +x
@@ -374,6 +385,7 @@ python-test:
echo "export SCRIPTS_DIR=./install" >> ${JOB_FOLDER}/set-job-env-vars.sh
echo "Variables passed through:"
cat ${JOB_FOLDER}/set-job-env-vars.sh
echo "export CI_JOB_JWT=${CI_JOB_JWT}" >> ${JOB_FOLDER}/set-job-env-vars.sh
set -x
# Copy the mesa install tarball to the job folder, for later extraction
@@ -389,16 +401,10 @@ python-test:
env PYTHONUNBUFFERED=1 executorctl \
run -w b2c.yml.jinja2 -j $(slugify "$CI_JOB_NAME") -s ${JOB_FOLDER} -i "$CI_RUNNER_DESCRIPTION"
# Anything our job places in results/ will be collected by the
# Gitlab coordinator for status presentation. results/junit.xml
# will be parsed by the UI for more detailed explanations of
# test execution.
after_script:
# Keep the results path the same as baremetal and LAVA
- mkdir -p "${JOB_FOLDER}"/results
- mv "${JOB_FOLDER}"/results results/
- !reference [default, after_script]
# Anything our job places in results/ will be collected by the
# Gitlab coordinator for status presentation. results/junit.xml
# will be parsed by the UI for more detailed explanations of
# test execution.
artifacts:
when: always
name: "mesa_${CI_JOB_NAME}"
@@ -407,7 +413,7 @@ python-test:
reports:
junit: results/**/junit.xml
.b2c-x86_64-test-vk:
.b2c-test-vk:
extends:
- .use-debian/x86_64_test-vk
- .b2c-test
@@ -416,7 +422,7 @@ python-test:
- debian-testing
- !reference [.required-for-hardware-jobs, needs]
.b2c-x86_64-test-gl:
.b2c-test-gl:
extends:
- .use-debian/x86_64_test-gl
- .b2c-test

View File

@@ -1,142 +0,0 @@
job_name: 'test-project: my_pipeline_info'
device_type: my_fastboot_device_type
visibility:
group:
- my_visibility_group
priority: 75
context:
extra_nfsroot_args: ' init=/init rootwait usbcore.quirks=0bda:8153:k'
timeouts:
job:
minutes: 10
actions:
depthcharge-retry:
minutes: 4
depthcharge-start:
minutes: 1
depthcharge-action:
minutes: 15
actions:
- deploy:
timeout:
minutes: 10
to: nfs
nfsrootfs:
url: None/lava-rootfs.tar.zst
compression: zstd
namespace: dut
- deploy:
timeout:
minutes: 5
to: downloads
os: oe
images:
kernel:
url: None/None
dtb:
url: None/my_dtb_filename.dtb
postprocess:
docker:
image: registry.gitlab.collabora.com/lava/health-check-docker
steps:
- cat Image.gz my_dtb_filename.dtb > Image.gz+dtb
- mkbootimg --kernel Image.gz+dtb --cmdline "root=/dev/nfs rw nfsroot=$NFS_SERVER_IP:$NFS_ROOTFS,tcp,hard
rootwait ip=dhcp init=/init" --pagesize 4096 --base 0x80000000 -o boot.img
namespace: dut
- deploy:
timeout:
minutes: 10
to: fastboot
docker:
image: registry.gitlab.collabora.com/lava/health-check-docker
images:
boot:
url: downloads://boot.img
namespace: dut
failure_retry: 3
- boot:
timeout:
minutes: 2
docker:
image: registry.gitlab.collabora.com/lava/health-check-docker
failure_retry: 3
method: fastboot
prompts:
- 'lava-shell:'
commands:
- set_active a
namespace: dut
auto_login:
login_commands:
- dropbear -R -B
- touch /dut_ready
login_prompt: 'ogin:'
username: ''
- test:
namespace: dut
definitions:
- from: inline
name: setup-ssh-server
path: inline-setup-ssh-server
repository:
metadata:
format: Lava-Test Test Definition 1.0
name: dut-env-export
run:
steps:
- |-
echo test FASTBOOT
- export -p > /dut-env-vars.sh
- test:
namespace: container
timeout:
minutes: 10
failure_retry: 3
definitions:
- name: docker_ssh_client
from: inline
path: inline/docker_ssh_client.yaml
repository:
metadata:
name: mesa
description: Mesa test plan
format: Lava-Test Test Definition 1.0
run:
steps:
- |-
set -ex
timeout 1m bash << EOF
while [ -z "$(lava-target-ip)" ]; do
echo Waiting for DUT to join LAN;
sleep 1;
done
EOF
ping -c 5 -w 60 $(lava-target-ip)
lava_ssh_test_case() {
set -x
local test_case="${1}"
shift
lava-test-case "${test_case}" --shell \
ssh ${SSH_PTY_ARGS:--T} \
-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
root@$(lava-target-ip) "${@}"
}
- lava_ssh_test_case 'wait_for_dut_login' << EOF
- while [ ! -e /dut_ready ]; do sleep 1; done;
- EOF
- |-
lava_ssh_test_case 'artifact_download' 'bash --' << EOF
source /dut-env-vars.sh
set -ex
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 None | tar -xz -C /
mkdir -p /ci/project/dir
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 None | tar --zstd -x -C /ci/project/dir
echo Could not find jwt file, disabling S3 requests...
sed -i '/S3_RESULTS_UPLOAD/d' /set-job-env-vars.sh
EOF
- export SSH_PTY_ARGS=-tt
- lava_ssh_test_case 'test-project_dut' '"cd / && /init-stage2.sh"'
docker:
image:

View File

@@ -1,96 +0,0 @@
job_name: 'test-project: my_pipeline_info'
device_type: my_fastboot_device_type
visibility:
group:
- my_visibility_group
priority: 75
context:
extra_nfsroot_args: ' init=/init rootwait usbcore.quirks=0bda:8153:k'
timeouts:
job:
minutes: 10
actions:
depthcharge-retry:
minutes: 4
depthcharge-start:
minutes: 1
depthcharge-action:
minutes: 15
actions:
- deploy:
timeout:
minutes: 10
to: nfs
nfsrootfs:
url: None/lava-rootfs.tar.zst
compression: zstd
- deploy:
timeout:
minutes: 5
to: downloads
os: oe
images:
kernel:
url: None/None
dtb:
url: None/my_dtb_filename.dtb
postprocess:
docker:
image: registry.gitlab.collabora.com/lava/health-check-docker
steps:
- cat Image.gz my_dtb_filename.dtb > Image.gz+dtb
- mkbootimg --kernel Image.gz+dtb --cmdline "root=/dev/nfs rw nfsroot=$NFS_SERVER_IP:$NFS_ROOTFS,tcp,hard
rootwait ip=dhcp init=/init" --pagesize 4096 --base 0x80000000 -o boot.img
- deploy:
timeout:
minutes: 2
to: fastboot
docker:
image: registry.gitlab.collabora.com/lava/health-check-docker
images:
boot:
url: downloads://boot.img
- boot:
timeout:
minutes: 2
docker:
image: registry.gitlab.collabora.com/lava/health-check-docker
failure_retry: 3
method: fastboot
prompts:
- 'lava-shell:'
commands:
- set_active a
- test:
timeout:
minutes: 10
failure_retry: 1
definitions:
- name: mesa
from: inline
lava-signal: kmsg
path: inline/mesa.yaml
repository:
metadata:
name: mesa
description: Mesa test plan
os:
- oe
scope:
- functional
format: Lava-Test Test Definition 1.0
run:
steps:
- echo test FASTBOOT
- set -ex
- curl -L --retry 4 -f --retry-all-errors --retry-delay 60 None | tar -xz
-C /
- mkdir -p /ci/project/dir
- curl -L --retry 4 -f --retry-all-errors --retry-delay 60 None | tar --zstd
-x -C /ci/project/dir
- echo Could not find jwt file, disabling S3 requests...
- sed -i '/S3_RESULTS_UPLOAD/d' /set-job-env-vars.sh
- mkdir -p /ci/project/dir
- curl None | tar --zstd -x -C /ci/project/dir
- sleep 1
- lava-test-case 'test-project_dut' --shell /init-stage2.sh

View File

@@ -1,114 +0,0 @@
job_name: 'test-project: my_pipeline_info'
device_type: my_uboot_device_type
visibility:
group:
- my_visibility_group
priority: 75
context:
extra_nfsroot_args: ' init=/init rootwait usbcore.quirks=0bda:8153:k'
timeouts:
job:
minutes: 10
actions:
depthcharge-retry:
minutes: 4
depthcharge-start:
minutes: 1
depthcharge-action:
minutes: 15
actions:
- deploy:
timeout:
minutes: 10
to: tftp
os: oe
kernel:
url: None/None
nfsrootfs:
url: None/lava-rootfs.tar.zst
compression: zstd
dtb:
url: None/my_dtb_filename.dtb
namespace: dut
failure_retry: 3
- boot:
failure_retry: 3
method: u-boot
prompts:
- 'lava-shell:'
commands: nfs
namespace: dut
auto_login:
login_commands:
- dropbear -R -B
- touch /dut_ready
login_prompt: 'ogin:'
username: ''
- test:
namespace: dut
definitions:
- from: inline
name: setup-ssh-server
path: inline-setup-ssh-server
repository:
metadata:
format: Lava-Test Test Definition 1.0
name: dut-env-export
run:
steps:
- |-
echo test UBOOT
- export -p > /dut-env-vars.sh
- test:
namespace: container
timeout:
minutes: 10
failure_retry: 3
definitions:
- name: docker_ssh_client
from: inline
path: inline/docker_ssh_client.yaml
repository:
metadata:
name: mesa
description: Mesa test plan
format: Lava-Test Test Definition 1.0
run:
steps:
- |-
set -ex
timeout 1m bash << EOF
while [ -z "$(lava-target-ip)" ]; do
echo Waiting for DUT to join LAN;
sleep 1;
done
EOF
ping -c 5 -w 60 $(lava-target-ip)
lava_ssh_test_case() {
set -x
local test_case="${1}"
shift
lava-test-case "${test_case}" --shell \
ssh ${SSH_PTY_ARGS:--T} \
-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
root@$(lava-target-ip) "${@}"
}
- lava_ssh_test_case 'wait_for_dut_login' << EOF
- while [ ! -e /dut_ready ]; do sleep 1; done;
- EOF
- |-
lava_ssh_test_case 'artifact_download' 'bash --' << EOF
source /dut-env-vars.sh
set -ex
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 None | tar -xz -C /
mkdir -p /ci/project/dir
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 None | tar --zstd -x -C /ci/project/dir
echo Could not find jwt file, disabling S3 requests...
sed -i '/S3_RESULTS_UPLOAD/d' /set-job-env-vars.sh
EOF
- export SSH_PTY_ARGS=-tt
- lava_ssh_test_case 'test-project_dut' '"cd / && /init-stage2.sh"'
docker:
image:

View File

@@ -1,70 +0,0 @@
job_name: 'test-project: my_pipeline_info'
device_type: my_uboot_device_type
visibility:
group:
- my_visibility_group
priority: 75
context:
extra_nfsroot_args: ' init=/init rootwait usbcore.quirks=0bda:8153:k'
timeouts:
job:
minutes: 10
actions:
depthcharge-retry:
minutes: 4
depthcharge-start:
minutes: 1
depthcharge-action:
minutes: 15
actions:
- deploy:
timeout:
minutes: 5
to: tftp
os: oe
kernel:
url: None/None
nfsrootfs:
url: None/lava-rootfs.tar.zst
compression: zstd
dtb:
url: None/my_dtb_filename.dtb
- boot:
failure_retry: 3
method: u-boot
prompts:
- 'lava-shell:'
commands: nfs
- test:
timeout:
minutes: 10
failure_retry: 1
definitions:
- name: mesa
from: inline
lava-signal: kmsg
path: inline/mesa.yaml
repository:
metadata:
name: mesa
description: Mesa test plan
os:
- oe
scope:
- functional
format: Lava-Test Test Definition 1.0
run:
steps:
- echo test UBOOT
- set -ex
- curl -L --retry 4 -f --retry-all-errors --retry-delay 60 None | tar -xz
-C /
- mkdir -p /ci/project/dir
- curl -L --retry 4 -f --retry-all-errors --retry-delay 60 None | tar --zstd
-x -C /ci/project/dir
- echo Could not find jwt file, disabling S3 requests...
- sed -i '/S3_RESULTS_UPLOAD/d' /set-job-env-vars.sh
- mkdir -p /ci/project/dir
- curl None | tar --zstd -x -C /ci/project/dir
- sleep 1
- lava-test-case 'test-project_dut' --shell /init-stage2.sh

View File

@@ -9,13 +9,12 @@ import os
import xmlrpc.client
from contextlib import nullcontext as does_not_raise
from datetime import datetime
from itertools import islice, repeat
from itertools import chain, repeat
from pathlib import Path
from typing import Generator
from unittest.mock import MagicMock, patch
import pytest
from lava.exceptions import MesaCIException, MesaCIRetryError, MesaCIFatalException
from lava.exceptions import MesaCIException, MesaCIRetryError
from lava.lava_job_submitter import (
DEVICE_HANGING_TIMEOUT_SEC,
NUMBER_OF_RETRIES_TIMEOUT_DETECTION,
@@ -24,7 +23,6 @@ from lava.lava_job_submitter import (
bootstrap_log_follower,
follow_job_execution,
retriable_follow_job,
wait_for_job_get_started,
)
from lava.utils import LogSectionType
@@ -84,7 +82,7 @@ def lava_job_submitter(
def test_submit_and_follow_respects_exceptions(mock_sleep, mock_proxy, exception):
with pytest.raises(MesaCIException):
proxy = mock_proxy(side_effect=exception)
job = LAVAJob(proxy, "")
job = LAVAJob(proxy, '')
log_follower = bootstrap_log_follower()
follow_job_execution(job, log_follower)
@@ -166,13 +164,21 @@ PROXY_SCENARIOS = {
mock_logs(result="pass"),
does_not_raise(),
"pass",
{"testsuite_results": [generate_testsuite_result(result="pass")]},
{
"testsuite_results": [
generate_testsuite_result(result="pass")
]
},
),
"no retries, but testsuite fails": (
mock_logs(result="fail"),
does_not_raise(),
"fail",
{"testsuite_results": [generate_testsuite_result(result="fail")]},
{
"testsuite_results": [
generate_testsuite_result(result="fail")
]
},
),
"no retries, one testsuite fails": (
generate_n_logs(n=1, tick_fn=0, result="fail"),
@@ -181,7 +187,7 @@ PROXY_SCENARIOS = {
{
"testsuite_results": [
generate_testsuite_result(result="fail"),
generate_testsuite_result(result="pass"),
generate_testsuite_result(result="pass")
]
},
),
@@ -258,27 +264,6 @@ def test_simulate_a_long_wait_to_start_a_job(
assert delta_time.total_seconds() >= wait_time
LONG_LAVA_QUEUE_SCENARIOS = {
"no_time_to_run": (0, pytest.raises(MesaCIFatalException)),
"enough_time_to_run": (9999999999, does_not_raise()),
}
@pytest.mark.parametrize(
"job_timeout, expectation",
LONG_LAVA_QUEUE_SCENARIOS.values(),
ids=LONG_LAVA_QUEUE_SCENARIOS.keys(),
)
def test_wait_for_job_get_started_no_time_to_run(monkeypatch, job_timeout, expectation):
monkeypatch.setattr("lava.lava_job_submitter.CI_JOB_TIMEOUT_SEC", job_timeout)
job = MagicMock()
# Make it escape the loop
job.is_started.side_effect = (False, False, True)
with expectation as e:
wait_for_job_get_started(job, 1)
if e:
job.cancel.assert_called_with()
CORRUPTED_LOG_SCENARIOS = {
"too much subsequent corrupted data": (
@@ -356,15 +341,13 @@ def test_full_yaml_log(mock_proxy, frozen_time, lava_job_submitter):
if not data_chunk:
return
first_log = lava_yaml.load(data_chunk[0])[0]
first_log_time = first_log["dt"]
first_log_time = data_chunk[0]["dt"]
frozen_time.move_to(first_log_time)
yield
last_log = lava_yaml.load(data_chunk[-1])[0]
last_log_time = last_log["dt"]
last_log_time = data_chunk[-1]["dt"]
frozen_time.move_to(last_log_time)
yield
return
def time_travel_to_test_time():
# Suppose that the first message timestamp of the entire LAVA job log is
@@ -374,31 +357,22 @@ def test_full_yaml_log(mock_proxy, frozen_time, lava_job_submitter):
first_log_time = lava_yaml.load(first_log)[0]["dt"]
frozen_time.move_to(first_log_time)
def load_lines() -> Generator[tuple[bool, str], None, None]:
def load_lines() -> list:
with open("/tmp/log.yaml", "r") as f:
# data = yaml.safe_load(f)
log_lines = f.readlines()
serial_message: str = ""
chunk_start_line = 0
chunk_end_line = 0
chunk_max_size = 100
data = f.readlines()
stream = chain(data)
try:
while True:
chunk_end_line = chunk_start_line + random.randint(1, chunk_max_size)
# split the log in chunks of random size
log_chunk = list(islice(log_lines, chunk_start_line, chunk_end_line))
chunk_start_line = chunk_end_line + 1
serial_message = "".join(log_chunk)
# time_traveller_gen will make the time trave according to the timestamp from
# the message
time_traveller_gen = time_travel_from_log_chunk(log_chunk)
data_chunk = [next(stream) for _ in range(random.randint(0, 50))]
serial_message = "".join(data_chunk)
# Suppose that the first message timestamp is the same of
# log fetch RPC call
next(time_traveller_gen)
time_travel_from_log_chunk(data_chunk)
yield False, "[]"
# Travel to the same datetime of the last fetched log line
# in the chunk
next(time_traveller_gen)
time_travel_from_log_chunk(data_chunk)
yield False, serial_message
except StopIteration:
yield True, serial_message
@@ -410,20 +384,11 @@ def test_full_yaml_log(mock_proxy, frozen_time, lava_job_submitter):
proxy.scheduler.jobs.logs.side_effect = load_lines()
proxy.scheduler.jobs.submit = reset_logs
try:
with pytest.raises(MesaCIRetryError):
time_travel_to_test_time()
start_time = datetime.now()
lava_job_submitter.submit()
retriable_follow_job(proxy, "")
finally:
try:
# If the job fails, maybe there will be no structured log
print(lava_job_submitter.structured_log_file.read_text())
finally:
end_time = datetime.now()
print("---- Reproduction log stats ----")
print(f"Start time: {start_time}")
print(f"End time: {end_time}")
print(f"Duration: {end_time - start_time}")
print(lava_job_submitter.structured_log_file.read_text())
@pytest.mark.parametrize(
@@ -452,7 +417,9 @@ def test_job_combined_status(
"lava.lava_job_submitter.retriable_follow_job"
) as mock_retriable_follow_job, patch(
"lava.lava_job_submitter.LAVAJobSubmitter._LAVAJobSubmitter__prepare_submission"
) as mock_prepare_submission, patch("sys.exit"):
) as mock_prepare_submission, patch(
"sys.exit"
):
from lava.lava_job_submitter import STRUCTURAL_LOG
mock_retriable_follow_job.return_value = MagicMock(status=finished_job_status)

View File

@@ -1,197 +0,0 @@
import importlib
import os
import re
from itertools import chain
from pathlib import Path
from typing import Any, Iterable, Literal
from unittest import mock
import lava.utils.constants
import pytest
from lava.lava_job_submitter import LAVAJobSubmitter
from lava.utils.lava_job_definition import LAVAJobDefinition
from ruamel.yaml import YAML
def flatten(iterable: Iterable[Iterable[Any]]) -> list[Any]:
return list(chain.from_iterable(iterable))
# mock shell file
@pytest.fixture(scope="session")
def shell_file(tmp_path_factory):
def create_shell_file(content: str = "# test"):
shell_file = tmp_path_factory.mktemp("data") / "shell_file.sh"
shell_file.write_text(content)
return shell_file
return create_shell_file
# fn to load the data file from $CWD/data using pathlib
def load_data_file(filename):
return Path(__file__).parent.parent / "data" / filename
def load_yaml_file(filename) -> dict:
with open(load_data_file(filename)) as f:
return YAML().load(f)
def job_submitter_factory(mode: Literal["UBOOT", "FASTBOOT"], shell_file):
if mode == "UBOOT":
boot_method = "u-boot"
device_type = "my_uboot_device_type"
elif mode == "FASTBOOT":
boot_method = "fastboot"
device_type = "my_fastboot_device_type"
job_timeout_min = 10
mesa_job_name = "dut test"
pipeline_info = "my_pipeline_info"
project_name = "test-project"
visibility_group = "my_visibility_group"
return LAVAJobSubmitter(
boot_method=boot_method,
ci_project_dir="/ci/project/dir",
device_type=device_type,
dtb_filename="my_dtb_filename",
first_stage_init=shell_file,
job_timeout_min=job_timeout_min,
mesa_job_name=mesa_job_name,
pipeline_info=pipeline_info,
visibility_group=visibility_group,
project_name=project_name,
)
@pytest.fixture
def clear_env_vars(autouse=True):
with mock.patch.dict(os.environ) as environ:
# Remove all LAVA-related environment variables to make the test more robust
# and deterministic, once a envvar is capable of overriding the default value
for key in environ:
if any(kw in key for kw in ("LAVA_", "CI_", "JOB_", "RUNNER_", "DEVICE_")):
del environ[key]
# reload lava.utils.constants to update the JOB_PRIORITY value
importlib.reload(lava.utils.constants)
importlib.reload(lava.utils.lava_job_definition)
yield
@pytest.fixture
def mock_collabora_farm(clear_env_vars, monkeypatch):
# Mock a Collabora farm-like device runner tag to enable SSH execution
monkeypatch.setenv("RUNNER_TAG", "mesa-ci-1234-lava-collabora")
@pytest.mark.parametrize("force_uart", [True, False], ids=["SSH", "UART"])
@pytest.mark.parametrize("mode", ["UBOOT", "FASTBOOT"])
def test_generate_lava_job_definition_sanity(
force_uart, mode, shell_file, mock_collabora_farm, monkeypatch
):
monkeypatch.setattr(lava.utils.lava_job_definition, "FORCE_UART", force_uart)
init_script_content = f"echo test {mode}"
job_submitter = job_submitter_factory(mode, shell_file(init_script_content))
job_definition = LAVAJobDefinition(job_submitter).generate_lava_job_definition()
# Load the YAML output and check that it contains the expected keys and values
yaml = YAML()
job_dict = yaml.load(job_definition)
yaml.dump(job_dict, Path(f"/tmp/{mode}_force_uart={force_uart}_job_definition.yaml"))
assert job_dict["device_type"] == job_submitter.device_type
assert job_dict["visibility"]["group"] == [job_submitter.visibility_group]
assert job_dict["timeouts"]["job"]["minutes"] == job_submitter.job_timeout_min
assert job_dict["context"]["extra_nfsroot_args"]
assert job_dict["timeouts"]["actions"]
assert len(job_dict["actions"]) == 3 if mode == "UART" else 5
last_test_action = job_dict["actions"][-1]["test"]
# TODO: Remove hardcoded "mesa" test name, as this submitter is being used by other projects
first_test_name = last_test_action["definitions"][0]["name"]
is_running_ssh = "ssh" in first_test_name
# if force_uart, is_ssh must be False. If is_ssh, force_uart must be False. Both can be False
assert not (is_running_ssh and force_uart)
assert last_test_action["failure_retry"] == 3 if is_running_ssh else 1
run_steps = "".join(last_test_action["definitions"][0]["repository"]["run"]["steps"])
# Check for project name in lava-test-case
assert re.search(rf"lava.?\S*.test.case.*{job_submitter.project_name}", run_steps)
action_names = flatten(j.keys() for j in job_dict["actions"])
if is_running_ssh:
assert action_names == (
[
"deploy",
"boot",
"test", # DUT: SSH server
"test", # Docker: SSH client
]
if mode == "UBOOT"
else [
"deploy", # NFS
"deploy", # Image generation
"deploy", # Image deployment
"boot",
"test", # DUT: SSH server
"test", # Docker: SSH client
]
)
test_action_server = job_dict["actions"][-2]["test"]
# SSH server in the DUT
assert test_action_server["namespace"] == "dut"
# SSH client via docker
assert last_test_action["namespace"] == "container"
boot_action = next(a["boot"] for a in job_dict["actions"] if "boot" in a)
assert boot_action["namespace"] == "dut"
# SSH server bootstrapping
assert "dropbear" in "".join(boot_action["auto_login"]["login_commands"])
return
# ---- Not SSH job
assert action_names == (
[
"deploy",
"boot",
"test",
]
if mode == "UBOOT"
else [
"deploy", # NFS
"deploy", # Image generation
"deploy", # Image deployment
"boot",
"test",
]
)
assert init_script_content in run_steps
# use yaml files from tests/data/ to test the job definition generation
@pytest.mark.parametrize("force_uart", [False, True], ids=["SSH", "UART"])
@pytest.mark.parametrize("mode", ["UBOOT", "FASTBOOT"])
def test_lava_job_definition(mode, force_uart, shell_file, mock_collabora_farm, monkeypatch):
monkeypatch.setattr(lava.utils.lava_job_definition, "FORCE_UART", force_uart)
yaml = YAML()
yaml.default_flow_style = False
# Load the YAML output and check that it contains the expected keys and values
expected_job_dict = load_yaml_file(f"{mode}_force_uart={force_uart}_job_definition.yaml")
init_script_content = f"echo test {mode}"
job_submitter = job_submitter_factory(mode, shell_file(init_script_content))
job_definition = LAVAJobDefinition(job_submitter).generate_lava_job_definition()
job_dict = yaml.load(job_definition)
# Uncomment the following to update the expected YAML files
# yaml.dump(job_dict, Path(f"../../data/{mode}_force_uart={force_uart}_job_definition.yaml"))
# Check that the generated job definition matches the expected one
assert job_dict == expected_job_dict

View File

@@ -16,15 +16,8 @@ from lava.utils import (
fix_lava_gitlab_section_log,
hide_sensitive_data,
)
from lava.utils.constants import KNOWN_ISSUE_R8152_MAX_CONSECUTIVE_COUNTER
from ..lava.helpers import (
create_lava_yaml_msg,
does_not_raise,
lava_yaml,
mock_lava_signal,
yaml_dump,
)
from ..lava.helpers import create_lava_yaml_msg, does_not_raise, lava_yaml, yaml_dump
GITLAB_SECTION_SCENARIOS = {
"start collapsed": (
@@ -318,56 +311,47 @@ def test_gitlab_section_id(case_name, expected_id):
assert gl.id == expected_id
def a618_network_issue_logs(level: str = "target") -> list:
net_error = create_lava_yaml_msg(
msg="[ 1733.599402] r8152 2-1.3:1.0 eth0: Tx status -71", lvl=level)
nfs_error = create_lava_yaml_msg(
msg="[ 1733.604506] nfs: server 192.168.201.1 not responding, still trying",
lvl=level,
)
return [
*(KNOWN_ISSUE_R8152_MAX_CONSECUTIVE_COUNTER*[net_error]),
nfs_error
]
TEST_PHASE_LAVA_SIGNAL = mock_lava_signal(LogSectionType.TEST_CASE)
A618_NET_ISSUE_BOOT = a618_network_issue_logs(level="feedback")
A618_NET_ISSUE_TEST = [TEST_PHASE_LAVA_SIGNAL, *a618_network_issue_logs(level="target")]
A618_NETWORK_ISSUE_LOGS = [
create_lava_yaml_msg(
msg="[ 1733.599402] r8152 2-1.3:1.0 eth0: Tx status -71", lvl="target"
),
create_lava_yaml_msg(
msg="[ 1733.604506] nfs: server 192.168.201.1 not responding, still trying",
lvl="target",
),
]
TEST_PHASE_LAVA_SIGNAL = create_lava_yaml_msg(
msg="Received signal: <STARTTC> mesa-ci_a618_vk", lvl="debug"
)
A618_NETWORK_ISSUE_SCENARIOS = {
"Fail - R8152 kmsg during boot phase": (
A618_NET_ISSUE_BOOT,
pytest.raises(MesaCIKnownIssueException),
),
"Pass - R8152 kmsg during boot": (A618_NETWORK_ISSUE_LOGS, does_not_raise()),
"Fail - R8152 kmsg during test phase": (
A618_NET_ISSUE_TEST,
[TEST_PHASE_LAVA_SIGNAL, *A618_NETWORK_ISSUE_LOGS],
pytest.raises(MesaCIKnownIssueException),
),
"Pass - Partial (1) R8152 kmsg during test phase": (
A618_NET_ISSUE_TEST[:1],
[TEST_PHASE_LAVA_SIGNAL, A618_NETWORK_ISSUE_LOGS[0]],
does_not_raise(),
),
"Pass - Partial (2) R8152 kmsg during test phase": (
A618_NET_ISSUE_TEST[:2],
[TEST_PHASE_LAVA_SIGNAL, A618_NETWORK_ISSUE_LOGS[1]],
does_not_raise(),
),
"Pass - Partial (3) subsequent R8152 kmsg during test phase": (
"Pass - Partial subsequent (3) R8152 kmsg during test phase": (
[
TEST_PHASE_LAVA_SIGNAL,
A618_NET_ISSUE_TEST[1],
A618_NET_ISSUE_TEST[1],
A618_NETWORK_ISSUE_LOGS[0],
A618_NETWORK_ISSUE_LOGS[0],
],
does_not_raise(),
),
"Pass - Partial (4) subsequent nfs kmsg during test phase": (
"Pass - Partial subsequent (4) R8152 kmsg during test phase": (
[
TEST_PHASE_LAVA_SIGNAL,
A618_NET_ISSUE_TEST[-1],
A618_NET_ISSUE_TEST[-1],
A618_NETWORK_ISSUE_LOGS[1],
A618_NETWORK_ISSUE_LOGS[1],
],
does_not_raise(),
),
@@ -380,13 +364,6 @@ A618_NETWORK_ISSUE_SCENARIOS = {
ids=A618_NETWORK_ISSUE_SCENARIOS.keys(),
)
def test_detect_failure(messages, expectation):
boot_section = GitlabSection(
id="lava_boot",
header="LAVA boot",
type=LogSectionType.LAVA_BOOT,
start_collapsed=True,
)
boot_section.start()
lf = LogFollower(starting_section=boot_section)
lf = LogFollower()
with expectation:
lf.feed(messages)

Some files were not shown because too many files have changed in this diff Show More