VERSION: bump for 23.0.0-rc2

intel/compiler: add cpp_std=c++17 when building tests
Otherwise build fails: "../src/intel/compiler/brw_private.h:40:4: note: ‘std::variant’ is only available from C++17 onwards" Fixes: 6c194ddd18 ("intel/compiler: Prepare SIMD selection helpers to handle different prog_datas") Signed-off-by: Tapani Pälli <tapani.palli@intel.com> Reviewed-by: José Roberto de Souza <jose.souza@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20725> (cherry picked from commit 53de48f1c4)
2023-01-18 11:04:34 -08:00 · 2023-01-17 10:16:40 -08:00 · 2023-01-17 10:16:40 -08:00 · 2023-01-17 10:16:40 -08:00 · 2023-01-17 10:16:39 -08:00 · 2023-01-17 10:16:39 -08:00
5961 changed files with 385692 additions and 1111128 deletions
--- a/.ci-farms-disabled/.placeholder
+++ b/.ci-farms-disabled/.placeholder
--- a/.ci-farms-disabled/anholt
+++ b/.ci-farms-disabled/anholt
--- a/.ci-farms/.placeholder
+++ b/.ci-farms/.placeholder
--- a/.ci-farms/austriancoder
+++ b/.ci-farms/austriancoder
--- a/.ci-farms/collabora
+++ b/.ci-farms/collabora
--- a/.ci-farms/freedreno
+++ b/.ci-farms/freedreno
--- a/.ci-farms/igalia
+++ b/.ci-farms/igalia
--- a/.ci-farms/lima
+++ b/.ci-farms/lima
--- a/.ci-farms/microsoft
+++ b/.ci-farms/microsoft
--- a/.ci-farms/ondracka
+++ b/.ci-farms/ondracka
--- a/.ci-farms/valve-kws
+++ b/.ci-farms/valve-kws
--- a/.ci-farms/valve-mupuf
+++ b/.ci-farms/valve-mupuf
--- a/.clang-format
+++ b/.clang-format
--- a/.clang-format-ignore
+++ b/.clang-format-ignore
@@ -1,2 +0,0 @@
-# Vendored code
-src/amd/vulkan/radix_sort/*
--- a/.clang-format-include
+++ b/.clang-format-include
@@ -1,8 +0,0 @@
-# The following files are opted into `ninja clang-format` and
-# enforcement in the CI.
-
-src/gallium/drivers/i915
-src/amd/vulkan/**/*
-src/amd/compiler/**/*
-src/egl/**/*
-src/etnaviv/isa/**/*
--- a/.git-blame-ignore-revs
+++ b/.git-blame-ignore-revs
@@ -1,67 +0,0 @@
-# List of commits to ignore when using `git blame`.
-# Enable with:
-#   git config blame.ignoreRevsFile .git-blame-ignore-revs
-#
-# Per git-blame(1):
-#   Ignore revisions listed in the file, one unabbreviated object name
-#   per line, in git-blame. Whitespace and comments beginning with # are
-#   ignored.
-#
-# Please keep these in chronological order :)
-#
-# You can add a new commit with the following command:
-#   git log -1 --pretty=format:'%n# %s%n%H%n' >> .git-blame-ignore-revs $COMMIT
-
-# pvr: Fix clang-format error.
-0ad5b0a74ef73f5fcbe1406ad9d57fe5dc00a5b1
-
-# panfrost: Fix up some formatting for clang-format
-a4705afe63412498d13ded73cba969c66be67907
-
-# asahi: clang-format the world again
-26c51bb8d8a33098b1990425a391f56ffba5728c
-
-# perfetto: Add a .clang-format for the directory.
-da78d5d729b1800136dd713b68492cb339993f4a
-
-# panfrost/winsys: Clang-format
-c90f036516a5376002be6550a917e8bad6a8a3b8
-
-# panfrost: Re-run clang-format
-4ccf174009af6732cbffa5d8ebb4687da7517505
-
-# panvk: Clang-format
-c7bf3b69ebc8f2252dbf724a4de638e6bb2ac402
-
-# pan/mdg: Fix icky formatting
-133af0d6c945d3aaca8989edd15283a2b7dcc6c7
-
-# mapi: clang-format _glapi_add_dispatch()
-30332529663268a6406e910848e906e725e6fda7
-
-# radv: reformat according to its .clang-format
-8b319c6db8bd93603b18bd783eb75225fcfd51b7
-
-# aco: reformat according to its .clang-format
-6b21653ab4d3a67e711fe10e3d403128b6d26eb2
-
-# egl: re-format using clang-format
-2f670d89db038d5a29f6b72732fd7ad63dfaf4c6
-
-# panfrost: clang-format the tree
-0afd691f29683f6e9dde60f79eca094373521806
-
-# aco: Format.
-1e2639026fec7069806449f9ba2a124ce4eb5569
-
-# radv: Format.
-59c501ca353f8ec9d2717c98af2bfa1a1dbf4d75
-
-# pvr: clang-format fixes
-953c04ebd39c52d457301bdd8ac803949001da2d
-
-# freedreno: Re-indent
-2d439343ea1aee146d4ce32800992cd389bd505d
-
-# ir3: Reformat source with clang-format
-177138d8cb0b4f6a42ef0a1f8593e14d79f17c54
--- a/.gitattributes
+++ b/.gitattributes
@@ -4,4 +4,3 @@
 *.png binary
 *.gif binary
 *.ico binary
-*.cl gitlab-language=c
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,68 +1,3 @@
-# Types of CI pipelines:
-# | pipeline name        | context   | description                                                 |
-# |----------------------|-----------|-------------------------------------------------------------|
-# | merge pipeline       | mesa/mesa | pipeline running for an MR; if it passes the MR gets merged |
-# | pre-merge pipeline   | mesa/mesa | same as above, except its status doesn't affect the MR      |
-# | post-merge pipeline  | mesa/mesa | pipeline immediately after merging                          |
-# | fork pipeline        | fork      | pipeline running in a user fork                             |
-# | scheduled pipeline   | mesa/mesa | nightly pipelines, running every morning at 4am UTC         |
-# | direct-push pipeline | mesa/mesa | when commits are pushed directly to mesa/mesa, bypassing Marge and its gating pipeline |
-#
-# Note that the release branches maintained by the release manager fall under
-# the "direct push" category.
-#
-# "context" indicates the permissions that the jobs get; notably, any
-# container created in mesa/mesa gets pushed immediately for everyone to use
-# as soon as the image tag change is merged.
-#
-# Merge pipelines contain all jobs that must pass before the MR can be merged.
-# Pre-merge pipelines contain the exact same jobs as merge pipelines.
-# Post-merge pipelines contain *only* the `pages` job that deploys the new
-# version of the website.
-# Fork pipelines contain everything.
-# Scheduled pipelines only contain the container+build jobs, and some extra
-# test jobs (typically "full" variants of pre-merge jobs that only run 1/X
-# test cases), but not a repeat of the merge pipeline jobs.
-# Direct-push pipelines contain the same jobs as merge pipelines.
-
-workflow:
-  rules:
-    # do not duplicate pipelines on merge pipelines
-    - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"
-      when: never
-    # merge pipeline
-    - if: &is-merge-attempt $GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event"
-      variables:
-        KERNEL_IMAGE_BASE: https://${S3_HOST}/mesa-lava/${KERNEL_REPO}/${KERNEL_TAG}
-        MESA_CI_PERFORMANCE_ENABLED: 1
-        VALVE_INFRA_VANGOGH_JOB_PRIORITY: ""  # Empty tags are ignored by gitlab
-    # post-merge pipeline
-    - if: &is-post-merge $GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "push"
-    # nightly pipeline
-    - if: &is-scheduled-pipeline $CI_PIPELINE_SOURCE == "schedule"
-      variables:
-        KERNEL_IMAGE_BASE: https://${S3_HOST}/mesa-lava/${KERNEL_REPO}/${KERNEL_TAG}
-        JOB_PRIORITY: 50
-        VALVE_INFRA_VANGOGH_JOB_PRIORITY: priority:low
-    # pipeline for direct pushes that bypassed the CI
-    - if: &is-direct-push $CI_PROJECT_NAMESPACE == "mesa" && $CI_PIPELINE_SOURCE == "push" && $GITLAB_USER_LOGIN != "marge-bot"
-      variables:
-        KERNEL_IMAGE_BASE: https://${S3_HOST}/mesa-lava/${KERNEL_REPO}/${KERNEL_TAG}
-        JOB_PRIORITY: 40
-        VALVE_INFRA_VANGOGH_JOB_PRIORITY: priority:low
-    # pre-merge or fork pipeline
-    - if: $FORCE_KERNEL_TAG != null
-      variables:
-        KERNEL_IMAGE_BASE: https://${S3_HOST}/mesa-lava/${KERNEL_REPO}/${FORCE_KERNEL_TAG}
-        JOB_PRIORITY: 50
-        VALVE_INFRA_VANGOGH_JOB_PRIORITY: priority:low
-    - if: $FORCE_KERNEL_TAG == null
-      variables:
-        KERNEL_IMAGE_BASE: https://${S3_HOST}/mesa-lava/${KERNEL_REPO}/${KERNEL_TAG}
-        JOB_PRIORITY: 50
-        VALVE_INFRA_VANGOGH_JOB_PRIORITY: priority:low
-
-
 variables:
  FDO_UPSTREAM_REPO: mesa/mesa
  MESA_TEMPLATES_COMMIT: &ci-templates-commit d5aa3941aa03c2f716595116354fb81eb8012acb
@@ -73,34 +8,32 @@ variables:
          rm download-git-cache.sh
          set +o xtrace
  CI_JOB_JWT_FILE: /minio_jwt
-  S3_HOST: s3.freedesktop.org
+  MINIO_HOST: s3.freedesktop.org
  # per-pipeline artifact storage on MinIO
-  PIPELINE_ARTIFACTS_BASE: ${S3_HOST}/artifacts/${CI_PROJECT_PATH}/${CI_PIPELINE_ID}
+  PIPELINE_ARTIFACTS_BASE: ${MINIO_HOST}/artifacts/${CI_PROJECT_PATH}/${CI_PIPELINE_ID}
  # per-job artifact storage on MinIO
  JOB_ARTIFACTS_BASE: ${PIPELINE_ARTIFACTS_BASE}/${CI_JOB_ID}
  # reference images stored for traces
-  PIGLIT_REPLAY_REFERENCE_IMAGES_BASE: "${S3_HOST}/mesa-tracie-results/$FDO_UPSTREAM_REPO"
-  # For individual CI farm status see .ci-farms folder
-  # Disable farm with   `git mv .ci-farms{,-disabled}/$farm_name`
-  # Re-enable farm with `git mv .ci-farms{-disabled,}/$farm_name`
-  # NEVER MIX FARM MAINTENANCE WITH ANY OTHER CHANGE IN THE SAME MERGE REQUEST!
-  ARTIFACTS_BASE_URL: https://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_NAME}/-/jobs/${CI_JOB_ID}/artifacts
-  # Python scripts for structured logger
-  PYTHONPATH: "$PYTHONPATH:$CI_PROJECT_DIR/install"
+  PIGLIT_REPLAY_REFERENCE_IMAGES_BASE: "${MINIO_HOST}/mesa-tracie-results/$FDO_UPSTREAM_REPO"
+  # Individual CI farm status, set to "offline" to disable jobs
+  # running on a particular CI farm (ie. for outages, etc):
+  FD_FARM: "online"
+  COLLABORA_FARM: "online"
+  MICROSOFT_FARM: "online"
+  LIMA_FARM: "online"
+  IGALIA_FARM: "online"
+  ANHOLT_FARM: "online"
+  VALVE_FARM: "online"
+  AUSTRIANCODER_FARM: "online"  # only etnaviv GPUs

 default:
  before_script:
-    - >
-      export SCRIPTS_DIR=$(mktemp -d) &&
-      curl -L -s --retry 4 -f --retry-all-errors --retry-delay 60 -O --output-dir "${SCRIPTS_DIR}" "${CI_PROJECT_URL}/-/raw/${CI_COMMIT_SHA}/.gitlab-ci/setup-test-env.sh" &&
-      . ${SCRIPTS_DIR}/setup-test-env.sh &&
-      echo -n "${CI_JOB_JWT}" > "${CI_JOB_JWT_FILE}" &&
-      unset CI_JOB_JWT  # Unsetting vulnerable env variables
+    - echo -e "\e[0Ksection_start:$(date +%s):unset_env_vars_section[collapsed=true]\r\e[0KUnsetting vulnerable environment variables"
+    - echo -n "${CI_JOB_JWT}" > "${CI_JOB_JWT_FILE}"
+    - unset CI_JOB_JWT
+    - echo -e "\e[0Ksection_end:$(date +%s):unset_env_vars_section\r\e[0K"

  after_script:
-    # Work around https://gitlab.com/gitlab-org/gitlab/-/issues/20338
-    - find -name '*.log' -exec mv {} {}.txt \;
-
    - >
      set +x

@@ -108,42 +41,15 @@ default:
      export CI_JOB_JWT="$(<${CI_JOB_JWT_FILE})" &&
      rm "${CI_JOB_JWT_FILE}"

-  # Retry when job fails. Failed jobs can be found in the Mesa CI Daily Reports:
-  # https://gitlab.freedesktop.org/mesa/mesa/-/issues/?sort=created_date&state=opened&label_name%5B%5D=CI%20daily
+  # Retry build or test jobs up to twice when the gitlab-runner itself fails somehow.
  retry:
-    max: 1
-    # Ignore runner_unsupported, stale_schedule, archived_failure, or
-    # unmet_prerequisites
+    max: 2
    when:
-      - api_failure
      - runner_system_failure
-      - script_failure
-      - job_execution_timeout
-      - scheduler_failure
-      - data_integrity_failure
-      - unknown_failure
-
-stages:
-  - sanity
-  - container
-  - git-archive
-  - build-x86_64
-  - build-misc
-  - code-validation
-  - amd
-  - intel
-  - nouveau
-  - arm
-  - broadcom
-  - freedreno
-  - etnaviv
-  - software-renderer
-  - layered-backends
-  - deploy

 include:
  - project: 'freedesktop/ci-templates'
-    ref: 16bc29078de5e0a067ff84a1a199a3760d3b3811
+    ref: ffe4d1b10aab7534489f0c4bbc4c5899df17d3f2
    file:
      - '/templates/ci-fairy.yml'
  - project: 'freedesktop/ci-templates'
@@ -157,9 +63,7 @@ include:
  - local: '.gitlab-ci/container/gitlab-ci.yml'
  - local: '.gitlab-ci/build/gitlab-ci.yml'
  - local: '.gitlab-ci/test/gitlab-ci.yml'
-  - local: '.gitlab-ci/farm-rules.yml'
  - local: '.gitlab-ci/test-source-dep.yml'
-  - local: 'docs/gitlab-ci.yml'
  - local: 'src/amd/ci/gitlab-ci.yml'
  - local: 'src/broadcom/ci/gitlab-ci.yml'
  - local: 'src/etnaviv/ci/gitlab-ci.yml'
@@ -167,7 +71,6 @@ include:
  - local: 'src/gallium/drivers/crocus/ci/gitlab-ci.yml'
  - local: 'src/gallium/drivers/d3d12/ci/gitlab-ci.yml'
  - local: 'src/gallium/drivers/i915/ci/gitlab-ci.yml'
-  - local: 'src/gallium/drivers/r300/ci/gitlab-ci.yml'
  - local: 'src/gallium/drivers/lima/ci/gitlab-ci.yml'
  - local: 'src/gallium/drivers/llvmpipe/ci/gitlab-ci.yml'
  - local: 'src/gallium/drivers/nouveau/ci/gitlab-ci.yml'
@@ -180,70 +83,141 @@ include:
  - local: 'src/panfrost/ci/gitlab-ci.yml'
  - local: 'src/virtio/ci/gitlab-ci.yml'

+stages:
+  - sanity
+  - container
+  - git-archive
+  - build-x86_64
+  - build-misc
+  - lint
+  - amd
+  - intel
+  - nouveau
+  - arm
+  - broadcom
+  - freedreno
+  - etnaviv
+  - software-renderer
+  - layered-backends
+  - deploy
+

 # YAML anchors for rule conditions
 # --------------------------------
 .rules-anchors:
-  # Pre-merge pipeline
-  - &is-pre-merge '$CI_PIPELINE_SOURCE == "merge_request_event"'
-
-
-.never-post-merge-rules:
  rules:
-    - if: *is-post-merge
-      when: never
-
-
-.container+build-rules:
-  rules:
-    # Run when re-enabling a disabled farm, but not when disabling it
-    - !reference [.disable-farm-mr-rules, rules]
-    # Never run immediately after merging, as we just ran everything
-    - !reference [.never-post-merge-rules, rules]
-    # Build everything in merge pipelines, if any files affecting the pipeline
-    # were changed
-    - if: *is-merge-attempt
-      changes: &all_paths
-        - VERSION
-        - bin/git_sha1_gen.py
-        - bin/install_megadrivers.py
-        - bin/symbols-check.py
-        # GitLab CI
-        - .gitlab-ci.yml
-        - .gitlab-ci/**/*
-        - .ci-farms/*
-        # Meson
-        - meson*
-        - build-support/**/*
-        - subprojects/**/*
-        # clang format
-        - .clang-format
-        - .clang-format-include
-        - .clang-format-ignore
-        # Source code
-        - include/**/*
-        - src/**/*
-      when: on_success
-    # Same as above, but for pre-merge pipelines
-    - if: *is-pre-merge
-      changes:
-        *all_paths
+    # Pipeline for forked project branch
+    - if: &is-forked-branch '$CI_COMMIT_BRANCH && $CI_PROJECT_NAMESPACE != "mesa"'
      when: manual
-    # Skip everything for pre-merge and merge pipelines which don't change
-    # anything in the build
-    - if: *is-merge-attempt
-      when: never
+    # Forked project branch / pre-merge pipeline not for Marge bot
+    - if: &is-forked-branch-or-pre-merge-not-for-marge '$CI_PROJECT_NAMESPACE != "mesa" || ($GITLAB_USER_LOGIN != "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event")'
+      when: manual
+    # Pipeline runs for the main branch of the upstream Mesa project
+    - if: &is-mesa-main '$CI_PROJECT_NAMESPACE == "mesa" && $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH && $CI_COMMIT_BRANCH'
+      when: always
+    # Post-merge pipeline
+    - if: &is-post-merge '$CI_PROJECT_NAMESPACE == "mesa" && $CI_COMMIT_BRANCH'
+      when: on_success
+    # Post-merge pipeline, not for Marge Bot
+    - if: &is-post-merge-not-for-marge '$CI_PROJECT_NAMESPACE == "mesa" && $GITLAB_USER_LOGIN != "marge-bot" && $CI_COMMIT_BRANCH'
+      when: on_success
+    # Pre-merge pipeline
+    - if: &is-pre-merge '$CI_PIPELINE_SOURCE == "merge_request_event"'
+      when: on_success
+    # Pre-merge pipeline for Marge Bot
+    - if: &is-pre-merge-for-marge '$GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event"'
+      when: on_success
+
+
+.docs-base:
+  variables:
+    BUILDER: html
+  extends:
+    - .fdo.ci-fairy
+    - .build-rules
+  artifacts:
+    expose_as: 'Documentation preview'
+    paths:
+    - public/
+  script:
+  - apk --no-cache add graphviz doxygen
+  - pip3 install sphinx===5.1.1 breathe===4.34.0 mako===1.2.3 sphinx_rtd_theme===1.0.0
+  - docs/doxygen-wrapper.py --out-dir=docs/doxygen_xml
+  - sphinx-build -W -b $BUILDER docs public
+
+pages:
+  extends: .docs-base
+  stage: deploy
+  needs: []
+  rules:
+    - !reference [.no_scheduled_pipelines-rules, rules]
+    - if: *is-mesa-main
+      changes: &docs-or-ci
+      - docs/**/*
+      - .gitlab-ci.yml
+      when: always
+    # Other cases default to never
+
+test-docs:
+  extends: .docs-base
+  # Cancel job if a newer commit is pushed to the same branch
+  interruptible: true
+  stage: deploy
+  needs: []
+  rules:
+    - !reference [.no_scheduled_pipelines-rules, rules]
+    - if: *is-forked-branch
+      changes: *docs-or-ci
+      when: manual
+    # Other cases default to never
+
+test-docs-mr:
+  extends:
+    - test-docs
+  needs:
+    - sanity
+  rules:
    - if: *is-pre-merge
-      when: never
-    # Build everything after someone bypassed the CI
-    - if: *is-direct-push
+      changes: *docs-or-ci
      when: on_success
-    # Build everything in scheduled pipelines
-    - if: *is-scheduled-pipeline
+    # Other cases default to never
+
+lincheck-docs:
+  extends: .docs-base
+  # Cancel job if a newer commit is pushed to the same branch
+  interruptible: true
+  stage: deploy
+  needs: []
+  rules:
+    - !reference [.scheduled_pipeline-rules, rules]
+  allow_failure: true
+  variables:
+    BUILDER: linkcheck
+
+# When to automatically run the CI for build jobs
+.build-rules:
+  rules:
+    - !reference [.no_scheduled_pipelines-rules, rules]
+    # If any files affecting the pipeline are changed, build/test jobs run
+    # automatically once all dependency jobs have passed
+    - changes: &all_paths
+      - VERSION
+      - bin/git_sha1_gen.py
+      - bin/install_megadrivers.py
+      - bin/meson_get_version.py
+      - bin/symbols-check.py
+      # GitLab CI
+      - .gitlab-ci.yml
+      - .gitlab-ci/**/*
+      # Meson
+      - meson*
+      - build-support/**/*
+      - subprojects/**/*
+      # Source code
+      - include/**/*
+      - src/**/*
      when: on_success
-    # Allow building everything in fork pipelines, but build nothing unless
-    # manually triggered
-    - when: manual
+    # Otherwise, build/test jobs won't run because no rule matched.


 .ci-deqp-artifacts:
@@ -258,6 +232,35 @@ include:
      - _build/meson-logs/*.txt
      - _build/meson-logs/strace

+.container-rules:
+  rules:
+    - !reference [.no_scheduled_pipelines-rules, rules]
+    # Run pipeline by default in the main project if any CI pipeline
+    # configuration files were changed, to ensure docker images are up to date
+    - if: *is-post-merge
+      changes:
+      - .gitlab-ci.yml
+      - .gitlab-ci/**/*
+      when: on_success
+    # Run pipeline by default if it was triggered by Marge Bot, is for a
+    # merge request, and any files affecting the pipeline were changed
+    - if: *is-pre-merge-for-marge
+      changes:
+        *all_paths
+      when: on_success
+    # Run pipeline by default in the main project if it was not triggered by
+    # Marge Bot, and any files affecting the pipeline were changed
+    - if: *is-post-merge-not-for-marge
+      changes:
+        *all_paths
+      when: on_success
+    # Allow triggering jobs manually in other cases if any files affecting the
+    # pipeline were changed
+    - changes:
+        *all_paths
+      when: manual
+    # Otherwise, container jobs won't run because no rule matched.
+
 # Git archive

 make git archive:
@@ -272,12 +275,10 @@ make git archive:
  script:
    # Compactify the .git directory
    - git gc --aggressive
-    # Download & cache the perfetto subproject as well.
-    - rm -rf subprojects/perfetto ; mkdir -p subprojects/perfetto && curl https://android.googlesource.com/platform/external/perfetto/+archive/$(grep 'revision =' subprojects/perfetto.wrap | cut -d ' ' -f3).tar.gz | tar zxf - -C subprojects/perfetto
    # compress the current folder
    - tar -cvzf ../$CI_PROJECT_NAME.tar.gz .

-    - ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" ../$CI_PROJECT_NAME.tar.gz https://$S3_HOST/git-cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/$CI_PROJECT_NAME.tar.gz
+    - ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" ../$CI_PROJECT_NAME.tar.gz https://$MINIO_HOST/git-cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/$CI_PROJECT_NAME.tar.gz


 # Sanity checks of MR settings and commit logs
@@ -288,7 +289,7 @@ sanity:
  rules:
    - if: *is-pre-merge
      when: on_success
-    - when: never
+    # Other cases default to never
  variables:
    GIT_STRATEGY: none
  script:
@@ -298,14 +299,18 @@ sanity:
    when: on_failure
    reports:
      junit: check-*.xml
-  tags:
-    - placeholder-job

+# Rules for tests that should not block merging, but should be available to
+# optionally run with the "play" button in the UI in pre-merge non-marge
+# pipelines.  This should appear in "extends:" after any includes of
+# test-source-dep.yml rules, so that these rules replace those.
+.test-manual-mr:
+  rules:
+    - !reference [.no_scheduled_pipelines-rules, rules]
+    - if: *is-forked-branch-or-pre-merge-not-for-marge
+      changes:
+        *all_paths
+      when: manual
+  variables:
+    JOB_TIMEOUT: 80

-# Jobs that need to pass before spending hardware resources on further testing
-.required-for-hardware-jobs:
-  needs:
-    - job: clang-format
-      optional: true
-    - job: rustfmt
-      optional: true
--- a/.gitlab-ci/all-skips.txt
+++ b/.gitlab-ci/all-skips.txt
@@ -2,13 +2,6 @@
 # non-zero-length and not starting with '#', will regex match to
 # delete lines from the test list.  Be careful.

-# This test checks the driver's reported conformance version against the
-# version of the CTS we're running. This check fails every few months
-# and everyone has to go and bump the number in every driver.
-# Running this check only makes sense while preparing a conformance
-# submission, so skip it in the regular CI.
-dEQP-VK.api.driver_properties.conformance_version
-
 # These are tremendously slow (pushing toward a minute), and aren't
 # reliable to be run in parallel with other tests due to CPU-side timing.
 dEQP-GLES[0-9]*.functional.flush_finish.*
@@ -22,58 +15,3 @@ glx@glx_arb_sync_control@timing.*

 # This test is not built with waffle, while we do build tests with waffle
 spec@!opengl 1.1@windowoverlap
-
-# These tests all read from the front buffer after a swap.  Given that we
-# run piglit tests in parallel in Mesa CI, and don't have a compositor
-# running, the frontbuffer reads may end up with undefined results from
-# windows overlapping us.
-#
-# Piglit does mark these tests as not to be run in parallel, but deqp-runner
-# doesn't respect that.  We need to extend deqp-runner to allow some tests to be
-# marked as single-threaded and run after the rayon loop if we want to support
-# them.
-#
-# Note that "glx-" tests don't appear in x11-skips.txt because they can be
-# run even if PIGLIT_PLATFORM=gbm (for example)
-glx@glx-copy-sub-buffer.*
-
-# A majority of the tests introduced in CTS 1.3.7.0 are experiencing failures and flakes.
-# Disable these tests until someone with a more deeper understanding of EGL examines them.
-#
-# Note: on sc8280xp/a690 I get identical results (same passes and fails)
-# between freedreno, zink, and llvmpipe, so I believe this is either a
-# deqp bug or egl/wayland bug, rather than driver issue.
-#
-# With llvmpipe, the failing tests have the error message:
-#
-#   "Illegal sampler view creation without bind flag"
-#
-# which might be a hint.  (But some passing tests also have the same
-# error message.)
-#
-# more context from David Heidelberg on IRC: the deqp commit where these
-# started failing is: https://github.com/KhronosGroup/VK-GL-CTS/commit/79b25659bcbced0cfc2c3fe318951c585f682abe
-# prior to that they were skipping.
-wayland-dEQP-EGL.functional.color_clears.single_context.gles1.other
-wayland-dEQP-EGL.functional.color_clears.single_context.gles2.other
-wayland-dEQP-EGL.functional.color_clears.single_context.gles3.other
-wayland-dEQP-EGL.functional.color_clears.multi_context.gles1.other
-wayland-dEQP-EGL.functional.color_clears.multi_context.gles2.other
-wayland-dEQP-EGL.functional.color_clears.multi_context.gles3.other
-wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2.other
-wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2_gles3.other
-wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1.other
-wayland-dEQP-EGL.functional.color_clears.multi_thread.gles2.other
-wayland-dEQP-EGL.functional.color_clears.multi_thread.gles3.other
-wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2.other
-wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2_gles3.other
-
-# Seems to be the same is as wayland-dEQP-EGL.functional.color_clears.*
-wayland-dEQP-EGL.functional.render.single_context.gles2.other
-wayland-dEQP-EGL.functional.render.single_context.gles3.other
-wayland-dEQP-EGL.functional.render.multi_context.gles2.other
-wayland-dEQP-EGL.functional.render.multi_context.gles3.other
-wayland-dEQP-EGL.functional.render.multi_context.gles2_gles3.other
-wayland-dEQP-EGL.functional.render.multi_thread.gles2.other
-wayland-dEQP-EGL.functional.render.multi_thread.gles3.other
-wayland-dEQP-EGL.functional.render.multi_thread.gles2_gles3.other
--- a/.gitlab-ci/b2c/b2c.yml.jinja2.jinja2
+++ b/.gitlab-ci/b2c/b2c.yml.jinja2.jinja2
@@ -2,7 +2,12 @@ version: 1

 # Rules to match for a machine to qualify
 target:
-  id: '{{ ci_runner_id }}'
+{% if tags %}
+  tags:
+{% for tag in tags %}
+    - '{{ tag | trim }}'
+{% endfor %}
+{% endif %}

 timeouts:
  first_console_activity:  # This limits the time it can take to receive the first console log
@@ -23,11 +28,9 @@ console_patterns:
    session_end:
        regex: >-
          {{ session_end_regex }}
-{% if session_reboot_regex %}
    session_reboot:
        regex: >-
          {{ session_reboot_regex }}
-{% endif %}
    job_success:
        regex: >-
          {{ job_success_regex }}
@@ -45,8 +48,8 @@ deployment:
        SALAD.machine_id={{ '{{' }} machine_id }}
        console={{ '{{' }} local_tty_device }},115200 earlyprintk=vga,keep
        loglevel={{ log_level }} no_hash_pointers
-        b2c.service="--privileged --tls-verify=false --pid=host docker://{{ '{{' }} fdo_proxy_registry }}/gfx-ci/ci-tron/telegraf:latest" b2c.hostname=dut-{{ '{{' }} machine.full_name }}
-        b2c.container="-ti --tls-verify=false docker://{{ '{{' }} fdo_proxy_registry }}/gfx-ci/ci-tron/machine-registration:latest check"
+        b2c.service="--privileged --tls-verify=false --pid=host docker://{{ '{{' }} fdo_proxy_registry }}/mupuf/valve-infra/telegraf-container:latest" b2c.hostname=dut-{{ '{{' }} machine.full_name }}
+        b2c.container="-ti --tls-verify=false docker://{{ '{{' }} fdo_proxy_registry }}/mupuf/valve-infra/machine_registration:latest check"
        b2c.ntp_peer=10.42.0.1 b2c.pipefail b2c.cache_device=auto b2c.poweroff_delay={{ poweroff_delay }}
        b2c.minio="gateway,{{ '{{' }} minio_url }},{{ '{{' }} job_bucket_access_key }},{{ '{{' }} job_bucket_secret_key }}"
        b2c.volume="{{ '{{' }} job_bucket }}-results,mirror=gateway/{{ '{{' }} job_bucket }},pull_on=pipeline_start,push_on=changes,overwrite{% for excl in job_volume_exclusions %},exclude={{ excl }}{% endfor %},remove,expiration=pipeline_end,preserve"
@@ -54,8 +57,8 @@ deployment:
        b2c.volume={{ volume }}
 {% endfor %}
        b2c.container="-v {{ '{{' }} job_bucket }}-results:{{ working_dir }} -w {{ working_dir }} {% for mount_volume in mount_volumes %} -v {{ mount_volume }}{% endfor %} --tls-verify=false docker://{{ local_container }} {{ container_cmd }}"
-        {% if kernel_cmdline_extras is defined %}
-        {{ kernel_cmdline_extras }}
+        {% if cmdline_extras is defined %}
+        {{ cmdline_extras }}
        {% endif %}

    initramfs:
--- a/.gitlab-ci/b2c/generate_b2c.py
+++ b/.gitlab-ci/b2c/generate_b2c.py
@@ -22,34 +22,86 @@
 # IN THE SOFTWARE.

 from jinja2 import Environment, FileSystemLoader
+from argparse import ArgumentParser
 from os import environ, path
+import json


-# Pass all the environment variables prefixed by B2C_
-values = {
-    key.removeprefix("B2C_").lower(): environ[key]
-    for key in environ if key.startswith("B2C_")
-}
+parser = ArgumentParser()
+parser.add_argument('--ci-job-id')
+parser.add_argument('--container-cmd')
+parser.add_argument('--initramfs-url')
+parser.add_argument('--job-success-regex')
+parser.add_argument('--job-warn-regex')
+parser.add_argument('--kernel-url')
+parser.add_argument('--log-level', type=int)
+parser.add_argument('--poweroff-delay', type=int)
+parser.add_argument('--session-end-regex')
+parser.add_argument('--session-reboot-regex')
+parser.add_argument('--tags', nargs='?', default='')
+parser.add_argument('--template', default='b2c.yml.jinja2.jinja2')
+parser.add_argument('--timeout-boot-minutes', type=int)
+parser.add_argument('--timeout-boot-retries', type=int)
+parser.add_argument('--timeout-first-minutes', type=int)
+parser.add_argument('--timeout-first-retries', type=int)
+parser.add_argument('--timeout-minutes', type=int)
+parser.add_argument('--timeout-overall-minutes', type=int)
+parser.add_argument('--timeout-retries', type=int)
+parser.add_argument('--job-volume-exclusions', nargs='?', default='')
+parser.add_argument('--volume', action='append')
+parser.add_argument('--mount-volume', action='append')
+parser.add_argument('--local-container', default=environ.get('B2C_LOCAL_CONTAINER', 'alpine:latest'))
+parser.add_argument('--working-dir')
+args = parser.parse_args()

-env = Environment(loader=FileSystemLoader(path.dirname(values['job_template'])),
+env = Environment(loader=FileSystemLoader(path.dirname(args.template)),
                  trim_blocks=True, lstrip_blocks=True)

-template = env.get_template(path.basename(values['job_template']))
+template = env.get_template(path.basename(args.template))

-values['ci_job_id'] = environ['CI_JOB_ID']
-values['ci_runner_id'] = environ['CI_RUNNER_ID']
-values['job_volume_exclusions'] = [excl for excl in values['job_volume_exclusions'].split(",") if excl]
-values['working_dir'] = environ['CI_PROJECT_DIR']
+values = {}
+values['ci_job_id'] = args.ci_job_id
+values['container_cmd'] = args.container_cmd
+values['initramfs_url'] = args.initramfs_url
+values['job_success_regex'] = args.job_success_regex
+values['job_warn_regex'] = args.job_warn_regex
+values['kernel_url'] = args.kernel_url
+values['log_level'] = args.log_level
+values['poweroff_delay'] = args.poweroff_delay
+values['session_end_regex'] = args.session_end_regex
+values['session_reboot_regex'] = args.session_reboot_regex
+try:
+    values['tags'] = json.loads(args.tags)
+except json.decoder.JSONDecodeError:
+    values['tags'] = args.tags.split(",")
+values['template'] = args.template
+values['timeout_boot_minutes'] = args.timeout_boot_minutes
+values['timeout_boot_retries'] = args.timeout_boot_retries
+values['timeout_first_minutes'] = args.timeout_first_minutes
+values['timeout_first_retries'] = args.timeout_first_retries
+values['timeout_minutes'] = args.timeout_minutes
+values['timeout_overall_minutes'] = args.timeout_overall_minutes
+values['timeout_retries'] = args.timeout_retries
+if len(args.job_volume_exclusions) > 0:
+    exclusions = args.job_volume_exclusions.split(",")
+    values['job_volume_exclusions'] = [excl for excl in exclusions if len(excl) > 0]
+if args.volume is not None:
+    values['volumes'] = args.volume
+if args.mount_volume is not None:
+    values['mount_volumes'] = args.mount_volume
+values['working_dir'] = args.working_dir
+
+assert(len(args.local_container) > 0)

 # Use the gateway's pull-through registry caches to reduce load on fd.o.
-values['local_container'] = environ['IMAGE_UNDER_TEST']
-values['local_container'] = values['local_container'].replace(
-    'registry.freedesktop.org',
-    '{{ fdo_proxy_registry }}'
-)
+values['local_container'] = args.local_container
+for url, replacement in [('registry.freedesktop.org', '{{ fdo_proxy_registry }}'),
+                         ('harbor.freedesktop.org', '{{ harbor_fdo_registry }}')]:
+    values['local_container'] = values['local_container'].replace(url, replacement)

-if 'kernel_cmdline_extras' not in values:
-    values['kernel_cmdline_extras'] = ''
+if 'B2C_KERNEL_CMDLINE_EXTRAS' in environ:
+    values['cmdline_extras'] = environ['B2C_KERNEL_CMDLINE_EXTRAS']

-with open(path.splitext(path.basename(values['job_template']))[0], "w") as f:
-    f.write(template.render(values))
+f = open(path.splitext(path.basename(args.template))[0], "w")
+f.write(template.render(values))
+f.close()
--- a/.gitlab-ci/bare-metal/cisco-2960-poe-off.sh
+++ b/.gitlab-ci/bare-metal/cisco-2960-poe-off.sh
@@ -1,5 +1,4 @@
 #!/bin/bash
-# shellcheck disable=SC2086 # we want word splitting

 if [ -z "$BM_POE_INTERFACE" ]; then
    echo "Must supply the PoE Interface to power down"
@@ -12,6 +11,7 @@ if [ -z "$BM_POE_ADDRESS" ]; then
 fi

 SNMP_KEY="1.3.6.1.4.1.9.9.402.1.2.1.1.1.$BM_POE_INTERFACE"
+SNMP_ON="i 1"
 SNMP_OFF="i 4"

-snmpset -v2c -r 3 -t 30 -cmesaci "$BM_POE_ADDRESS" "$SNMP_KEY" $SNMP_OFF
+snmpset -v2c -r 3 -t 30 -cmesaci $BM_POE_ADDRESS $SNMP_KEY $SNMP_OFF
--- a/.gitlab-ci/bare-metal/cisco-2960-poe-on.sh
+++ b/.gitlab-ci/bare-metal/cisco-2960-poe-on.sh
@@ -1,5 +1,4 @@
 #!/bin/bash
-# shellcheck disable=SC2086 # we want word splitting

 if [ -z "$BM_POE_INTERFACE" ]; then
    echo "Must supply the PoE Interface to power up"
@@ -17,6 +16,6 @@ SNMP_KEY="1.3.6.1.4.1.9.9.402.1.2.1.1.1.$BM_POE_INTERFACE"
 SNMP_ON="i 1"
 SNMP_OFF="i 4"

-snmpset -v2c -r 3 -t 10 -cmesaci "$BM_POE_ADDRESS" "$SNMP_KEY" $SNMP_OFF
+snmpset -v2c -r 3 -t 10 -cmesaci $BM_POE_ADDRESS $SNMP_KEY $SNMP_OFF
 sleep 3s
-snmpset -v2c -r 3 -t 10 -cmesaci "$BM_POE_ADDRESS" "$SNMP_KEY" $SNMP_ON
+snmpset -v2c -r 3 -t 10 -cmesaci $BM_POE_ADDRESS $SNMP_KEY $SNMP_ON
--- a/.gitlab-ci/bare-metal/cros-servo.sh
+++ b/.gitlab-ci/bare-metal/cros-servo.sh
@@ -1,7 +1,4 @@
 #!/bin/bash
-# shellcheck disable=SC1091 # The relative paths in this file only become valid at runtime.
-# shellcheck disable=SC2034
-# shellcheck disable=SC2086 # we want word splitting

 # Boot script for Chrome OS devices attached to a servo debug connector, using
 # NFS and TFTP to boot.
@@ -9,7 +6,6 @@
 # We're run from the root of the repo, make a helper var for our paths
 BM=$CI_PROJECT_DIR/install/bare-metal
 CI_COMMON=$CI_PROJECT_DIR/install/common
-CI_INSTALL=$CI_PROJECT_DIR/install

 # Runner config checks
 if [ -z "$BM_SERIAL" ]; then
@@ -84,41 +80,23 @@ mkdir -p /nfs/results

 rm -rf /tftp/*
 if echo "$BM_KERNEL" | grep -q http; then
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-      $BM_KERNEL -o /tftp/vmlinuz
-elif [ -n "${FORCE_KERNEL_TAG}" ]; then
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-    "${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${BM_KERNEL}" -o /tftp/vmlinuz
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-    "${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/modules.tar.zst" -o modules.tar.zst
-  tar --keep-directory-symlink --zstd -xf modules.tar.zst -C "/nfs/"
-  rm modules.tar.zst &
+  apt install -y wget
+  wget $BM_KERNEL -O /tftp/vmlinuz
 else
-  cp /baremetal-files/"$BM_KERNEL" /tftp/vmlinuz
+  cp $BM_KERNEL /tftp/vmlinuz
 fi
 echo "$BM_CMDLINE" > /tftp/cmdline

 set +e
-STRUCTURED_LOG_FILE=job_detail.json
-python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update dut_job_type "${DEVICE_TYPE}"
-python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update farm "${FARM}"
-python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --create-dut-job dut_name "${CI_RUNNER_DESCRIPTION}"
-python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update-dut-time submit "${CI_JOB_STARTED_AT}"
 python3 $BM/cros_servo_run.py \
        --cpu $BM_SERIAL \
        --ec $BM_SERIAL_EC \
        --test-timeout ${TEST_PHASE_TIMEOUT:-20}
 ret=$?
-python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --close-dut-job
-python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --close
 set -e

 # Bring artifacts back from the NFS dir to the build dir where gitlab-runner
 # will look for them.
 cp -Rp /nfs/results/. results/
-if [ -f "${STRUCTURED_LOG_FILE}" ]; then
-  cp -p ${STRUCTURED_LOG_FILE} results/
-  echo "Structured log file is available at https://${CI_PROJECT_ROOT_NAMESPACE}.pages.freedesktop.org/-/${CI_PROJECT_NAME}/-/jobs/${CI_JOB_ID}/artifacts/results/${STRUCTURED_LOG_FILE}"
-fi

 exit $ret
--- a/.gitlab-ci/bare-metal/cros_servo_run.py
+++ b/.gitlab-ci/bare-metal/cros_servo_run.py
@@ -1,18 +1,37 @@
+
 #!/usr/bin/env python3
 #
 # Copyright © 2020 Google LLC
-# SPDX-License-Identifier: MIT
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.

 import argparse
+import queue
 import re
-import sys
-
-from custom_logger import CustomLogger
 from serial_buffer import SerialBuffer
+import sys
+import threading


 class CrosServoRun:
-    def __init__(self, cpu, ec, test_timeout, logger):
+    def __init__(self, cpu, ec, test_timeout):
        self.cpu_ser = SerialBuffer(
            cpu, "results/serial.txt", "R SERIAL-CPU> ")
        # Merge the EC serial into the cpu_ser's line stream so that we can
@@ -20,7 +39,6 @@ class CrosServoRun:
        self.ec_ser = SerialBuffer(
            ec, "results/serial-ec.txt", "R SERIAL-EC> ", line_queue=self.cpu_ser.line_queue)
        self.test_timeout = test_timeout
-        self.logger = logger

    def close(self):
        self.ec_ser.close()
@@ -38,7 +56,6 @@ class CrosServoRun:
        RED = '\033[0;31m'
        NO_COLOR = '\033[0m'
        print(RED + message + NO_COLOR)
-        self.logger.update_status_fail(message)

    def run(self):
        # Flush any partial commands in the EC's prompt, then ask for a reboot.
@@ -46,8 +63,6 @@ class CrosServoRun:
        self.ec_write("reboot\n")

        bootloader_done = False
-        self.logger.create_job_phase("boot")
-        tftp_failures = 0
        # This is emitted right when the bootloader pauses to check for input.
        # Emit a ^N character to request network boot, because we don't have a
        # direct-to-netboot firmware on cheza.
@@ -57,17 +72,6 @@ class CrosServoRun:
                bootloader_done = True
                break

-            # The Cheza firmware seems to occasionally get stuck looping in
-            # this error state during TFTP booting, possibly based on amount of
-            # network traffic around it, but it'll usually recover after a
-            # reboot. Currently mostly visible on google-freedreno-cheza-14.
-            if re.search("R8152: Bulk read error 0xffffffbf", line):
-                tftp_failures += 1
-                if tftp_failures >= 10:
-                    self.print_error(
-                        "Detected intermittent tftp failure, restarting run.")
-                    return 1
-
            # If the board has a netboot firmware and we made it to booting the
            # kernel, proceed to processing of the test run.
            if re.search("Booting Linux", line):
@@ -79,30 +83,41 @@ class CrosServoRun:
            # in the farm.
            if re.search("POWER_GOOD not seen in time", line):
                self.print_error(
-                    "Detected intermittent poweron failure, abandoning run.")
-                return 1
+                    "Detected intermittent poweron failure, restarting run...")
+                return 2

        if not bootloader_done:
-            self.print_error("Failed to make it through bootloader, abandoning run.")
-            return 1
+            print("Failed to make it through bootloader, restarting run...")
+            return 2

-        self.logger.create_job_phase("test")
+        tftp_failures = 0
        for line in self.cpu_ser.lines(timeout=self.test_timeout, phase="test"):
            if re.search("---. end Kernel panic", line):
                return 1

+            # The Cheza firmware seems to occasionally get stuck looping in
+            # this error state during TFTP booting, possibly based on amount of
+            # network traffic around it, but it'll usually recover after a
+            # reboot.
+            if re.search("R8152: Bulk read error 0xffffffbf", line):
+                tftp_failures += 1
+                if tftp_failures >= 100:
+                    self.print_error(
+                        "Detected intermittent tftp failure, restarting run...")
+                    return 2
+
            # There are very infrequent bus errors during power management transitions
            # on cheza, which we don't expect to be the case on future boards.
            if re.search("Kernel panic - not syncing: Asynchronous SError Interrupt", line):
                self.print_error(
-                    "Detected cheza power management bus error, abandoning run.")
-                return 1
+                    "Detected cheza power management bus error, restarting run...")
+                return 2

            # If the network device dies, it's probably not graphics's fault, just try again.
            if re.search("NETDEV WATCHDOG", line):
                self.print_error(
-                    "Detected network device failure, abandoning run.")
-                return 1
+                    "Detected network device failure, restarting run...")
+                return 2

            # These HFI response errors started appearing with the introduction
            # of piglit runs.  CosmicPenguin says:
@@ -115,30 +130,28 @@ class CrosServoRun:
            # break many tests after that, just restart the whole run.
            if re.search("a6xx_hfi_send_msg.*Unexpected message id .* on the response queue", line):
                self.print_error(
-                    "Detected cheza power management bus error, abandoning run.")
-                return 1
+                    "Detected cheza power management bus error, restarting run...")
+                return 2

            if re.search("coreboot.*bootblock starting", line):
                self.print_error(
-                    "Detected spontaneous reboot, abandoning run.")
-                return 1
+                    "Detected spontaneous reboot, restarting run...")
+                return 2

            if re.search("arm-smmu 5040000.iommu: TLB sync timed out -- SMMU may be deadlocked", line):
-                self.print_error("Detected cheza MMU fail, abandoning run.")
-                return 1
+                self.print_error("Detected cheza MMU fail, restarting run...")
+                return 2

            result = re.search("hwci: mesa: (\S*)", line)
            if result:
                if result.group(1) == "pass":
-                    self.logger.update_dut_job("status", "pass")
                    return 0
                else:
-                    self.logger.update_status_fail("test fail")
                    return 1

        self.print_error(
            "Reached the end of the CPU serial log without finding a result")
-        return 1
+        return 2


 def main():
@@ -151,17 +164,16 @@ def main():
        '--test-timeout', type=int, help='Test phase timeout (minutes)', required=True)
    args = parser.parse_args()

-    logger = CustomLogger("job_detail.json")
-    logger.update_dut_time("start", None)
-    servo = CrosServoRun(args.cpu, args.ec, args.test_timeout * 60, logger)
-    retval = servo.run()
+    while True:
+        servo = CrosServoRun(args.cpu, args.ec, args.test_timeout * 60)
+        retval = servo.run()

-    # power down the CPU on the device
-    servo.ec_write("power off\n")
-    logger.update_dut_time("end", None)
-    servo.close()
+        # power down the CPU on the device
+        servo.ec_write("power off\n")
+        servo.close()

-    sys.exit(retval)
+        if retval != 2:
+            sys.exit(retval)


 if __name__ == '__main__':
--- a/.gitlab-ci/bare-metal/eth008-power-down.sh
+++ b/.gitlab-ci/bare-metal/eth008-power-down.sh
@@ -7,4 +7,4 @@ if [ -z "$relay" ]; then
    exit 1
 fi

-"$CI_PROJECT_DIR"/install/bare-metal/eth008-power-relay.py "$ETH_HOST" "$ETH_PORT" off "$relay"
+$CI_PROJECT_DIR/install/bare-metal/eth008-power-relay.py $ETH_HOST $ETH_PORT off $relay
--- a/.gitlab-ci/bare-metal/eth008-power-up.sh
+++ b/.gitlab-ci/bare-metal/eth008-power-up.sh
@@ -7,6 +7,6 @@ if [ -z "$relay" ]; then
    exit 1
 fi

-"$CI_PROJECT_DIR"/install/bare-metal/eth008-power-relay.py "$ETH_HOST" "$ETH_PORT" off "$relay"
+$CI_PROJECT_DIR/install/bare-metal/eth008-power-relay.py $ETH_HOST $ETH_PORT off $relay
 sleep 5
-"$CI_PROJECT_DIR"/install/bare-metal/eth008-power-relay.py "$ETH_HOST" "$ETH_PORT" on "$relay"
+$CI_PROJECT_DIR/install/bare-metal/eth008-power-relay.py $ETH_HOST $ETH_PORT on $relay
--- a/.gitlab-ci/bare-metal/expect-output.sh
+++ b/.gitlab-ci/bare-metal/expect-output.sh
@@ -5,27 +5,26 @@ set -e
 STRINGS=$(mktemp)
 ERRORS=$(mktemp)

-trap 'rm $STRINGS; rm $ERRORS;' EXIT
+trap "rm $STRINGS; rm $ERRORS;" EXIT

 FILE=$1
 shift 1

 while getopts "f:e:" opt; do
  case $opt in
-    f) echo "$OPTARG" >> "$STRINGS";;
-    e) echo "$OPTARG" >> "$STRINGS" ; echo "$OPTARG" >> "$ERRORS";;
-    *) exit
+    f) echo "$OPTARG" >> $STRINGS;;
+    e) echo "$OPTARG" >> $STRINGS ; echo "$OPTARG" >> $ERRORS;;
  esac
 done
 shift $((OPTIND -1))

 echo "Waiting for $FILE to say one of following strings"
-cat "$STRINGS"
+cat $STRINGS

-while ! grep -E -wf "$STRINGS" "$FILE"; do
+while ! egrep -wf $STRINGS $FILE; do
  sleep 2
 done

-if grep -E -wf "$ERRORS" "$FILE"; then
+if egrep -wf $ERRORS $FILE; then
  exit 1
 fi
--- a/.gitlab-ci/bare-metal/fastboot.sh
+++ b/.gitlab-ci/bare-metal/fastboot.sh
@@ -1,14 +1,9 @@
 #!/bin/bash
-# shellcheck disable=SC1091 # The relative paths in this file only become valid at runtime.
-# shellcheck disable=SC2034
-# shellcheck disable=SC2086 # we want word splitting
-
-. "$SCRIPTS_DIR"/setup-test-env.sh

 BM=$CI_PROJECT_DIR/install/bare-metal
 CI_COMMON=$CI_PROJECT_DIR/install/common

-if [ -z "$BM_SERIAL" ] && [ -z "$BM_SERIAL_SCRIPT" ]; then
+if [ -z "$BM_SERIAL" -a -z "$BM_SERIAL_SCRIPT" ]; then
  echo "Must set BM_SERIAL OR BM_SERIAL_SCRIPT in your gitlab-runner config.toml [[runners]] environment"
  echo "BM_SERIAL:"
  echo "  This is the serial device to talk to for waiting for fastboot to be ready and logging from the kernel."
@@ -87,42 +82,34 @@ else
  fi

  pushd rootfs
-  find -H . | \
-    grep -E -v "external/(openglcts|vulkancts|amber|glslang|spirv-tools)" |
-    grep -E -v "traces-db|apitrace|renderdoc" | \
-    grep -E -v $EXCLUDE_FILTER | \
+  find -H | \
+    egrep -v "external/(openglcts|vulkancts|amber|glslang|spirv-tools)" |
+    egrep -v "traces-db|apitrace|renderdoc" | \
+    egrep -v $EXCLUDE_FILTER | \
    cpio -H newc -o | \
    xz --check=crc32 -T4 - > $CI_PROJECT_DIR/rootfs.cpio.gz
  popd
 fi

+# Make the combined kernel image and dtb for passing to fastboot.  For normal
+# Mesa development, we build the kernel and store it in the docker container
+# that this script is running in.
+#
+# However, container builds are expensive, so when you're hacking on the
+# kernel, it's nice to be able to skip the half hour container build and plus
+# moving that container to the runner.  So, if BM_KERNEL+BM_DTB are URLs,
+# fetch them instead of looking in the container.
 if echo "$BM_KERNEL $BM_DTB" | grep -q http; then
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-      "$BM_KERNEL" -o kernel
-  # FIXME: modules should be supplied too
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-      "$BM_DTB" -o dtb
+  apt install -y wget
+
+  wget $BM_KERNEL -O kernel
+  wget $BM_DTB -O dtb

  cat kernel dtb > Image.gz-dtb
-
-elif [ -n "${FORCE_KERNEL_TAG}" ]; then
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-      "${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${BM_KERNEL}" -o kernel
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-      "${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/modules.tar.zst" -o modules.tar.zst
-
-  if [ -n "$BM_DTB" ]; then
-    curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-	"${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${BM_DTB}.dtb" -o dtb
-  fi
-
-  cat kernel dtb > Image.gz-dtb || echo "No DTB available, using pure kernel."
  rm kernel
-  tar --keep-directory-symlink --zstd -xf modules.tar.zst -C "$BM_ROOTFS/"
-  rm modules.tar.zst &
 else
-  cat /baremetal-files/"$BM_KERNEL" /baremetal-files/"$BM_DTB".dtb > Image.gz-dtb
-  cp /baremetal-files/"$BM_DTB".dtb dtb
+  cat $BM_KERNEL $BM_DTB > Image.gz-dtb
+  cp $BM_DTB dtb
 fi

 export PATH=$BM:$PATH
--- a/.gitlab-ci/bare-metal/fastboot_run.py
+++ b/.gitlab-ci/bare-metal/fastboot_run.py
@@ -51,8 +51,8 @@ class FastbootRun:
        try:
            return subprocess.call(cmd, shell=True, timeout=timeout)
        except subprocess.TimeoutExpired:
-            self.print_error("timeout, abandoning run.")
-            return 1
+            self.print_error("timeout, restarting run...")
+            return 2

    def run(self):
        if ret := self.logged_system(self.powerup):
@@ -60,20 +60,20 @@ class FastbootRun:

        fastboot_ready = False
        for line in self.ser.lines(timeout=2 * 60, phase="bootloader"):
-            if re.search("[Ff]astboot: [Pp]rocessing commands", line) or \
+            if re.search("fastboot: processing commands", line) or \
                    re.search("Listening for fastboot command on", line):
                fastboot_ready = True
                break

            if re.search("data abort", line):
                self.print_error(
-                    "Detected crash during boot, abandoning run.")
-                return 1
+                    "Detected crash during boot, restarting run...")
+                return 2

        if not fastboot_ready:
            self.print_error(
-                "Failed to get to fastboot prompt, abandoning run.")
-            return 1
+                "Failed to get to fastboot prompt, restarting run...")
+            return 2

        if ret := self.logged_system(self.fastboot):
            return ret
@@ -81,7 +81,7 @@ class FastbootRun:
        print_more_lines = -1
        for line in self.ser.lines(timeout=self.test_timeout, phase="test"):
            if print_more_lines == 0:
-                return 1
+                return 2
            if print_more_lines > 0:
                print_more_lines -= 1

@@ -92,20 +92,20 @@ class FastbootRun:
            # when if we see a reboot after we got past fastboot.
            if re.search("PON REASON", line):
                self.print_error(
-                    "Detected spontaneous reboot, abandoning run.")
-                return 1
+                    "Detected spontaneous reboot, restarting run...")
+                return 2

            # db820c sometimes wedges around iommu fault recovery
            if re.search("watchdog: BUG: soft lockup - CPU.* stuck", line):
                self.print_error(
-                    "Detected kernel soft lockup, abandoning run.")
-                return 1
+                    "Detected kernel soft lockup, restarting run...")
+                return 2

            # If the network device dies, it's probably not graphics's fault, just try again.
            if re.search("NETDEV WATCHDOG", line):
                self.print_error(
-                    "Detected network device failure, abandoning run.")
-                return 1
+                    "Detected network device failure, restarting run...")
+                return 2

            # A3xx recovery doesn't quite work. Sometimes the GPU will get
            # wedged and recovery will fail (because power can't be reset?)
@@ -115,7 +115,7 @@ class FastbootRun:
            # of the hang. Once a hang happens, it's pretty chatty.
            if "[drm:adreno_recover] *ERROR* gpu hw init failed: -22" in line:
                self.print_error(
-                    "Detected GPU hang, abandoning run.")
+                    "Detected GPU hang, restarting run...")
                if print_more_lines == -1:
                    print_more_lines = 30

@@ -127,8 +127,8 @@ class FastbootRun:
                    return 1

        self.print_error(
-            "Reached the end of the CPU serial log without finding a result, abandoning run.")
-        return 1
+            "Reached the end of the CPU serial log without finding a result, restarting run...")
+        return 2


 def main():
@@ -147,8 +147,13 @@ def main():

    fastboot = FastbootRun(args, args.test_timeout * 60)

-    retval = fastboot.run()
-    fastboot.close()
+    while True:
+        retval = fastboot.run()
+        fastboot.close()
+        if retval != 2:
+            break
+
+        fastboot = FastbootRun(args, args.test_timeout * 60)

    fastboot.logged_system(args.powerdown)

--- a/.gitlab-ci/bare-metal/google-power-down.sh
+++ b/.gitlab-ci/bare-metal/google-power-down.sh
@@ -7,4 +7,4 @@ if [ -z "$relay" ]; then
    exit 1
 fi

-"$CI_PROJECT_DIR"/install/bare-metal/google-power-relay.py off "$relay"
+$CI_PROJECT_DIR/install/bare-metal/google-power-relay.py off $relay
--- a/.gitlab-ci/bare-metal/google-power-up.sh
+++ b/.gitlab-ci/bare-metal/google-power-up.sh
@@ -7,6 +7,6 @@ if [ -z "$relay" ]; then
    exit 1
 fi

-"$CI_PROJECT_DIR"/install/bare-metal/google-power-relay.py off "$relay"
+$CI_PROJECT_DIR/install/bare-metal/google-power-relay.py off $relay
 sleep 5
-"$CI_PROJECT_DIR"/install/bare-metal/google-power-relay.py on "$relay"
+$CI_PROJECT_DIR/install/bare-metal/google-power-relay.py on $relay
--- a/.gitlab-ci/bare-metal/poe-off
+++ b/.gitlab-ci/bare-metal/poe-off
@@ -10,7 +10,8 @@ if [ -z "$BM_POE_ADDRESS" ]; then
    exit 1
 fi

-SNMP_KEY="SNMPv2-SMI::mib-2.105.1.1.1.3.1.$((48 + BM_POE_INTERFACE))"
+SNMP_KEY="SNMPv2-SMI::mib-2.105.1.1.1.3.1.`expr 48 + $BM_POE_INTERFACE`"
+SNMP_ON="i 1"
 SNMP_OFF="i 2"

 flock /var/run/poe.lock -c "snmpset -v2c -r 3 -t 30 -cmesaci $BM_POE_ADDRESS $SNMP_KEY $SNMP_OFF"
--- a/.gitlab-ci/bare-metal/poe-on
+++ b/.gitlab-ci/bare-metal/poe-on
@@ -10,7 +10,7 @@ if [ -z "$BM_POE_ADDRESS" ]; then
    exit 1
 fi

-SNMP_KEY="SNMPv2-SMI::mib-2.105.1.1.1.3.1.$((48 + BM_POE_INTERFACE))"
+SNMP_KEY="SNMPv2-SMI::mib-2.105.1.1.1.3.1.`expr 48 + $BM_POE_INTERFACE`"
 SNMP_ON="i 1"
 SNMP_OFF="i 2"

--- a/.gitlab-ci/bare-metal/poe-powered.sh
+++ b/.gitlab-ci/bare-metal/poe-powered.sh
@@ -1,10 +1,4 @@
 #!/bin/bash
-# shellcheck disable=SC1091
-# shellcheck disable=SC2034
-# shellcheck disable=SC2059
-# shellcheck disable=SC2086 # we want word splitting
-
-. "$SCRIPTS_DIR"/setup-test-env.sh

 # Boot script for devices attached to a PoE switch, using NFS for the root
 # filesystem.
@@ -12,7 +6,6 @@
 # We're run from the root of the repo, make a helper var for our paths
 BM=$CI_PROJECT_DIR/install/bare-metal
 CI_COMMON=$CI_PROJECT_DIR/install/common
-CI_INSTALL=$CI_PROJECT_DIR/install

 # Runner config checks
 if [ -z "$BM_SERIAL" ]; then
@@ -61,8 +54,8 @@ if [ -z "$BM_ROOTFS" ]; then
  exit 1
 fi

-if [ -z "$BM_BOOTFS" ] && { [ -z "$BM_KERNEL" ] || [ -z "$BM_DTB" ]; } ; then
-  echo "Must set /boot files for the TFTP boot in the job's variables or set kernel and dtb"
+if [ -z "$BM_BOOTFS" ]; then
+  echo "Must set /boot files for the TFTP boot in the job's variables"
  exit 1
 fi

@@ -71,9 +64,12 @@ if [ -z "$BM_CMDLINE" ]; then
  exit 1
 fi

-set -ex
+if [ -z "$BM_BOOTCONFIG" ]; then
+  echo "Must set BM_BOOTCONFIG to your board's required boot configuration arguments"
+  exit 1
+fi

-date +'%F %T'
+set -ex

 # Clear out any previous run's artifacts.
 rm -rf results/
@@ -83,64 +79,27 @@ mkdir -p results
 # state, since it's volume-mounted on the host.
 rsync -a --delete $BM_ROOTFS/ /nfs/

-date +'%F %T'
-
 # If BM_BOOTFS is an URL, download it
 if echo $BM_BOOTFS | grep -q http; then
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-    "${FDO_HTTP_CACHE_URI:-}$BM_BOOTFS" -o /tmp/bootfs.tar
+  apt install -y wget
+  wget ${FDO_HTTP_CACHE_URI:-}$BM_BOOTFS -O /tmp/bootfs.tar
  BM_BOOTFS=/tmp/bootfs.tar
 fi

-date +'%F %T'
-
 # If BM_BOOTFS is a file, assume it is a tarball and uncompress it
-if [ -f "${BM_BOOTFS}" ]; then
+if [ -f $BM_BOOTFS ]; then
  mkdir -p /tmp/bootfs
  tar xf $BM_BOOTFS -C /tmp/bootfs
  BM_BOOTFS=/tmp/bootfs
 fi

-# If BM_KERNEL and BM_DTS is present
-if [ -n "${FORCE_KERNEL_TAG}" ]; then
-  if [ -z "${BM_KERNEL}" ] || [ -z "${BM_DTB}" ]; then
-    echo "This machine cannot be tested with external kernel since BM_KERNEL or BM_DTB missing!"
-    exit 1
-  fi
-
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-      "${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${BM_KERNEL}" -o "${BM_KERNEL}"
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-      "${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${BM_DTB}.dtb" -o "${BM_DTB}.dtb"
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-      "${FDO_HTTP_CACHE_URI:-}${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/modules.tar.zst" -o modules.tar.zst
-fi
-
-date +'%F %T'
-
 # Install kernel modules (it could be either in /lib/modules or
 # /usr/lib/modules, but we want to install in the latter)
-if [ -n "${FORCE_KERNEL_TAG}" ]; then
-  tar --keep-directory-symlink --zstd -xf modules.tar.zst -C /nfs/
-  rm modules.tar.zst &
-elif [ -n "${BM_BOOTFS}" ]; then
-  [ -d $BM_BOOTFS/usr/lib/modules ] && rsync -a $BM_BOOTFS/usr/lib/modules/ /nfs/usr/lib/modules/
-  [ -d $BM_BOOTFS/lib/modules ] && rsync -a $BM_BOOTFS/lib/modules/ /nfs/lib/modules/
-else
-  echo "No modules!"
-fi
-
-
-date +'%F %T'
+[ -d $BM_BOOTFS/usr/lib/modules ] && rsync -a $BM_BOOTFS/usr/lib/modules/ /nfs/usr/lib/modules/
+[ -d $BM_BOOTFS/lib/modules ] && rsync -a $BM_BOOTFS/lib/modules/ /nfs/lib/modules/

 # Install kernel image + bootloader files
-if [ -n "${FORCE_KERNEL_TAG}" ] || [ -z "$BM_BOOTFS" ]; then
-  mv "${BM_KERNEL}" "${BM_DTB}.dtb" /tftp/
-else  # BM_BOOTFS
-  rsync -aL --delete $BM_BOOTFS/boot/ /tftp/
-fi
-
-date +'%F %T'
+rsync -aL --delete $BM_BOOTFS/boot/ /tftp/

 # Set up the pxelinux config for Jetson Nano
 mkdir -p /tftp/pxelinux.cfg
@@ -170,31 +129,17 @@ LABEL primary
 EOF

 # Create the rootfs in the NFS directory
+mkdir -p /nfs/results
 . $BM/rootfs-setup.sh /nfs

-date +'%F %T'
-
 echo "$BM_CMDLINE" > /tftp/cmdline.txt

-# Add some options in config.txt, if defined
-if [ -n "$BM_BOOTCONFIG" ]; then
-  printf "$BM_BOOTCONFIG" >> /tftp/config.txt
-fi
+# Add some required options in config.txt
+printf "$BM_BOOTCONFIG" >> /tftp/config.txt

 set +e
-STRUCTURED_LOG_FILE=job_detail.json
-python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update dut_job_type "${DEVICE_TYPE}"
-python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update farm "${FARM}"
-ATTEMPTS=3
-first_attempt=True
+ATTEMPTS=10
 while [ $((ATTEMPTS--)) -gt 0 ]; do
-  python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --create-dut-job dut_name "${CI_RUNNER_DESCRIPTION}"
-  # Update subtime time to CI_JOB_STARTED_AT only for the first run
-  if [ "$first_attempt" = "True" ]; then
-    python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update-dut-time submit "${CI_JOB_STARTED_AT}"
-  else
-    python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --update-dut-time submit
-  fi
  python3 $BM/poe_run.py \
          --dev="$BM_SERIAL" \
          --powerup="$BM_POWERUP" \
@@ -204,26 +149,14 @@ while [ $((ATTEMPTS--)) -gt 0 ]; do

  if [ $ret -eq 2 ]; then
    echo "Did not detect boot sequence, retrying..."
-    python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --close-dut-job
-    first_attempt=False
  else
    ATTEMPTS=0
  fi
 done
-python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --close-dut-job
-python3 $CI_INSTALL/custom_logger.py ${STRUCTURED_LOG_FILE} --close
 set -e

-date +'%F %T'
-
 # Bring artifacts back from the NFS dir to the build dir where gitlab-runner
 # will look for them.
 cp -Rp /nfs/results/. results/
-if [ -f "${STRUCTURED_LOG_FILE}" ]; then
-  cp -p ${STRUCTURED_LOG_FILE} results/
-  echo "Structured log file is available at ${ARTIFACTS_BASE_URL}/results/${STRUCTURED_LOG_FILE}"
-fi
-
-date +'%F %T'

 exit $ret
--- a/.gitlab-ci/bare-metal/poe_run.py
+++ b/.gitlab-ci/bare-metal/poe_run.py
@@ -24,26 +24,23 @@
 import argparse
 import os
 import re
+from serial_buffer import SerialBuffer
 import sys
 import threading

-from custom_logger import CustomLogger
-from serial_buffer import SerialBuffer

 class PoERun:
-    def __init__(self, args, test_timeout, logger):
+    def __init__(self, args, test_timeout):
        self.powerup = args.powerup
        self.powerdown = args.powerdown
        self.ser = SerialBuffer(
            args.dev, "results/serial-output.txt", "")
        self.test_timeout = test_timeout
-        self.logger = logger

    def print_error(self, message):
        RED = '\033[0;31m'
        NO_COLOR = '\033[0m'
        print(RED + message + NO_COLOR)
-        self.logger.update_status_fail(message)

    def logged_system(self, cmd):
        print("Running '{}'".format(cmd))
@@ -51,11 +48,9 @@ class PoERun:

    def run(self):
        if self.logged_system(self.powerup) != 0:
-            self.logger.update_status_fail("powerup failed")
            return 1

        boot_detected = False
-        self.logger.create_job_phase("boot")
        for line in self.ser.lines(timeout=5 * 60, phase="bootloader"):
            if re.search("Booting Linux", line):
                boot_detected = True
@@ -64,12 +59,10 @@ class PoERun:
        if not boot_detected:
            self.print_error(
                "Something wrong; couldn't detect the boot start up sequence")
-            return 1
+            return 2

-        self.logger.create_job_phase("test")
        for line in self.ser.lines(timeout=self.test_timeout, phase="test"):
            if re.search("---. end Kernel panic", line):
-                self.logger.update_status_fail("kernel panic")
                return 1

            # Binning memory problems
@@ -78,26 +71,24 @@ class PoERun:
                return 1

            if re.search("nouveau 57000000.gpu: bus: MMIO read of 00000000 FAULT at 137000", line):
-                self.print_error("nouveau jetson boot bug, abandoning run.")
-                return 1
+                self.print_error("nouveau jetson boot bug, retrying.")
+                return 2

            # network fail on tk1
            if re.search("NETDEV WATCHDOG:.* transmit queue 0 timed out", line):
-                self.print_error("nouveau jetson tk1 network fail, abandoning run.")
-                return 1
+                self.print_error("nouveau jetson tk1 network fail, retrying.")
+                return 2

            result = re.search("hwci: mesa: (\S*)", line)
            if result:
                if result.group(1) == "pass":
-                    self.logger.update_dut_job("status", "pass")
                    return 0
                else:
-                    self.logger.update_status_fail("test fail")
                    return 1

        self.print_error(
            "Reached the end of the CPU serial log without finding a result")
-        return 1
+        return 2


 def main():
@@ -112,13 +103,10 @@ def main():
        '--test-timeout', type=int, help='Test phase timeout (minutes)', required=True)
    args = parser.parse_args()

-    logger = CustomLogger("job_detail.json")
-    logger.update_dut_time("start", None)
-    poe = PoERun(args, args.test_timeout * 60, logger)
+    poe = PoERun(args, args.test_timeout * 60)
    retval = poe.run()

    poe.logged_system(args.powerdown)
-    logger.update_dut_time("end", None)

    sys.exit(retval)

--- a/.gitlab-ci/bare-metal/rootfs-setup.sh
+++ b/.gitlab-ci/bare-metal/rootfs-setup.sh
@@ -1,5 +1,4 @@
-#!/usr/bin/env bash
-# shellcheck disable=SC2086 # we want word splitting
+#!/bin/bash

 rootfs_dst=$1

@@ -9,29 +8,23 @@ mkdir -p $rootfs_dst/results
 cp $BM/bm-init.sh $rootfs_dst/init
 cp $CI_COMMON/init*.sh $rootfs_dst/

-date +'%F %T'
-
 # Make JWT token available as file in the bare-metal storage to enable access
 # to MinIO
 cp "${CI_JOB_JWT_FILE}" "${rootfs_dst}${CI_JOB_JWT_FILE}"

-date +'%F %T'
-
 cp $CI_COMMON/capture-devcoredump.sh $rootfs_dst/
 cp $CI_COMMON/intel-gpu-freq.sh $rootfs_dst/
-cp $CI_COMMON/kdl.sh $rootfs_dst/
-cp "$SCRIPTS_DIR/setup-test-env.sh" "$rootfs_dst/"

 set +x

 # Pass through relevant env vars from the gitlab job to the baremetal init script
+"$CI_COMMON"/generate-env.sh > $rootfs_dst/set-job-env-vars.sh
+chmod +x $rootfs_dst/set-job-env-vars.sh
 echo "Variables passed through:"
-"$CI_COMMON"/generate-env.sh | tee $rootfs_dst/set-job-env-vars.sh
+cat $rootfs_dst/set-job-env-vars.sh

 set -x

 # Add the Mesa drivers we built, and make a consistent symlink to them.
 mkdir -p $rootfs_dst/$CI_PROJECT_DIR
 rsync -aH --delete $CI_PROJECT_DIR/install/ $rootfs_dst/$CI_PROJECT_DIR/install/
-
-date +'%F %T'
--- a/.gitlab-ci/build/compiler-wrapper-clang++-15.sh
+++ b/.gitlab-ci/build/compiler-wrapper-clang++-15.sh
@@ -1,7 +0,0 @@
-#!/bin/sh
-# shellcheck disable=SC1091
-
-set -e
-
-_COMPILER=clang++-15
-. compiler-wrapper.sh
--- a/.gitlab-ci/build/compiler-wrapper-clang++.sh
+++ b/.gitlab-ci/build/compiler-wrapper-clang++.sh
@@ -1,7 +0,0 @@
-#!/bin/sh
-# shellcheck disable=SC1091
-
-set -e
-
-_COMPILER=clang++
-. compiler-wrapper.sh
--- a/.gitlab-ci/build/compiler-wrapper-clang-15.sh
+++ b/.gitlab-ci/build/compiler-wrapper-clang-15.sh
@@ -1,7 +0,0 @@
-#!/bin/sh
-# shellcheck disable=SC1091
-
-set -e
-
-_COMPILER=clang-15
-. compiler-wrapper.sh
--- a/.gitlab-ci/build/compiler-wrapper-clang.sh
+++ b/.gitlab-ci/build/compiler-wrapper-clang.sh
@@ -1,7 +0,0 @@
-#!/bin/sh
-# shellcheck disable=SC1091
-
-set -e
-
-_COMPILER=clang
-. compiler-wrapper.sh
--- a/.gitlab-ci/build/compiler-wrapper-g++.sh
+++ b/.gitlab-ci/build/compiler-wrapper-g++.sh
@@ -1,7 +0,0 @@
-#!/bin/sh
-# shellcheck disable=SC1091
-
-set -e
-
-_COMPILER=g++
-. compiler-wrapper.sh
--- a/.gitlab-ci/build/compiler-wrapper-gcc.sh
+++ b/.gitlab-ci/build/compiler-wrapper-gcc.sh
@@ -1,7 +0,0 @@
-#!/bin/sh
-# shellcheck disable=SC1091
-
-set -e
-
-_COMPILER=gcc
-. compiler-wrapper.sh
--- a/.gitlab-ci/build/compiler-wrapper.sh
+++ b/.gitlab-ci/build/compiler-wrapper.sh
@@ -1,21 +0,0 @@
-# shellcheck disable=SC1091
-# shellcheck disable=SC2086 # we want word splitting
-if command -V ccache >/dev/null 2>/dev/null; then
-  CCACHE=ccache
-else
-  CCACHE=
-fi
-
-if echo "$@" | grep -E 'meson-private/tmp[^ /]*/testfile.c' >/dev/null; then
-    # Invoked for meson feature check
-    exec $CCACHE $_COMPILER "$@"
-fi
-
-if [ "$(eval printf "'%s'" "\"\${$(($#-1))}\"")" = "-c" ]; then
-    # Not invoked for linking
-    exec $CCACHE $_COMPILER "$@"
-fi
-
-# Compiler invoked by ninja for linking. Add -Werror to turn compiler warnings into errors
-# with LTO. (meson's werror should arguably do this, but meanwhile we need to)
-exec $CCACHE $_COMPILER "$@" -Werror
--- a/.gitlab-ci/build/gitlab-ci.yml
+++ b/.gitlab-ci/build/gitlab-ci.yml
@@ -1,16 +1,8 @@
 # Shared between windows and Linux
 .build-common:
-  extends: .container+build-rules
+  extends: .build-rules
  # Cancel job if a newer commit is pushed to the same branch
  interruptible: true
-  # Build jobs don't take more than 1-3 minutes. 5-8 min max on a fresh runner
-  # without a populated ccache.
-  # These jobs are never slow, either they finish within reasonable time or
-  # something has gone wrong and the job will never terminate, so we should
-  # instead timeout so that the retry mechanism can kick in.
-  # A few exception are made, see `timeout:` overrides in the rest of this
-  # file.
-  timeout: 30m
  artifacts:
    name: "mesa_${CI_JOB_NAME}"
    when: always
@@ -18,7 +10,6 @@
      - _build/meson-logs/*.txt
      - _build/meson-logs/strace
      - shader-db
-      - artifacts

 # Just Linux
 .build-linux:
@@ -30,22 +21,24 @@
  # Use ccache transparently, and print stats before/after
  before_script:
    - !reference [default, before_script]
-    - |
-      export PATH="/usr/lib/ccache:$PATH"
-      export CCACHE_BASEDIR="$PWD"
-      if test -x /usr/bin/ccache; then
-        section_start ccache_before "ccache stats before build"
-        ccache --show-stats
-        section_end ccache_before
-      fi
+    - export PATH="/usr/lib/ccache:$PATH"
+    - export CCACHE_BASEDIR="$PWD"
+    - echo -e "\e[0Ksection_start:$(date +%s):ccache_before[collapsed=true]\r\e[0Kccache stats before build"
+    - ccache --show-stats
+    - echo -e "\e[0Ksection_end:$(date +%s):ccache_before\r\e[0K"
  after_script:
-    - if test -x /usr/bin/ccache; then ccache --show-stats | grep "Hits:"; fi
+    - echo -e "\e[0Ksection_start:$(date +%s):ccache_after[collapsed=true]\r\e[0Kccache stats after build"
+    - ccache --show-stats
+    - echo -e "\e[0Ksection_end:$(date +%s):ccache_after\r\e[0K"
    - !reference [default, after_script]

 .build-windows:
-  extends:
-    - .build-common
-    - .windows-docker-tags
+  extends: .build-common
+  tags:
+    - windows
+    - docker
+    - "2022"
+    - mesa
  cache:
    key: ${CI_JOB_NAME}
    paths:
@@ -54,10 +47,19 @@
 .meson-build:
  extends:
    - .build-linux
-    - .use-debian/x86_64_build
+    - .use-debian/x86_build
  stage: build-x86_64
  variables:
-    LLVM_VERSION: 15
+    LLVM_VERSION: 11
+  script:
+    - .gitlab-ci/meson/build.sh
+
+.meson-build_mingw:
+  extends:
+    - .build-linux
+    - .use-debian/x86_build_mingw
+    - .use-wine
+  stage: build-x86_64
  script:
    - .gitlab-ci/meson/build.sh

@@ -71,21 +73,18 @@ debian-testing:
      -D glx=dri
      -D gbm=enabled
      -D egl=enabled
-      -D platforms=x11,wayland
+      -D platforms=x11
    GALLIUM_ST: >
      -D dri3=enabled
      -D gallium-va=enabled
-      -D gallium-rusticl=true
-    GALLIUM_DRIVERS: "swrast,virgl,radeonsi,zink,crocus,iris,i915,r300"
-    VULKAN_DRIVERS: "swrast,amd,intel,intel_hasvk,virtio,nouveau-experimental"
+    GALLIUM_DRIVERS: "swrast,virgl,radeonsi,zink,crocus,iris,i915"
+    VULKAN_DRIVERS: "swrast,amd,intel,virtio-experimental"
    BUILDTYPE: "debugoptimized"
    EXTRA_OPTION: >
      -D spirv-to-dxil=true
-      -D valgrind=disabled
-      -D perfetto=true
-      -D tools=drm-shim
-    S3_ARTIFACT_NAME: mesa-x86_64-default-${BUILDTYPE}
-    LLVM_VERSION: 15
+      -D valgrind=false
+    MINIO_ARTIFACT_NAME: mesa-amd64
+    LLVM_VERSION: "13"
  script:
    - .gitlab-ci/meson/build.sh
    - .gitlab-ci/prepare-artifacts.sh
@@ -101,14 +100,12 @@ debian-testing-asan:
      -Wno-error=stringop-truncation
    EXTRA_OPTION: >
      -D b_sanitize=address
-      -D valgrind=disabled
+      -D valgrind=false
      -D tools=dlclose-skip
-    S3_ARTIFACT_NAME: ""
+    MINIO_ARTIFACT_NAME: ""
    ARTIFACTS_DEBUG_SYMBOLS: 1

 debian-testing-msan:
-  # https://github.com/google/sanitizers/wiki/MemorySanitizerLibcxxHowTo
-  # msan cannot fully work until it's used together with msan libc
  extends:
    - debian-clang
  variables:
@@ -116,19 +113,54 @@ debian-testing-msan:
    EXTRA_OPTION:
      -D b_sanitize=memory
      -D b_lundef=false
-    S3_ARTIFACT_NAME: ""
+    MINIO_ARTIFACT_NAME: ""
    ARTIFACTS_DEBUG_SYMBOLS: 1
    # Don't run all the tests yet:
    # GLSL has some issues in sexpression reading.
    # gtest has issues in its test initialization.
-    MESON_TEST_ARGS: "--suite glcpp --suite format"
-    GALLIUM_DRIVERS: "freedreno,iris,nouveau,kmsro,r300,r600,swrast,svga,v3d,vc4,virgl,etnaviv,panfrost,lima,zink,radeonsi,tegra,d3d12,crocus"
-    VULKAN_DRIVERS: intel,amd,broadcom,virtio
+    MESON_TEST_ARGS: "--suite glcpp --suite gallium  --suite format"
+    # Freedreno dropped because freedreno tools fail at msan.
+    GALLIUM_DRIVERS: "iris,nouveau,kmsro,r300,r600,swrast,svga,v3d,vc4,virgl,etnaviv,panfrost,lima,zink,radeonsi,tegra,d3d12,crocus"
+    VULKAN_DRIVERS: intel,amd,broadcom,virtio-experimental
+
+.debian-cl-testing:
+  extends:
+    - .meson-build
+    - .ci-deqp-artifacts
+  variables:
+    LLVM_VERSION: "13"
+    UNWIND: "enabled"
+    DRI_LOADERS: >
+      -D glx=disabled
+      -D egl=disabled
+      -D gbm=disabled
+    GALLIUM_DRIVERS: "swrast"
+    BUILDTYPE: "debugoptimized"
+    EXTRA_OPTION: >
+      -D valgrind=false
+  script:
+    - .gitlab-ci/meson/build.sh
+    - .gitlab-ci/prepare-artifacts.sh
+
+debian-clover-testing:
+  extends:
+    - .debian-cl-testing
+  variables:
+    GALLIUM_ST: >
+      -D gallium-opencl=icd
+      -D opencl-spirv=true
+
+debian-rusticl-testing:
+  extends:
+    - .debian-cl-testing
+  variables:
+    GALLIUM_ST: >
+      -D gallium-rusticl=true
+      -D opencl-spirv=true

 debian-build-testing:
  extends: .meson-build
  variables:
-    BUILDTYPE: debug
    UNWIND: "enabled"
    DRI_LOADERS: >
      -D glx=dri
@@ -143,6 +175,7 @@ debian-build-testing:
      -D gallium-va=enabled
      -D gallium-xa=enabled
      -D gallium-nine=true
+      -D gallium-opencl=disabled
      -D gallium-rusticl=false
    GALLIUM_DRIVERS: "iris,nouveau,kmsro,r300,r600,freedreno,swrast,svga,v3d,vc4,virgl,etnaviv,panfrost,lima,zink,d3d12,asahi,crocus"
    VULKAN_DRIVERS: swrast
@@ -151,28 +184,20 @@ debian-build-testing:
      -D osmesa=true
      -D tools=drm-shim,etnaviv,freedreno,glsl,intel,intel-ui,nir,nouveau,lima,panfrost,asahi
      -D b_lto=true
-    LLVM_VERSION: 15
-  script: |
-    section_start lava-pytest "lava-pytest"
-    .gitlab-ci/lava/lava-pytest.sh
-    section_switch shellcheck "shellcheck"
-    .gitlab-ci/run-shellcheck.sh
-    section_switch yamllint "yamllint"
-    .gitlab-ci/run-yamllint.sh
-    section_switch meson "meson"
-    .gitlab-ci/meson/build.sh
-    section_switch shader-db "shader-db"
-    .gitlab-ci/run-shader-db.sh
-  timeout: 30m
+    LLVM_VERSION: 13
+  script:
+    - .gitlab-ci/lava/lava-pytest.sh
+    - .gitlab-ci/run-shellcheck.sh
+    - .gitlab-ci/run-yamllint.sh
+    - .gitlab-ci/meson/build.sh
+    - .gitlab-ci/run-shader-db.sh

 # Test a release build with -Werror so new warnings don't sneak in.
 debian-release:
  extends: .meson-build
  variables:
-    LLVM_VERSION: 15
+    LLVM_VERSION: "13"
    UNWIND: "enabled"
-    C_ARGS: >
-      -Wno-error=stringop-overread
    DRI_LOADERS: >
      -D glx=dri
      -D gbm=enabled
@@ -186,26 +211,25 @@ debian-release:
      -D gallium-va=enabled
      -D gallium-xa=enabled
      -D gallium-nine=false
+      -D gallium-opencl=disabled
      -D gallium-rusticl=false
      -D llvm=enabled
    GALLIUM_DRIVERS: "i915,iris,nouveau,kmsro,freedreno,r300,svga,swrast,v3d,vc4,virgl,etnaviv,panfrost,lima,zink,d3d12,crocus"
    VULKAN_DRIVERS: "amd,imagination-experimental,microsoft-experimental"
+    BUILDTYPE: "release"
    EXTRA_OPTION: >
      -D spirv-to-dxil=true
      -D osmesa=true
      -D tools=all
      -D intel-clc=enabled
      -D imagination-srv=true
-    BUILDTYPE: "release"
-    S3_ARTIFACT_NAME: "mesa-x86_64-default-${BUILDTYPE}"
  script:
    - .gitlab-ci/meson/build.sh
-    - 'if [ -n "$MESA_CI_PERFORMANCE_ENABLED" ]; then .gitlab-ci/prepare-artifacts.sh; fi'

 alpine-build-testing:
  extends:
    - .meson-build
-    - .use-alpine/x86_64_build
+    - .use-alpine/x86_build
  stage: build-x86_64
  variables:
    BUILDTYPE: "release"
@@ -237,21 +261,17 @@ alpine-build-testing:
      -D shared-llvm=enabled
    UNWIND: "disabled"
    VULKAN_DRIVERS: "amd,broadcom,freedreno,intel,imagination-experimental"
+  script:
+    - .gitlab-ci/meson/build.sh

 fedora-release:
  extends:
    - .meson-build
-    - .use-fedora/x86_64_build
+    - .use-fedora/x86_build
  variables:
    BUILDTYPE: "release"
-    C_LINK_ARGS: >
-      -Wno-error=stringop-overflow
-      -Wno-error=stringop-overread
-    CPP_ARGS: >
-      -Wno-error=dangling-reference
-      -Wno-error=overloaded-virtual
-    CPP_LINK_ARGS: >
-      -Wno-error=stringop-overflow
+    C_ARGS: >
+      -Wno-error=array-bounds
      -Wno-error=stringop-overread
    DRI_LOADERS: >
      -D glx=dri
@@ -259,15 +279,15 @@ fedora-release:
      -D egl=enabled
      -D glvnd=true
      -D platforms=x11,wayland
+    # intel-clc disabled, we need llvm-spirv-translator 13.0+, Fedora 34 only packages 12.0.
    EXTRA_OPTION: >
-      -D b_lto=true
      -D osmesa=true
      -D selinux=true
      -D tools=drm-shim,etnaviv,freedreno,glsl,intel,nir,nouveau,lima,panfrost,imagination
      -D vulkan-layers=device-select,overlay
-      -D intel-clc=enabled
+      -D intel-clc=disabled
      -D imagination-srv=true
-    GALLIUM_DRIVERS: "crocus,etnaviv,freedreno,i915,iris,kmsro,lima,nouveau,panfrost,r300,r600,radeonsi,svga,swrast,tegra,v3d,vc4,virgl,zink"
+    GALLIUM_DRIVERS: "crocus,etnaviv,freedreno,iris,kmsro,lima,nouveau,panfrost,r300,r600,radeonsi,svga,swrast,tegra,v3d,vc4,virgl,zink"
    GALLIUM_ST: >
      -D dri3=enabled
      -D gallium-extra-hud=true
@@ -276,7 +296,8 @@ fedora-release:
      -D gallium-va=enabled
      -D gallium-xa=enabled
      -D gallium-nine=false
-      -D gallium-rusticl=true
+      -D gallium-opencl=icd
+      -D gallium-rusticl=false
      -D gles1=disabled
      -D gles2=enabled
      -D llvm=enabled
@@ -284,27 +305,23 @@ fedora-release:
      -D shared-llvm=enabled
    LLVM_VERSION: ""
    UNWIND: "disabled"
-    VULKAN_DRIVERS: "amd,broadcom,freedreno,imagination-experimental,intel,intel_hasvk"
+    VULKAN_DRIVERS: "amd,broadcom,freedreno,intel,imagination-experimental"
+  script:
+    - .gitlab-ci/meson/build.sh

 debian-android:
  extends:
    - .meson-cross
    - .use-debian/android_build
-    - .ci-deqp-artifacts
  variables:
-    BUILDTYPE: debug
    UNWIND: "disabled"
    C_ARGS: >
      -Wno-error=asm-operand-widths
      -Wno-error=constant-conversion
      -Wno-error=enum-conversion
      -Wno-error=initializer-overrides
+      -Wno-error=missing-braces
      -Wno-error=sometimes-uninitialized
-    CPP_ARGS: >
-      -Wno-error=c99-designator
-      -Wno-error=unused-variable
-      -Wno-error=unused-but-set-variable
-      -Wno-error=self-assign
    DRI_LOADERS: >
      -D glx=disabled
      -D gbm=disabled
@@ -313,9 +330,8 @@ debian-android:
    EXTRA_OPTION: >
      -D android-stub=true
      -D llvm=disabled
-      -D platform-sdk-version=33
-      -D valgrind=disabled
-      -D android-libbacktrace=disabled
+      -D platform-sdk-version=29
+      -D valgrind=false
    GALLIUM_ST: >
      -D dri3=disabled
      -D gallium-vdpau=disabled
@@ -323,20 +339,18 @@ debian-android:
      -D gallium-va=disabled
      -D gallium-xa=disabled
      -D gallium-nine=false
+      -D gallium-opencl=disabled
      -D gallium-rusticl=false
    LLVM_VERSION: ""
    PKG_CONFIG_LIBDIR: "/disable/non/android/system/pc/files"
-    ARTIFACTS_DEBUG_SYMBOLS: 1
-    S3_ARTIFACT_NAME: mesa-x86_64-android-${BUILDTYPE}
  script:
-    - CROSS=aarch64-linux-android GALLIUM_DRIVERS=etnaviv,freedreno,lima,panfrost,vc4,v3d VULKAN_DRIVERS=freedreno,broadcom,virtio .gitlab-ci/meson/build.sh
+    - PKG_CONFIG_PATH=/usr/local/lib/aarch64-linux-android/pkgconfig/:/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/pkgconfig/ CROSS=aarch64-linux-android GALLIUM_DRIVERS=etnaviv,freedreno,lima,panfrost,vc4,v3d VULKAN_DRIVERS=freedreno,broadcom,virtio-experimental .gitlab-ci/meson/build.sh
    # x86_64 build:
    # Can't do Intel because gen_decoder.c currently requires libexpat, which
    # is not a dependency that AOSP wants to accept.  Can't do Radeon Gallium
    # drivers because they requires LLVM, which we don't have an Android build
    # of.
-    - CROSS=x86_64-linux-android GALLIUM_DRIVERS=iris,virgl VULKAN_DRIVERS=amd,intel .gitlab-ci/meson/build.sh
-    - .gitlab-ci/prepare-artifacts.sh
+    - PKG_CONFIG_PATH=/usr/local/lib/x86_64-linux-android/pkgconfig/:/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/x86_64-linux-android/pkgconfig/ CROSS=x86_64-linux-android GALLIUM_DRIVERS=iris VULKAN_DRIVERS=amd,intel .gitlab-ci/meson/build.sh

 .meson-cross:
  extends:
@@ -361,9 +375,9 @@ debian-android:
 .meson-arm:
  extends:
    - .meson-cross
-    - .use-debian/arm64_build
+    - .use-debian/arm_build
  needs:
-    - debian/arm64_build
+    - debian/arm_build
  variables:
    VULKAN_DRIVERS: freedreno,broadcom
    GALLIUM_DRIVERS: "etnaviv,freedreno,kmsro,lima,nouveau,panfrost,swrast,tegra,v3d,vc4,zink"
@@ -371,7 +385,7 @@ debian-android:
  tags:
    - aarch64

-debian-arm32:
+debian-armhf:
  extends:
    - .meson-arm
    - .ci-deqp-artifacts
@@ -379,8 +393,8 @@ debian-arm32:
    CROSS: armhf
    EXTRA_OPTION: >
      -D llvm=disabled
-      -D valgrind=disabled
-    S3_ARTIFACT_NAME: mesa-arm32-default-${BUILDTYPE}
+      -D valgrind=false
+    MINIO_ARTIFACT_NAME: mesa-armhf
    # The strip command segfaults, failing to strip the binary and leaving
    # tempfiles in our artifacts.
    ARTIFACTS_DEBUG_SYMBOLS: 1
@@ -388,35 +402,17 @@ debian-arm32:
    - .gitlab-ci/meson/build.sh
    - .gitlab-ci/prepare-artifacts.sh

-debian-arm32-asan:
-  extends:
-    - debian-arm32
-  variables:
-    EXTRA_OPTION: >
-      -D llvm=disabled
-      -D b_sanitize=address
-      -D valgrind=disabled
-      -D tools=dlclose-skip
-    ARTIFACTS_DEBUG_SYMBOLS: 1
-    S3_ARTIFACT_NAME: mesa-arm32-asan-${BUILDTYPE}
-    MESON_TEST_ARGS: "--no-suite mesa:compiler --no-suite mesa:util"
-
 debian-arm64:
  extends:
    - .meson-arm
    - .ci-deqp-artifacts
  variables:
-    C_ARGS: >
-      -Wno-error=array-bounds
-      -Wno-error=stringop-truncation
    VULKAN_DRIVERS: "freedreno,broadcom,panfrost,imagination-experimental"
    EXTRA_OPTION: >
      -D llvm=disabled
-      -D valgrind=disabled
+      -D valgrind=false
      -D imagination-srv=true
-      -D perfetto=true
-      -D freedreno-kmds=msm,virtio
-    S3_ARTIFACT_NAME: mesa-arm64-default-${BUILDTYPE}
+    MINIO_ARTIFACT_NAME: mesa-arm64
  script:
    - .gitlab-ci/meson/build.sh
    - .gitlab-ci/prepare-artifacts.sh
@@ -428,10 +424,10 @@ debian-arm64-asan:
    EXTRA_OPTION: >
      -D llvm=disabled
      -D b_sanitize=address
-      -D valgrind=disabled
+      -D valgrind=false
      -D tools=dlclose-skip
    ARTIFACTS_DEBUG_SYMBOLS: 1
-    S3_ARTIFACT_NAME: mesa-arm64-asan-${BUILDTYPE}
+    MINIO_ARTIFACT_NAME: mesa-arm64-asan
    MESON_TEST_ARGS: "--no-suite mesa:compiler"

 debian-arm64-build-test:
@@ -442,35 +438,24 @@ debian-arm64-build-test:
    VULKAN_DRIVERS: "amd"
    EXTRA_OPTION: >
      -Dtools=panfrost,imagination
-
-debian-arm64-release:
-  extends:
-    - debian-arm64
-  variables:
-    BUILDTYPE: release
-    S3_ARTIFACT_NAME: mesa-arm64-default-${BUILDTYPE}
-    C_ARGS: >
-      -Wno-error=array-bounds
-      -Wno-error=stringop-truncation
-      -Wno-error=stringop-overread
  script:
    - .gitlab-ci/meson/build.sh
-    - 'if [ -n "$MESA_CI_PERFORMANCE_ENABLED" ]; then .gitlab-ci/prepare-artifacts.sh; fi'

 debian-clang:
  extends: .meson-build
  variables:
-    BUILDTYPE: debug
-    LLVM_VERSION: 15
+    LLVM_VERSION: "13"
    UNWIND: "enabled"
+    GALLIUM_DUMP_CPU: "true"
    C_ARGS: >
      -Wno-error=constant-conversion
      -Wno-error=enum-conversion
+      -Wno-error=implicit-const-int-float-conversion
      -Wno-error=initializer-overrides
      -Wno-error=sometimes-uninitialized
-      -Werror=misleading-indentation
    CPP_ARGS: >
      -Wno-error=c99-designator
+      -Wno-error=implicit-const-int-float-conversion
      -Wno-error=overloaded-virtual
      -Wno-error=tautological-constant-out-of-range-compare
      -Wno-error=unused-private-field
@@ -488,15 +473,15 @@ debian-clang:
      -D gallium-va=enabled
      -D gallium-xa=enabled
      -D gallium-nine=true
+      -D gallium-opencl=icd
      -D gles1=enabled
      -D gles2=enabled
      -D llvm=enabled
-      -D microsoft-clc=disabled
+      -D microsoft-clc=enabled
      -D shared-llvm=enabled
      -D opencl-spirv=true
-      -D shared-glapi=enabled
    GALLIUM_DRIVERS: "iris,nouveau,kmsro,r300,r600,freedreno,swrast,svga,v3d,vc4,virgl,etnaviv,panfrost,lima,zink,radeonsi,tegra,d3d12,crocus,i915,asahi"
-    VULKAN_DRIVERS: intel,amd,freedreno,broadcom,virtio,swrast,panfrost,imagination-experimental,microsoft-experimental,nouveau-experimental
+    VULKAN_DRIVERS: intel,amd,freedreno,broadcom,virtio-experimental,swrast,panfrost,imagination-experimental,microsoft-experimental
    EXTRA_OPTION:
      -D spirv-to-dxil=true
      -D osmesa=true
@@ -506,8 +491,8 @@ debian-clang:
      -D build-aco-tests=true
      -D intel-clc=enabled
      -D imagination-srv=true
-    CC: clang-${LLVM_VERSION}
-    CXX: clang++-${LLVM_VERSION}
+    CC: clang
+    CXX: clang++

 debian-clang-release:
  extends: debian-clang
@@ -516,26 +501,11 @@ debian-clang-release:
    DRI_LOADERS: >
      -D glx=xlib
      -D platforms=x11,wayland
-    GALLIUM_ST: >
-      -D dri3=enabled
-      -D gallium-extra-hud=true
-      -D gallium-vdpau=enabled
-      -D gallium-omx=bellagio
-      -D gallium-va=enabled
-      -D gallium-xa=enabled
-      -D gallium-nine=true
-      -D gles1=disabled
-      -D gles2=disabled
-      -D llvm=enabled
-      -D microsoft-clc=disabled
-      -D shared-llvm=enabled
-      -D opencl-spirv=true
-      -D shared-glapi=disabled

-windows-msvc:
+windows-vs2019:
  extends:
    - .build-windows
-    - .use-windows_build_msvc
+    - .use-windows_build_vs2019
    - .windows-build-rules
  stage: build-misc
  script:
@@ -545,19 +515,55 @@ windows-msvc:
      - _build/meson-logs/*.txt
      - _install/

+.debian-cl:
+  extends: .meson-build
+  variables:
+    LLVM_VERSION: "13"
+    UNWIND: "enabled"
+    DRI_LOADERS: >
+      -D glx=disabled
+      -D egl=disabled
+      -D gbm=disabled
+    EXTRA_OPTION: >
+      -D valgrind=false
+
+debian-clover:
+  extends: .debian-cl
+  variables:
+    GALLIUM_DRIVERS: "r600,radeonsi,swrast"
+    GALLIUM_ST: >
+      -D dri3=disabled
+      -D gallium-vdpau=disabled
+      -D gallium-omx=disabled
+      -D gallium-va=disabled
+      -D gallium-xa=disabled
+      -D gallium-nine=false
+      -D gallium-opencl=icd
+      -D gallium-rusticl=false
+
+debian-rusticl:
+  extends: .debian-cl
+  variables:
+    GALLIUM_DRIVERS: "iris,swrast"
+    GALLIUM_ST: >
+      -D dri3=disabled
+      -D gallium-vdpau=disabled
+      -D gallium-omx=disabled
+      -D gallium-va=disabled
+      -D gallium-xa=disabled
+      -D gallium-nine=false
+      -D gallium-opencl=disabled
+      -D gallium-rusticl=true
+
 debian-vulkan:
  extends: .meson-build
  variables:
-    BUILDTYPE: debug
-    LLVM_VERSION: 15
+    LLVM_VERSION: "13"
    UNWIND: "disabled"
    DRI_LOADERS: >
      -D glx=disabled
      -D gbm=disabled
      -D egl=disabled
-      -D opengl=false
-      -D gles1=disabled
-      -D gles2=disabled
      -D platforms=x11,wayland
      -D osmesa=false
    GALLIUM_ST: >
@@ -567,28 +573,28 @@ debian-vulkan:
      -D gallium-va=disabled
      -D gallium-xa=disabled
      -D gallium-nine=false
+      -D gallium-opencl=disabled
      -D gallium-rusticl=false
      -D b_sanitize=undefined
      -D c_args=-fno-sanitize-recover=all
      -D cpp_args=-fno-sanitize-recover=all
    UBSAN_OPTIONS: "print_stacktrace=1"
-    VULKAN_DRIVERS: amd,broadcom,freedreno,intel,intel_hasvk,virtio,imagination-experimental,microsoft-experimental
+    VULKAN_DRIVERS: intel,amd,freedreno,broadcom,virtio-experimental,imagination-experimental,microsoft-experimental
    EXTRA_OPTION: >
      -D vulkan-layers=device-select,overlay
      -D build-aco-tests=true
      -D intel-clc=disabled
      -D imagination-srv=true

-debian-x86_32:
+debian-i386:
  extends:
    - .meson-cross
-    - .use-debian/x86_32_build
+    - .use-debian/i386_build
  variables:
-    BUILDTYPE: debug
    CROSS: i386
-    VULKAN_DRIVERS: intel,amd,swrast,virtio
-    GALLIUM_DRIVERS: "iris,nouveau,r300,r600,radeonsi,swrast,virgl,zink,crocus,d3d12"
-    LLVM_VERSION: 15
+    VULKAN_DRIVERS: intel,amd,swrast,virtio-experimental
+    GALLIUM_DRIVERS: "iris,nouveau,r300,r600,radeonsi,swrast,virgl,zink,crocus"
+    LLVM_VERSION: 13
    EXTRA_OPTION: >
      -D vulkan-layers=device-select,overlay

@@ -602,7 +608,7 @@ debian-s390x:
  variables:
    CROSS: s390x
    GALLIUM_DRIVERS: "swrast,zink"
-    LLVM_VERSION: 15
+    LLVM_VERSION: 13
    VULKAN_DRIVERS: "swrast"

 debian-ppc64el:
@@ -611,7 +617,48 @@ debian-ppc64el:
    - .use-debian/ppc64el_build
    - .ppc64el-rules
  variables:
-    BUILDTYPE: debug
    CROSS: ppc64el
    GALLIUM_DRIVERS: "nouveau,radeonsi,swrast,virgl,zink"
    VULKAN_DRIVERS: "amd,swrast"
+
+debian-mingw32-x86_64:
+  extends: .meson-build_mingw
+  stage: build-misc
+  variables:
+    UNWIND: "disabled"
+    C_ARGS: >
+      -Wno-error=format
+      -Wno-error=format-extra-args
+      -Wno-error=deprecated-declarations
+      -Wno-error=unused-function
+      -Wno-error=unused-variable
+      -Wno-error=unused-but-set-variable
+      -Wno-error=unused-value
+      -Wno-error=switch
+      -Wno-error=parentheses
+      -Wno-error=missing-prototypes
+      -Wno-error=sign-compare
+      -Wno-error=narrowing
+      -Wno-error=overflow
+    CPP_ARGS: $C_ARGS
+    GALLIUM_DRIVERS: "swrast,d3d12,zink"
+    VULKAN_DRIVERS: "swrast,amd,microsoft-experimental"
+    GALLIUM_ST: >
+      -D gallium-opencl=icd
+      -D gallium-rusticl=false
+      -D opencl-spirv=true
+      -D microsoft-clc=enabled
+      -D static-libclc=all
+      -D llvm=enabled
+      -D gallium-va=true
+      -D video-codecs=h264dec,h264enc,h265dec,h265enc,vc1dec
+    EXTRA_OPTION: >
+      -D min-windows-version=7
+      -D spirv-to-dxil=true
+      -D gles1=enabled
+      -D gles2=enabled
+      -D osmesa=true
+      -D cpp_rtti=true
+      -D shared-glapi=enabled
+      -D zlib=enabled
+      --cross-file=.gitlab-ci/x86_64-w64-mingw32
--- a/.gitlab-ci/common/capture-devcoredump.sh
+++ b/.gitlab-ci/common/capture-devcoredump.sh
@@ -1,10 +1,7 @@
-#!/usr/bin/env bash
-# shellcheck disable=SC2035
-# shellcheck disable=SC2061
-# shellcheck disable=SC2086 # we want word splitting
+#!/bin/sh

 while true; do
-  devcds=$(find /sys/devices/virtual/devcoredump/ -name data 2>/dev/null)
+  devcds=`find /sys/devices/virtual/devcoredump/ -name data 2>/dev/null`
  for i in $devcds; do
    echo "Found a devcoredump at $i."
    if cp $i /results/first.devcore; then
@@ -13,23 +10,5 @@ while true; do
      exit 0
    fi
  done
-  i915_error_states=$(find /sys/devices/ -path */drm/card*/error)
-  for i in $i915_error_states; do
-    tmpfile=$(mktemp)
-    cp "$i" "$tmpfile"
-    filesize=$(stat --printf="%s" "$tmpfile")
-    # Does the file contain "No error state collected" ?
-    if [ "$filesize" = 25 ]; then
-        rm "$tmpfile"
-    else
-        echo "Found an i915 error state at $i size=$filesize."
-        if cp "$tmpfile" /results/first.i915_error_state; then
-            rm "$tmpfile"
-            echo 1 > "$i"
-            echo "Saved to the job artifacts at /first.i915_error_state"
-            exit 0
-        fi
-    fi
-  done
  sleep 10
 done
--- a/.gitlab-ci/common/generate-env.sh
+++ b/.gitlab-ci/common/generate-env.sh
@@ -2,7 +2,6 @@

 for var in \
    ACO_DEBUG \
-    ARTIFACTS_BASE_URL \
    ASAN_OPTIONS \
    BASE_SYSTEM_FORK_HOST_PREFIX \
    BASE_SYSTEM_MAINLINE_HOST_PREFIX \
@@ -11,7 +10,6 @@ for var in \
    CI_COMMIT_TITLE \
    CI_JOB_ID \
    CI_JOB_JWT_FILE \
-    CI_JOB_STARTED_AT \
    CI_JOB_NAME \
    CI_JOB_URL \
    CI_MERGE_REQUEST_SOURCE_BRANCH_NAME \
@@ -29,7 +27,6 @@ for var in \
    CI_SERVER_URL \
    CROSVM_GALLIUM_DRIVER \
    CROSVM_GPU_ARGS \
-    CURRENT_SECTION \
    DEQP_BIN_DIR \
    DEQP_CONFIG \
    DEQP_EXPECTED_RENDERER \
@@ -70,7 +67,7 @@ for var in \
    JOB_ARTIFACTS_BASE \
    JOB_RESULTS_PATH \
    JOB_ROOTFS_OVERLAY_PATH \
-    KERNEL_IMAGE_BASE \
+    KERNEL_IMAGE_BASE_URL \
    KERNEL_IMAGE_NAME \
    LD_LIBRARY_PATH \
    LP_NUM_THREADS \
@@ -86,18 +83,19 @@ for var in \
    MESA_LOADER_DRIVER_OVERRIDE \
    MESA_TEMPLATES_COMMIT \
    MESA_VK_IGNORE_CONFORMANCE_WARNING \
-    S3_HOST \
-    S3_RESULTS_UPLOAD \
+    MESA_SPIRV_LOG_LEVEL \
+    MINIO_HOST \
+    MINIO_RESULTS_UPLOAD \
    NIR_DEBUG \
    PAN_I_WANT_A_BROKEN_VULKAN_DRIVER \
    PAN_MESA_DEBUG \
-    PANVK_DEBUG \
    PIGLIT_FRACTION \
    PIGLIT_NO_WINDOW \
    PIGLIT_OPTIONS \
    PIGLIT_PLATFORM \
    PIGLIT_PROFILES \
    PIGLIT_REPLAY_ARTIFACTS_BASE_URL \
+    PIGLIT_REPLAY_DESCRIPTION_FILE \
    PIGLIT_REPLAY_DEVICE_NAME \
    PIGLIT_REPLAY_EXTRA_ARGS \
    PIGLIT_REPLAY_LOOP_TIMES \
@@ -105,25 +103,18 @@ for var in \
    PIGLIT_REPLAY_SUBCOMMAND \
    PIGLIT_RESULTS \
    PIGLIT_TESTS \
-    PIGLIT_TRACES_FILE \
    PIPELINE_ARTIFACTS_BASE \
-    RADEON_DEBUG \
    RADV_DEBUG \
    RADV_PERFTEST \
    SKQP_ASSETS_DIR \
    SKQP_BACKENDS \
    TU_DEBUG \
-    USE_ANGLE \
    VIRGL_HOST_API \
    WAFFLE_PLATFORM \
    VK_CPU \
    VK_DRIVER \
    VK_ICD_FILENAMES \
    VKD3D_PROTON_RESULTS \
-    VKD3D_CONFIG \
-    ZINK_DESCRIPTORS \
-    ZINK_DEBUG \
-    LVP_POISON_MEMORY \
    ; do
  if [ -n "${!var+x}" ]; then
    echo "export $var=${!var@Q}"
--- a/.gitlab-ci/common/init-stage1.sh
+++ b/.gitlab-ci/common/init-stage1.sh
@@ -7,14 +7,12 @@ set -ex

 cd /

-findmnt --mountpoint /proc || mount -t proc none /proc
-findmnt --mountpoint /sys || mount -t sysfs none /sys
+mount -t proc none /proc
+mount -t sysfs none /sys
 mount -t debugfs none /sys/kernel/debug
-findmnt --mountpoint /dev || mount -t devtmpfs none /dev
+mount -t devtmpfs none /dev || echo possibly already mounted
 mkdir -p /dev/pts
 mount -t devpts devpts /dev/pts
-mkdir /dev/shm
-mount -t tmpfs -o noexec,nodev,nosuid tmpfs /dev/shm
 mount -t tmpfs tmpfs /tmp

 echo "nameserver 8.8.8.8" > /etc/resolv.conf
@@ -22,4 +20,4 @@ echo "nameserver 8.8.8.8" > /etc/resolv.conf

 # Set the time so we can validate certificates before we fetch anything;
 # however as not all DUTs have network, make this non-fatal.
-for _ in 1 2 3; do sntp -sS pool.ntp.org && break || sleep 2; done || true
+for i in 1 2 3; do sntp -sS pool.ntp.org && break || sleep 2; done || true
--- a/.gitlab-ci/common/init-stage2.sh
+++ b/.gitlab-ci/common/init-stage2.sh
@@ -1,11 +1,4 @@
-#!/bin/bash
-# shellcheck disable=SC1090
-# shellcheck disable=SC1091
-# shellcheck disable=SC2086 # we want word splitting
-# shellcheck disable=SC2155
-
-# Second-stage init, used to set up devices and our job environment before
-# running tests.
+#!/bin/sh

 # Make sure to kill itself and all the children process from this script on
 # exiting, since any console output may interfere with LAVA signals handling,
@@ -40,10 +33,10 @@ trap cleanup INT TERM EXIT
 BACKGROUND_PIDS=


-for path in '/dut-env-vars.sh' '/set-job-env-vars.sh' './set-job-env-vars.sh'; do
-    [ -f "$path" ] && source "$path"
-done
-. "$SCRIPTS_DIR"/setup-test-env.sh
+# Second-stage init, used to set up devices and our job environment before
+# running tests.
+
+. /set-job-env-vars.sh

 set -ex

@@ -54,7 +47,7 @@ set -ex

 # Set up ZRAM
 HWCI_ZRAM_SIZE=2G
-if /sbin/zramctl --find --size $HWCI_ZRAM_SIZE -a zstd; then
+if zramctl --find --size $HWCI_ZRAM_SIZE -a zstd; then
    mkswap /dev/zram0
    swapon /dev/zram0
    echo "zram: $HWCI_ZRAM_SIZE activated"
@@ -71,22 +64,17 @@ fi
 #
 if [ "$HWCI_KVM" = "true" ]; then
    unset KVM_KERNEL_MODULE
-    {
-      grep -qs '\bvmx\b' /proc/cpuinfo && KVM_KERNEL_MODULE=kvm_intel
-    } || {
-      grep -qs '\bsvm\b' /proc/cpuinfo && KVM_KERNEL_MODULE=kvm_amd
+    grep -qs '\bvmx\b' /proc/cpuinfo && KVM_KERNEL_MODULE=kvm_intel || {
+        grep -qs '\bsvm\b' /proc/cpuinfo && KVM_KERNEL_MODULE=kvm_amd
    }

-    {
-      [ -z "${KVM_KERNEL_MODULE}" ] && \
-      echo "WARNING: Failed to detect CPU virtualization extensions"
-    } || \
+    [ -z "${KVM_KERNEL_MODULE}" ] && \
+        echo "WARNING: Failed to detect CPU virtualization extensions" || \
        modprobe ${KVM_KERNEL_MODULE}

    mkdir -p /lava-files
-    curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-	-o "/lava-files/${KERNEL_IMAGE_NAME}" \
-        "${KERNEL_IMAGE_BASE}/amd64/${KERNEL_IMAGE_NAME}"
+    wget -S --progress=dot:giga -O /lava-files/${KERNEL_IMAGE_NAME} \
+        "${KERNEL_IMAGE_BASE_URL}/${KERNEL_IMAGE_NAME}"
 fi

 # Fix prefix confusion: the build installs to $CI_PROJECT_DIR, but we expect
@@ -95,11 +83,6 @@ ln -sf $CI_PROJECT_DIR/install /install
 export LD_LIBRARY_PATH=/install/lib
 export LIBGL_DRIVERS_PATH=/install/lib/dri

-# https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22495#note_1876691
-# The navi21 boards seem to have trouble with ld.so.cache, so try explicitly
-# telling it to look in /usr/local/lib.
-export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
-
 # Store Mesa's disk cache under /tmp, rather than sending it out over NFS.
 export XDG_CACHE_HOME=/tmp

@@ -111,29 +94,20 @@ if [ "$HWCI_FREQ_MAX" = "true" ]; then
  head -0 /dev/dri/renderD128

  # Disable GPU frequency scaling
-  DEVFREQ_GOVERNOR=$(find /sys/devices -name governor | grep gpu || true)
+  DEVFREQ_GOVERNOR=`find /sys/devices -name governor | grep gpu || true`
  test -z "$DEVFREQ_GOVERNOR" || echo performance > $DEVFREQ_GOVERNOR || true

  # Disable CPU frequency scaling
  echo performance | tee -a /sys/devices/system/cpu/cpufreq/policy*/scaling_governor || true

  # Disable GPU runtime power management
-  GPU_AUTOSUSPEND=$(find /sys/devices -name autosuspend_delay_ms | grep gpu | head -1)
+  GPU_AUTOSUSPEND=`find /sys/devices -name autosuspend_delay_ms | grep gpu | head -1`
  test -z "$GPU_AUTOSUSPEND" || echo -1 > $GPU_AUTOSUSPEND || true
  # Lock Intel GPU frequency to 70% of the maximum allowed by hardware
  # and enable throttling detection & reporting.
  # Additionally, set the upper limit for CPU scaling frequency to 65% of the
  # maximum permitted, as an additional measure to mitigate thermal throttling.
-  /intel-gpu-freq.sh -s 70% --cpu-set-max 65% -g all -d
-fi
-
-# Start a little daemon to capture sysfs records and produce a JSON file
-if [ -x /kdl.sh ]; then
-  echo "launch kdl.sh!"
-  /kdl.sh &
-  BACKGROUND_PIDS="$! $BACKGROUND_PIDS"
-else
-  echo "kdl.sh not found!"
+  ./intel-gpu-freq.sh -s 70% --cpu-set-max 65% -g all -d
 fi

 # Increase freedreno hangcheck timer because it's right at the edge of the
@@ -144,10 +118,8 @@ fi

 # Start a little daemon to capture the first devcoredump we encounter.  (They
 # expire after 5 minutes, so we poll for them).
-if [ -x /capture-devcoredump.sh ]; then
-  /capture-devcoredump.sh &
-  BACKGROUND_PIDS="$! $BACKGROUND_PIDS"
-fi
+/capture-devcoredump.sh &
+BACKGROUND_PIDS="$! $BACKGROUND_PIDS"

 # If we want Xorg to be running for the test, then we start it up before the
 # HWCI_TEST_SCRIPT because we need to use xinit to start X (otherwise
@@ -156,12 +128,12 @@ fi
 if [ -n "$HWCI_START_XORG" ]; then
  echo "touch /xorg-started; sleep 100000" > /xorg-script
  env \
-    VK_ICD_FILENAMES="/install/share/vulkan/icd.d/${VK_DRIVER}_icd.$(uname -m).json" \
+    VK_ICD_FILENAMES=/install/share/vulkan/icd.d/${VK_DRIVER}_icd.`uname -m`.json \
    xinit /bin/sh /xorg-script -- /usr/bin/Xorg -noreset -s 0 -dpms -logfile /Xorg.0.log &
  BACKGROUND_PIDS="$! $BACKGROUND_PIDS"

  # Wait for xorg to be ready for connections.
-  for _ in 1 2 3 4 5; do
+  for i in 1 2 3 4 5; do
    if [ -e /xorg-started ]; then
      break
    fi
@@ -171,27 +143,23 @@ if [ -n "$HWCI_START_XORG" ]; then
 fi

 if [ -n "$HWCI_START_WESTON" ]; then
-  WESTON_X11_SOCK="/tmp/.X11-unix/X0"
-  if [ -n "$HWCI_START_XORG" ]; then
-    echo "Please consider dropping HWCI_START_XORG and instead using Weston XWayland for testing."
-    WESTON_X11_SOCK="/tmp/.X11-unix/X1"
-  fi
-  export WAYLAND_DISPLAY=wayland-0
+  export XDG_RUNTIME_DIR=/run/user
+  mkdir -p $XDG_RUNTIME_DIR

-  # Display server is Weston Xwayland when HWCI_START_XORG is not set or Xorg when it's
+  # Xwayland to be used when HWCI_START_XORG is not set
  export DISPLAY=:0
  mkdir -p /tmp/.X11-unix

  env \
-    VK_ICD_FILENAMES="/install/share/vulkan/icd.d/${VK_DRIVER}_icd.$(uname -m).json" \
-    weston -Bheadless-backend.so --use-gl -Swayland-0 --xwayland --idle-time=0 &
-  BACKGROUND_PIDS="$! $BACKGROUND_PIDS"
-
-  while [ ! -S "$WESTON_X11_SOCK" ]; do sleep 1; done
+    VK_ICD_FILENAMES=/install/share/vulkan/icd.d/${VK_DRIVER}_icd.`uname -m`.json \
+    weston -Bheadless-backend.so --use-gl -Swayland-0 --xwayland &
+  export WAYLAND_DISPLAY=wayland-0
+  sleep 1
 fi

+RESULT=fail
 set +e
-bash -c ". $SCRIPTS_DIR/setup-test-env.sh && $HWCI_TEST_SCRIPT"
+sh -c "$HWCI_TEST_SCRIPT"
 EXIT_CODE=$?
 set -e

@@ -206,21 +174,17 @@ mv -f ${CI_PROJECT_DIR}/results ./ 2>/dev/null || true
 cleanup

 # upload artifacts
-if [ -n "$S3_RESULTS_UPLOAD" ]; then
+if [ -n "$MINIO_RESULTS_UPLOAD" ]; then
  tar --zstd -cf results.tar.zst results/;
-  ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" results.tar.zst https://"$S3_RESULTS_UPLOAD"/results.tar.zst;
+  ci-fairy s3cp --token-file "${CI_JOB_JWT_FILE}" results.tar.zst https://"$MINIO_RESULTS_UPLOAD"/results.tar.zst;
 fi

 # We still need to echo the hwci: mesa message, as some scripts rely on it, such
 # as the python ones inside the bare-metal folder
-[ ${EXIT_CODE} -eq 0 ] && RESULT=pass || RESULT=fail
+[ ${EXIT_CODE} -eq 0 ] && RESULT=pass

 set +x
-
-# Print the final result; both bare-metal and LAVA look for this string to get
-# the result of our run, so try really hard to get it out rather than losing
-# the run. The device gets shut down right at this point, and a630 seems to
-# enjoy corrupting the last line of serial output before shutdown.
-for _ in $(seq 0 3); do echo "hwci: mesa: $RESULT"; sleep 1; echo; done
-
+echo "hwci: mesa: $RESULT"
+# Sleep a bit to avoid kernel dump message interleave from LAVA ENDTC signal
+sleep 1
 exit $EXIT_CODE
--- a/.gitlab-ci/common/intel-gpu-freq.sh
+++ b/.gitlab-ci/common/intel-gpu-freq.sh
@@ -1,14 +1,4 @@
-#!/usr/bin/env bash
-# shellcheck disable=SC2013
-# shellcheck disable=SC2015
-# shellcheck disable=SC2034
-# shellcheck disable=SC2046
-# shellcheck disable=SC2059
-# shellcheck disable=SC2086 # we want word splitting
-# shellcheck disable=SC2154
-# shellcheck disable=SC2155
-# shellcheck disable=SC2162
-# shellcheck disable=SC2229
+#!/bin/sh
 #
 # This is an utility script to manage Intel GPU frequencies.
 # It can be used for debugging performance problems or trying to obtain a stable
@@ -203,7 +193,7 @@ compute_freq_set() {
        val=${FREQ_RPn}
        ;;
    *%)
-        val=$((${1%?} * FREQ_RP0 / 100))
+        val=$((${1%?} * ${FREQ_RP0} / 100))
        # Adjust freq to comply with 50 MHz increments
        val=$((val / 50 * 50))
        ;;
@@ -252,12 +242,12 @@ set_freq_max() {

    [ -z "${DRY_RUN}" ] || return 0

-    if ! printf "%s" ${SET_MAX_FREQ} | tee $(print_freq_sysfs_path max) \
-        $(print_freq_sysfs_path boost) > /dev/null;
-    then
+    printf "%s" ${SET_MAX_FREQ} | tee $(print_freq_sysfs_path max) \
+        $(print_freq_sysfs_path boost) > /dev/null
+    [ $? -eq 0 ] || {
        log ERROR "Failed to set GPU max frequency"
        return 1
-    fi
+    }
 }

 #
@@ -282,11 +272,11 @@ set_freq_min() {

    [ -z "${DRY_RUN}" ] || return 0

-    if ! printf "%s" ${SET_MIN_FREQ} > $(print_freq_sysfs_path min);
-    then
+    printf "%s" ${SET_MIN_FREQ} > $(print_freq_sysfs_path min)
+    [ $? -eq 0 ] || {
        log ERROR "Failed to set GPU min frequency"
        return 1
-    fi
+    }
 }

 #
@@ -505,7 +495,7 @@ compute_cpu_freq_set() {
        val=${CPU_FREQ_cpuinfo_min}
        ;;
    *%)
-        val=$((${1%?} * CPU_FREQ_cpuinfo_max / 100))
+        val=$((${1%?} * ${CPU_FREQ_cpuinfo_max} / 100))
        ;;
    *[!0-9]*)
        log ERROR "Cannot set CPU freq to invalid value: %s" "$1"
@@ -548,11 +538,11 @@ set_cpu_freq_max() {
    local pstate_info=$(printf "${CPU_PSTATE_SYSFS_PATTERN}" max_perf_pct)
    [ -e "${pstate_info}" ] && {
        log INFO "Setting intel_pstate max perf to %s" "${target_freq}%"
-        if ! printf "%s" "${target_freq}" > "${pstate_info}";
-	then
+        printf "%s" "${target_freq}" > "${pstate_info}"
+        [ $? -eq 0 ] || {
            log ERROR "Failed to set intel_pstate max perf"
            res=1
-	fi
+        }
    }

    local cpu_index
@@ -565,11 +555,11 @@ set_cpu_freq_max() {
        log INFO "Setting CPU%s max scaling freq to %s Hz" ${cpu_index} "${target_freq}"
        [ -n "${DRY_RUN}" ] && continue

-        if ! printf "%s" ${target_freq} > $(print_cpu_freq_sysfs_path scaling_max ${cpu_index});
-	then
+        printf "%s" ${target_freq} > $(print_cpu_freq_sysfs_path scaling_max ${cpu_index})
+        [ $? -eq 0 ] || {
            res=1
            log ERROR "Failed to set CPU%s max scaling frequency" ${cpu_index}
-	fi
+        }
    done

    return ${res}
--- a/.gitlab-ci/common/kdl.sh
+++ b/.gitlab-ci/common/kdl.sh
@@ -1,24 +0,0 @@
-#!/usr/bin/env bash
-# shellcheck disable=SC1091  # the path is created in build-kdl and
-# here is check if exist
-
-terminate() {
-  echo "ci-kdl.sh caught SIGTERM signal! propagating to child processes"
-  for job in $(jobs -p)
-  do
-    kill -15 "$job"
-  done
-}
-
-trap terminate SIGTERM
-
-if [ -f /ci-kdl.venv/bin/activate ]; then
-  source /ci-kdl.venv/bin/activate
-  /ci-kdl.venv/bin/python /ci-kdl.venv/bin/ci-kdl | tee -a /results/kdl.log &
-  child=$!
-  wait $child
-  mv kdl_*.json /results/kdl.json
-else
-  echo -e "Not possible to activate ci-kdl virtual environment"
-fi
-
--- a/.gitlab-ci/common/start-x.sh
+++ b/.gitlab-ci/common/start-x.sh
@@ -13,7 +13,7 @@ fi
 xinit /bin/sh "${_XORG_SCRIPT}" -- /usr/bin/Xorg vt45 -noreset -s 0 -dpms -logfile /Xorg.0.log &

 # Wait for xorg to be ready for connections.
-for _ in 1 2 3 4 5; do
+for i in 1 2 3 4 5; do
    if [ -e "${_FLAG_FILE}" ]; then
        break
    fi
--- a/.gitlab-ci/container/alpine/x86_64_build.sh
+++ b/.gitlab-ci/container/alpine/x86_64_build.sh
@@ -1,58 +0,0 @@
-#!/usr/bin/env bash
-# shellcheck disable=SC1091
-
-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# ALPINE_X86_64_BUILD_TAG
-
-set -e
-set -o xtrace
-
-EPHEMERAL=(
-)
-
-
-DEPS=(
-    bash
-    bison
-    ccache
-    cmake
-    clang-dev
-    coreutils
-    curl
-    flex
-    gcc
-    g++
-    git
-    gettext
-    glslang
-    linux-headers
-    llvm16-dev
-    meson
-    expat-dev
-    elfutils-dev
-    libdrm-dev
-    libselinux-dev
-    libva-dev
-    libpciaccess-dev
-    zlib-dev
-    python3-dev
-    py3-mako
-    py3-ply
-    vulkan-headers
-    spirv-tools-dev
-    util-macros
-    wayland-dev
-    wayland-protocols
-)
-
-apk --no-cache add "${DEPS[@]}" "${EPHEMERAL[@]}"
-
-. .gitlab-ci/container/container_pre_build.sh
-
-
-############### Uninstall the build software
-
-apk del "${EPHEMERAL[@]}"
-
-. .gitlab-ci/container/container_post_build.sh
--- a/.gitlab-ci/container/alpine/x86_64_lava_ssh_client.sh
+++ b/.gitlab-ci/container/alpine/x86_64_lava_ssh_client.sh
@@ -1,29 +0,0 @@
-#!/usr/bin/env bash
-
-# This is a ci-templates build script to generate a container for LAVA SSH client.
-
-# shellcheck disable=SC1091
-set -e
-set -o xtrace
-
-EPHEMERAL=(
-)
-
-# We only need these very basic packages to run the tests.
-DEPS=(
-    openssh-client  # for ssh
-    iputils         # for ping
-    bash
-    curl
-)
-
-
-apk --no-cache add "${DEPS[@]}" "${EPHEMERAL[@]}"
-
-. .gitlab-ci/container/container_pre_build.sh
-
-############### Uninstall the build software
-
-apk del "${EPHEMERAL[@]}"
-
-. .gitlab-ci/container/container_post_build.sh
--- a/.gitlab-ci/container/alpine/x86_build.sh
+++ b/.gitlab-ci/container/alpine/x86_build.sh
@@ -0,0 +1,70 @@
+#!/bin/bash
+# shellcheck disable=SC2086 # we want word splitting
+
+set -e
+set -o xtrace
+
+
+EPHEMERAL="
+        autoconf
+        automake
+        bzip2
+        cmake
+        git
+        libtool
+        libepoxy-dev
+        libtbb-dev
+        make
+        openssl-dev
+        unzip
+        wget
+        xz
+	zstd-dev
+        "
+
+apk add \
+    bison \
+    ccache \
+    clang-dev \
+    flex \
+    gcc \
+    g++ \
+    gettext \
+    glslang \
+    linux-headers \
+    llvm15-dev \
+    meson \
+    expat-dev \
+    elfutils-dev \
+    libselinux-dev \
+    libva-dev \
+    libpciaccess-dev \
+    zlib-dev \
+    python3-dev \
+    py3-mako \
+    py3-ply \
+    vulkan-headers \
+    spirv-tools-dev \
+    util-macros \
+    $EPHEMERAL
+
+
+. .gitlab-ci/container/container_pre_build.sh
+
+. .gitlab-ci/container/build-libdrm.sh
+
+. .gitlab-ci/container/build-wayland.sh
+
+pushd /usr/local
+git clone https://gitlab.freedesktop.org/mesa/shader-db.git --depth 1
+rm -rf shader-db/.git
+cd shader-db
+make
+popd
+
+
+############### Uninstall the build software
+
+apk del $EPHEMERAL
+
+. .gitlab-ci/container/container_post_build.sh
--- a/.gitlab-ci/container/arm.config
+++ b/.gitlab-ci/container/arm.config
@@ -0,0 +1,69 @@
+CONFIG_LOCALVERSION_AUTO=y
+CONFIG_DEBUG_KERNEL=y
+
+CONFIG_CRYPTO_ZSTD=y
+CONFIG_ZRAM_MEMORY_TRACKING=y
+CONFIG_ZRAM_WRITEBACK=y
+CONFIG_ZRAM=y
+CONFIG_ZSMALLOC_STAT=y
+
+# abootimg with a 'dummy' rootfs fails with root=/dev/nfs
+CONFIG_BLK_DEV_INITRD=n
+
+CONFIG_DEVFREQ_GOV_PERFORMANCE=y
+CONFIG_DEVFREQ_GOV_POWERSAVE=y
+CONFIG_DEVFREQ_GOV_USERSPACE=y
+CONFIG_DEVFREQ_GOV_PASSIVE=y
+CONFIG_DEVFREQ_GOV_SIMPLE_ONDEMAND=y
+
+CONFIG_DRM=y
+CONFIG_DRM_ETNAVIV=y
+CONFIG_DRM_ROCKCHIP=y
+CONFIG_DRM_PANFROST=y
+CONFIG_DRM_LIMA=y
+CONFIG_DRM_PANEL_SIMPLE=y
+CONFIG_PWM_CROS_EC=y
+CONFIG_BACKLIGHT_PWM=y
+
+CONFIG_ROCKCHIP_CDN_DP=n
+
+CONFIG_SPI_ROCKCHIP=y
+CONFIG_PWM_ROCKCHIP=y
+CONFIG_PHY_ROCKCHIP_DP=y
+CONFIG_DWMAC_ROCKCHIP=y
+
+CONFIG_MFD_RK808=y
+CONFIG_REGULATOR_RK808=y
+CONFIG_RTC_DRV_RK808=y
+CONFIG_COMMON_CLK_RK808=y
+
+CONFIG_REGULATOR_FAN53555=y
+CONFIG_REGULATOR=y
+
+CONFIG_REGULATOR_VCTRL=y
+
+CONFIG_KASAN=n
+CONFIG_KASAN_INLINE=n
+CONFIG_STACKTRACE=n
+
+CONFIG_TMPFS=y
+
+CONFIG_PROVE_LOCKING=n
+CONFIG_DEBUG_LOCKDEP=n
+CONFIG_SOFTLOCKUP_DETECTOR=n
+CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=n
+
+CONFIG_FW_LOADER_COMPRESS=y
+
+CONFIG_USB_USBNET=y
+CONFIG_NETDEVICES=y
+CONFIG_USB_NET_DRIVERS=y
+CONFIG_USB_RTL8152=y
+CONFIG_USB_NET_AX8817X=y
+CONFIG_USB_NET_SMSC95XX=y
+
+# TK1
+CONFIG_ARM_TEGRA_DEVFREQ=y
+
+# 32-bit build failure
+CONFIG_DRM_MSM=n
--- a/.gitlab-ci/container/arm64.config
+++ b/.gitlab-ci/container/arm64.config
@@ -0,0 +1,183 @@
+CONFIG_LOCALVERSION_AUTO=y
+CONFIG_DEBUG_KERNEL=y
+
+CONFIG_CRYPTO_ZSTD=y
+CONFIG_ZRAM_MEMORY_TRACKING=y
+CONFIG_ZRAM_WRITEBACK=y
+CONFIG_ZRAM=y
+CONFIG_ZSMALLOC_STAT=y
+
+# abootimg with a 'dummy' rootfs fails with root=/dev/nfs
+CONFIG_BLK_DEV_INITRD=n
+
+CONFIG_DEVFREQ_GOV_PERFORMANCE=y
+CONFIG_DEVFREQ_GOV_POWERSAVE=y
+CONFIG_DEVFREQ_GOV_USERSPACE=y
+CONFIG_DEVFREQ_GOV_PASSIVE=y
+
+CONFIG_DRM=y
+CONFIG_DRM_ROCKCHIP=y
+CONFIG_DRM_PANFROST=y
+CONFIG_DRM_LIMA=y
+CONFIG_DRM_PANEL_SIMPLE=y
+CONFIG_DRM_PANEL_EDP=y
+CONFIG_DRM_MSM=y
+CONFIG_DRM_ETNAVIV=y
+CONFIG_DRM_I2C_ADV7511=y
+CONFIG_PWM_CROS_EC=y
+CONFIG_BACKLIGHT_PWM=y
+
+CONFIG_ROCKCHIP_CDN_DP=n
+
+CONFIG_SPI_ROCKCHIP=y
+CONFIG_PWM_ROCKCHIP=y
+CONFIG_PHY_ROCKCHIP_DP=y
+CONFIG_DWMAC_ROCKCHIP=y
+CONFIG_STMMAC_ETH=y
+CONFIG_TYPEC_FUSB302=y
+CONFIG_TYPEC=y
+CONFIG_TYPEC_TCPM=y
+
+# MSM platform bits
+
+# For CONFIG_QCOM_LMH
+CONFIG_OF=y
+
+CONFIG_QCOM_COMMAND_DB=y
+CONFIG_QCOM_RPMHPD=y
+CONFIG_QCOM_RPMPD=y
+CONFIG_SDM_GPUCC_845=y
+CONFIG_SDM_VIDEOCC_845=y
+CONFIG_SDM_DISPCC_845=y
+CONFIG_SDM_LPASSCC_845=y
+CONFIG_SDM_CAMCC_845=y
+CONFIG_RESET_QCOM_PDC=y
+CONFIG_DRM_TI_SN65DSI86=y
+CONFIG_I2C_QCOM_GENI=y
+CONFIG_SPI_QCOM_GENI=y
+CONFIG_PHY_QCOM_QUSB2=y
+CONFIG_PHY_QCOM_QMP=y
+CONFIG_QCOM_CLK_APCC_MSM8996=y
+CONFIG_QCOM_LLCC=y
+CONFIG_QCOM_LMH=y
+CONFIG_QCOM_SPMI_TEMP_ALARM=y
+CONFIG_QCOM_WDT=y
+CONFIG_POWER_RESET_QCOM_PON=y
+CONFIG_RTC_DRV_PM8XXX=y
+CONFIG_INTERCONNECT=y
+CONFIG_INTERCONNECT_QCOM=y
+CONFIG_INTERCONNECT_QCOM_MSM8996=y
+CONFIG_INTERCONNECT_QCOM_SDM845=y
+CONFIG_INTERCONNECT_QCOM_MSM8916=y
+CONFIG_INTERCONNECT_QCOM_OSM_L3=y
+CONFIG_INTERCONNECT_QCOM_SC7180=y
+CONFIG_CRYPTO_DEV_QCOM_RNG=y
+CONFIG_SC_DISPCC_7180=y
+CONFIG_SC_GPUCC_7180=y
+CONFIG_QCOM_SPMI_ADC5=y
+CONFIG_DRM_PARADE_PS8640=y
+CONFIG_PHY_QCOM_USB_HS=y
+
+# db410c ethernet
+CONFIG_USB_RTL8152=y
+# db820c ethernet
+CONFIG_ATL1C=y
+
+CONFIG_ARCH_ALPINE=n
+CONFIG_ARCH_BCM2835=n
+CONFIG_ARCH_BCM_IPROC=n
+CONFIG_ARCH_BERLIN=n
+CONFIG_ARCH_BRCMSTB=n
+CONFIG_ARCH_EXYNOS=n
+CONFIG_ARCH_K3=n
+CONFIG_ARCH_LAYERSCAPE=n
+CONFIG_ARCH_LG1K=n
+CONFIG_ARCH_HISI=n
+CONFIG_ARCH_MVEBU=n
+CONFIG_ARCH_SEATTLE=n
+CONFIG_ARCH_SYNQUACER=n
+CONFIG_ARCH_RENESAS=n
+CONFIG_ARCH_R8A774A1=n
+CONFIG_ARCH_R8A774C0=n
+CONFIG_ARCH_R8A7795=n
+CONFIG_ARCH_R8A7796=n
+CONFIG_ARCH_R8A77965=n
+CONFIG_ARCH_R8A77970=n
+CONFIG_ARCH_R8A77980=n
+CONFIG_ARCH_R8A77990=n
+CONFIG_ARCH_R8A77995=n
+CONFIG_ARCH_STRATIX10=n
+CONFIG_ARCH_TEGRA=n
+CONFIG_ARCH_SPRD=n
+CONFIG_ARCH_THUNDER=n
+CONFIG_ARCH_THUNDER2=n
+CONFIG_ARCH_UNIPHIER=n
+CONFIG_ARCH_VEXPRESS=n
+CONFIG_ARCH_XGENE=n
+CONFIG_ARCH_ZX=n
+CONFIG_ARCH_ZYNQMP=n
+
+# Strip out some stuff we don't need for graphics testing, to reduce
+# the build.
+CONFIG_CAN=n
+CONFIG_WIRELESS=n
+CONFIG_RFKILL=n
+CONFIG_WLAN=n
+
+CONFIG_REGULATOR_FAN53555=y
+CONFIG_REGULATOR=y
+
+CONFIG_REGULATOR_VCTRL=y
+
+CONFIG_KASAN=n
+CONFIG_KASAN_INLINE=n
+CONFIG_STACKTRACE=n
+
+CONFIG_TMPFS=y
+
+CONFIG_PROVE_LOCKING=n
+CONFIG_DEBUG_LOCKDEP=n
+CONFIG_SOFTLOCKUP_DETECTOR=y
+CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=y
+
+CONFIG_DETECT_HUNG_TASK=y
+
+CONFIG_FW_LOADER_COMPRESS=y
+CONFIG_FW_LOADER_USER_HELPER=n
+
+CONFIG_USB_USBNET=y
+CONFIG_NETDEVICES=y
+CONFIG_USB_NET_DRIVERS=y
+CONFIG_USB_RTL8152=y
+CONFIG_USB_NET_AX8817X=y
+CONFIG_USB_NET_SMSC95XX=y
+
+# For amlogic
+CONFIG_MESON_GXL_PHY=y
+CONFIG_MDIO_BUS_MUX_MESON_G12A=y
+CONFIG_DRM_MESON=y
+
+# For Mediatek
+CONFIG_DRM_MEDIATEK=y
+CONFIG_PWM_MEDIATEK=y
+CONFIG_DRM_MEDIATEK_HDMI=y
+CONFIG_GNSS=y
+CONFIG_GNSS_MTK_SERIAL=y
+CONFIG_HW_RANDOM=y
+CONFIG_HW_RANDOM_MTK=y
+CONFIG_MTK_DEVAPC=y
+CONFIG_PWM_MTK_DISP=y
+CONFIG_MTK_CMDQ=y
+
+# For nouveau.  Note that DRM must be a module so that it's loaded after NFS is up to provide the firmware.
+CONFIG_ARCH_TEGRA=y
+CONFIG_DRM_NOUVEAU=m
+CONFIG_DRM_TEGRA=m
+CONFIG_R8169=y
+CONFIG_STAGING=y
+CONFIG_DRM_TEGRA_STAGING=y
+CONFIG_TEGRA_HOST1X=y
+CONFIG_ARM_TEGRA_DEVFREQ=y
+CONFIG_TEGRA_SOCTHERM=y
+CONFIG_DRM_TEGRA_DEBUG=y
+CONFIG_PWM_TEGRA=y
--- a/.gitlab-ci/container/baremetal_build.sh
+++ b/.gitlab-ci/container/baremetal_build.sh
@@ -1,20 +1,19 @@
-#!/usr/bin/env bash
+#!/bin/bash

 set -e
 set -o xtrace

-# Fetch the arm-built rootfs image and unpack it in our x86_64 container (saves
+# Fetch the arm-built rootfs image and unpack it in our x86 container (saves
 # network transfer, disk usage, and runtime on test jobs)

 # shellcheck disable=SC2154 # arch is assigned in previous scripts
-if curl -X HEAD -s "${ARTIFACTS_PREFIX}/${FDO_UPSTREAM_REPO}/${ARTIFACTS_SUFFIX}/${arch}/done"; then
+if wget -q --method=HEAD "${ARTIFACTS_PREFIX}/${FDO_UPSTREAM_REPO}/${ARTIFACTS_SUFFIX}/${arch}/done"; then
  ARTIFACTS_URL="${ARTIFACTS_PREFIX}/${FDO_UPSTREAM_REPO}/${ARTIFACTS_SUFFIX}/${arch}"
 else
  ARTIFACTS_URL="${ARTIFACTS_PREFIX}/${CI_PROJECT_PATH}/${ARTIFACTS_SUFFIX}/${arch}"
 fi

-curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-    "${ARTIFACTS_URL}"/lava-rootfs.tar.zst -o rootfs.tar.zst
+wget "${ARTIFACTS_URL}"/lava-rootfs.tar.zst -O rootfs.tar.zst
 mkdir -p /rootfs-"$arch"
 tar -C /rootfs-"$arch" '--exclude=./dev/*' --zstd -xf rootfs.tar.zst
 rm rootfs.tar.zst
@@ -23,12 +22,9 @@ if [[ $arch == "arm64" ]]; then
    mkdir -p /baremetal-files
    pushd /baremetal-files

-    curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-	-O "${KERNEL_IMAGE_BASE}"/arm64/Image
-    curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-        -O "${KERNEL_IMAGE_BASE}"/arm64/Image.gz
-    curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-        -O "${KERNEL_IMAGE_BASE}"/arm64/cheza-kernel
+    wget "${ARTIFACTS_URL}"/Image
+    wget "${ARTIFACTS_URL}"/Image.gz
+    wget "${ARTIFACTS_URL}"/cheza-kernel

    DEVICE_TREES=""
    DEVICE_TREES="$DEVICE_TREES apq8016-sbc.dtb"
@@ -37,8 +33,7 @@ if [[ $arch == "arm64" ]]; then
    DEVICE_TREES="$DEVICE_TREES imx8mq-nitrogen.dtb"

    for DTB in $DEVICE_TREES; do
-	curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-            -O "${KERNEL_IMAGE_BASE}/arm64/$DTB"
+        wget "${ARTIFACTS_URL}/$DTB"
    done

    popd
@@ -46,16 +41,14 @@ elif [[ $arch == "armhf" ]]; then
    mkdir -p /baremetal-files
    pushd /baremetal-files

-    curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-        -O "${KERNEL_IMAGE_BASE}"/armhf/zImage
+    wget "${ARTIFACTS_URL}"/zImage

    DEVICE_TREES=""
    DEVICE_TREES="$DEVICE_TREES imx6q-cubox-i.dtb"
    DEVICE_TREES="$DEVICE_TREES tegra124-jetson-tk1.dtb"

    for DTB in $DEVICE_TREES; do
-	curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-            -O "${KERNEL_IMAGE_BASE}/armhf/$DTB"
+        wget "${ARTIFACTS_URL}/$DTB"
    done

    popd
--- a/.gitlab-ci/container/build-angle.sh
+++ b/.gitlab-ci/container/build-angle.sh
@@ -1,58 +0,0 @@
-#!/usr/bin/env bash
-
-set -ex
-
-ANGLE_REV="0518a3ff4d4e7e5b2ce8203358f719613a31c118"
-
-# DEPOT tools
-git clone --depth 1 https://chromium.googlesource.com/chromium/tools/depot_tools.git
-PWD=$(pwd)
-export PATH=$PWD/depot_tools:$PATH
-export DEPOT_TOOLS_UPDATE=0
-
-mkdir /angle-build
-pushd /angle-build
-git init
-git remote add origin https://chromium.googlesource.com/angle/angle.git
-git fetch --depth 1 origin "$ANGLE_REV"
-git checkout FETCH_HEAD
-
-# source preparation
-python3 scripts/bootstrap.py
-mkdir -p build/config
-gclient sync
-
-sed -i "/catapult/d" testing/BUILD.gn
-
-mkdir -p out/Release
-echo '
-is_debug = false
-angle_enable_swiftshader = false
-angle_enable_null = false
-angle_enable_gl = false
-angle_enable_vulkan = true
-angle_has_histograms = false
-build_angle_trace_perf_tests = false
-build_angle_deqp_tests = false
-angle_use_custom_libvulkan = false
-dcheck_always_on=true
-' > out/Release/args.gn
-
-if [[ "$DEBIAN_ARCH" = "arm64" ]]; then
-  build/linux/sysroot_scripts/install-sysroot.py --arch=arm64
-fi
-
-gn gen out/Release
-# depot_tools overrides ninja with a version that doesn't work.  We want
-# ninja with FDO_CI_CONCURRENT anyway.
-/usr/local/bin/ninja -C out/Release/
-
-mkdir /angle
-cp out/Release/lib*GL*.so /angle/
-ln -s libEGL.so /angle/libEGL.so.1
-ln -s libGLESv2.so /angle/libGLESv2.so.2
-
-rm -rf out
-
-popd
-rm -rf ./depot_tools
--- a/.gitlab-ci/container/build-apitrace.sh
+++ b/.gitlab-ci/container/build-apitrace.sh
@@ -1,15 +1,9 @@
-#!/usr/bin/env bash
+#!/bin/bash
 # shellcheck disable=SC2086 # we want word splitting

-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_X86_64_TEST_GL_TAG
-# DEBIAN_X86_64_TEST_VK_TAG
-# KERNEL_ROOTFS_TAG
-
 set -ex

-APITRACE_VERSION="0a6506433e1f9f7b69757b4e5730326970c4321a"
+APITRACE_VERSION="790380e05854d5c9d315555444ffcc7acb8f4037"

 git clone https://github.com/apitrace/apitrace.git --single-branch --no-checkout /apitrace
 pushd /apitrace
--- a/.gitlab-ci/container/build-crosvm.sh
+++ b/.gitlab-ci/container/build-crosvm.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/bash
 # shellcheck disable=SC2086 # we want word splitting

 set -ex
@@ -6,32 +6,29 @@ set -ex
 git config --global user.email "mesa@example.com"
 git config --global user.name "Mesa CI"

-CROSVM_VERSION=e3815e62d675ef436956a992e0ed58b7309c759d
+CROSVM_VERSION=504899212d626ecf42b1c459e5592891dde5bf91
 git clone --single-branch -b main --no-checkout https://chromium.googlesource.com/crosvm/crosvm /platform/crosvm
 pushd /platform/crosvm
 git checkout "$CROSVM_VERSION"
 git submodule update --init

-VIRGLRENDERER_VERSION=747c6ae5b194ca551a79958a9a86c42bddcc4553
+VIRGLRENDERER_VERSION=3f2685355f71201f22b98c19aa778b43732c8435
 rm -rf third_party/virglrenderer
-git clone --single-branch -b main --no-checkout https://gitlab.freedesktop.org/virgl/virglrenderer.git third_party/virglrenderer
+git clone --single-branch -b master --no-checkout https://gitlab.freedesktop.org/virgl/virglrenderer.git third_party/virglrenderer
 pushd third_party/virglrenderer
 git checkout "$VIRGLRENDERER_VERSION"
-meson setup build/ -D libdir=lib -D render-server-worker=process -D venus=true $EXTRA_MESON_ARGS
-meson install -C build
+meson build/ -Drender-server=true -Drender-server-worker=process -Dvenus-experimental=true $EXTRA_MESON_ARGS
+ninja -C build install
 popd

-cargo update -p pkg-config@0.3.26 --precise 0.3.27
-
 RUSTFLAGS='-L native=/usr/local/lib' cargo install \
-  bindgen-cli \
-  --locked \
+  bindgen \
  -j ${FDO_CI_CONCURRENT:-4} \
  --root /usr/local \
-  --version 0.65.1 \
+  --version 0.60.1 \
  $EXTRA_CARGO_ARGS

-CROSVM_USE_SYSTEM_VIRGLRENDERER=1 RUSTFLAGS='-L native=/usr/local/lib' cargo install \
+RUSTFLAGS='-L native=/usr/local/lib' cargo install \
  -j ${FDO_CI_CONCURRENT:-4} \
  --locked \
  --features 'default-no-sandbox gpu x virgl_renderer virgl_renderer_next' \
--- a/.gitlab-ci/container/build-deqp-runner.sh
+++ b/.gitlab-ci/container/build-deqp-runner.sh
@@ -1,16 +1,8 @@
-#!/usr/bin/env bash
+#!/bin/sh
 # shellcheck disable=SC2086 # we want word splitting

-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_BASE_TAG
-# DEBIAN_X86_64_TEST_ANDROID_TAG
-# KERNEL_ROOTFS_TAG
-
 set -ex

-DEQP_RUNNER_VERSION=0.18.0
-
 if [ -n "${DEQP_RUNNER_GIT_TAG}${DEQP_RUNNER_GIT_REV}" ]; then
    # Build and install from source
    DEQP_RUNNER_CARGO_ARGS="--git ${DEQP_RUNNER_GIT_URL:-https://gitlab.freedesktop.org/anholt/deqp-runner.git}"
@@ -24,38 +16,13 @@ if [ -n "${DEQP_RUNNER_GIT_TAG}${DEQP_RUNNER_GIT_REV}" ]; then
    DEQP_RUNNER_CARGO_ARGS="${DEQP_RUNNER_CARGO_ARGS} ${EXTRA_CARGO_ARGS}"
 else
    # Install from package registry
-    DEQP_RUNNER_CARGO_ARGS="--version ${DEQP_RUNNER_VERSION} ${EXTRA_CARGO_ARGS} -- deqp-runner"
+    DEQP_RUNNER_CARGO_ARGS="--version 0.16.0 ${EXTRA_CARGO_ARGS} -- deqp-runner"
 fi

-if [ -z "$ANDROID_NDK_HOME" ]; then
-    cargo install --locked  \
-        -j ${FDO_CI_CONCURRENT:-4} \
-        --root /usr/local \
-        ${DEQP_RUNNER_CARGO_ARGS}
-else
-    mkdir -p /deqp-runner
-    pushd /deqp-runner
-    git clone --branch v${DEQP_RUNNER_VERSION} --depth 1 https://gitlab.freedesktop.org/anholt/deqp-runner.git deqp-runner-git
-    pushd deqp-runner-git
-
-    cargo install --locked  \
-        -j ${FDO_CI_CONCURRENT:-4} \
-        --root /usr/local --version 2.10.0 \
-        cargo-ndk
-
-    rustup target add x86_64-linux-android
-    RUSTFLAGS='-C target-feature=+crt-static' cargo ndk --target x86_64-linux-android build
-
-    mv target/x86_64-linux-android/debug/deqp-runner /deqp-runner
-
-    cargo uninstall --locked  \
-        --root /usr/local \
-        cargo-ndk
-
-    popd
-    rm -rf deqp-runner-git
-    popd
-fi
+cargo install --locked  \
+    -j ${FDO_CI_CONCURRENT:-4} \
+    --root /usr/local \
+    ${DEQP_RUNNER_CARGO_ARGS}

 # remove unused test runners to shrink images for the Mesa CI build (not kernel,
 # which chooses its own deqp branch)
--- a/.gitlab-ci/container/build-deqp.sh
+++ b/.gitlab-ci/container/build-deqp.sh
@@ -1,119 +1,59 @@
-#!/usr/bin/env bash
+#!/bin/bash
 # shellcheck disable=SC2086 # we want word splitting

-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_X86_64_TEST_ANDROID_TAG
-# DEBIAN_X86_64_TEST_GL_TAG
-# DEBIAN_X86_64_TEST_VK_TAG
-# KERNEL_ROOTFS_TAG
-
-set -ex -o pipefail
-
-DEQP_VERSION=vulkan-cts-1.3.7.0
+set -ex

 git config --global user.email "mesa@example.com"
 git config --global user.name "Mesa CI"
 git clone \
    https://github.com/KhronosGroup/VK-GL-CTS.git \
-    -b $DEQP_VERSION \
+    -b vulkan-cts-1.3.3.0 \
    --depth 1 \
    /VK-GL-CTS
 pushd /VK-GL-CTS

-mkdir -p /deqp
-
-echo "dEQP base version $DEQP_VERSION" > /deqp/version-log
-
-# Patches to VulkanCTS may come from commits in their repo (listed in
-# cts_commits_to_backport) or patch files stored in our repo (in the patch
-# directory `$OLDPWD/.gitlab-ci/container/patches/` listed in cts_patch_files).
-# Both list variables would have comments explaining the reasons behind the
-# patches.
-
-cts_commits_to_backport=(
-    # Take multiview into account for task shader inv. stats
-    22aa3f4c59f6e1d4daebd5a8c9c05bce6cd3b63b
-
-    # Remove illegal mesh shader query tests
-    2a87f7b25dc27188be0f0a003b2d7aef69d9002e
-
-    # Relax fragment shader invocations result verifications
-    0d8bf6a2715f95907e9cf86a86876ff1f26c66fe
-
-    # Fix several issues in dynamic rendering basic tests
-    c5453824b498c981c6ba42017d119f5de02a3e34
-)
-
-for commit in "${cts_commits_to_backport[@]}"
-do
-  PATCH_URL="https://github.com/KhronosGroup/VK-GL-CTS/commit/$commit.patch"
-  echo "Apply patch to VK-GL-CTS from $PATCH_URL"
-  curl -L --retry 4 -f --retry-all-errors --retry-delay 60 $PATCH_URL | \
+# Apply a patch to update zlib link to an available version.
+# vulkan-cts-1.3.3.0 uses zlib 1.2.12 which was removed from zlib server due to
+# a CVE. See https://zlib.net/
+# FIXME: Remove this patch when uprev to 1.3.4.0+
+wget -O- https://github.com/KhronosGroup/VK-GL-CTS/commit/6bb2e7d64261bedb503947b1b251b1eeeb49be73.patch |
    git am -
-done
-
-cts_patch_files=(
-  # Android specific patches.
-  build-deqp_Allow-running-on-Android-from-the-command-line.patch
-  build-deqp_Android-prints-to-stdout-instead-of-logcat.patch
-)
-
-for patch in "${cts_patch_files[@]}"
-do
-  echo "Apply patch to VK-GL-CTS from $patch"
-  git am < $OLDPWD/.gitlab-ci/container/patches/$patch
-done
-
-echo "The following local patches are applied on top:" >> /deqp/version-log
-git log --reverse --oneline $DEQP_VERSION.. --format=%s | sed 's/^/- /' >> /deqp/version-log

 # --insecure is due to SSL cert failures hitting sourceforge for zlib and
 # libpng (sigh).  The archives get their checksums checked anyway, and git
 # always goes through ssh or https.
 python3 external/fetch_sources.py --insecure

+mkdir -p /deqp
+
 # Save the testlog stylesheets:
 cp doc/testlog-stylesheet/testlog.{css,xsl} /deqp
 popd

 pushd /deqp
-
-if [ "${DEQP_TARGET}" != 'android' ]; then
-    # When including EGL/X11 testing, do that build first and save off its
-    # deqp-egl binary.
-    cmake -S /VK-GL-CTS -B . -G Ninja \
-        -DDEQP_TARGET=x11_egl_glx \
-        -DCMAKE_BUILD_TYPE=Release \
-        $EXTRA_CMAKE_ARGS
-    ninja modules/egl/deqp-egl
-    mv /deqp/modules/egl/deqp-egl /deqp/modules/egl/deqp-egl-x11
-
-    cmake -S /VK-GL-CTS -B . -G Ninja \
-        -DDEQP_TARGET=wayland \
-        -DCMAKE_BUILD_TYPE=Release \
-        $EXTRA_CMAKE_ARGS
-    ninja modules/egl/deqp-egl
-    mv /deqp/modules/egl/deqp-egl /deqp/modules/egl/deqp-egl-wayland
-fi
-
+# When including EGL/X11 testing, do that build first and save off its
+# deqp-egl binary.
 cmake -S /VK-GL-CTS -B . -G Ninja \
-      -DDEQP_TARGET=${DEQP_TARGET:-default} \
+      -DDEQP_TARGET=x11_egl_glx \
      -DCMAKE_BUILD_TYPE=Release \
      $EXTRA_CMAKE_ARGS
+ninja modules/egl/deqp-egl
+cp /deqp/modules/egl/deqp-egl /deqp/modules/egl/deqp-egl-x11

-# Make sure `default` doesn't silently stop detecting one of the platforms we care about
-if [ "${DEQP_TARGET}" = 'default' ]; then
-  grep -q DEQP_SUPPORT_WAYLAND=1 build.ninja
-  grep -q DEQP_SUPPORT_X11=1 build.ninja
-  grep -q DEQP_SUPPORT_XCB=1 build.ninja
-fi
+cmake -S /VK-GL-CTS -B . -G Ninja \
+      -DDEQP_TARGET=wayland \
+      -DCMAKE_BUILD_TYPE=Release \
+      $EXTRA_CMAKE_ARGS
+ninja modules/egl/deqp-egl
+cp /deqp/modules/egl/deqp-egl /deqp/modules/egl/deqp-egl-wayland

-mold --run ninja
+cmake -S /VK-GL-CTS -B . -G Ninja \
+      -DDEQP_TARGET=${DEQP_TARGET:-x11_glx} \
+      -DCMAKE_BUILD_TYPE=Release \
+      $EXTRA_CMAKE_ARGS
+ninja

-if [ "${DEQP_TARGET}" = 'android' ]; then
-    mv /deqp/modules/egl/deqp-egl /deqp/modules/egl/deqp-egl-android
-fi
+mv /deqp/modules/egl/deqp-egl-x11 /deqp/modules/egl/deqp-egl

 # Copy out the mustpass lists we want.
 mkdir /deqp/mustpass
@@ -122,30 +62,28 @@ for mustpass in $(< /VK-GL-CTS/external/vulkancts/mustpass/main/vk-default.txt)
        >> /deqp/mustpass/vk-master.txt
 done

-if [ "${DEQP_TARGET}" != 'android' ]; then
-    cp \
-        /deqp/external/openglcts/modules/gl_cts/data/mustpass/gles/aosp_mustpass/3.2.6.x/*.txt \
-        /deqp/mustpass/.
-    cp \
-        /deqp/external/openglcts/modules/gl_cts/data/mustpass/egl/aosp_mustpass/3.2.6.x/egl-master.txt \
-        /deqp/mustpass/.
-    cp \
-        /deqp/external/openglcts/modules/gl_cts/data/mustpass/gles/khronos_mustpass/3.2.6.x/*-master.txt \
-        /deqp/mustpass/.
-    cp \
-        /deqp/external/openglcts/modules/gl_cts/data/mustpass/gl/khronos_mustpass/4.6.1.x/*-master.txt \
-        /deqp/mustpass/.
-    cp \
-        /deqp/external/openglcts/modules/gl_cts/data/mustpass/gl/khronos_mustpass_single/4.6.1.x/*-single.txt \
-        /deqp/mustpass/.
+cp \
+    /deqp/external/openglcts/modules/gl_cts/data/mustpass/gles/aosp_mustpass/3.2.6.x/*.txt \
+    /deqp/mustpass/.
+cp \
+    /deqp/external/openglcts/modules/gl_cts/data/mustpass/egl/aosp_mustpass/3.2.6.x/egl-master.txt \
+    /deqp/mustpass/.
+cp \
+    /deqp/external/openglcts/modules/gl_cts/data/mustpass/gles/khronos_mustpass/3.2.6.x/*-master.txt \
+    /deqp/mustpass/.
+cp \
+    /deqp/external/openglcts/modules/gl_cts/data/mustpass/gl/khronos_mustpass/4.6.1.x/*-master.txt \
+    /deqp/mustpass/.
+cp \
+    /deqp/external/openglcts/modules/gl_cts/data/mustpass/gl/khronos_mustpass_single/4.6.1.x/*-single.txt \
+    /deqp/mustpass/.

-    # Save *some* executor utils, but otherwise strip things down
-    # to reduct deqp build size:
-    mkdir /deqp/executor.save
-    cp /deqp/executor/testlog-to-* /deqp/executor.save
-    rm -rf /deqp/executor
-    mv /deqp/executor.save /deqp/executor
-fi
+# Save *some* executor utils, but otherwise strip things down
+# to reduct deqp build size:
+mkdir /deqp/executor.save
+cp /deqp/executor/testlog-to-* /deqp/executor.save
+rm -rf /deqp/executor
+mv /deqp/executor.save /deqp/executor

 # Remove other mustpass files, since we saved off the ones we wanted to conventient locations above.
 rm -rf /deqp/external/openglcts/modules/gl_cts/data/mustpass
@@ -156,7 +94,8 @@ rm -rf /deqp/external/openglcts/modules/cts-runner
 rm -rf /deqp/modules/internal
 rm -rf /deqp/execserver
 rm -rf /deqp/framework
-find . -depth \( -iname '*cmake*' -o -name '*ninja*' -o -name '*.o' -o -name '*.a' \) -exec rm -rf {} \;
+# shellcheck disable=SC2038,SC2185 # TODO: rewrite find
+find -iname '*cmake*' -o -name '*ninja*' -o -name '*.o' -o -name '*.a' | xargs rm -rf
 ${STRIP_CMD:-strip} external/vulkancts/modules/vulkan/deqp-vk
 ${STRIP_CMD:-strip} external/openglcts/modules/glcts
 ${STRIP_CMD:-strip} modules/*/deqp-*
--- a/.gitlab-ci/container/build-directx-headers.sh
+++ b/.gitlab-ci/container/build-directx-headers.sh
@@ -1,15 +0,0 @@
-#!/usr/bin/env bash
-# shellcheck disable=SC2086 # we want word splitting
-
-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_BUILD_TAG
-
-set -ex
-
-git clone https://github.com/microsoft/DirectX-Headers -b v1.611.0 --depth 1
-pushd DirectX-Headers
-meson setup build --backend=ninja --buildtype=release -Dbuild-test=false $EXTRA_MESON_ARGS
-meson install -C build
-popd
-rm -rf DirectX-Headers
--- a/.gitlab-ci/container/build-fossilize.sh
+++ b/.gitlab-ci/container/build-fossilize.sh
@@ -1,15 +1,10 @@
 #!/bin/bash

-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_X86_64_TEST_VK_TAG
-# KERNEL_ROOTFS_TAG
-
 set -ex

 git clone https://github.com/ValveSoftware/Fossilize.git
 cd Fossilize
-git checkout b43ee42bbd5631ea21fe9a2dee4190d5d875c327
+git checkout 16fba1b8b5d9310126bb02323d7bae3227338461
 git submodule update --init
 mkdir build
 cd build
--- a/.gitlab-ci/container/build-gfxreconstruct.sh
+++ b/.gitlab-ci/container/build-gfxreconstruct.sh
@@ -1,8 +1,8 @@
-#!/usr/bin/env bash
+#!/bin/bash

 set -ex

-GFXRECONSTRUCT_VERSION=761837794a1e57f918a85af7000b12e531b178ae
+GFXRECONSTRUCT_VERSION=5ed3caeecc46e976c4df31e263df8451ae176c26

 git clone https://github.com/LunarG/gfxreconstruct.git \
    --single-branch \
--- a/.gitlab-ci/container/build-kdl.sh
+++ b/.gitlab-ci/container/build-kdl.sh
@@ -1,23 +0,0 @@
-#!/usr/bin/env bash
-# shellcheck disable=SC1091  # the path is created by the script
-
-set -ex
-
-KDL_REVISION="5056f71b100a68b72b285c6fc845a66a2ed25985"
-
-mkdir ci-kdl.git
-pushd ci-kdl.git
-git init
-git remote add origin https://gitlab.freedesktop.org/gfx-ci/ci-kdl.git
-git fetch --depth 1 origin ${KDL_REVISION}
-git checkout FETCH_HEAD
-popd
-
-python3 -m venv ci-kdl.venv
-source ci-kdl.venv/bin/activate
-pushd ci-kdl.git
-pip install -r requirements.txt
-pip install .
-popd
-
-rm -rf ci-kdl.git
--- a/.gitlab-ci/container/build-kernel.sh
+++ b/.gitlab-ci/container/build-kernel.sh
@@ -1,30 +1,52 @@
-#!/usr/bin/env bash
+#!/bin/bash
 # shellcheck disable=SC2086 # we want word splitting
-# shellcheck disable=SC2153

 set -ex

 mkdir -p kernel
+wget -qO- ${KERNEL_URL} | tar -xj --strip-components=1 -C kernel
 pushd kernel

-if [[ ${DEBIAN_ARCH} = "arm64" ]]; then
-    KERNEL_IMAGE_NAME+=" cheza-kernel"
-fi
-
-for image in ${KERNEL_IMAGE_NAME}; do
-    curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-      -o "/lava-files/${image}" "${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${image}"
+# The kernel doesn't like the gold linker (or the old lld in our debians).
+# Sneak in some override symlinks during kernel build until we can update
+# debian (they'll get blown away by the rm of the kernel dir at the end).
+mkdir -p ld-links
+for i in /usr/bin/*-ld /usr/bin/ld; do
+    i=$(basename $i)
+    ln -sf /usr/bin/$i.bfd ld-links/$i
 done

-for dtb in ${DEVICE_TREES}; do
-    curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-      -o "/lava-files/${dtb}" "${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/${dtb}"
-  done
+NEWPATH=$(pwd)/ld-links
+export PATH=$NEWPATH:$PATH

-mkdir -p "/lava-files/rootfs-${DEBIAN_ARCH}"
-curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-  -O "${KERNEL_IMAGE_BASE}/${DEBIAN_ARCH}/modules.tar.zst"
-tar --keep-directory-symlink --zstd -xf modules.tar.zst -C "/lava-files/rootfs-${DEBIAN_ARCH}/"
+KERNEL_FILENAME=$(basename $KERNEL_URL)
+export LOCALVERSION="$KERNEL_FILENAME"
+./scripts/kconfig/merge_config.sh ${DEFCONFIG} ../.gitlab-ci/container/${KERNEL_ARCH}.config
+make ${KERNEL_IMAGE_NAME}
+for image in ${KERNEL_IMAGE_NAME}; do
+    cp arch/${KERNEL_ARCH}/boot/${image} /lava-files/.
+done
+
+if [[ -n ${DEVICE_TREES} ]]; then
+    make dtbs
+    cp ${DEVICE_TREES} /lava-files/.
+fi
+
+make modules
+INSTALL_MOD_PATH=/lava-files/rootfs-${DEBIAN_ARCH}/ make modules_install
+
+if [[ ${DEBIAN_ARCH} = "arm64" ]]; then
+    make Image.lzma
+    mkimage \
+        -f auto \
+        -A arm \
+        -O linux \
+        -d arch/arm64/boot/Image.lzma \
+        -C lzma\
+        -b arch/arm64/boot/dts/qcom/sdm845-cheza-r3.dtb \
+        /lava-files/cheza-kernel
+    KERNEL_IMAGE_NAME+=" cheza-kernel"
+fi

 popd
 rm -rf kernel
--- a/.gitlab-ci/container/build-libclc.sh
+++ b/.gitlab-ci/container/build-libclc.sh
@@ -1,9 +1,8 @@
-#!/usr/bin/env bash
+#!/bin/bash

 set -ex

-export LLVM_CONFIG="llvm-config-${LLVM_VERSION:?"llvm unset!"}"
-LLVM_TAG="llvmorg-15.0.7"
+export LLVM_CONFIG="llvm-config-11"

 $LLVM_CONFIG --version

@@ -12,12 +11,12 @@ git config --global user.name "Mesa CI"
 git clone \
    https://github.com/llvm/llvm-project \
    --depth 1 \
-    -b "${LLVM_TAG}" \
+    -b llvmorg-12.0.0-rc3 \
    /llvm-project

 mkdir /libclc
 pushd /libclc
-cmake -S /llvm-project/libclc -B . -G Ninja -DLLVM_CONFIG="$LLVM_CONFIG" -DLIBCLC_TARGETS_TO_BUILD="spirv-mesa3d-;spirv64-mesa3d-" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DLLVM_SPIRV=/usr/bin/llvm-spirv
+cmake -S /llvm-project/libclc -B . -G Ninja -DLLVM_CONFIG=$LLVM_CONFIG -DLIBCLC_TARGETS_TO_BUILD="spirv-mesa3d-;spirv64-mesa3d-" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DLLVM_SPIRV=/usr/bin/llvm-spirv
 ninja
 ninja install
 popd
--- a/.gitlab-ci/container/build-libdrm.sh
+++ b/.gitlab-ci/container/build-libdrm.sh
@@ -1,16 +1,14 @@
-#!/usr/bin/env bash
-# Script used for Android and Fedora builds
+#!/bin/bash
 # shellcheck disable=SC2086 # we want word splitting

 set -ex

-export LIBDRM_VERSION=libdrm-2.4.119
+export LIBDRM_VERSION=libdrm-2.4.110

-curl -L -O --retry 4 -f --retry-all-errors --retry-delay 60 \
-    https://dri.freedesktop.org/libdrm/"$LIBDRM_VERSION".tar.xz
+wget https://dri.freedesktop.org/libdrm/"$LIBDRM_VERSION".tar.xz
 tar -xvf "$LIBDRM_VERSION".tar.xz && rm "$LIBDRM_VERSION".tar.xz
 cd "$LIBDRM_VERSION"
-meson setup build -D vc4=disabled -D freedreno=disabled -D etnaviv=disabled $EXTRA_MESON_ARGS
-meson install -C build
+meson build -D vc4=false -D freedreno=false -D etnaviv=false $EXTRA_MESON_ARGS
+ninja -C build install
 cd ..
 rm -rf "$LIBDRM_VERSION"
--- a/.gitlab-ci/container/build-llvm-spirv.sh
+++ b/.gitlab-ci/container/build-llvm-spirv.sh
@@ -1,15 +1,12 @@
-#!/usr/bin/env bash
+#!/bin/bash

 set -ex

-VER="${LLVM_VERSION:?llvm not set}.0.0"
+wget https://github.com/KhronosGroup/SPIRV-LLVM-Translator/archive/refs/tags/v13.0.0.tar.gz
+tar -xvf v13.0.0.tar.gz && rm v13.0.0.tar.gz

-curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-    -O "https://github.com/KhronosGroup/SPIRV-LLVM-Translator/archive/refs/tags/v${VER}.tar.gz"
-tar -xvf "v${VER}.tar.gz" && rm "v${VER}.tar.gz"
-
-mkdir "SPIRV-LLVM-Translator-${VER}/build"
-pushd "SPIRV-LLVM-Translator-${VER}/build"
+mkdir SPIRV-LLVM-Translator-13.0.0/build
+pushd SPIRV-LLVM-Translator-13.0.0/build
 cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr
 ninja
 ninja install
@@ -18,5 +15,5 @@ ninja llvm-spirv
 cp tools/llvm-spirv/llvm-spirv /usr/bin/
 popd

-du -sh "SPIRV-LLVM-Translator-${VER}"
-rm -rf "SPIRV-LLVM-Translator-${VER}"
+du -sh SPIRV-LLVM-Translator-13.0.0
+rm -rf SPIRV-LLVM-Translator-13.0.0
--- a/.gitlab-ci/container/build-mold.sh
+++ b/.gitlab-ci/container/build-mold.sh
@@ -2,14 +2,12 @@

 set -ex

-MOLD_VERSION="1.11.0"
+MOLD_VERSION="1.9.0"

 git clone -b v"$MOLD_VERSION" --single-branch --depth 1 https://github.com/rui314/mold.git
 pushd mold
-
 cmake -DCMAKE_BUILD_TYPE=Release -D BUILD_TESTING=OFF -D MOLD_LTO=ON
 cmake --build . --parallel
 cmake --install .
-
 popd
 rm -rf mold
--- a/.gitlab-ci/container/build-piglit.sh
+++ b/.gitlab-ci/container/build-piglit.sh
@@ -1,33 +1,26 @@
 #!/bin/bash
 # shellcheck disable=SC2086 # we want word splitting
+
 set -ex

-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_X86_64_TEST_GL_TAG
-# DEBIAN_X86_64_TEST_VK_TAG
-# KERNEL_ROOTFS_TAG
-
-REV="f7db20b03de6896d013826c0a731bc4417c1a5a0"
-
 git clone https://gitlab.freedesktop.org/mesa/piglit.git --single-branch --no-checkout /piglit
 pushd /piglit
-git checkout "$REV"
+git checkout 1cd716180cfb6ef0c1fc54702460ef49e5115791
 patch -p1 <$OLDPWD/.gitlab-ci/piglit/disable-vs_in.diff
 cmake -S . -B . -G Ninja -DCMAKE_BUILD_TYPE=Release $PIGLIT_OPTS $EXTRA_CMAKE_ARGS
 ninja $PIGLIT_BUILD_TARGETS
-find . -depth \( -name .git -o -name '*ninja*' -o -iname '*cmake*' -o -name '*.[chao]' \) -exec rm -rf {} \;
+# shellcheck disable=SC2038,SC2185 # TODO: rewrite find
+find -name .git -o -name '*ninja*' -o -iname '*cmake*' -o -name '*.[chao]' | xargs rm -rf
 rm -rf target_api
 if [ "$PIGLIT_BUILD_TARGETS" = "piglit_replayer" ]; then
-    find . -depth \
-         ! -regex "^\.$" \
+    # shellcheck disable=SC2038,SC2185 # TODO: rewrite find
+    find ! -regex "^\.$" \
         ! -regex "^\.\/piglit.*" \
         ! -regex "^\.\/framework.*" \
         ! -regex "^\.\/bin$" \
         ! -regex "^\.\/bin\/replayer\.py" \
         ! -regex "^\.\/templates.*" \
         ! -regex "^\.\/tests$" \
-         ! -regex "^\.\/tests\/replay\.py" \
-         -exec rm -rf {} \; 2>/dev/null
+         ! -regex "^\.\/tests\/replay\.py" 2>/dev/null | xargs rm -rf
 fi
 popd
--- a/.gitlab-ci/container/build-rust.sh
+++ b/.gitlab-ci/container/build-rust.sh
@@ -11,22 +11,21 @@ set -ex
 mkdir -p "$HOME"/.cargo
 ln -s /usr/local/bin "$HOME"/.cargo/bin

-# Rusticl requires at least Rust 1.66.0 and NAK requires 1.73.0
+# Rusticl requires at least Rust 1.59.0
 #
-# Also, pick a specific snapshot from rustup so the compiler doesn't drift on
+# Also, oick a specific snapshot from rustup so the compiler doesn't drift on
 # us.
-RUST_VERSION=1.73.0-2023-10-05
+RUST_VERSION=1.59.0-2022-02-24

 # For rust in Mesa, we use rustup to install.  This lets us pick an arbitrary
 # version of the compiler, rather than whatever the container's Debian comes
 # with.
-curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-    --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- \
-      --default-toolchain $RUST_VERSION \
-      --profile minimal \
-      -y
+wget https://sh.rustup.rs -O - | sh -s -- \
+   --default-toolchain $RUST_VERSION \
+   --profile minimal \
+   -y

-rustup component add clippy rustfmt
+rustup component add rustfmt

 # Set up a config script for cross compiling -- cargo needs your system cc for
 # linking in cross builds, but doesn't know what you want to use for system cc.
--- a/.gitlab-ci/container/build-shader-db.sh
+++ b/.gitlab-ci/container/build-shader-db.sh
@@ -1,14 +0,0 @@
-#!/usr/bin/env bash
-
-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_BUILD_TAG
-
-set -ex
-
-pushd /usr/local
-git clone https://gitlab.freedesktop.org/mesa/shader-db.git --depth 1
-rm -rf shader-db/.git
-cd shader-db
-make
-popd
--- a/.gitlab-ci/container/build-skqp.sh
+++ b/.gitlab-ci/container/build-skqp.sh
@@ -1,20 +1,27 @@
-#!/usr/bin/env bash
-# SPDX-License-Identifier: MIT
+#!/bin/bash
 #
-# Copyright © 2022 Collabora Limited
+# Copyright (C) 2022 Collabora Limited
 # Author: Guilherme Gallo <guilherme.gallo@collabora.com>
 #
-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# KERNEL_ROOTFS_TAG
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice (including the next
+# paragraph) shall be included in all copies or substantial portions of the
+# Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.

-SKQP_BRANCH=android-cts-12.1_r5
-
-# hack for skqp see the clang
-pushd /usr/bin/
-ln -s ../lib/llvm-15/bin/clang clang
-ln -s ../lib/llvm-15/bin/clang++ clang++
-popd

 create_gn_args() {
    # gn can be configured to cross-compile skia and its tools
@@ -34,6 +41,7 @@ download_skia_source() {
    # Skia cloned from https://android.googlesource.com/platform/external/skqp
    # has all needed assets tracked on git-fs
    SKQP_REPO=https://android.googlesource.com/platform/external/skqp
+    SKQP_BRANCH=android-cts-11.0_r7

    git clone --branch "${SKQP_BRANCH}" --depth 1 "${SKQP_REPO}" "${SKIA_DIR}"
 }
@@ -41,7 +49,7 @@ download_skia_source() {
 set -ex

 SCRIPT_DIR=$(realpath "$(dirname "$0")")
-SKQP_PATCH_DIR="${SCRIPT_DIR}/patches"
+SKQP_PATCH_DIR="${SCRIPT_DIR}"
 BASE_ARGS_GN_FILE="${SCRIPT_DIR}/build-skqp_base.gn"

 SKQP_ARCH=${SKQP_ARCH:-x64}
--- a/.gitlab-ci/container/patches/build-skqp_BUILD.gn.patch
+++ b/.gitlab-ci/container/patches/build-skqp_BUILD.gn.patch
--- a/.gitlab-ci/container/build-skqp_base.gn
+++ b/.gitlab-ci/container/build-skqp_base.gn
@@ -1,12 +1,7 @@
 cc = "clang"
 cxx = "clang++"

-extra_cflags = [
-        "-Wno-error",
-
-        "-DSK_ENABLE_DUMP_GPU",
-        "-DSK_BUILD_FOR_SKQP"
-    ]
+extra_cflags = [ "-DSK_ENABLE_DUMP_GPU", "-DSK_BUILD_FOR_SKQP" ]
 extra_cflags_cc = [
        "-Wno-error",

@@ -27,13 +22,6 @@ extra_cflags_cc = [
        "-Wno-suggest-destructor-override",
        "-Wno-return-std-move-in-c++11",
        "-Wno-extra-semi-stmt",
-        "-Wno-reserved-identifier",
-        "-Wno-bitwise-instead-of-logical",
-        "-Wno-reserved-identifier",
-        "-Wno-psabi",
-        "-Wno-unused-but-set-variable",
-        "-Wno-sizeof-array-div",
-        "-Wno-string-concatenation",
    ]

 cc_wrapper = "ccache"
--- a/.gitlab-ci/container/patches/build-skqp_fetch_gn.patch
+++ b/.gitlab-ci/container/patches/build-skqp_fetch_gn.patch
--- a/.gitlab-ci/container/patches/build-skqp_git-sync-deps.patch
+++ b/.gitlab-ci/container/patches/build-skqp_git-sync-deps.patch
--- a/.gitlab-ci/container/patches/build-skqp_gl.patch
+++ b/.gitlab-ci/container/patches/build-skqp_gl.patch
--- a/.gitlab-ci/container/patches/build-skqp_is_clang.py.patch
+++ b/.gitlab-ci/container/patches/build-skqp_is_clang.py.patch
--- a/.gitlab-ci/container/patches/build-skqp_nima.patch
+++ b/.gitlab-ci/container/patches/build-skqp_nima.patch
--- a/.gitlab-ci/container/build-va-tools.sh
+++ b/.gitlab-ci/container/build-va-tools.sh
@@ -1,25 +1,18 @@
-#!/usr/bin/env bash
+#!/bin/bash
 # shellcheck disable=SC2086 # we want word splitting
-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# KERNEL_ROOTFS_TAG

 set -ex

 git config --global user.email "mesa@example.com"
 git config --global user.name "Mesa CI"
-
 git clone \
    https://github.com/intel/libva-utils.git \
-    -b 2.18.1 \
+    -b 2.13.0 \
    --depth 1 \
    /va-utils

 pushd /va-utils
-# Too old libva in Debian 11. TODO: when this PR gets in, refer to the patch.
-curl -L https://github.com/intel/libva-utils/pull/329.patch | git am
-
-meson setup build -D tests=true -Dprefix=/va $EXTRA_MESON_ARGS
-meson install -C build
+meson build -D tests=true  -Dprefix=/va $EXTRA_MESON_ARGS
+ninja -C build install
 popd
 rm -rf /va-utils
--- a/.gitlab-ci/container/build-vkd3d-proton.sh
+++ b/.gitlab-ci/container/build-vkd3d-proton.sh
@@ -1,12 +1,8 @@
 #!/bin/bash

-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_X86_64_TEST_VK_TAG
-# KERNEL_ROOTFS_TAG
 set -ex

-VKD3D_PROTON_COMMIT="a0ccc383937903f4ca0997ce53e41ccce7f2f2ec"
+VKD3D_PROTON_COMMIT="804751ee1cb108a2ec59e182ce0c052bafef268e"

 VKD3D_PROTON_DST_DIR="/vkd3d-proton-tests"
 VKD3D_PROTON_SRC_DIR="/vkd3d-proton-src"
--- a/.gitlab-ci/container/build-vulkan-validation.sh
+++ b/.gitlab-ci/container/build-vulkan-validation.sh
@@ -1,18 +0,0 @@
-#!/usr/bin/env bash
-
-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_X86_64_TEST_GL_TAG
-# KERNEL_ROOTFS_TAG:
-
-set -ex
-
-VALIDATION_TAG="v1.3.269"
-
-git clone -b "$VALIDATION_TAG" --single-branch --depth 1 https://github.com/KhronosGroup/Vulkan-ValidationLayers.git
-pushd Vulkan-ValidationLayers
-python3 scripts/update_deps.py --dir external --config debug
-cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_TESTS=OFF -DBUILD_WERROR=OFF -C external/helper.cmake -S . -B build
-ninja -C build install
-popd
-rm -rf Vulkan-ValidationLayers
--- a/.gitlab-ci/container/build-wayland.sh
+++ b/.gitlab-ci/container/build-wayland.sh
@@ -1,23 +1,23 @@
-#!/usr/bin/env bash
+#!/bin/bash
 # shellcheck disable=SC2086 # we want word splitting

 set -ex

-export LIBWAYLAND_VERSION="1.21.0"
-export WAYLAND_PROTOCOLS_VERSION="1.31"
+export LIBWAYLAND_VERSION="1.18.0"
+export WAYLAND_PROTOCOLS_VERSION="1.24"

 git clone https://gitlab.freedesktop.org/wayland/wayland
 cd wayland
 git checkout "$LIBWAYLAND_VERSION"
-meson setup -Ddocumentation=false -Ddtd_validation=false -Dlibraries=true _build $EXTRA_MESON_ARGS
-meson install -C _build
+meson -Ddocumentation=false -Ddtd_validation=false -Dlibraries=true _build $EXTRA_MESON_ARGS
+ninja -C _build install
 cd ..
 rm -rf wayland

 git clone https://gitlab.freedesktop.org/wayland/wayland-protocols
 cd wayland-protocols
 git checkout "$WAYLAND_PROTOCOLS_VERSION"
-meson setup _build $EXTRA_MESON_ARGS
-meson install -C _build
+meson _build $EXTRA_MESON_ARGS
+ninja -C _build install
 cd ..
 rm -rf wayland-protocols
--- a/.gitlab-ci/container/container_post_build.sh
+++ b/.gitlab-ci/container/container_post_build.sh
@@ -1,14 +1,10 @@
-#!/usr/bin/env bash
+#!/bin/sh

 if test -f /etc/debian_version; then
    apt-get autoremove -y --purge
 fi

-# Clean up any build cache
-rm -rf /root/.cache
-rm -rf /root/.cargo
+# Clean up any build cache for rust.
 rm -rf /.cargo

-if test -x /usr/bin/ccache; then
-    ccache --show-stats
-fi
+ccache --show-stats
--- a/.gitlab-ci/container/container_pre_build.sh
+++ b/.gitlab-ci/container/container_pre_build.sh
@@ -1,36 +1,34 @@
 #!/bin/sh

-if test -x /usr/bin/ccache; then
-    if test -f /etc/debian_version; then
-        CCACHE_PATH=/usr/lib/ccache
-    elif test -f /etc/alpine-release; then
-        CCACHE_PATH=/usr/lib/ccache/bin
-    else
-        CCACHE_PATH=/usr/lib64/ccache
-    fi
-
-    # Common setup among container builds before we get to building code.
-
-    export CCACHE_COMPILERCHECK=content
-    export CCACHE_COMPRESS=true
-    export CCACHE_DIR=/cache/$CI_PROJECT_NAME/ccache
-    export PATH=$CCACHE_PATH:$PATH
-
-    # CMake ignores $PATH, so we have to force CC/GCC to the ccache versions.
-    export CC="${CCACHE_PATH}/gcc"
-    export CXX="${CCACHE_PATH}/g++"
-
-    ccache --show-stats
+if test -f /etc/debian_version; then
+    CCACHE_PATH=/usr/lib/ccache
+elif test -f /etc/alpine-release; then
+    CCACHE_PATH=/usr/lib/ccache/bin
+else
+    CCACHE_PATH=/usr/lib64/ccache
 fi

+# Common setup among container builds before we get to building code.
+
+export CCACHE_COMPILERCHECK=content
+export CCACHE_COMPRESS=true
+export CCACHE_DIR=/cache/$CI_PROJECT_NAME/ccache
+export PATH=$CCACHE_PATH:$PATH
+
+# CMake ignores $PATH, so we have to force CC/GCC to the ccache versions.
+export CC="${CCACHE_PATH}/gcc"
+export CXX="${CCACHE_PATH}/g++"
+
 # When not using the mold linker (e.g. unsupported architecture), force
 # linkers to gold, since it's so much faster for building.  We can't use
-# lld because we're on old debian and it's buggy.  mingw fails meson builds
+# lld because we're on old debian and it's buggy.  ming fails meson builds
 # with it with "meson.build:21:0: ERROR: Unable to determine dynamic linker"
 find /usr/bin -name \*-ld -o -name ld | \
    grep -v mingw | \
    xargs -n 1 -I '{}' ln -sf '{}.gold' '{}'

+ccache --show-stats
+
 # Make a wrapper script for ninja to always include the -j flags
 {
    echo '#!/bin/sh -x'
--- a/.gitlab-ci/container/create-android-cross-file.sh
+++ b/.gitlab-ci/container/create-android-cross-file.sh
@@ -5,10 +5,9 @@ arch=$2
 cpu_family=$3
 cpu=$4
 cross_file="/cross_file-$arch.txt"
-sdk_version=$5

 # armv7 has the toolchain split between two names.
-arch2=${6:-$2}
+arch2=${5:-$2}

 # Note that we disable C++ exceptions, because Mesa doesn't use exceptions,
 # and allowing it in code generation means we get unwind symbols that break
@@ -16,22 +15,21 @@ arch2=${6:-$2}

 cat > "$cross_file" <<EOF
 [binaries]
-ar = '$ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar'
-c = ['ccache', '$ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/${arch2}${sdk_version}-clang', '-fno-exceptions', '-fno-unwind-tables', '-fno-asynchronous-unwind-tables']
-cpp = ['ccache', '$ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/${arch2}${sdk_version}-clang++', '-fno-exceptions', '-fno-unwind-tables', '-fno-asynchronous-unwind-tables', '-static-libstdc++']
+ar = '$ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/$arch-ar'
+c = ['ccache', '$ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/${arch2}29-clang', '-fno-exceptions', '-fno-unwind-tables', '-fno-asynchronous-unwind-tables']
+cpp = ['ccache', '$ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/${arch2}29-clang++', '-fno-exceptions', '-fno-unwind-tables', '-fno-asynchronous-unwind-tables']
 c_ld = 'lld'
 cpp_ld = 'lld'
-strip = '$ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip'
-pkgconfig = ['/usr/bin/pkgconf']
+strip = '$ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/$arch-strip'
+pkgconfig = ['/usr/bin/pkg-config']

 [host_machine]
-system = 'android'
+system = 'linux'
 cpu_family = '$cpu_family'
 cpu = '$cpu'
 endian = 'little'

 [properties]
 needs_exe_wrapper = true
-pkg_config_libdir = '/usr/local/lib/${arch2}/pkgconfig/:/${ndk}/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/${arch2}/pkgconfig/'

 EOF
--- a/.gitlab-ci/container/create-android-ndk-pc.sh
+++ b/.gitlab-ci/container/create-android-ndk-pc.sh
@@ -10,7 +10,6 @@ pc="$2"
 cflags="$3"
 libs="$4"
 version="$5"
-sdk_version="$6"

 sysroot=$ndk/toolchains/llvm/prebuilt/linux-x86_64/sysroot

@@ -25,7 +24,7 @@ for arch in \
    cat >$pcdir/$pc <<EOF
 prefix=$sysroot
 exec_prefix=$sysroot
-libdir=$sysroot/usr/lib/$arch/$sdk_version
+libdir=$sysroot/usr/lib/$arch/29
 sharedlibdir=$sysroot/usr/lib/$arch
 includedir=$sysroot/usr/include

@@ -34,7 +33,7 @@ Description: zlib compression library
 Version: $version

 Requires:
-Libs: -L$sysroot/usr/lib/$arch/$sdk_version $libs
+Libs: -L$sysroot/usr/lib/$arch/29 $libs
 Cflags: -I$sysroot/usr/include $cflags
 EOF
 done
--- a/.gitlab-ci/container/create-cross-file.sh
+++ b/.gitlab-ci/container/create-cross-file.sh
@@ -47,8 +47,7 @@ if [[ -n "$GCC_ARCH" ]]; then
        echo "set(CMAKE_SYSTEM_PROCESSOR arm)";
        echo "set(CMAKE_C_COMPILER /usr/lib/ccache/$GCC_ARCH-gcc)";
        echo "set(CMAKE_CXX_COMPILER /usr/lib/ccache/$GCC_ARCH-g++)";
-        echo "set(CMAKE_CXX_FLAGS_INIT \"-Wno-psabi\")";  # makes ABI warnings quiet for ARMv7
-        echo "set(ENV{PKG_CONFIG} \"/usr/bin/$GCC_ARCH-pkgconf\")";
+        echo "set(ENV{PKG_CONFIG} \"/usr/bin/$GCC_ARCH-pkg-config\")";
        echo "set(DE_CPU $DE_CPU)";
    } > "$toolchain_file"
 fi
--- a/.gitlab-ci/container/create-rootfs.sh
+++ b/.gitlab-ci/container/create-rootfs.sh
@@ -0,0 +1,325 @@
+#!/bin/bash
+# shellcheck disable=SC2140  # ugly array, remove later
+# shellcheck disable=SC2288  # ugly array, remove later
+# shellcheck disable=SC2086 # we want word splitting
+
+set -ex
+
+if [ $DEBIAN_ARCH = arm64 ]; then
+    ARCH_PACKAGES="firmware-qcom-media
+                   firmware-linux-nonfree
+                   libfontconfig1
+                   libgl1
+                   libglu1-mesa
+                   libvulkan-dev
+    "
+elif [ $DEBIAN_ARCH = amd64 ]; then
+    # Add llvm 13 to the build image
+    apt-get -y install --no-install-recommends wget gnupg2 software-properties-common
+    apt-key add /llvm-snapshot.gpg.key
+    add-apt-repository "deb https://apt.llvm.org/bullseye/ llvm-toolchain-bullseye-13 main"
+    # Debian bullseye has older wine 5.0, we want >= 7.0 for traces.
+    apt-key add /winehq.gpg.key
+    apt-add-repository https://dl.winehq.org/wine-builds/debian/
+
+
+    ARCH_PACKAGES="firmware-amd-graphics
+                   inetutils-syslogd
+                   iptables
+                   libcap2
+                   libfontconfig1
+                   libelf1
+                   libfdt1
+                   libgl1
+                   libglu1-mesa
+                   libllvm13
+                   libllvm11
+                   libva2
+                   libva-drm2
+                   libvulkan-dev
+                   socat
+                   spirv-tools
+                   sysvinit-core
+                  "
+
+elif [ $DEBIAN_ARCH = armhf ]; then
+    ARCH_PACKAGES="firmware-misc-nonfree
+                  "
+fi
+
+INSTALL_CI_FAIRY_PACKAGES="git
+                           python3-dev
+                           python3-pip
+                           python3-setuptools
+                           python3-wheel
+                           "
+
+apt-get update
+apt-get -y install --no-install-recommends \
+    $ARCH_PACKAGES \
+    $INSTALL_CI_FAIRY_PACKAGES \
+    $EXTRA_LOCAL_PACKAGES \
+    bash \
+    ca-certificates \
+    firmware-realtek \
+    initramfs-tools \
+    jq \
+    libasan6 \
+    libexpat1 \
+    libpng16-16 \
+    libpython3.9 \
+    libsensors5 \
+    libvulkan1 \
+    libwaffle-1-0 \
+    libx11-6 \
+    libx11-xcb1 \
+    libxcb-dri2-0 \
+    libxcb-dri3-0 \
+    libxcb-glx0 \
+    libxcb-present0 \
+    libxcb-randr0 \
+    libxcb-shm0 \
+    libxcb-sync1 \
+    libxcb-xfixes0 \
+    libxdamage1 \
+    libxext6 \
+    libxfixes3 \
+    libxkbcommon0 \
+    libxrender1 \
+    libxshmfence1 \
+    libxxf86vm1 \
+    netcat-openbsd \
+    python3 \
+    python3-lxml \
+    python3-mako \
+    python3-numpy \
+    python3-packaging \
+    python3-pil \
+    python3-renderdoc \
+    python3-requests \
+    python3-simplejson \
+    python3-yaml \
+    sntp \
+    strace \
+    waffle-utils \
+    weston \
+    wget \
+    xinit \
+    xserver-xorg-core \
+    xwayland \
+    zstd
+
+
+if [ "$DEBIAN_ARCH" = "amd64" ]; then
+  # workaround wine needing 32-bit
+  # https://bugs.winehq.org/show_bug.cgi?id=53393
+  apt-get install -y --no-remove wine-stable-amd64  # a requirement for wine-stable
+  WINE_PKG="wine-stable"
+  WINE_PKG_DROP="wine-stable-i386"
+  apt download "${WINE_PKG}"
+  dpkg --ignore-depends="${WINE_PKG_DROP}" -i "${WINE_PKG}"*.deb
+  rm "${WINE_PKG}"*.deb
+  sed -i "/${WINE_PKG_DROP}/d" /var/lib/dpkg/status
+  apt-get install -y --no-remove winehq-stable  # symlinks-only, depends on wine-stable
+fi
+
+# Needed for ci-fairy, this revision is able to upload files to
+# MinIO and doesn't depend on git
+pip3 install git+http://gitlab.freedesktop.org/freedesktop/ci-templates@ffe4d1b10aab7534489f0c4bbc4c5899df17d3f2
+
+# Needed for manipulation with traces yaml files.
+pip3 install yq
+
+apt-get purge -y \
+        $INSTALL_CI_FAIRY_PACKAGES
+
+passwd root -d
+chsh -s /bin/sh
+
+cat > /init <<EOF
+#!/bin/sh
+export PS1=lava-shell:
+exec sh
+EOF
+chmod +x  /init
+
+#######################################################################
+# Strip the image to a small minimal system without removing the debian
+# toolchain.
+
+# Copy timezone file and remove tzdata package
+rm -rf /etc/localtime
+cp /usr/share/zoneinfo/Etc/UTC /etc/localtime
+
+UNNEEDED_PACKAGES="
+        libfdisk1
+        "
+
+export DEBIAN_FRONTEND=noninteractive
+
+# Removing unused packages
+for PACKAGE in ${UNNEEDED_PACKAGES}
+do
+	echo ${PACKAGE}
+	if ! apt-get remove --purge --yes "${PACKAGE}"
+	then
+		echo "WARNING: ${PACKAGE} isn't installed"
+	fi
+done
+
+apt-get autoremove --yes || true
+
+# Dropping logs
+rm -rf /var/log/*
+
+# Dropping documentation, localization, i18n files, etc
+rm -rf /usr/share/doc/*
+rm -rf /usr/share/locale/*
+rm -rf /usr/share/X11/locale/*
+rm -rf /usr/share/man
+rm -rf /usr/share/i18n/*
+rm -rf /usr/share/info/*
+rm -rf /usr/share/lintian/*
+rm -rf /usr/share/common-licenses/*
+rm -rf /usr/share/mime/*
+
+# Dropping reportbug scripts
+rm -rf /usr/share/bug
+
+# Drop udev hwdb not required on a stripped system
+rm -rf /lib/udev/hwdb.bin /lib/udev/hwdb.d/*
+
+# Drop all gconv conversions && binaries
+rm -rf usr/bin/iconv
+rm -rf usr/sbin/iconvconfig
+rm -rf usr/lib/*/gconv/
+
+# Remove libusb database
+rm -rf usr/sbin/update-usbids
+rm -rf var/lib/usbutils/usb.ids
+rm -rf usr/share/misc/usb.ids
+
+rm -rf /root/.pip
+
+#######################################################################
+# Crush into a minimal production image to be deployed via some type of image
+# updating system.
+# IMPORTANT: The Debian system is not longer functional at this point,
+# for example, apt and dpkg will stop working
+
+UNNEEDED_PACKAGES="apt libapt-pkg6.0 "\
+"ncurses-bin ncurses-base libncursesw6 libncurses6 "\
+"perl-base "\
+"debconf libdebconfclient0 "\
+"e2fsprogs e2fslibs libfdisk1 "\
+"insserv "\
+"udev "\
+"init-system-helpers "\
+"cpio "\
+"passwd "\
+"libsemanage1 libsemanage-common "\
+"libsepol1 "\
+"gpgv "\
+"hostname "\
+"adduser "\
+"debian-archive-keyring "\
+"libegl1-mesa-dev "\
+"libegl-mesa0 "\
+"libgl1-mesa-dev "\
+"libgl1-mesa-dri "\
+"libglapi-mesa "\
+"libgles2-mesa-dev "\
+"libglx-mesa0 "\
+"mesa-common-dev "\
+"gnupg2 "\
+"software-properties-common " \
+
+# Removing unneeded packages
+for PACKAGE in ${UNNEEDED_PACKAGES}
+do
+	echo "Forcing removal of ${PACKAGE}"
+	if ! dpkg --purge --force-remove-essential --force-depends "${PACKAGE}"
+	then
+		echo "WARNING: ${PACKAGE} isn't installed"
+	fi
+done
+
+# Show what's left package-wise before dropping dpkg itself
+COLUMNS=300 dpkg-query -W --showformat='${Installed-Size;10}\t${Package}\n' | sort -k1,1n
+
+# Drop dpkg
+dpkg --purge --force-remove-essential --force-depends  dpkg
+
+# No apt or dpkg, no need for its configuration archives
+rm -rf etc/apt
+rm -rf etc/dpkg
+
+# Drop directories not part of ostree
+# Note that /var needs to exist as ostree bind mounts the deployment /var over
+# it
+rm -rf var/* srv share
+
+# ca-certificates are in /etc drop the source
+rm -rf usr/share/ca-certificates
+
+# No need for completions
+rm -rf usr/share/bash-completion
+
+# No zsh, no need for comletions
+rm -rf usr/share/zsh/vendor-completions
+
+# drop gcc python helpers
+rm -rf usr/share/gcc
+
+# Drop sysvinit leftovers
+rm -rf etc/init.d
+rm -rf etc/rc[0-6S].d
+
+# Drop upstart helpers
+rm -rf etc/init
+
+# Various xtables helpers
+rm -rf usr/lib/xtables
+
+# Drop all locales
+# TODO: only remaining locale is actually "C". Should we really remove it?
+rm -rf usr/lib/locale/*
+
+# partition helpers
+rm -rf usr/sbin/*fdisk
+
+# local compiler
+rm -rf usr/bin/localedef
+
+# Systemd dns resolver
+find usr etc -name '*systemd-resolve*' -prune -exec rm -r {} \;
+
+# Systemd network configuration
+find usr etc -name '*networkd*' -prune -exec rm -r {} \;
+
+# systemd ntp client
+find usr etc -name '*timesyncd*' -prune -exec rm -r {} \;
+
+# systemd hw database manager
+find usr etc -name '*systemd-hwdb*' -prune -exec rm -r {} \;
+
+# No need for fuse
+find usr etc -name '*fuse*' -prune -exec rm -r {} \;
+
+# lsb init function leftovers
+rm -rf usr/lib/lsb
+
+# Only needed when adding libraries
+rm -rf usr/sbin/ldconfig*
+
+# Games, unused
+rmdir usr/games
+
+# Remove pam module to authenticate against a DB
+# plus libdb-5.3.so that is only used by this pam module
+rm -rf usr/lib/*/security/pam_userdb.so
+rm -rf usr/lib/*/libdb-5.3.so
+
+# remove NSS support for nis, nisplus and hesiod
+rm -rf usr/lib/*/libnss_hesiod*
+rm -rf usr/lib/*/libnss_nis*
--- a/.gitlab-ci/container/cross_build.sh
+++ b/.gitlab-ci/container/cross_build.sh
@@ -1,68 +1,61 @@
-#!/usr/bin/env bash
+#!/bin/bash
 # shellcheck disable=SC2086 # we want word splitting

 set -e
 set -o xtrace

 export DEBIAN_FRONTEND=noninteractive
-export LLVM_VERSION="${LLVM_VERSION:=15}"

 # Ephemeral packages (installed for this script and removed again at the end)
-EPHEMERAL=(
-)
-
-DEPS=(
-    "crossbuild-essential-$arch"
-    "pkgconf:$arch"
-    "libasan8:$arch"
-    "libdrm-dev:$arch"
-    "libelf-dev:$arch"
-    "libexpat1-dev:$arch"
-    "libffi-dev:$arch"
-    "libpciaccess-dev:$arch"
-    "libstdc++6:$arch"
-    "libvulkan-dev:$arch"
-    "libx11-dev:$arch"
-    "libx11-xcb-dev:$arch"
-    "libxcb-dri2-0-dev:$arch"
-    "libxcb-dri3-dev:$arch"
-    "libxcb-glx0-dev:$arch"
-    "libxcb-present-dev:$arch"
-    "libxcb-randr0-dev:$arch"
-    "libxcb-shm0-dev:$arch"
-    "libxcb-xfixes0-dev:$arch"
-    "libxdamage-dev:$arch"
-    "libxext-dev:$arch"
-    "libxrandr-dev:$arch"
-    "libxshmfence-dev:$arch"
-    "libxxf86vm-dev:$arch"
-    "libwayland-dev:$arch"
-)
+STABLE_EPHEMERAL=" \
+        "

 dpkg --add-architecture $arch
-
-echo "deb [trusted=yes] https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/raw/${PKG_REPO_REV}/ ${FDO_DISTRIBUTION_VERSION%-*} main" | tee /etc/apt/sources.list.d/gfx-ci_.list
-
 apt-get update

-apt-get install -y --no-remove "${DEPS[@]}" "${EPHEMERAL[@]}" \
-    $EXTRA_LOCAL_PACKAGES
+apt-get install -y --no-remove \
+        $STABLE_EPHEMERAL \
+        crossbuild-essential-$arch \
+        libelf-dev:$arch \
+        libexpat1-dev:$arch \
+        libffi-dev:$arch \
+        libpciaccess-dev:$arch \
+        libstdc++6:$arch \
+        libvulkan-dev:$arch \
+        libx11-dev:$arch \
+        libx11-xcb-dev:$arch \
+        libxcb-dri2-0-dev:$arch \
+        libxcb-dri3-dev:$arch \
+        libxcb-glx0-dev:$arch \
+        libxcb-present-dev:$arch \
+        libxcb-randr0-dev:$arch \
+        libxcb-shm0-dev:$arch \
+        libxcb-xfixes0-dev:$arch \
+        libxdamage-dev:$arch \
+        libxext-dev:$arch \
+        libxrandr-dev:$arch \
+        libxshmfence-dev:$arch \
+        libxxf86vm-dev:$arch \
+        libwayland-dev:$arch \
+        wget

 if [[ $arch != "armhf" ]]; then
-    # We don't need clang-format for the crossbuilds, but the installed amd64
-    # package will conflict with libclang. Uninstall clang-format (and its
-    # problematic dependency) to fix.
-    apt-get remove -y "clang-format-${LLVM_VERSION}" "libclang-cpp${LLVM_VERSION}"
+    # See the list of available architectures in https://apt.llvm.org/bullseye/dists/llvm-toolchain-bullseye-13/main/
+    if [[ $arch == "s390x" ]] || [[ $arch == "i386" ]] || [[ $arch == "arm64" ]]; then
+        LLVM=13
+    else
+        LLVM=11
+    fi

    # llvm-*-tools:$arch conflicts with python3:amd64. Install dependencies only
    # with apt-get, then force-install llvm-*-{dev,tools}:$arch with dpkg to get
    # around this.
    apt-get install -y --no-remove --no-install-recommends \
-            "libclang-cpp${LLVM_VERSION}:$arch" \
-            "libgcc-s1:$arch" \
-            "libtinfo-dev:$arch" \
-            "libz3-dev:$arch" \
-            "llvm-${LLVM_VERSION}:$arch" \
+            libclang-cpp${LLVM}:$arch \
+            libgcc-s1:$arch \
+            libtinfo-dev:$arch \
+            libz3-dev:$arch \
+            llvm-${LLVM}:$arch \
            zlib1g
 fi

@@ -73,19 +66,19 @@ fi


 # dependencies where we want a specific version
-MULTIARCH_PATH=$(dpkg-architecture -A $arch -qDEB_TARGET_MULTIARCH)
-export EXTRA_MESON_ARGS="--cross-file=/cross_file-${arch}.txt -D libdir=lib/${MULTIARCH_PATH}"
+EXTRA_MESON_ARGS="--cross-file=/cross_file-${arch}.txt -D libdir=lib/$(dpkg-architecture -A $arch -qDEB_TARGET_MULTIARCH)"
+. .gitlab-ci/container/build-libdrm.sh
+
 . .gitlab-ci/container/build-wayland.sh

-. .gitlab-ci/container/build-directx-headers.sh
-
-apt-get purge -y "${EPHEMERAL[@]}"
+apt-get purge -y \
+        $STABLE_EPHEMERAL

 . .gitlab-ci/container/container_post_build.sh

 # This needs to be done after container_post_build.sh, or apt-get breaks in there
 if [[ $arch != "armhf" ]]; then
-    apt-get download llvm-"${LLVM_VERSION}"-{dev,tools}:"$arch"
-    dpkg -i --force-depends llvm-"${LLVM_VERSION}"-*_"${arch}".deb
-    rm llvm-"${LLVM_VERSION}"-*_"${arch}".deb
+    apt-get download llvm-${LLVM}-{dev,tools}:$arch
+    dpkg -i --force-depends llvm-${LLVM}-*_${arch}.deb
+    rm llvm-${LLVM}-*_${arch}.deb
 fi
--- a/.gitlab-ci/container/debian/android_build.sh
+++ b/.gitlab-ci/container/debian/android_build.sh
@@ -1,53 +1,67 @@
-#!/usr/bin/env bash
+#!/bin/bash
 # shellcheck disable=SC2086 # we want word splitting

-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_BUILD_TAG
-
 set -ex

-EPHEMERAL=(
-    autoconf
-    rdfind
-    unzip
-)
+EPHEMERAL="\
+         autoconf \
+         rdfind \
+         unzip \
+         "

-apt-get install -y --no-remove "${EPHEMERAL[@]}"
+apt-get install -y --no-remove $EPHEMERAL

 # Fetch the NDK and extract just the toolchain we want.
-ndk=$ANDROID_NDK
-curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-  -o $ndk.zip https://dl.google.com/android/repository/$ndk-linux.zip
+ndk=android-ndk-r21d
+wget -O $ndk.zip https://dl.google.com/android/repository/$ndk-linux-x86_64.zip
 unzip -d / $ndk.zip "$ndk/toolchains/llvm/*"
 rm $ndk.zip
 # Since it was packed as a zip file, symlinks/hardlinks got turned into
 # duplicate files.  Turn them into hardlinks to save on container space.
-rdfind -makehardlinks true -makeresultsfile false /${ndk}/
+rdfind -makehardlinks true -makeresultsfile false /android-ndk-r21d/
 # Drop some large tools we won't use in this build.
-find /${ndk}/ -type f \( -iname '*clang-check*' -o -iname '*clang-tidy*' -o -iname '*lldb*' \) -exec rm -f {} \;
+find /android-ndk-r21d/ -type f | grep -E -i "clang-check|clang-tidy|lldb" | xargs rm -f

-sh .gitlab-ci/container/create-android-ndk-pc.sh /$ndk zlib.pc "" "-lz" "1.2.3" $ANDROID_SDK_VERSION
+sh .gitlab-ci/container/create-android-ndk-pc.sh /$ndk zlib.pc "" "-lz" "1.2.3"

-sh .gitlab-ci/container/create-android-cross-file.sh /$ndk x86_64-linux-android x86_64 x86_64 $ANDROID_SDK_VERSION
-sh .gitlab-ci/container/create-android-cross-file.sh /$ndk i686-linux-android x86 x86 $ANDROID_SDK_VERSION
-sh .gitlab-ci/container/create-android-cross-file.sh /$ndk aarch64-linux-android aarch64 armv8 $ANDROID_SDK_VERSION
-sh .gitlab-ci/container/create-android-cross-file.sh /$ndk arm-linux-androideabi arm armv7hl $ANDROID_SDK_VERSION armv7a-linux-androideabi
+sh .gitlab-ci/container/create-android-cross-file.sh /$ndk x86_64-linux-android x86_64 x86_64
+sh .gitlab-ci/container/create-android-cross-file.sh /$ndk i686-linux-android x86 x86
+sh .gitlab-ci/container/create-android-cross-file.sh /$ndk aarch64-linux-android arm armv8
+sh .gitlab-ci/container/create-android-cross-file.sh /$ndk arm-linux-androideabi arm armv7hl armv7a-linux-androideabi
+
+# Not using build-libdrm.sh because we don't want its cleanup after building
+# each arch.  Fetch and extract now.
+export LIBDRM_VERSION=libdrm-2.4.110
+wget https://dri.freedesktop.org/libdrm/$LIBDRM_VERSION.tar.xz
+tar -xf $LIBDRM_VERSION.tar.xz && rm $LIBDRM_VERSION.tar.xz

 for arch in \
        x86_64-linux-android \
        i686-linux-android \
        aarch64-linux-android \
        arm-linux-androideabi ; do
-    EXTRA_MESON_ARGS="--cross-file=/cross_file-$arch.txt --libdir=lib/$arch -Dnouveau=disabled -Dintel=disabled" \
-    . .gitlab-ci/container/build-libdrm.sh
+
+    cd $LIBDRM_VERSION
+    rm -rf build-$arch
+    meson build-$arch \
+          --cross-file=/cross_file-$arch.txt \
+          --libdir=lib/$arch \
+          -Dlibkms=false \
+          -Dnouveau=false \
+          -Dvc4=false \
+          -Detnaviv=false \
+          -Dfreedreno=false \
+          -Dintel=false \
+          -Dcairo-tests=false \
+          -Dvalgrind=false
+    ninja -C build-$arch install
+    cd ..
 done

 rm -rf $LIBDRM_VERSION

 export LIBELF_VERSION=libelf-0.8.13
-curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-  -O https://fossies.org/linux/misc/old/$LIBELF_VERSION.tar.gz
+wget https://fossies.org/linux/misc/old/$LIBELF_VERSION.tar.gz

 # Not 100% sure who runs the mirror above so be extra careful
 if ! echo "4136d7b4c04df68b686570afa26988ac ${LIBELF_VERSION}.tar.gz" | md5sum -c -; then
@@ -73,11 +87,11 @@ for arch in \
       ccarch=armv7a-linux-androideabi
    fi

-    export CC=/${ndk}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar
-    export CC=/${ndk}/toolchains/llvm/prebuilt/linux-x86_64/bin/${ccarch}${ANDROID_SDK_VERSION}-clang
-    export CXX=/${ndk}/toolchains/llvm/prebuilt/linux-x86_64/bin/${ccarch}${ANDROID_SDK_VERSION}-clang++
-    export LD=/${ndk}/toolchains/llvm/prebuilt/linux-x86_64/bin/${arch}-ld
-    export RANLIB=/${ndk}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ranlib
+    export CC=/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/${arch}-ar
+    export CC=/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/${ccarch}29-clang
+    export CXX=/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/${ccarch}29-clang++
+    export LD=/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/${arch}-ld
+    export RANLIB=/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/${arch}-ranlib

    # The configure script doesn't know about android, but doesn't really use the host anyway it
    # seems
@@ -90,4 +104,4 @@ done
 cd ..
 rm -rf $LIBELF_VERSION

-apt-get purge -y "${EPHEMERAL[@]}"
+apt-get purge -y $EPHEMERAL
--- a/.gitlab-ci/container/debian/arm64_build.sh
+++ b/.gitlab-ci/container/debian/arm64_build.sh
@@ -1,87 +0,0 @@
-#!/usr/bin/env bash
-# shellcheck disable=SC2086 # we want word splitting
-
-set -e
-set -o xtrace
-
-export LLVM_VERSION="${LLVM_VERSION:=15}"
-
-apt-get -y install ca-certificates
-sed -i -e 's/http:\/\/deb/https:\/\/deb/g' /etc/apt/sources.list.d/*
-echo "deb [trusted=yes] https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/raw/${PKG_REPO_REV}/ ${FDO_DISTRIBUTION_VERSION%-*} main" | tee /etc/apt/sources.list.d/gfx-ci_.list
-apt-get update
-
-# Ephemeral packages (installed for this script and removed again at the end)
-EPHEMERAL=(
-    libssl-dev
-)
-
-DEPS=(
-    apt-utils
-    android-libext4-utils
-    autoconf
-    automake
-    bc
-    bison
-    ccache
-    cmake
-    curl
-    fastboot
-    flex
-    g++
-    git
-    glslang-tools
-    kmod
-    libasan8
-    libdrm-dev
-    libelf-dev
-    libexpat1-dev
-    libvulkan-dev
-    libx11-dev
-    libx11-xcb-dev
-    libxcb-dri2-0-dev
-    libxcb-dri3-dev
-    libxcb-glx0-dev
-    libxcb-present-dev
-    libxcb-randr0-dev
-    libxcb-shm0-dev
-    libxcb-xfixes0-dev
-    libxdamage-dev
-    libxext-dev
-    libxrandr-dev
-    libxshmfence-dev
-    libxxf86vm-dev
-    libwayland-dev
-    libwayland-egl-backend-dev
-    "llvm-${LLVM_VERSION}-dev"
-    ninja-build
-    meson
-    openssh-server
-    pkgconf
-    python3-mako
-    python3-pil
-    python3-pip
-    python3-requests
-    python3-setuptools
-    u-boot-tools
-    xz-utils
-    zlib1g-dev
-    zstd
-)
-
-apt-get -y install "${DEPS[@]}" "${EPHEMERAL[@]}"
-
-pip3 install --break-system-packages git+http://gitlab.freedesktop.org/freedesktop/ci-templates@ffe4d1b10aab7534489f0c4bbc4c5899df17d3f2
-
-arch=armhf
-. .gitlab-ci/container/cross_build.sh
-
-. .gitlab-ci/container/container_pre_build.sh
-
-. .gitlab-ci/container/build-mold.sh
-
-. .gitlab-ci/container/build-wayland.sh
-
-apt-get purge -y "${EPHEMERAL[@]}"
-
-. .gitlab-ci/container/container_post_build.sh
--- a/.gitlab-ci/container/debian/arm_build.sh
+++ b/.gitlab-ci/container/debian/arm_build.sh
@@ -0,0 +1,92 @@
+#!/bin/bash
+# shellcheck disable=SC2086 # we want word splitting
+
+set -e
+set -o xtrace
+
+apt-get -y install ca-certificates
+sed -i -e 's/http:\/\/deb/https:\/\/deb/g' /etc/apt/sources.list
+echo 'deb https://deb.debian.org/debian buster main' >/etc/apt/sources.list.d/buster.list
+apt-get update
+
+# Ephemeral packages (installed for this script and removed again at
+# the end)
+STABLE_EPHEMERAL=" \
+        libssl-dev \
+        "
+
+apt-get -y install \
+	${EXTRA_LOCAL_PACKAGES} \
+	${STABLE_EPHEMERAL} \
+	autoconf \
+	automake \
+	bc \
+	bison \
+	ccache \
+	cmake \
+	debootstrap \
+	fastboot \
+	flex \
+	g++ \
+	git \
+	glslang-tools \
+	kmod \
+	libasan6 \
+	libdrm-dev \
+	libelf-dev \
+	libexpat1-dev \
+	libvulkan-dev \
+	libx11-dev \
+	libx11-xcb-dev \
+	libxcb-dri2-0-dev \
+	libxcb-dri3-dev \
+	libxcb-glx0-dev \
+	libxcb-present-dev \
+	libxcb-randr0-dev \
+	libxcb-shm0-dev \
+	libxcb-xfixes0-dev \
+	libxdamage-dev \
+	libxext-dev \
+	libxrandr-dev \
+	libxshmfence-dev \
+	libxxf86vm-dev \
+	libwayland-dev \
+	llvm-11-dev \
+	ninja-build \
+	pkg-config \
+	python3-mako \
+	python3-pil \
+	python3-pip \
+	python3-requests \
+	python3-setuptools \
+	u-boot-tools \
+	wget \
+	xz-utils \
+	zlib1g-dev \
+	zstd
+
+# Not available anymore in bullseye
+apt-get install -y --no-remove -t buster \
+        android-sdk-ext4-utils
+
+pip3 install git+http://gitlab.freedesktop.org/freedesktop/ci-templates@ffe4d1b10aab7534489f0c4bbc4c5899df17d3f2
+
+# We need at least 0.61.4 for proper Rust; 0.62 for modern meson env2mfile
+pip3 install meson==0.63.3
+
+arch=armhf
+. .gitlab-ci/container/cross_build.sh
+
+. .gitlab-ci/container/container_pre_build.sh
+
+. .gitlab-ci/container/build-mold.sh
+
+# dependencies where we want a specific version
+EXTRA_MESON_ARGS=
+. .gitlab-ci/container/build-libdrm.sh
+
+. .gitlab-ci/container/build-wayland.sh
+
+apt-get purge -y $STABLE_EPHEMERAL
+
+. .gitlab-ci/container/container_post_build.sh
--- a/.gitlab-ci/container/debian/arm_test.sh
+++ b/.gitlab-ci/container/debian/arm_test.sh
@@ -1,45 +1,39 @@
-#!/usr/bin/env bash
+#!/bin/bash
 # shellcheck disable=SC2154 # arch is assigned in previous scripts
-# When changing this file, you need to bump the following
-# .gitlab-ci/image-tags.yml tags:
-# DEBIAN_BASE_TAG
-# KERNEL_ROOTFS_TAG

 set -e
 set -o xtrace

 ############### Install packages for baremetal testing
-DEPS=(
-    cpio
-    curl
-    fastboot
-    netcat-openbsd
-    openssh-server
-    procps
-    python3-distutils
-    python3-filelock
-    python3-fire
-    python3-minimal
-    python3-serial
-    rsync
-    snmp
-    zstd
-)
-
 apt-get install -y ca-certificates
-sed -i -e 's/http:\/\/deb/https:\/\/deb/g' /etc/apt/sources.list.d/*
-echo "deb [trusted=yes] https://gitlab.freedesktop.org/gfx-ci/ci-deb-repo/-/raw/${PKG_REPO_REV}/ ${FDO_DISTRIBUTION_VERSION%-*} main" | tee /etc/apt/sources.list.d/gfx-ci_.list
+sed -i -e 's/http:\/\/deb/https:\/\/deb/g' /etc/apt/sources.list
 apt-get update

-apt-get install -y --no-remove "${DEPS[@]}"
+apt-get install -y --no-remove \
+        cpio \
+        fastboot \
+        netcat \
+        procps \
+        python3-distutils \
+        python3-minimal \
+        python3-serial \
+        rsync \
+        snmp \
+        wget \
+        zstd

 # setup SNMPv2 SMI MIB
-curl -L --retry 4 -f --retry-all-errors --retry-delay 60 \
-    https://raw.githubusercontent.com/net-snmp/net-snmp/master/mibs/SNMPv2-SMI.txt \
-    -o /usr/share/snmp/mibs/SNMPv2-SMI.txt
+wget https://raw.githubusercontent.com/net-snmp/net-snmp/master/mibs/SNMPv2-SMI.txt \
+    -O /usr/share/snmp/mibs/SNMPv2-SMI.txt

 . .gitlab-ci/container/baremetal_build.sh

+if [[ "$arch" == "arm64" ]]; then
+    # This firmware file from Debian bullseye causes hangs
+    wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/qcom/a530_pfp.fw?id=d5f9eea5a251d43412b07f5295d03e97b89ac4a5 \
+      -O /rootfs-arm64/lib/firmware/qcom/a530_pfp.fw
+fi
+
 mkdir -p /baremetal-files/jetson-nano/boot/
 ln -s \
    /baremetal-files/Image \
--- a/.gitlab-ci/container/debian/armhf_test.sh
+++ b/.gitlab-ci/container/debian/armhf_test.sh
--- a/.gitlab-ci/container/debian/x86_32_build.sh
+++ b/.gitlab-ci/container/debian/x86_32_build.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/bash

 arch=i386

--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Dylan Baker	667218a8d2	VERSION: bump for 23.0.0-rc2	2023-01-18 11:04:34 -08:00
Tapani Pälli	05e9555046	intel/compiler: add cpp_std=c++17 when building tests Otherwise build fails: "../src/intel/compiler/brw_private.h:40:4: note: ‘std::variant’ is only available from C++17 onwards" Fixes: `6c194ddd18` ("intel/compiler: Prepare SIMD selection helpers to handle different prog_datas") Signed-off-by: Tapani Pälli <tapani.palli@intel.com> Reviewed-by: José Roberto de Souza <jose.souza@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20725> (cherry picked from commit `53de48f1c4`)	2023-01-17 10:16:40 -08:00
Alyssa Rosenzweig	c1393d1430	panfrost: Fix clears with conditional rendering batch can be invalidated by the render condition check. Fixes nv_conditional_render-clear. Fixes: `638b22354e` ("panfrost: Clear with a quad to avoid flushing") Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20689> (cherry picked from commit `02f9cddb61`)	2023-01-17 10:16:40 -08:00
Alyssa Rosenzweig	7dd8437e09	panfrost: Fix logic ops on Bifrost opaque should not be set when logicops are enabled, that needs blending even on Bifrost. Fixes is for when I believe the bug became possible to hit. The logical error is older. Fixes Piglit logicop tests again. Fixes: `d849d9779a` ("panfrost: Avoid blend shader when not blending") Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20685> (cherry picked from commit `41d99c10d1`)	2023-01-17 10:16:40 -08:00
Alyssa Rosenzweig	6b139c2be7	panfrost: Enable NV_primitive_restart on Valhall Unlike literally every other mesa/st emulation, for some inexplicable reason we need to pretend to support the CAP and then set a different EMULATE cap instead of the emulation keying off the lack of support for the CAP. Set the CAPs accordingly so we get NV_primitive_restart (with emulation of non-fixed indices). This gets Mesa to advertise GL 3.1 on Mali-G57 as intended. Fixes: `30c14f54cf` ("panfrost: Disable PIPE_CAP_PRIMITIVE_RESTART on v9") Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20702> (cherry picked from commit `fe4dc59e99`)	2023-01-17 10:16:39 -08:00
Alyssa Rosenzweig	0b4c673810	pan/bi: Fix incorrect compilation of fsat(reg.yx) Future changes to nir_lower_blend cause fsat(reg.yx) instructions to be generated, which correspond to "FCLAMP.v2f16 x.h10" pseudoinstructions. These get their swizzles lowered, but we forgot to clear the swizzle out, so we end up with extra swap (cancelling out the intended swizzle). Fix the lowering logic. Fixes: `ac636f5adb` ("pan/bi: Use FCLAMP pseudo op for clamp prop") Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20683> (cherry picked from commit `ed46c617b0`)	2023-01-17 10:16:39 -08:00
Friedrich Vock	e8dd7a275e	radv/bvh: Prevent NANs when computing node cost Otherwise the degenerate geometry workaround never triggers, leading to bad performance. Fixes: `6f45c98b` ("radv/bvh: Adjust sah cost based on depth") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20723> (cherry picked from commit `684eee0748`)	2023-01-17 10:16:39 -08:00
Lionel Landwerlin	20f8f6ebc6	Revert "ci: build hasvk if we're building anv" This reverts commit `b4d3d11e43`. We're seeing problems on the builders running the hasvk tests. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20726> (cherry picked from commit `4121aa43c4`)	2023-01-17 10:16:38 -08:00
Alejandro Piñeiro	ec70534827	vulkan/wsi: check if image info was already freed We set the different data being freed to NULL after freeing it, and checks for NULL before freeing it. This fixes several double free crash with v3dv, when running OOM wsi tests, like for example: dEQP-VK.wsi.xlib.swapchain.simulate_oom.composite_alpha Although note that only one person got those on a new fresh install of the Raspbian OS, so this problem was rare. Fixes: `5b13d74583` ("vulkan/wsi/drm: Break create_native_image in pieces") Reviewed-by: Eric Engestrom <eric@igalia.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20695> (cherry picked from commit `b27e42dcb5`)	2023-01-17 10:16:23 -08:00
Vinson Lee	fdf09a06ca	radv: Fix memory leak. Fix defect reported by Coverity Scan. Resource leak (RESOURCE_LEAK) leaked_storage: Variable page going out of scope leaks the storage it points to. Fixes: `8d0e6c02c7` ("radv: Add RMV tracing utilities") Signed-off-by: Vinson Lee <vlee@freedesktop.org> Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20711> (cherry picked from commit `6986332c80`)	2023-01-17 10:16:18 -08:00
Friedrich Vock	35213f18b8	radv/rt: Divide by the correct workgroup size Improves build performance by around 25%. Fixes: `9369b407` ("radv: Use PLOC for BVH building") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20717> (cherry picked from commit `eab2c39951`)	2023-01-17 10:16:17 -08:00
Alyssa Rosenzweig	bbf4675847	panfrost: Don't use AFBC of sRGB luminance-alpha This isn't allowed for the same reason that AFBC of regular luminance-alpha isn't allowed (and will raise DATA_INVALID_FAULTs). Reorder the checks to ensure these formats are checked. Fixes Piglit texwrap GL_EXT_texture_sRGB-s3tc. Fixes: `476be5cb27` ("panfrost: Don't use texture format swizzles on v7") Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20686> (cherry picked from commit `5fdfd8044d`)	2023-01-17 10:16:17 -08:00
Jesse Natalie	668503534d	dzn: Fix clear bind flag logic This is the patch I had meant to merge Fixes: `8b79e6fb` ("dzn: No need to add another bind flag for clears if one is already present") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20361> (cherry picked from commit `f1faf30a5f`)	2023-01-17 10:16:16 -08:00
Dylan Baker	4b0a1e19d6	.pick_status.json: Update to `8084b412ca`	2023-01-17 10:16:15 -08:00
Bas Nieuwenhuizen	2114fdf5eb	aco: Pass correct number of coords to Vega 1D LOD instruction. If we pass a physical 2D texture descriptor we should also pass 2 coords. Otherwise it just uses the random content in the second register which ends up funny sometimes. Cc: mesa-stable Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20696> (cherry picked from commit `edca10e9c9`)	2023-01-13 13:52:42 -08:00
t0b3	54cfb552ab	nir/nir_opt_move: fix ALWAYS_INLINE compiler error Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Reviewed-by: Adam Jackson <ajax@redhat.com> Closes: #6825 Fixes: `f1d20ec6` ("nir/nir_opt_move: handle non-SSA defs ") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17439> (cherry picked from commit `267dd1f4d5`)	2023-01-13 13:52:42 -08:00
Samuel Pitoiset	264d64cc6c	radv: fix re-emitting RB+ when the non-compacted color format changes If the previously emitted graphics pipeline uses the value A for col_format_non_compacted and the new bound graphics pipeline uses B. At bind time, radv_cmd_state::col_format_non_compacted will be set to B and the rbplus flag will be dirtied. But if there is no draws and a new graphics pipeline is bound with the same value as A, the next draw will emit the rbplus state with B instead of A. This can be basically triggered with meta operations after drawing because the driver saves/restores the bound pipeline. Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8073 Fixes: `11469f7553` ("radv: copy the non-compacted color format at pipeline bind time") Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20692> (cherry picked from commit `5b3fb44ecc`)	2023-01-13 13:52:42 -08:00
Dylan Baker	a56dcf635e	.pick_status.json: Update to `a5a19903ab`	2023-01-13 13:52:38 -08:00
Dylan Baker	f081fa4047	VERSION: bump for rc1	2023-01-13 09:54:13 -08:00
Dylan Baker	376f8131d0	docs: add missing updates to new_features.txt I looked through the changes to features.txt since the 22.3 branchpoint and added those to the new_features list as well. It was pointed out that VK_KHR_present_wait wasn't in the list of new features.	2023-01-13 09:51:01 -08:00