Commit Graph

24 Commits

Author SHA1 Message Date
Vignesh Raman
c3531d5fea ci: rename FORCE_KERNEL_TAG to EXTERNAL_KERNEL_TAG
FORCE_KERNEL_TAG allows testing kernel uprevs without rebuilding
containers by supplying an external kernel directly for booting on
hardware devices. Renaming it to EXTERNAL_KERNEL_TAG clarifies its
purpose, distinguishing it from KERNEL_TAG which rebuilds containers.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31795>
2024-10-28 02:18:27 +00:00
Daniel Stone
a9b03a8fc4 ci/bare-metal: Move preparation steps into section
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31602>
2024-10-20 11:32:42 +01:00
Daniel Stone
cff886d808 ci: Don't print structured log data URL
People who want to get it will know where to find it. Most people will
not care.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31602>
2024-10-20 11:32:42 +01:00
Daniel Stone
111c15ae4a ci/bare-metal: Don't move structured log file
Just create it in the right place to begin with.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31110>
2024-09-13 10:12:08 +01:00
Eric Engestrom
771e07ad93 ci/bare-metal: rename fastboot & cros-servo TEST_PHASE_TIMEOUT to TEST_PHASE_TIMEOUT_MINUTES to be coherent
Avoids the risk of accidentally copy/pasting the wrong variable name
from another baremetal job.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30409>
2024-07-31 09:00:25 +02:00
Vignesh Raman
dc66de3029 ci: bare-metal: cros-servo: Create strutured logs for a630
Use the CustomLogger class and CLI tool to create strutured
logs for cros-servo scripts. Below is an example for a630.

{
  "_timestamp": "2023-10-18T18:14:16.114117",
  "dut_job_type": "sdm845-cheza-r3",
  "farm": "google",
  "dut_jobs": [
    {
      "status": "pass",
      "submitter_start_time": "2023-10-18T18:14:16.255163",
      "dut_start_time": "2023-10-18T18:14:16.328396",
      "dut_submit_time": "2023-10-18T18:14:16.330469",
      "dut_end_time": "2023-10-18T18:33:47.034774",
      "dut_name": "google-freedreno-cheza-18",
      "dut_state": "finished",
      "dut_job_phases": [
        {
          "name": "boot",
          "start_time": "2023-10-18T18:14:16.329576",
          "end_time": "2023-10-18T18:14:24.495826"
        },
        {
          "name": "test",
          "start_time": "2023-10-18T18:14:24.495855",
          "end_time": "2023-10-18T18:33:47.136979"
        }
      ],
      "submitter_end_time": "2023-10-18T18:33:47.207702"
    }
  ],
  "job_combined_status": "pass",
  "dut_attempt_counter": 1
}

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25807>
2023-12-21 19:54:44 +00:00
David Heidelberg
5e44cee47d ci: inject gfx-ci/linux S3 artifacts without rebuilding containers
We need update kernel often. We need test kernel changes often.

Introduced `KERNEL_EXTERNAL_TAG` to differ between `KERNEL_TAG` which is
also used to rebuild the containers. We don't need rebuild containers
for the external kernel, so this way we don't have to.

Updating kernel goes wruuuuuum.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23563>
2023-11-07 12:22:09 +00:00
David Heidelberg
5174cae8c2 ci/baremetal: shorten BM_KERNEL to filename and BM_DTB to name only
We don't need the path, not at all when we use external kernel.

Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24646>
2023-08-14 17:18:04 +00:00
David Heidelberg
795a099f38 ci/baremetal: do not install curl, it's already there
curl is already installed in these images, drop it.

Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24646>
2023-08-14 17:18:04 +00:00
David Heidelberg
5941fc8ccd ci: enable shellcheck on whole .gitlab-ci
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21977>
2023-05-25 16:06:53 +02:00
David Heidelberg
796686af1b ci: migrate from wget to curl
Better error handling is more reliable.

Options:
 -L, follow location
 --retry, number of retries
 --retry-all-errors, does not fail on ALL errors, that's why there is -f
 -f, fail fast with no output at all on server errors
 --retry-delay, make curl sleep this amount of time before each retry

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20788>
2023-01-19 23:46:44 +00:00
Emma Anholt
5f09b1ebe9 ci/bare-metal: Add test phase timeouts to all boards.
This should help with "marge got stuck for an hour and all I got was this
failed job with no results/" when a system intermittently wedges.

This replaces the BM_POE_TIMEOUT ("did we get something on serial in the
last 3 minutes?") that rpi had, in favor of checking that the whole test
job gets through in 20 minutes.

Acked-by: Juan A. Suarez <jasuarez@igalia.com>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17096>
2022-06-21 21:38:25 +00:00
Daniel Stone
a86449cb96 ci/bare-metal: Factor out environment to a separate script
This will let us reuse the same environment generation for both
bare-metal and LAVA.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-11 12:13:00 +00:00
Eric Anholt
0f61f0142a ci/bare-metal: Allow wget of the kernel/dtb for kernel development.
It's useful for kernel dev to be able throw all of our testing
infrastructure at a risky kernel change, but it's expensive (time and
bandwidth) to roll new containers every time your rev your kernel.  Make
it so you can just point the env vars to your personal build you've
uploaded.

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6592>
2020-09-09 17:25:38 +00:00
Eric Anholt
0453a46f66 ci/bare-metal: Fix capturing of serial output as job artifacts.
I tried to put them in the wrong directory -- everything needs to go in
results/, which we want clean and ready before we start our job.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6529>
2020-09-03 23:22:43 +00:00
Eric Anholt
c63648121e ci/bare-metal: Convert the main cros-servo boot code to python
Switching this part to python makes the code clearer and cleans up our
logs as well.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6398>
2020-08-21 20:10:18 +00:00
Eric Anholt
b4374080d7 ci/bare-metal: Use a new serial buffer tool.
This one uses python threads to move some of our logic from shell
pipelines to python, and opens the door to doing better serial output
tracking in the future (the SerialBuffer.lines() method)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6398>
2020-08-21 20:10:18 +00:00
Eric Anholt
63805ccd3f ci/bare-metal: Fail early when we get stuck powering on a cheza.
I think I've seen about 3 of this error total so far, but waiting 60
minutes for the scripts to give up wastes marge time.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5651>
2020-06-26 16:54:07 +00:00
Eric Anholt
32143cba4d ci/bare-metal: Terminate the job with an error on kernel panic.
Otherwise, we'll time out after 60 minutes of waiting for the run to
complete.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2651
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
2020-06-12 23:34:44 +00:00
Eric Anholt
72fe7b98ea ci/bare-metal: Stop fetching the git tree.
Like for LAVA, make the tradeoff of moving the test scripts and data (55k)
into the artifacts in order to make the per-build jobs not have to pull
down the git tree (hundreds of MB when you don't hit a cached container
for your specific user, which I see happen multiple times a day in my CI
runs).

To do this, we have to be a bit more careful in some places about our
working directory potentially being dirty.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
2020-06-12 23:34:44 +00:00
Eric Anholt
a13209bdec ci/bare-metal: Reword the final output of the init script on the board.
I'm going to be adding tracie, which isn't deqp.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
2020-06-12 23:34:44 +00:00
Eric Anholt
3d5429d646 ci: Use rsync for initial nfsroot population on cheza.
rm -rf and then copying over all the contents again is a waste of time
when we'll almost always be using the same rootfs.  Saves about 30s of job
time.

Closes: #3065
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>
2020-06-09 18:28:18 +00:00
Christian Gmeiner
06d8171994 ci: extend expect-output.sh
We need to support different fastboot fetch strings for different
bootloader solutions. Lets extend expect-output.sh to support
multiple fetch strings (-f) and add support for error catch
strings (-e) to stop the CI run early.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5258>
2020-06-05 11:02:35 +00:00
Eric Anholt
c89a749f66 ci: Add scripts for controlling bare-metal chezas.
This will let us:

- deploy kernels for testing code depending on new kernel featuers
- Ensure a pristine state in the HW before starting our tests
- Avoid disk rot on the chezas taking them out (we'd lost 3/9 in a few
  months).

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5247>
2020-05-29 16:46:44 +00:00