Replace global path variables with ProjectPaths dataclass
- Add explicit file existence check before loading YAML
- Enhance tag retrieval by checking environment variables first
- Add logging for better debugging of tag selection process
- Remove redundant file existence check in main function
- Improve error handling for missing conditional tags file
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33863>
Make structural tagging functions available for both test and build
scripts.
Introduces the update_tag.sh helper for listing, checking, and updating
deterministic tags.
Also adds the ci_tag_build_time_check and ci_tag_test_time_check
functions to validate tags during build and test phases, ensuring
consistent component versioning.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33421>
Currently marge will ignore an error message if it is just the word
"error" without any further information. However, if she never finds a
more informative message, then she will just print that meaningless
error message.
Instead of an empty error message, just leave the message blank.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32773>
If marge doesn't find deqp-runner results or trace errors, she will parse
a failed job log for error messages. The parsing ignores certain phrases
in an attempt to find more relevant errors.
Add a test to check marge's error log parsing to make sure that we don't
create regressions when adding new phrases to ignore.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32773>
Use the CustomLogger class and CLI tool to create strutured logs
for poe scripts which are used by broadcom and nouveau jobs.
Renamed stage lint to code-validation and added python-test job
which runs the tests for structured and customer logger to ci.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25179>
This commit introduces the CustomLogger class, which provides methods
for updating, creating, and managing dut jobs and phases in a structured
log in below json format.
{
"_timestamp": "2023-10-05T06:16:42.603921",
"dut_job_type": "rpi3",
"farm": "igalia",
"dut_jobs": [
{
"status": "pass",
"submitter_start_time": "2023-10-05T06:16:42.745862",
"dut_start_time": "2023-10-05T06:16:42.819964",
"dut_submit_time": "2023-10-05T06:16:45.866096",
"dut_end_time": "2023-10-05T06:24:13.533394",
"dut_name": "igalia-ci01-rpi3-1gb",
"dut_state": "finished",
"dut_job_phases": [
{
"name": "boot",
"start_time": "2023-10-05T06:16:45.865863",
"end_time": "2023-10-05T06:17:14.801002"
},
{
"name": "test",
"start_time": "2023-10-05T06:17:14.801009",
"end_time": "2023-10-05T06:24:13.610296"
}
],
"submitter_end_time": "2023-10-05T06:24:13.680729"
}
],
"job_combined_status": "pass",
"dut_attempt_counter": 1
}
This class uses the existing StructuredLogger module,
which provides a robust and flexible logging utility supporting multiple
formats. It also includes a command-line tool for updating, creating,
and modifying dut jobs and phases through command-line arguments.
This can be used in ci job scripts to update information in structured logs.
Unit tests also have been added for the new class.
Currently, only LAVA jobs create structured log files, and this will be
extended for other jobs using this tool.
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25179>
This commit introduces the StructuredLogger module, which provides a
robust and flexible logging utility supporting multiple data formats
(CSV, JSON, and YAML). By incorporating this module into our CI system,
we enhance our log management capabilities, making it easier to:
1. Monitor and analyze logs: The StructuredLogger is a dict-like data
abstraction which autosaves into a structured data file, whenever it
is updated. With this file, one can easily know specifics of the job
execution without having to grep it in the traces logs or exploring
the job artifacts. The autosave feature makes it useful even when the
CI job fails unexpectedly, since the partial dict is always written
back to the disk.
2. Maintain data integrity: The module includes context managers for
file locking and editing log data, ensuring data integrity and
preventing race conditions.
3. Support multiple formats: With built-in support for CSV, JSON, and
YAML formats, this module caters to a wide range of use cases and
user preferences.
4. Increase maintainability: The modular design of the StructuredLogger
and its corresponding strategies simplifies maintenance and allows
for seamless integration of additional formats in the future.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22500>
