feat: add support for structured log info #216
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implementation notes:
structured_logs
array property to theLog
schema that allows implementers to provide well-defined structure log information for run (viaRunLog.run_log
) and task logs (viaRunLog.task_logs[]
).structured_logs
property uses a custom schema of typeStructuredLog
that defines a model or schema (log_type
) of a given version (log_type_version
) that describe the shape of the log contents (logs
). Optional properties comprise pointers to additional information, in particular a URL to the specification of the model or schema itself (spec_url
) and a URL to human-readable implementation notes (doc_url
) describing how to consume the log contents.log_type
are defined in theenum
LogType
and their mapping to the log contents inlogs
should be sufficiently well described in theenum
description such that server-side implementers can reasonably be expected to provide the logs in a uniform way, while client-side implementers can reasonably be expected to consume them as long as they provide support for the indicatedlog_type_version
.enum
LogType
is nullable. Specifyingnull
can be used to provide information on structured log formats that are not (yet) explicitly supported by WES. In such cases, it is strongly advised that at least one ofspec_url
ordoc_url
are provided to allow clients to interpret the log concents.Open questions:
StructureLog
andLogType
schemas could also be defined in a separate repository where schemas are kept that can be reused across multiple APIs (perhaps governed by the Cloud WS or DaMaSC?).logs
property was defined in a deliberately promiscuous way (it can take a string, an object or an array of arbitrary values), it is not clear if the proposed schema is sufficient for supporting a wide range of external log format models and schemas.string
item. However, if adding a set of concrete log formats goes too far at this point, these can also be added at a later point.enum
values viaoneOf
andconst
(see here). Adopting this would greatly improve upon the maintainability of theLogType
enum
. However, it would require to upgrade WES to OpenAPI 3.1.Resolves #215