Reduce time to validate data in CI

[demeringo]

Problem

Validating data in CI takes a long time, mainly to setup the validation environment.

Most of the time is spent install frictionless validation tools (https://github.com/frictionlessdata/frictionless-ci) using a dedicated github action which does not leverage a pre-backed container image.

Additionally, the action does not seem to be well maintained (use old version of node).

And last but not least, the output of current validation is not very practical:

• the result of validation is a json file that is stored as an artefact of the workflow and is difficut to access and read.
• the link to the dashboard that is expected to display the resuts does not work (See No access to frictionless runs reports #47)

Solution

See if we can use a custom container (build separately) for the validation step.

Ensure the result of the validation appears directly in the logs of the workflow.

Alternatives

Additional context or elements

It may also permit us to use an different tool to validate data like qsv #55

• https://aschmelyun.com/blog/using-docker-run-inside-of-github-actions/
• https://github.com/frictionlessdata/frictionless-ci

[demeringo]

I build such feature and docker image here: https://github.com/demeringo/csv-validator-docker-image

[demeringo]

An example of output for a failed validation can be viewed on this PR:
https://github.com/demeringo/csv-validator-docker-image/actions/runs/12000137639/job/33448873520?pr=1