Skip to content

Commit

Permalink
feat: validate csv file
Browse files Browse the repository at this point in the history
  • Loading branch information
demeringo committed Nov 13, 2024
1 parent f821863 commit 39fe33a
Show file tree
Hide file tree
Showing 3 changed files with 59 additions and 6 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/validate-project-list-example.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:

jobs:

# Validate the example project-list
# Validate the format of the csv data
validate-sample-project:
runs-on: ubuntu-latest
steps:
Expand All @@ -19,4 +19,4 @@ jobs:
- name: Validate data
uses: frictionlessdata/repository@v2
with:
resources: old-examples/project-list/data/project-list.resources.yaml
resources: ictst/data/tools.resources.yaml
25 changes: 23 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ It defines a data model that can be used to build an inventory of the tools.
- [Components](#components)
- [Data format](#data-format)
- [draft dataset](#draft-dataset)
- [Example Datami widgets](#example-datami-widgets)
- [data validation in CI](#data-validation-in-ci)
- [Other examples of Datami widgets configration](#other-examples-of-datami-widgets-configration)
- [Mini server for local development](#mini-server-for-local-development)
- [Run local server](#run-local-server)

Expand Down Expand Up @@ -90,7 +91,27 @@ We use a [Datami](https://datami-docs.multi.coop/?locale=en) widget to display d

See [draft dataset](old-examples/ict-sustainability-tools.csv)

### Example Datami widgets
### data validation in CI

Data _format_ (not content) is automatically validated in CI using frictionless-ci in a github action triggered on PR's.

See [frictionless-ci | Frictionless Repository](https://repository.frictionlessdata.io/index.html)

```sh
# Local data validation
# See https://framework.frictionlessdata.io/docs/guides/validating-data.html

# Install frictionless cli
pip install frictionless
#
cd ictst/data
# validate just csv _global_ structure
frictionless validate tools.csv
# validate that format of fields matches description
frictionless validate tools.resources.yaml
```

### Other examples of Datami widgets configration

- A CSV file displayed without any customization [old-examples/csv/csv-widget-basic.html](old-examples/csv/csv-widget-basic.html)
- A CSV file displayed with some additional constraints on fields [old-examples/csv/csv-widget-with-constraints.html](old-examples/csv/csv-widget-with-constraints.html).
Expand Down
36 changes: 34 additions & 2 deletions ictst/data/tools.resources.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,16 @@ schema:
# The name of the organization that produces the tools
- name: organization
type: string
- name: website
type: string
- name: sub_module
type: string
- name: environmental_indicator
type: string
- name: quick_description
type: string
- name: assessment_type
type: string
#What is the general scope covered by the tool. Several values are possible but one should be chosen as the primary one for visual classification purpose.
- name: main_perimeter
type: string
Expand All @@ -35,5 +45,27 @@ schema:
- Software | Web Content
- Global | Digital Workspace
- To Be Qualified

# TODO: Add other fields
- name: other_perimeters
- name: assessed_functional_unit
- name: data_entry_points
- name: life_cycle_steps
- name: focus
- name: licence
- name: self_hostable
- name: originating_country
- name: logo
- name: inside_models
- name: methodology_transparency
- name: ref_sources
- name: status
- name: Creator
- name: Creation date
- name: Creator_comment
- name: reviewer_1
- name: review_1_date
- name: review_1_explanation
- name: reviewer_2
- name: review_2_date
- name: review_2_explanation
- name: Linked Publications
# Todo detail constraints on the fields.

0 comments on commit 39fe33a

Please sign in to comment.