Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data sources should share consistent metdata record types/entries in configuration #472

Open
carlhiggs opened this issue Aug 8, 2024 · 0 comments

Comments

@carlhiggs
Copy link
Collaborator

Currently, there are similar but different records gathered for different data sources when these are configured.

For example,

  • data paths (some entries ask for data, others for data_dir, regardless of whether a file or folder can be configured. Should just be consistent, e.g. 'data')
  • some ask for 'name', but others don't
  • some ask for 'licence', while others don't (e.g. in custom aggregation)
  • ditto citation

et cetera.

Really, there should be a base generic class for data sources that has the minimum required information that should be associated with data, and then specific datasets could build off this extending with data-specific attributes as required.

This will make configuration easier to both complete and code for (e.g. #414), by being consistent, and more maintainable as the type definitions will be centralised in shared classes.

Implementing this would mean a breaking change, as configuration formatting would be updated to use the more consistent record gathering. Potentially, a script to update/translate older configuration files to a newer format could be developed, but may not be necessary, e.g. if configuration files work with a specific software version. However, that may not be necessary, as we already record the study region template version in the header comment (e.g. v4.2.2). Future configuration versions could have this not in a comment, but rather as a parameter up the top that can be checked. In this way, older configurations could be distinguished from new ones. This is a seperate issue really.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant