Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Implement first outline of real time CI retrieval from Electricity maps #96

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

tillenglert
Copy link
Collaborator

Draft of real time ci retrieval, based on location!

So far only location of Germany is pulled (hardcoded), we need to see how to actually implement access, and check what kind of location/region codes we use. Apart from that usage of token is used, and if no access to the internet is given the ci table is used to collect the ci.

@skrakau
Copy link
Collaborator

skrakau commented Dec 10, 2024

Would need to check if API call for each Nextflow run would cause problem and how to circumvent this.

@tillenglert
Copy link
Collaborator Author

Some comments what needs to be done here:

  1. First of all ElectricityMaps needs to be contacted to check in
    1.1. if their servers can handle api calls masses if all nextflow pipelines would use the plugin.
    1.2. if we can distribute their data like that (we do not necessarily use one api key for all, but each user would (optionally) register and create their own API access token.
    1.3. We would need to access yearly data and aggregate it to a table which can be used offline (or without access tokens)
  2. We need to align our location coding to ElecricityMaps. Therefore, we would need a backup table (mentioned in 1.3) that can be used for offline runs or users without api access. The user would then choose their region based of the Electricity maps app, and use the code, it is similar to what is implemented currently, but not by country-region, but e.g. in America by ?provider? and one location of our table is not mapping to one region of electricity maps.
    2.1. The documentation needs to be adjusted and ElecticityMaps needs to be cited, attributed!
  3. A hierarchical handling needs to be implemented to choose the appropriate locations ci: zone>country>world

@tillenglert
Copy link
Collaborator Author

Also I just realised, I implemented it, so that the CI is pulled right at the beginning of the pipeline run, it would be better to pull a new CI for each task actually, especially for longer pipeline runs! Downside -> the burden on ElectricityMaps server would be even greater! As not each pipeline run requests an CI but each task will request one!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants