Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add kaya_variables, kaya_factors, and kaya_lmdi methods to the comput… #884

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

zacharyschmidt
Copy link

…e module. Also add the kaya subdirectory that contains the implementation for the kaya methods. (#875)

Please confirm that this PR has done the following:

  • Tests Added
  • Documentation Added
  • Name of contributors Added to AUTHORS.rst
  • Description in RELEASE_NOTES.md Added

Description of PR

This PR adds three methods to the public api of the compute module: kaya_variables(), kaya_factors(), and kaya_lmdi(). The implementation for these methods is in the new kaya submodule.

…e module. Also add the kaya subdirectory that contains the implementation for the kaya methods. (IAMconsortium#875)
Copy link

codecov bot commented Nov 2, 2024

Codecov Report

Attention: Patch coverage is 96.14325% with 14 lines in your changes missing coverage. Please review.

Project coverage is 95.1%. Comparing base (ddbb88e) to head (b741dbb).
Report is 33 commits behind head on main.

Files with missing lines Patch % Lines
pyam/compute.py 87.2% 7 Missing ⚠️
pyam/kaya/kaya_factors.py 88.8% 4 Missing ⚠️
pyam/kaya/kaya_variables.py 94.9% 3 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #884     +/-   ##
=======================================
+ Coverage   95.0%   95.1%   +0.1%     
=======================================
  Files         64      74     +10     
  Lines       6134    6579    +445     
=======================================
+ Hits        5828    6260    +432     
- Misses       306     319     +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@danielhuppmann
Copy link
Member

Thanks for the PR! Looks like the legacy-test is failing is because the unit's are ordered differently

model   scenario region variable unit                        year              
model_a scen_a   World  GNP/P    USD * billion / a / million 2010    NaN  0.006
                                 USD * billion / million / a 2010  0.006    NaN

Will take a look more closely next week, don't worry about the failing test for now, will think about how to avoid that.

Copy link
Member

@danielhuppmann danielhuppmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a first look at the PR and made a few comments for improvements, Quite a lot of material here...

It would be helpful if you could remove the LMDI decomposition and add that as a separate PR later.

pyam/compute.py Outdated
Comment on lines 260 to 261
scenarios : iterable of tuples (model, scenario, region)
The (model, scenario, region) combinations to be included.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering about two issues with the scenarios filter:

  1. Why not just compute kaya-variables for all data in the IamDataFrame?
  2. If it's necessary to filter, why not allow filter-kwargs and add those below, like?
    def kaya_variables(self, append=False, **kwargs)
        _data = self._df.filter(**kwargs)
       ...

I'm concerned about adding an argument "scenarios" that is actually a combination of model-scenario-region, that's quite confusing...

Copy link
Author

@zacharyschmidt zacharyschmidt Dec 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the scenarios filter and followed suggestion 1--we'll try to compute kaya-variables for all scenario/model/region combinations in the dataframe. I didn't try to implement returning partial sets of kaya-variables for scenario/model/region combinations that have only partial input data (though, if you think this is an important feature I'll give it a try). There's a new helper function in the kaya_variables module, _is_input_data_incomplete(), that relies on the require_data() method to make sure the data is complete enough so that none of the arithmetic operations in later processing will throw errors.

require_data() only considers model/scenario combinations (it does not care if all variables are present in all regions), but I want to alert the user when regions without full input data are present, so there is extra logic in _is_input_data_incomplete() to log this information.

pyam/compute.py Outdated
@@ -249,6 +251,248 @@ def bias(self, name, method, axis):
"""
_compute_bias(self._df, name, method, axis)

def kaya_variables(self, scenarios, append=False):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a method kaya_variables and an imported module with the same name, which then also has a method of that name... Hard to follow the code logic here.

tests/test_feature_kaya_factors.py Outdated Show resolved Hide resolved
pyam/kaya/kaya_variables.py Outdated Show resolved Hide resolved
pyam/kaya/kaya_factors.py Outdated Show resolved Hide resolved
pyam/kaya/lmdi_names.py Outdated Show resolved Hide resolved
pyam/compute.py Outdated Show resolved Hide resolved
pyam/compute.py Outdated Show resolved Hide resolved
pyam/kaya/kaya_factors.py Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants