Transport improvements for 2024-W36 #225

khaeru · 2024-09-05T08:20:17Z

Postprocess SSP 2024 reporting output to compute aviation emissions.
- Initial implementation using a fixed share of total transportation emissions.
- ~~Compute absolute emissions using:~~ Postponed; next PR
  - Emission factors from GAINS, at the highest possible resolution.
  - Aviation share of total transport final energy, computed using latest IEA EWEB data.
Update freight technology representation
- Add transport mode F RAIL with two technologies.
- Add a technology for F ROAD that consumes commodity=electr.
Add scenarios:
- Add "SSP1-LED", "SSP2-LED", "EDITS-CA", "EDITS-HA" scenarios to transport workflow.
  - For LED: replace PDT per capita (with mode dimension) from Schafer logic with values computed based on LED narrative; this is then multiplied by SSP-specific population.
  - For EDITS: replace PDT (absolute/not per-capita; with mode dimension) aggregated from ITF data. No multiplication is needed.
Update usage of IEA Extended World Energy Balances data for preparation of base-model data:
- Switch to the (IEA, 2024) edition of this data, from (OECD, 2022) edition.
- Use values for y=2019 to avoid calibrating to the “COVID-19 dip”.

Housekeeping:

Add a emission code list.
Update Read The Docs config.
Many others mentioned in doc/whatsnew.

How to review

View the preview build of the docs, particularly the “What's new?” sections 'Transport' and 'Others'.
Note that the CI checks all pass.
Skim the diff.
~~Run the command described here and confirm (a) it works and (b) the output is as desired.~~

PR checklist

Continuous integration checks all ✅
Add or expand tests; coverage checks both ✅
Add, expand, or update documentation.
Update doc/whatsnew.

codecov · 2024-09-05T08:27:00Z

Codecov Report

Attention: Patch coverage is 92.64706% with 80 lines in your changes missing coverage. Please review.

Project coverage is 76.6%. Comparing base (5993c55) to head (42edfce).
Report is 85 commits behind head on main.

Files with missing lines	Patch %	Lines
message_ix_models/report/operator.py	7.6%	12 Missing ⚠️
message_ix_models/model/transport/ustimes_ma3t.py	54.1%	11 Missing ⚠️
..._models/tests/model/transport/test_ustimes_ma3t.py	47.0%	9 Missing ⚠️
message_ix_models/util/_dataclasses.py	80.0%	8 Missing ⚠️
message_ix_models/types.py	0.0%	5 Missing ⚠️
...sage_ix_models/tests/model/transport/test_build.py	42.8%	4 Missing ⚠️
message_ix_models/util/context.py	96.5%	4 Missing ⚠️
message_ix_models/model/transport/operator.py	95.7%	3 Missing ⚠️
message_ix_models/testing/__init__.py	40.0%	3 Missing ⚠️
message_ix_models/util/scenarioinfo.py	86.9%	3 Missing ⚠️
... and 12 more

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #225     +/-   ##
=======================================
+ Coverage   75.8%   76.6%   +0.8%     
=======================================
  Files        203     211      +8     
  Lines      15733   16069    +336     
=======================================
+ Hits       11931   12321    +390     
+ Misses      3802    3748     -54

Files with missing lines	Coverage Δ
message_ix_models/cli.py	`93.4% <ø> (ø)`
message_ix_models/model/config.py	`100.0% <100.0%> (ø)`
message_ix_models/model/transport/base.py	`97.9% <100.0%> (ø)`
message_ix_models/model/transport/build.py	`95.3% <100.0%> (ø)`
message_ix_models/model/transport/config.py	`97.6% <100.0%> (-1.5%)`	⬇️
message_ix_models/model/transport/disutility.py	`100.0% <100.0%> (ø)`
message_ix_models/model/transport/emission.py	`95.1% <100.0%> (-0.2%)`	⬇️
message_ix_models/model/transport/factor.py	`91.1% <100.0%> (+<0.1%)`	⬆️
message_ix_models/model/transport/files.py	`98.6% <100.0%> (+<0.1%)`	⬆️
message_ix_models/model/transport/freight.py	`100.0% <100.0%> (ø)`
... and 61 more

- Use standard collections, e.g. list[str] instead of typing.List[str]. - Import certain types from collections.abc, instead of deprecated aliases in typing. - Omit .model.transport, handled in #225.

- Ubuntu 22.04 → 24.04 (latest available via RTD) - Python 3.10 → 3.13 (latest) - Sphinx 6.2.1 → 8.1.3 (latest) - sphinx-rtd-theme 1.2.2 → 3.0.2 - sphinxcontrib-bibtex 2.5.0 → 2.6.3

- Global variable becomes an instance variable for .report.Config instances. This allows the use of different callbacks within the same program scope. - Retain same defaults. - Mark .report.register() function as deprecated. - Adjust all usage of register(). - Remove fixture preserve_report_callbacks (no longer needed) and all usage. - Adjust and add tests.

- Use left-to-right order. - Add debug code to only display the step(s) to the target key(s).

- Transform IEA 'COUNTRY' codes to MESSAGEix-GLOBIOM convention of ISO 3166-1 alpha-3. - Handle "..", "x", and "c" null values. - Add utility functions get_mapping(), dir_fallback(). - Update and expand tests.

…in .transport.base for scaling of outputs to match the base model.

Update the R12_NAM values to the values calculated based on US & Canada data and aligned with IEA-EWEB values for 2019. The earlier lower value was a result of aligning with 2020 IEA-EWEB data, that inadvertently imported the COVID dip.

Update R12_NAM mode share to avoid covid dip from iea-eweb 2020 values, rather align with 2019 values

@r-aneeque

Per @r-aneeque update in Slack: https://iiasa-ece.slack.com/archives/CCFHDNA6P/p1733134486407199?thread_ts=1730218237.960269&cid=CCFHDNA6P

- Allow the pytest.MarkDecorator to attach itself to the subject function. - Handle AssertionError by default. - Add raises=… parameter to allow specifying additional exception classes.

Work around python/cpython#79721 on these versions.

Same work-around for pyam-iamc packaging issue as in .github/workflows/pytest.yaml; see #250.

khaeru added enh New features or functionality transport MESSAGEix-Transport variant or transport in the base model labels Sep 5, 2024

khaeru assigned khaeru and r-aneeque Sep 5, 2024

khaeru force-pushed the transport/2024-W36 branch 3 times, most recently from 0280909 to 755138c Compare October 2, 2024 15:03

khaeru added the p:SSP-2024 2024 SSP updates and ScenarioMIP label Oct 2, 2024

khaeru force-pushed the transport/2024-W36 branch from 755138c to ecd58ac Compare October 3, 2024 08:47

khaeru mentioned this pull request Oct 30, 2024

Add scripts to modify "Low" and "Very Low" ScenarioMIP/SSP scenarios #239

Merged

khaeru force-pushed the transport/2024-W36 branch 2 times, most recently from e2c4833 to 2d28acb Compare November 5, 2024 14:32

khaeru force-pushed the transport/2024-W36 branch 4 times, most recently from bbcce57 to bf145a3 Compare November 19, 2024 23:22

khaeru force-pushed the transport/2024-W36 branch 2 times, most recently from 24b599e to 38bbcfa Compare November 20, 2024 18:22

khaeru force-pushed the transport/2024-W36 branch 4 times, most recently from a8ecba5 to e83dc13 Compare November 27, 2024 11:32

khaeru force-pushed the transport/2024-W36 branch 2 times, most recently from 5eaa074 to 98da0eb Compare November 29, 2024 13:36

khaeru and others added 15 commits December 3, 2024 12:35

Update docs and RTD requirements

f1993d5

- Ubuntu 22.04 → 24.04 (latest available via RTD) - Python 3.10 → 3.13 (latest) - Sphinx 6.2.1 → 8.1.3 (latest) - sphinx-rtd-theme 1.2.2 → 3.0.2 - sphinxcontrib-bibtex 2.5.0 → 2.6.3

Adjust workflow diagram in make_click_command()

d3d0c49

- Use left-to-right order. - Add debug code to only display the step(s) to the target key(s).

Add IEA WEB 2024 code lists in SDMX format

1f194f0

Handle 2024 data via IEA in .iea.web

0ff6ee6

- Transform IEA 'COUNTRY' codes to MESSAGEix-GLOBIOM convention of ISO 3166-1 alpha-3. - Handle "..", "x", and "c" null values. - Add utility functions get_mapping(), dir_fallback(). - Update and expand tests.

Use y=2019 data from (IEA, 2024) edition of EWEB

03c2c71

…in .transport.base for scaling of outputs to match the base model.

Add fuzzed private data for testing .iea.web 2024 edition

403a1eb

Tolerate missing labels in MappingAdapter.adapt

416eb9d

Adjust test marks for #225

8047a1c

Improve verbosity in CliRunner.assert_exit_0

75b50b5

Handle function when computing cache keys

b194af7

Update pdt-cap-ref for R12_NAM

c9cb436

Update the R12_NAM values to the values calculated based on US & Canada data and aligned with IEA-EWEB values for 2019. The earlier lower value was a result of aligning with 2020 IEA-EWEB data, that inadvertently imported the COVID dip.

Update R12_NAM mode share

6ffd7c0

Update R12_NAM mode share to avoid covid dip from iea-eweb 2020 values, rather align with 2019 values

Update R12/pdt-cap.csv values for s=LED

3d016c3

Per @r-aneeque update in Slack: https://iiasa-ece.slack.com/archives/CCFHDNA6P/p1733134486407199?thread_ts=1730218237.960269&cid=CCFHDNA6P

Improve .util.minimum_version()

b8d9622

- Allow the pytest.MarkDecorator to attach itself to the subject function. - Handle AssertionError by default. - Add raises=… parameter to allow specifying additional exception classes.

khaeru added a commit that referenced this pull request Dec 3, 2024

Add #225 to doc/whatsnew

a03767f

khaeru force-pushed the transport/2024-W36 branch from 3816930 to a03767f Compare December 3, 2024 11:39

khaeru added a commit that referenced this pull request Dec 3, 2024

Add #225 to doc/whatsnew

84ac564

khaeru force-pushed the transport/2024-W36 branch from a03767f to 84ac564 Compare December 3, 2024 13:02

khaeru added 4 commits December 3, 2024 15:05

Backport dataclasses.asdict(…) for Python ≤ 3.11

d5d7778

Work around python/cpython#79721 on these versions.

Adjust RTD config for Python 3.13

e77764f

Same work-around for pyam-iamc packaging issue as in .github/workflows/pytest.yaml; see #250.

Add subheadings to doc/whatsnew “Next release”

ffaf4ff

Add #225 to doc/whatsnew

42edfce

khaeru force-pushed the transport/2024-W36 branch from 84ac564 to 42edfce Compare December 3, 2024 14:05

khaeru merged commit c5e275c into main Dec 3, 2024
30 checks passed

khaeru deleted the transport/2024-W36 branch December 3, 2024 14:49

khaeru mentioned this pull request Dec 3, 2024

Fix issue with materials and cooling CCS technologies becoming cheaper than non-CCS counterpart #258

Merged

4 tasks

GamzeUnlu95 pushed a commit that referenced this pull request Dec 4, 2024

Adjust test marks for #225

b1cf6a4

GamzeUnlu95 pushed a commit that referenced this pull request Dec 4, 2024

Add #225 to doc/whatsnew

5ac2c59

khaeru mentioned this pull request Dec 4, 2024

Collect pieces for SSP 2024/ScenarioMIP #235

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Transport improvements for 2024-W36 #225

Transport improvements for 2024-W36 #225

khaeru commented Sep 5, 2024 •

edited

Loading

codecov bot commented Sep 5, 2024 •

edited

Loading

Transport improvements for 2024-W36 #225

Transport improvements for 2024-W36 #225

Conversation

khaeru commented Sep 5, 2024 • edited Loading

How to review

PR checklist

codecov bot commented Sep 5, 2024 • edited Loading

Codecov Report

khaeru commented Sep 5, 2024 •

edited

Loading

codecov bot commented Sep 5, 2024 •

edited

Loading