Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

6 generate graphs from icosahedral meshes #11

Merged
merged 84 commits into from
Jul 25, 2024
Merged

6 generate graphs from icosahedral meshes #11

merged 84 commits into from
Jul 25, 2024

Conversation

theissenhelen
Copy link
Collaborator

@theissenhelen theissenhelen commented Jul 2, 2024 •
edited
Loading

This PR adds the following functionality:

  • create nodes in icosahedral arrangement on the fly using two different methods (hexagonal and icosahedral).
  • option to create a multi scale graph with nodes on incrementally higher resolution levels
  • the new functionality adds nodes and edges to a HeteroData PyTorch-geometric object similar to [Feature] Global Graph Generation #1 and [Feature] Basic implementation of Node weighting #2.
  • The resulting HeteroData object can be used in Anemoi-training
  • corresponding unit tests

Checklist before requesting a review

  • I have performed a self-review of my code
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation and docstrings to reflect the changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have ensured that the code is still pip-installable after the changes and runs
  • I have ran this on single GPU
  • I have ran this on multi-GPU or multi-node

Issue ticket number and link

Issues #5, #6

This work would not have been possible without @JPXKQX and @JesperDramsch and the rest of the AIFS team!

@theissenhelen theissenhelen self-assigned this Jul 2, 2024
@theissenhelen theissenhelen linked an issue Jul 2, 2024 that may be closed by this pull request
Generate graphs from Icosahedral meshes #6
Closed
@theissenhelen theissenhelen added this to the Multi-Scale Grids milestone Jul 2, 2024
@theissenhelen theissenhelen linked an issue Jul 2, 2024 that may be closed by this pull request
Generate graphs from Hexagonal meshes #5
Closed
@theissenhelen theissenhelen added the enhancement New feature or request label Jul 2, 2024
JPXKQX and others added 25 commits July 5, 2024 16:36
@JPXKQX
Global Encoder-Processor-Decoder graph (#9)
3609681 
* feat: Initial implementation of global graphs

Co-authored by: Mario Santa Cruz <[email protected]>
Co-authored-by: Helen Theissen <[email protected]>
Co-authored-by: Sara Hahner <[email protected]>
Co-authored-by: Jesper Dramsch <[email protected]>
@JPXKQX
fix: attributes as torch.float32
a973c2d
@JPXKQX
new test: attributes must be float32
af111a6
@JPXKQX
fix typo
a8a1620
@JPXKQX
Homogeneize base builders
926c75b
@JPXKQX
improve test docstrings
f342073
@JPXKQX
homogeneize (name as class attribute)
9d9fea8
@JPXKQX
new input config
1b20845
@JPXKQX
new default
4e62431
@JPXKQX
Merge branch 'develop' into fix/global-version
6825850
@theissenhelen
feat: Initial implementation of global graphs
d25c47b 
Co-authored by: Mario Santa Cruz <[email protected]>
@JPXKQX @theissenhelen
add cli command
8f0415e
@JPXKQX @theissenhelen
Ignore .pt files
22fba0e
@JPXKQX @theissenhelen
run pre-commit
2879c7c
@JPXKQX @theissenhelen
docstring + log erros
1ca0633
@JPXKQX @theissenhelen
initial tests
277c231
@theissenhelen
feat: initial version of AttributeBuilder
7a4e9cb
@theissenhelen
refactor: separate into node edge attribute builders
96a6ed5
@JPXKQX @theissenhelen
feat: edge_length moved to edges/attributes.py
1c88ee8
@JPXKQX @theissenhelen
remove __init__
92988d7
@JPXKQX @theissenhelen
bugfix (encoder edge lengths) + refector
de07488
@JPXKQX @theissenhelen
feat: support path and dict for config argument
c900bfd
@JPXKQX @theissenhelen
fix: error
9768204
@theissenhelen
refactor: naming
74ef4cc
@theissenhelen
fix: pre-commit
91e15b3
JesperDramsch
JesperDramsch previously requested changes Jul 23, 2024
View reviewed changes
Copy link
Member

@JesperDramsch JesperDramsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very nice, just some additional things and then it's good to go.

Naming is much better thanks for the adjustments as well.

pyproject.toml Show resolved Hide resolved
src/anemoi/graphs/generate/hexagonal.py Show resolved Hide resolved
src/anemoi/graphs/generate/hexagonal.py Show resolved Hide resolved
src/anemoi/graphs/edges/builder.py Show resolved Hide resolved
src/anemoi/graphs/generate/icosahedral.py Outdated Show resolved Hide resolved
src/anemoi/graphs/generate/icosahedral.py Outdated Show resolved Hide resolved
src/anemoi/graphs/generate/icosahedral.py Show resolved Hide resolved
@JPXKQX JPXKQX requested a review from mchantry July 25, 2024 13:21
@mchantry mchantry dismissed JesperDramsch’s stale review July 25, 2024 13:45

Verbal approval from Jesper on remaining issues.

mchantry
mchantry approved these changes Jul 25, 2024
View reviewed changes
Copy link
Member

@mchantry mchantry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, looks good. Please merge.

@theissenhelen theissenhelen merged commit 746ea2b into develop Jul 25, 2024
4 checks passed
@theissenhelen theissenhelen deleted the 6-generate-graphs-from-icosahedral-meshes branch July 26, 2024 13:10
theissenhelen added a commit that referenced this pull request Sep 11, 2024
@theissenhelen @JPXKQX
@JesperDramsch @gmertes @iainrussell @Rilwan-Adewoyin @anaprietonem @pre-commit-ci @github-actions
Latest release 0.3.0 (#47)
17c2847 
* Fix (global version) (#12)

* Global Encoder-Processor-Decoder graph (#9)

* feat: Initial implementation of global graphs + fixes

Co-authored by: Mario Santa Cruz <[email protected]>
Co-authored-by: Helen Theissen <[email protected]>
Co-authored-by: Sara Hahner <[email protected]>
Co-authored-by: Jesper Dramsch <[email protected]>

* fix: attributes as torch.float32

* new test: attributes must be float32

* fix typo

* Homogeneize base builders

* improve test docstrings

* homogeneize (name as class attribute)

* new input config

* new default

* remove dataclass from attribute classes

* fix: config nodes name

* 6 generate graphs from icosahedral meshes (#11)

* Global Encoder-Processor-Decoder graph (#9)

* feat: Initial implementation of global graphs

Co-authored by: Mario Santa Cruz <[email protected]>
Co-authored-by: Helen Theissen <[email protected]>
Co-authored-by: Jesper Dramsch <[email protected]>

* fix: attributes as torch.float32

* new test: attributes must be float32

* fix typo

* Homogeneize base builders

* improve test docstrings

* homogeneize (name as class attribute)

* new input config

* new default

* feat: Initial implementation of global graphs

Co-authored by: Mario Santa Cruz <[email protected]>

* add cli command

* Ignore .pt files

* run pre-commit

* docstring + log erros

* initial tests

* feat: initial version of AttributeBuilder

* refactor: separate into node edge attribute builders

* feat: edge_length moved to edges/attributes.py

* remove __init__

* bugfix (encoder edge lengths) + refector

* feat: support path and dict for `config` argument

* fix: error

* refactor: naming

* fix: pre-commit

* feat: builders icosahedral

* feat: Add icosahedral graph generation

Co-authored-by: Mario Santa Cruz <[email protected]>

* refactor: remove create_shere

* feat: Icosahedral edge builder

* feat: hexagonal graph generation

Co-authored-by: Mario Santa Cruz <[email protected]>

* feat: hexagonal builders

* fix: AOI not implemented yet

* fix: abstractmethod and renaming

* chore: add dependencies

* test: add tests for trimesh

* test: add tests for hex (h3)

* fix: imports

* fix: output type

* refactor: delete unused file

* refactor: renaming and positioning

* feat: ensure src and dst always the same

* fix: imports

* fix: edge_name not supported

* test: add tests for TriIcosahedralEdges

* fix: assert missing for Hexagonal edges

* test: hexagonal edges

* fix: avoid same name

* fix: imports

* fix: conflicts

* update tests

* Include xhops to hexagonal edges

* docs: update docstrings

* fix: update attribute name

* refactor: rename multiscale nodes

* refactor: rename icosahedral nodes

* improve: clarity of function

* improve: function syntax

* refactor: simplify resolution assignment

* refactor: improve variable naming in icosahedral graph generation

* more comments

* refactor: naming

* refactor: separate into functions

* refactor: remove unused code

* refactor: remove unused options and rename

* doc: clarify cells and nodes

* fix: add return statements

* homogeneize: tri & hex edges

* naming: xhops to x_hops

* naming: x_hops in docstring

* docstring

* blank lines

* fix: add return statement

* simplify icoshaedral edges

* feat: select edge builder method based on ico node type

* test: adjust tests to new MultiScaleEdges

* docs: improve docstrings

* fix: remove LAM filtering for icosahedral, leave for next PR

* h3 (v4) not supported

---------

* [feature] add changelog (#21)

* ci: changelog check

* feat: add changelog

* fix: on develop

* Skip no-commit-to-branch in code QA (#17)

* Clean nodes after building the graph (#23)

* feat: clean graph of unneeded attributes after creation

Co-authored-by: Mario Santa Cruz <[email protected]>
Co-authored-by: Helen Theissen <[email protected]>
Co-authored-by: Jesper Dramsch <[email protected]>

* Make graph filename argument optional (#24)

* hotfix: make graph saving optional

Co-authored-by: Jesper Dramsch <[email protected]>
Co-authored-by: Mario Santa Cruz <[email protected]>
Co-authored-by: Helen Theissen <[email protected]>

* doc: update CHANGELOG

Co-authored-by: Jesper Dramsch <[email protected]>

* chore: add rtd pr preview

* HEALPix nodes (#16)

[feature] HEALPix node builder

Co-authored-by: theissenhelen <[email protected]>

* 25 integrate reusable workflows (#26)

* ci: add public pr label

* ci: add downstream-ci workflow

* ci: add ci-config

* fix: pre-commit

* docs: update changelog

* ci: switch main on downstream-ci

* ci: run ci on push on main

* Hotfix/clean graph (#28)

* Add binary dependencies to ci-config.yml (#29)

* Add binary dependencies to ci-config.yml

* docs: update changelog

---------

Co-authored-by: theissenhelen <[email protected]>

* ci: inherit pypi publish flow (#27)

* ci: inherit pypi publish flow

Co-authored-by: Jesper Dramsch <[email protected]>

* docs: update changelog

* ci: only downstream-ci for changes in src and tests

---------

Co-authored-by: Jesper Dramsch <[email protected]>

* Bugfix/31 ensure latlon input numpy array (#32)

* #31 Ensure lat lon values passed as numpy arrays

* fix formatting

* Updated CHANGELOG.md

* Remove bug in the graph cleaning function (#33)

* fix: remove the bug in the graph cleaning function

Co-authored-by: Jesper Dramsch <[email protected]>

* docs: update changelog

---------

Co-authored-by: Jesper Dramsch <[email protected]>
Co-authored-by: Gert Mertes <[email protected]>

* Move to reusable workflows for QA and Testing (#34)

* ci: move to reusable workflows for QA and Testing

* docs: changelog add ci reusable workflow

* ci: downstream ci ignore docs

* ci: remove anchor, unsupported by github

* ci: add changelog release updater (#35)

* ci: changelog updater PR (#36)

* ci: correct inputs typo

* ci: fix changelog updater

* docs: ci fixes changelog

* ci: premissions

* ci: typo

* Ci/fix pr permission (#37)

* ci: add pull-requests permissions

* ci: add on workflow dispatch

* ci: remove 3.9 tests (#39)

* ci: remove 3.9 tests

* ci: remove testing of python 3.9

* dics: update changelog

* Fix support for Python 3.9 (#38)

* fix: support py3.9

Co-authored-by: Jesper Dramsch <[email protected]>

* fix: update & homogeneize changelog

* fix: update tests to support py39

* fix: style

* ci: rollback 3.9 tests

* refactor: annotations not used in tests

---------

Co-authored-by: Jesper Dramsch <[email protected]>

* Ci/fix_double_notes (#40)

* ci: avoid duplication of release notes

* ci: try write-all

* ci: fix permissions

* docs: update changelog

* ci: fix indent

* Inspection tools (#22)


Co-authored-by: Ana Prieto Nemesio <[email protected]>
Co-authored-by: Helen Theissen <[email protected]>

* [pre-commit.ci] pre-commit autoupdate (#41)

updates:
- [github.com/psf/black-pre-commit-mirror: 24.4.2 → 24.8.0](psf/black-pre-commit-mirror@24.4.2...24.8.0)
- [github.com/astral-sh/ruff-pre-commit: v0.4.6 → v0.6.2](astral-sh/ruff-pre-commit@v0.4.6...v0.6.2)
- [github.com/tox-dev/pyproject-fmt: 2.1.3 → 2.2.1](tox-dev/pyproject-fmt@2.1.3...2.2.1)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [pre-commit.ci] pre-commit autoupdate (#43)

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.6.2 → v0.6.3](astral-sh/ruff-pre-commit@v0.6.2...v0.6.3)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* First version of documentation (#18)

Documentation web page
---------
Co-authored-by: Helen Theissen <[email protected]>
Co-authored-by: Jesper Dramsch <[email protected]>

* [Changelog] Update to 0.3.0 (#44)

* [create-pull-request] automated change

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: theissenhelen <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

---------

Co-authored-by: Mario Santa Cruz <[email protected]>
Co-authored-by: Mario Santa Cruz <[email protected]>
Co-authored-by: Jesper Dramsch <[email protected]>
Co-authored-by: Gert Mertes <[email protected]>
Co-authored-by: Mario Santa Cruz <[email protected]>
Co-authored-by: Iain Russell <[email protected]>
Co-authored-by: Rilwan (Akanni) Adewoyin <[email protected]>
Co-authored-by: Ana Prieto Nemesio <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: theissenhelen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JPXKQX JPXKQX JPXKQX left review comments

@mchantry mchantry mchantry approved these changes

@JesperDramsch JesperDramsch Awaiting requested review from JesperDramsch JesperDramsch is a code owner

Assignees

@theissenhelen theissenhelen

Labels
enhancement New feature or request
Projects
None yet
Milestone
Multi-Scale Grids
Development

Successfully merging this pull request may close these issues.

Generate graphs from Icosahedral meshes Generate graphs from Hexagonal meshes
4 participants
@theissenhelen @JesperDramsch @mchantry @JPXKQX