Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CoreNrn: support for rebalanced files.dat #3147

Merged
merged 2 commits into from
Dec 5, 2024
Merged

Conversation

ferdonline
Copy link
Member

@ferdonline ferdonline commented Oct 29, 2024

For MMB (BBPP134-917) we need to be able to load a set of .dat files distributed in given way.
However for the moment CoreNeuron only reads it in plain RoundRobin, which is not suitable to load ranks a different number of files.

This PR

We introduce a mark value (-1 in this case) indicating no-data so that, even with round-robin, we can define precisely which rank will load a given data file. All ranks will be able to jump strides of size N_RANKS (at their respective offset) and find entries targeted at them. Upon seeing a -1 the rank knows it is done and can stop reading.

This is a minor addition which enabled us to load balance and run the MMB simulation

Copy link

codecov bot commented Oct 29, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 67.07%. Comparing base (2abbd25) to head (5a9d014).
Report is 70 commits behind head on master.

Files with missing lines Patch % Lines
src/coreneuron/io/nrn_setup.cpp 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3147      +/-   ##
==========================================
+ Coverage   67.06%   67.07%   +0.01%     
==========================================
  Files         571      569       -2     
  Lines      111319   111172     -147     
==========================================
- Hits        74658    74573      -85     
+ Misses      36661    36599      -62     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

✔️ d0f0cfe -> Azure artifacts URL

@bbpbuildbot

This comment has been minimized.

@ferdonline ferdonline changed the title Intoduce a sentinel value to allow loading rebalanced files Introduce a sentinel value to allow loading rebalanced files.dat Oct 29, 2024
@ferdonline ferdonline changed the title Introduce a sentinel value to allow loading rebalanced files.dat CoreNrn: support for rebalanced files.dat Oct 29, 2024
@ferdonline ferdonline requested a review from WeinaJi October 30, 2024 18:04
@ferdonline ferdonline marked this pull request as ready for review October 30, 2024 18:05
@ferdonline ferdonline requested a review from pramodk November 6, 2024 17:22
@ferdonline ferdonline force-pushed the leite/balanced_files2 branch from c72d108 to d0f0cfe Compare November 6, 2024 18:30
@ferdonline ferdonline force-pushed the leite/balanced_files2 branch from 1d5fab0 to 005577e Compare November 6, 2024 18:47
Copy link

✔️ 005577e -> Azure artifacts URL

@bbpbuildbot

This comment has been minimized.

@pramodk pramodk marked this pull request as draft November 10, 2024 08:04
@pramodk
Copy link
Member

pramodk commented Nov 10, 2024

@ferdonline : I am converting this to a draft assuming this is specific to our custom manipulation of files.dat for MMB simulation.

Copy link

✔️ 5a9d014 -> Azure artifacts URL

@ferdonline ferdonline marked this pull request as ready for review November 20, 2024 09:27
@ferdonline
Copy link
Member Author

@pramodk PR updated with a more informative comment

Copy link
Member

@pramodk pramodk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once Michael confirms above, we can merge this.

src/coreneuron/io/nrn_setup.cpp Show resolved Hide resolved
src/coreneuron/io/nrn_setup.cpp Show resolved Hide resolved
@1uc 1uc merged commit 942d5ef into master Dec 5, 2024
35 checks passed
@1uc 1uc deleted the leite/balanced_files2 branch December 5, 2024 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants