Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Megan CTSM #96

Closed
wants to merge 227 commits into from
Closed

Conversation

rosiealice
Copy link
Collaborator

Description of changes

Megan changes without the fire emissions commits.

Specific notes

Same notes as the previous now closed PR
#74

Contributors other than yourself, if any:

CTSM Issues Fixed (include github issue #):
#36

Are answers expected to change (and if so in what way)?
Yes, all the MEG_* fields will change, but nothing else.

Any User Interface Changes (namelist or namelist defaults changes)?
No

Does this create a need to change or add documentation? Did you do so?
Yes, but haven;'t changed yet pending further coupled model testing.

Testing performed, if any:
(List what testing you did to show your changes worked as expected)
(This can be manual testing or running of the different test suites)
(Documentation on system testing is here: https://github.com/ESCOMP/ctsm/wiki/System-Testing-Guide)
(aux_clm on derecho for intel/gnu and izumi for intel/gnu/nag/nvhpc is the standard for tags on master)

NOTE: Be sure to check your coding style against the standard
(https://github.com/ESCOMP/ctsm/wiki/CTSM-coding-guidelines) and review
the list of common problems to watch out for
(https://github.com/ESCOMP/CTSM/wiki/List-of-common-problems).

johnpaulalex and others added 30 commits June 21, 2023 11:16
FATES API update to facilitate fates refactor

This updates a number of FATES type names and module use statements
which correspond with a refactoring effort that moves FATES
patches and cohorts into their own respective modules.

With the FATES update is a minor science update, so there are
changes to answers for FATES.

This also incorporates a minor update to a more recent version
of the ccs config external.
Bring b4b-dev branch to main CTSM development.

- PLUMBER2 for ctsm5.2 datasets
- Last bit of PPE changes for namelist and parameter file settings
- Update run_sys_tests on Derecho for compiler jobs to run using 16 tasks
- Bring in a fix for dust emissions for coupling with CAM

Update cs.status parsing script to make expected BASELINE fails more obvious

Fix some issues with finding IC files for certain lnd_tuning_modes: all for cam7,
   clm5_0_cam6.0, and clm6_0_cam6.0
hs-uii: Merge in latest b4b-dev
Add a namelist warning when hillslope hydrology is used with initial interpolation.
Fix comments on urban thermal variables (no testing done)
Also makes it accessible from table of contents. Not actually changing file contents at this point.
* The experience of making a mesh from an input with 1d vs. 2d coordinates is identical, so only provide one example.
* In example, use as input file the 5x5_amazon surface dataset included with CTSM.
* Various wording and typo fixes.
@mvertens
Copy link

Thanks @rosiealice. It also looks like the latest version of CIME does not permit you to do any tests with month bounds (i.e. _LmXX in the test name). We are using close to latest version of CIME in noresm2_5_alpha07 - whereas CTSM is using a version that is 6 months old. So I will change our tests for aux_clm_noresm to make sure we have no month specification in the test names.

@rosiealice
Copy link
Collaborator Author

rosiealice commented Nov 12, 2024

So, the model is trying to find the 'fates_turnover_leaf_canopy' variable in the parameter file. This is a quite recently introduced parameter, and so I would not be surprised if the cause was that we were using the wrong default fates_paramfile.

** In addition** to this parameter, the MEGAN code also introduces new parameters.

The solution to both of these is broadly to run with the parameter file that goes with this code which is here:
/cluster/work/users/rosief/git/NorESM_MEGAN_test/components/clm/src/fates/parameter_files/fates_params_default_megan.nc

How to integrate the parameter file in the tests is not clear to me. How does that normally work when you test a branch that needs a new parameter file? I guess this is something to do with the rest of the errors:
07/shell_commands' failed with error 'can't open file /cluster/projects/nn10013k/mvertens/src/noresm2_5_alpha07/src/fates/parameter_files/fates_params_default.cdl for reading:
which seem to be trying to do the automatic file generation thing (FATES stores its parameter files as a .cdl and then makes the netcdf files 'on the fly) that we were talking about with @mvdebolskiy the other day (and which he potentially correctly asserted would be annoying on account of their need for NCO modules, (or similar). But here it seems like the actual problem is with the fact that it can't find the parameter file in the path "/cluster/projects/nn10013k/mvertens/src/noresm2_5_alpha07/src/fates/parameter_files/fates_params_default.cdl". To ask an obvious question, is the file there?

@rosiealice
Copy link
Collaborator Author

I guess I was going to add, is it worth 'testing' the default version of the code first to troubleshoot these types of things?

@mvertens
Copy link

I agree. I can try to fire off a test with the default version. Are you referring to the latest version on master?

@rosiealice
Copy link
Collaborator Author

The version I used is the one linked from the alpha07 tag. So probably best to use that?

It was this CTSM
https://github.com/NorESMhub/CTSM/tree/ctsm5.3.0-noresm_v0

and this FATES
https://github.com/NGEET/fatessci.1.78.2_api.36.0.0

@mvertens
Copy link

mvertens commented Nov 12, 2024

@rosiealice - the problem seems to be that in lnd_in we are trying to read in
fates_paramfile = '/cluster/shared/noresm/inputdata/lnd/clm2/paramdata/fates_params_api.36.0.0_12pft_c240517.nc'.
And the same holds for the head of the ctsm noresm branch.

Since the fates branch you are pointing to is off of sci.1.78.3_api.36.1.0 (doing this from a git log on your branch).
The fates_paramfile is set in namelist_files/namelist_defaults_ctsm.xml. So to get your parameter file - all we have to do (I think) is simply modify this setting. @mvdebolskiy - do you agree with that?

@rosiealice
Copy link
Collaborator Author

Sounds about right @mvertens. Hopefully changing the setting will work. I wonder how we normally test things that need new parameter defaults?

@glemieux
Copy link

glemieux commented Nov 12, 2024

Sounds about right @mvertens. Hopefully changing the setting will work. I wonder how we normally test things that need new parameter defaults?

If I understand the question, when we have non-default parameter file values that need to be routinely tested, we use the on-the-fly netcdf generation scheme that you mentioned above. There is a PR that is 'stuck' to try and make this a little more robust, but we need to get back to it now that some upstream issues have been addressed: ESCOMP#2336.

I should note that alternate ways of generating and storing the files for testing has also been discussed over on the elm side of things: E3SM-Project/E3SM#6639 (comment)

@mvertens
Copy link

@glemieux - thanks for the clarification. That is super helpful.

Just to clarify - this is a parameter file that will need to accompany this PR - and not one that just needs to be routinely tested. So my understanding would be that we actually need to do the following:

  • copy /cluster/work/users/rosief/git/NorESM_MEGAN_test/components/clm/src/fates/parameter_files/fates_params_default_megan.nc in the standard CTSM inputdata with a new name
  • point to the above new param dataset in the namelist_defaults_ctsm.xml file entry for fates_paramfile
    Does that make sense?

@mvertens
Copy link

@rosiealice - what I have done is

  • create a personal CTSM branch from your branch in this PR
  • update this branch to ctsm5.3.11
  • add additional modifications in cime_config and bld that are needed for new noresm testing as well as compatibility for FATES to use megan.
  • start the testing again
    I'll keep you posted on the results.

@mvertens
Copy link

@mvdebolskiy - I cannot create a case with a stand-alone checkout of ctsm-noresm. I get the following error:
Batch_system_type is slurm
job is case.run USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None WALLTIME_FORMAT %H:%M:%S
WARNING: No queue on this system met the requirements for this job. Falling back to defaults
ERROR: No queues found
I am trying the following command:
./create_newcase --case foo --compset 2000_DATM%GSWP3v1_CLM60%FATES-SP_SICE_SOCN_SROF_SGLC_SWAV --res ne30pg3_ne30pg3_mtn14 --project nn9039k --run-unsupported --mach betzy
I have updated all of the externals in ctsm to be the same as in noresm2_5_alpha07.

@glemieux
Copy link

Just to clarify - this is a parameter file that will need to accompany this PR - and not one that just needs to be routinely tested. So my understanding would be that we actually need to do the following:

  • copy /cluster/work/users/rosief/git/NorESM_MEGAN_test/components/clm/src/fates/parameter_files/fates_params_default_megan.nc in the standard CTSM inputdata with a new name
  • point to the above new param dataset in the namelist_defaults_ctsm.xml file entry for fates_paramfile
    Does that make sense?

Gotcha, thanks for the clarification. What you've laid out is equivalent to what we do as well.

@mvertens
Copy link

@rosiealice - it turns out that we need this PR for noresm2_5_alpha08. What I found out today is the following:

  • running the test - SMS_D_Ld1.ne30pg3_tn14.N1850fates-sp.betzy_intel.allactive-defaultio - caused a crash in VOCEmissionMod.F90. Previously we were turning off megan in this test. With megan on, it does not work. Removing the debug flag caused the test to pass.
  • using the code for this PR along with some cime_config/ and bld/ modifications from me - SMS_D_Ld1.ne30pg3_tn14.N1850fates-sp.betzy_intel.allactive-defaultio passes.
  • What I want to do is the following:
    • accept your FATES PR
    • do a PR to your branch with my changes
    • then accept this PR and use that as the ctsm tag in noresm2_5_alpha08 There are still issues with some of the stand-alone tests - but most of the tests are now passing and the key test needed for alpha08 is fine.

@mvertens mvertens self-requested a review November 14, 2024 16:45
@mvertens mvertens mentioned this pull request Nov 17, 2024
@mvertens
Copy link

@rosiealice - it turns out that it will be much easier for me to close this PR and issue a new one with my changes on top of yours as well (which had some more relative to what you merged last week). I'll copy the PR summary exactly as you have it here. First I need to do some additional testing then I'll issue the PR again and have both you and @mvdebolskiy review it.
There should not be that many files since I just made a PR that had updated the noresm branch to ctsm5.3.011 - and these changes should be on top of that.

@rosiealice
Copy link
Collaborator Author

OK thanks @mvertens Let me know if I can help out at all...

@mvertens
Copy link

Closing in favor of #99

@mvertens
Copy link

closing in favor of #99

@mvertens mvertens closed this Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.