Error when saving iris Cube : "ValueError: Cell measures with missing data are not supported."

[mpmoine]

Dear support,

When testing recipe_sea_surface_salinity.yml on MPI-ESM1-2-HR/historical dataset, preprocessing stops at step 6 (06_add_fx_variables.nc) with the following error message:

ValueError: Cell measures with missing data are not supported.

Whereas all seem to be fine with the preprocessing of the reference dataset that is ESACCI-SEA-SURFACE-SALINITY : preprocessor runs until the end, last 09_remove_fx_variables.nc preproc file being generated.

The preprocessing operator called is area_statitics.mean

I am wondering whether this is an iris library problem or rather a miss-use of the recipe from my side?

Here attached are the recipe and the log.

Many thanks,
Marie-Pierre.
main_log_debug.txt
recipe_sea_surface_salinity.yml.txt

[remi-kazeroni]

Hi @mpmoine, thanks for reporting the issue and providing the files. I could run your recipe successfully at DKRZ with the same version of the MPI-ESM1-2-HR/historical dataset, namely v20190710, but using a more recent installation of ESMValTool. I wonder if your issue is related to errors that we had also encountered regarding the netcdf library if netcdf>4.6.0, see #2907

Could you please post the output of the command mamba list netcdf to see which versions are used in your environment? The recommendation could be to create a new environment. This would include the bugfix release esmvalcore 2.7.1 (your environment seems to have 2.7.0) where we don't get this type of errors. @valeriupredoi, feel free to chip in if you could think of other approaches 👍

[valeriupredoi]

@remi-kazeroni it could be a side-effect of this issue (that you opened) SciTools/iris#5119 - a side-effect in the sense that iris have definitely made changes to the handling of cell measures recently, and that issue and this error may stem from those changes; this is most definitely not related to the netCDF4 version past 1.6.0 since we have repo-patched the whole of iris not to use those, but by all means @mpmoine if you could tell us the version of iris and netCDF4 you are using that'd help: just do as Remi says conda list netCDF4 && conda list iris. Cheers 🍺

[mpmoine]

Hi @remi-kazeroni @valeriupredoi . Thanks for your replies 👍 . Indeed, we are running under singularity virtual env. I a not familiar with. How can I access the netcdf and iris version in such environment?
Cheers 🍺

[valeriupredoi]

@mpmoine you must have a working conda executable in your path, just run conda list iris and conda list netcdf4 and give us the output please 👍

[mpmoine]

Not so vebose 😕

$ --> conda list iris
# packages in environment at /data/softs/anaconda3:
#
# Name                    Version                   Build  Channel

 $ --> conda list netcdf
# packages in environment at /data/softs/anaconda3:
#
# Name       

[valeriupredoi]

cheers @mpmoine - neither iris nor netcdf4 are installed in /data/softs/anaconda3 which is the root of conda - could you please activate the environment before running the queries (ie conda activate esmvaltool)? You should see something along the lines of packages in environment at /data/softs/anaconda3/envs/esmvaltool: (or whatever the env is called). At any rate, I think this should be upgraded to an Issue rather than a Discussions item

[valeriupredoi]

very good catch @mpmoine - you discovered a fairly sleepy bug in ESMValCore ESMValGroup/ESMValCore#1917 - cheers!