File Not Downloaded but No Error Raised by Herbie

[RBhupi]

Herbie does not raise an error when attempting to download a file, when the file is not downloaded successfully. The function download() gives the impression that the data is downloaded, but the file is missing from tha path.

from herbie import Herbie
import os

H = Herbie("2020-08-10", model='hrrr', product='prs', fxx=0)
download_path = H.download('TMP:1000mb')
# check path
if os.path.exists(download_path):
    print(f"file found in {download_path}")
else:
    print(f"No file in {download_path}")

I get this
✅ Found ┊ model=hrrr ┊ product=prs ┊ 2020-Aug-10 00:00 UTC F00 ┊ GRIB2 @ local ┊ IDX @ aws

No file in /Users/bhupendra/data/hrrr/20200810/subset_58efb1fe__hrrr.t00z.wrfprsf00.grib2

Also tried with xarray ds = H.xarray('TMP:1000mb') and got the FileNotFoundError: [Errno 2] No such file or directory: '/Users/bhupendra/data/hrrr/20200810/subset_58efb1fe__hrrr.t00z.wrfprsf00.grib2'

Herbie should raise an error or warning if a file is not downloaded. The download() method should return none as download_path.

[vwgeiser]

Did you try updating the .config file that is made when Herbie was first called?

"Users\vwgei.config\herbie\config.toml"

You can open the .toml file with notepad on Windows. I changed the default save location of mine so it looks like this:

# Herbie defaults

[default]
model = "gfs"
fxx = 0
save_dir = "D:\\gfs_data"
overwrite = false
verbose = true

# =============================================================================
# You can set the priority order for checking data sources.
# If you don't specify a default priority, Herbie will check each source in the
# order listed in the model template file. Beware: setting a default priority
# might prevent you from checking all available sources.
#
#priority = ['aws', 'nomads', 'google', 'etc.']

[RBhupi]

Thanks for the response @vwgeiser !

Do you think setting the priority would help getting more data?

My config file looks like this

# Herbie defaults

[default]
model = "hrrr"
fxx = 0
save_dir = "/Users/bhupendra/data"
overwrite = false
verbose = true

# =============================================================================
# You can set the priority order for checking data sources.
# If you don't specify a default priority, Herbie will check each source in the
# order listed in the model template file. Beware: setting a default priority
# might prevent you from checking all available sources.
#
#priority = ['aws', 'nomads', 'google', 'etc.']

Checked with changing priority. No files are available as the data in aws and google are same. I checked some other discussion about missing files.

However, the file should be saved here /Users/bhupendra/data/hrrr/20200810/subset_*, if not it should return None or False so that the user can take care of that.

Thanks for the suggestions,

Bhupendra Raut