Skip to content

Commit

Permalink
Silent argument for CDIP processing (MHKiT-Software#273)
Browse files Browse the repository at this point in the history 
* add silent kwarg to request_parse_workflow and get_netcdf_variables

* describe optional use of silent in the cdip example

* fix typo

* fix netcdf typo
  • Loading branch information
@akeeste
akeeste authored Nov 6, 2023
1 parent 6d4739f commit f5f25a2
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 8 deletions.
4 changes: 3 additions & 1 deletion examples/cdip_example.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -566,7 +566,9 @@
"source": [
"## 4.b. `request_parse_workflow`\n",
"\n",
"In the previous example we requested the NetCDF file and then processed the data. This workflow has been codified into a single function to streamline the process and adds additional functionality as well. The `request_parse_workflow` function accepts a netCDF object or a station number. This means the user may pass a CDIP ndetCDF file loaded from file, pull the data with `request_netcdf` and then pass, or just pass a station number letting the function know what data to parse and return. Secondly, the `request_parse_workflow` function accepts parameters allowing the user to specify to only return specific parameters reducing processing requirements. This is especially useful for processing 2D data which is only processed is specifically requested due to the amount of time it takes to process all the 2D data. Next, `request_parse_workflow` will slice on time by years, start_date, or end date. Years can be a single integer or a list of integers and is not required to be consecutive. If specified the start date will remove any data prior to the specified string (e.g. '2011-01-01') and end_date will remove any data after the speficied date. start_date and end_date may be used together, seperatly or not at all. Years works indpendently of start and end date. Next, the data_type defaults to historic but specifying this as realtime will return realtime data from the buoy. Lastly, there is a the boolean `all_2D_variables`. If set to true the function will return all of the wave 2D variables. It is not reccomended to do this due to the computational expense to do so, Instead it is reccomended to specify 2D quantities of interest using the `parameters` keyword.\n",
"In the previous example we requested the NetCDF file and then processed the data. This workflow has been codified into a single function to streamline the process and adds additional functionality as well. The `request_parse_workflow` function accepts a netCDF object or a station number. This means the user may pass a CDIP netCDF file loaded from file, pull the data with `request_netcdf` and then pass, or just pass a station number letting the function know what data to parse and return. Secondly, the `request_parse_workflow` function accepts parameters allowing the user to specify to only return specific parameters reducing processing requirements. This is especially useful for processing 2D data which is only processed is specifically requested due to the amount of time it takes to process all the 2D data. A print statement indicates whether the function is currently still processing 2D variables. Use the `silent=True` keyword argument to turn off this print statement in production environments.\n",
"\n",
"Next, `request_parse_workflow` will slice on time by years, start_date, or end date. Years can be a single integer or a list of integers and is not required to be consecutive. If specified the start date will remove any data prior to the specified string (e.g. '2011-01-01') and end_date will remove any data after the speficied date. start_date and end_date may be used together, seperatly or not at all. Years works indpendently of start and end date. Next, the data_type defaults to historic but specifying this as realtime will return realtime data from the buoy. Lastly, there is a the boolean `all_2D_variables`. If set to true the function will return all of the wave 2D variables. It is not recommended to do this due to the computational expense to do so, Instead it is recommended to specify 2D quantities of interest using the `parameters` keyword.\n",
"\n",
"For an example we will create a compendium of HS for the year 2011 from the nc file requested earlier. In this case we can use the years parameter instead of start and end dates.\n"
]
Expand Down
26 changes: 19 additions & 7 deletions mhkit/wave/io/cdip.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,8 @@ def request_netCDF(station_number, data_type):

def request_parse_workflow(nc=None, station_number=None, parameters=None,
years=None, start_date=None, end_date=None,
data_type='historic', all_2D_variables=False):
data_type='historic', all_2D_variables=False,
silent=False):
'''
Parses a passed CDIP netCDF file or requests a station number
from http://cdip.ucsd.edu/) and parses. This function can return specific
Expand All @@ -200,7 +201,7 @@ def request_parse_workflow(nc=None, station_number=None, parameters=None,
request_netCDF
station_number: string
Station number of CDIP wave buoy
parameters: string or list of stings
parameters: string or list of strings
Parameters to return. If None will return all varaibles except
2D-variables.
years: int or list of int
Expand All @@ -216,6 +217,9 @@ def request_parse_workflow(nc=None, station_number=None, parameters=None,
processing time. If all 2D variables are not needed it is
recomended to pass 2D parameters of interest using the
'parameters' keyword and leave this set to False. Default False.
silent: boolean
Set to True to prevent the print statement that announces when 2D
variable processing begins. Default False.
Returns
-------
Expand Down Expand Up @@ -305,7 +309,8 @@ def request_parse_workflow(nc=None, station_number=None, parameters=None,
data = get_netcdf_variables(nc,
start_date=start_date, end_date=end_date,
parameters=parameters,
all_2D_variables=all_2D_variables)
all_2D_variables=all_2D_variables,
silent=silent)
handle_caching(hash_params, cache_dir, data=data)
else:
data = data[0]
Expand All @@ -324,7 +329,8 @@ def request_parse_workflow(nc=None, station_number=None, parameters=None,
year_data = get_netcdf_variables(nc,
start_date=start_date, end_date=end_date,
parameters=parameters,
all_2D_variables=all_2D_variables)
all_2D_variables=all_2D_variables,
silent=silent)
# Cache the individual year's data
handle_caching(hash_params, cache_dir, data=year_data)
else:
Expand Down Expand Up @@ -354,7 +360,8 @@ def request_parse_workflow(nc=None, station_number=None, parameters=None,


def get_netcdf_variables(nc, start_date=None, end_date=None,
parameters=None, all_2D_variables=False):
parameters=None, all_2D_variables=False,
silent=False):
'''
Iterates over and extracts variables from CDIP bouy data. See
the MHKiT CDiP example Jupyter notbook for information on available
Expand All @@ -368,14 +375,17 @@ def get_netcdf_variables(nc, start_date=None, end_date=None,
Data of interest start in seconds since epoch
end_stamp: float
Data of interest end in seconds since epoch
parameters: string or list of stings
parameters: string or list of strings
Parameters to return. If None will return all varaibles except
2D-variables. Default None.
all_2D_variables: boolean
Will return all 2D data. Enabling this will add significant
processing time. If all 2D variables are not needed it is
recomended to pass 2D parameters of interest using the
'parameters' keyword and leave this set to False. Default False.
silent: boolean
Set to True to prevent the print statement that announces when 2D
variable processing begins. Default False.
Returns
-------
Expand Down Expand Up @@ -495,7 +505,9 @@ def get_netcdf_variables(nc, start_date=None, end_date=None,

if (prefix == 'wave') and (include_2D_variables):

print('Processing 2D Variables:')
if not silent:
print('Processing 2D Variables:')

vars2D = {}
columns = metadata['waveFrequency']
N_time = len(time_slice)
Expand Down

0 comments on commit f5f25a2

Please sign in to comment.