Single Component Dependency end-point function

[rburghol]

This is provisionally completed pending review of PR #150

• function def model_order_recursive(model_object, model_object_cache, model_exec_list, model_touch_list = []):
  • Use: `model_order_recursive([End point hdf path], [model classes stored as {}], [a list of elements that have already been ordered (may be blank)], [a list of elements integer state ID that have been seen already (may be blank)])
  • Returns: nothing, but modifies model_exec_list and model_touch_list in place
• Called by model_domain_dependencies()
  • Use: domain_exec_list = model_domain_dependencies([the state dict], [domain, i.e. common path root], [end point names (to appens to root)] )
• Ex: From SEDTRN

        # Set the list of variables that are allowed to be changed by STATE in this function
	ep_list = ['RSED4', 'RSED5', 'RSED6']
        # Note: state_info['domain'] = '/STATE/RCHRES_R001' 
        # To allow ep_list to have multiple distinct endpoints
	model_exec_list = model_domain_dependencies(state, state_info['domain'], ep_list)
	model_exec_list = np.asarray(model_exec_list, dtype="i8") # format for use in numba

The STATE operational model system provides for "dependency execution" which assembles a list of integer IDs ordered according to their components data dependency (from independent to dependent variables), in order to insure proper order execution.

Currently, this is done in the same function that tokenizes all operational model and special action objects. But it would be best if there were a function to do this to allow us to extract different "execution chains" for a given endpoint to avoid redundant executions and the potential for erroneous data due to executing code outside of it's model domain.

Conceptual Example

What are the special actions/OM ops that must be executed each timestep for OD (withdrawal) during HYDR simulation in a given RCHRES 1?

• RCHRES 1:
  • The stream channel of interest
  • State path: /STATE/RCHRES_R001
  • OVOL: value of outflow from the previous timestep in RCHRES 1
  • OD = demand: the OD input (OD1, OD2, OD3) is the demand for water from the RCHRES at the current timestep
• month: the numerical index for current timestep month (1-12)
• RCHRES 1 contains a water withdrawal intake with the following components:
  • flowby = 10.0: a constant, in cfs, indicating the minimum value of instream flow that must pass the intake
  • available_flow: OVOL - flowby;
  • demand_monthly: a table that contains a monthly varying base demand
    • [ 1: 2.0, 2: 2.0, 3: 2.2, 4: 2.2, 5: 2.2, 6: 2.3, 7:2.4, 8:2.4, 9: 2.1, 10: 2.1, 11: 2.0, 12: 2.0 ]
  • demand: a lookup with the demand sent to the RCHRES
    • demand = min(demand_monthly, available_flow)
  • a push linkage OD1 = demand
  • Independent variables:
    • month: self-explanatory
    • flowby: in this case it is a single numerical value, and thus depends on no variable
  • OVOL since it is from the last time step
• dependent
• end-point (the final dependent variable used by the module HYDR: OD = demand: the OD input (OD1, OD2, OD3) is the demand for water from the RCHRES at the current timestep
  Since the allowable withdrawal amount varies as a function of a monthly minimum flow requirement (i.e. a "flow-by") OD must be calculated after demand_monthly, flowby and available_flow. Similarly, available_flow must be calculated after flowby.

Code

Example: Find SEDTRN related executables in a single RCHRES via model_order_recursive()

• Included in develop-specact-depend branch here: https://github.com/respec/HSPsquared/blob/ef729177cfb66cb193a9b087ae09cd9d70885e11/tests/testcbp/HSP2results/check_depends_endpoint.py

# Must be run from the HSPsquared source directory, the h5 file has already been setup with hsp import_uci test10.uci
# bare bones tester - must be run from the HSPsquared source directory
import os
from HSP2.main import *
from HSP2.om import *
import HSP2IO
import numpy
from HSP2IO.hdf import HDF5
from HSP2IO.io import IOManager
fpath = './tests/test10/HSP2results/test10.h5' 
# try also:
# fpath = './tests/testcbp/HSP2results/PL3_5250_0001.h5' 
# sometimes when testing you may need to close the file, so try:
# f = h5py.File(fpath,'a') # use mode 'a' which allows read, write, modify
# # f.close()
hdf5_instance = HDF5(fpath)
io_manager = IOManager(hdf5_instance)
uci_obj = io_manager.read_uci()
siminfo = uci_obj.siminfo
opseq = uci_obj.opseq
# - finally stash specactions in state, not domain (segment) dependent so do it once
# now load state and the special actions
state = init_state_dicts()
state_initialize_om(state)
state['specactions'] = uci_obj.specactions # stash the specaction dict in state

state_siminfo_hsp2(uci_obj, siminfo)
# Add support for dynamic functions to operate on STATE
# - Load any dynamic components if present, and store variables on objects 
state_load_dynamics_hsp2(state, io_manager, siminfo)
# Iterate through all segments and add crucial paths to state 
# before loading dynamic components that may reference them
state_init_hsp2(state, opseq, activities)
state_load_dynamics_specl(state, io_manager, siminfo) # traditional special actions
state_load_dynamics_om(state, io_manager, siminfo) # operational model for custom python
state_om_model_run_prep(state, io_manager, siminfo) # this creates all objects from the UCI and previous loads
# state['model_root_object'].find_var_path('RCHRES_R001')

# Aggregate the list of all SEDTRN end point dependencies
domain = '/STATE/RCHRES_R005'
ep_list = ['RSED4', 'RSED5', 'RSED6']
mello = model_domain_dependencies(state, domain, ep_list)
print("Dependency ordered execution for RSED constants and runnables influencing", domain, "=", mello)
mel_runnable = ModelObject.runnable_op_list(state['op_tokens'], mello)
print("Dependency ordered execution of RSED depemndencies (all)", domain, "=", 
model_element_paths(mello, state))
print("Dependency ordered execution of RSED runnables only for", domain, "=", 
model_element_paths(mel_runnable, state))

Extra testing output to show details of dependencies.

# Show order of ops based on dependencies
endpoint = state['model_object_cache']['/STATE/RCHRES_R005/RSED5']
mel = []
mtl = []
model_order_recursive(endpoint, state['model_object_cache'], mel, mtl)
print("Dependency ordered execution for constants and runnables influencing", endpoint.name)
model_element_paths(mel, state)
mel_runnable = ModelObject.runnable_op_list(state['op_tokens'], mel)
print("Dependency ordered execution of runnables only for", endpoint.name)
model_element_paths(mel_runnable, state)


# Just for grins, we can show the dependency using the special action as an end point
specl2 = state['model_object_cache']['/STATE/SPECACTION2']
mel = []
mtl = []
print("Dependency ordered execution for constants and runnables influencing ", rsed4.name)
model_order_recursive(specl2, state['model_object_cache'], mel, mtl)
model_element_paths(mel, state)
mel_runnable = ModelObject.runnable_op_list(state['op_tokens'], mel)

[rburghol]

Created model_order_recursive(), which is based almost verbatim on model_tokenizer_recursive() but with no calls to tokenize or load. Test the existing base routine for the ability to trace requirements for only a sub-section with an end point:

• Basic process:
  • Retrieve the desired end-point object from model_object_cache
  • send the end-point object as the model_object argument
  • pass the full model_object_cache arg
  • pass an empty dict as model_exec_list - mel = []
  • pass an empty dict as model_touch_list - mtl = []
• Example 1: the Runit variable from the outlet watershed

# run the first 43 lines of 
mel = []
mtl = []
outlet_runit_object = model_object_cache['/STATE/RCHRES_R001/Runit']
model_order_recursive(outlet_runit_object, model_object_cache, mel, mtl)
print(mel)
[233, 231, 48, 32, 235, 236, 31, 73, 57, 238, 239, 56, 95, 79, 241, 242, 78, 117, 101, 244, 245, 100, 139, 123, 247, 248, 122, 161, 145, 250, 251, 144, 183, 167, 253, 254, 166, 172, 173, 182, 186, 150, 151, 160, 164, 128, 129, 138, 142, 106, 107, 116, 120, 84, 85, 94, 98, 205, 189, 256, 257, 188, 227, 211, 259, 260, 210, 216, 217, 226, 230, 194, 195, 204, 208, 62, 63, 72, 76, 37, 38, 47, 232] 

• This is a voluminous list of elements, since the drainage_area_sqmi equation relies upon trib_area_sqmi, which is influenced by all of the nested tributaries own drainage_area_sqmi equations.
• Now, repeat the same process, but using the drainage_area_sqmi equation from a headwater catchment, which will only be influenced by the elements of area in the local catchement since nothing is upstream of it:

mtl = []
mel = []
headwater_area_object = model_object_cache['/STATE/RCHRES_R001/nhd_8566779/nhd_8566793/nhd_8566791/drainage_area_sqmi']
model_order_recursive(headwater_area_object, model_object_cache, mel, mtl)
# see the names/paths 
print(mel)
[227, 211, 259, 260, 210, 216, 217, 226]