Add run_reps.jl helper to run RME #26
Conversation
|
I think this looks good to me. I can add the function to remove the duplicate counterfactual runs before saving in another branch when this is merged |
You can create a branch from this one and create you PR already even before this one is merged, if you prefer. |
|
When I try to use the latest code with this PR I get this error adding the package. Is it a fault on my end in packages? |
Minor point: Unless you're not in the sandbox directory at all? But the error seems to be pointing to ReefModEngine.jl requiring Julia v1.11 but you seem to be running Julia v1.10.4 |
| """ | ||
| run_rme(rme_path::String, n_threads::Int64, reps::Int64, result_path::String; start_year::Int64=2022, end_year::Int64=2099, batch_size::Int64=10, start_batch::Int64=1, RCP_scen::String="SSP 2.45", gcm::String="CNRM_ESM2_1", rnd_seed::Int64=1234)::Nothing | ||
|
|
||
| Run counterfactual scenarios with ReefModEngine.jl and save result set to desired dir. |
There was a problem hiding this comment.
| Run counterfactual scenarios with ReefModEngine.jl and save result set to desired dir. | |
| Run counterfactual scenarios with ReefModEngine.jl and save result set to desired directory. |
| - `batch_size` : Number of repetitions to be run in each batch. | ||
| - `RCP_scen` : RCP scenario to be used for RME runs. | ||
| - `gcm` : GCM to be used for RME runs. | ||
| - `rnd_seed` : Random seed. |
There was a problem hiding this comment.
Prefer to accept an RNG instead:
| - `rnd_seed` : Random seed. | |
| - `rng` : Random number generator (default: Globally set RNG). |
rng::AbstractRNG=Random.GLOBAL_RNG
| init_rme(rme_path) | ||
| set_option("thread_count", n_threads) # Set number of threads | ||
| set_option("use_fixed_seed", 1) # Turn on use of a fixed seed value |
There was a problem hiding this comment.
I feel these should be set outside this function by the user and the settings reused across runs if possible.
Currently, RME is re-initialized every time run_rme() is called. In recent versions it takes milliseconds, but in the past (and potentially in the future) it may take 10s of seconds to minutes.
| function _rnd_seeds(rnd_seed::Int64, batch_size::Int64, reps::Int64)::Vector{Int64} | ||
| Random.seed!(rnd_seed) | ||
| n_seeds = Int(ceil(reps / batch_size)) | ||
| return Int.(floor.(rand(n_seeds) .* 1e6)) | ||
| end |
There was a problem hiding this comment.
I think use StatsBase and sample without replacement.
Right now there is a very (very) small possibility of a clash (multiple scenarios with same seed).
function _rnd_seeds(rnd_seed::Int64, batch_size::Int64, reps::Int64; rng::AbstractRNG=Random.GLOBAL_RNG, max_seed::Int64=1_000_000)::Vector{Int64}
n_seeds = Int(ceil(reps / batch_size))
if n_seeds > max_seed
# Protect against possibly number of seed values exceed seed range
max_seed = max_seed + (n_seeds * 2)
end
return sample(rng, 1:max_seed, n_seeds; replace=false)
end| end | ||
|
|
||
| function _resultset_dir_name()::String | ||
| timestamp = Dates.format(now(), "yyyymmdd_HHMMSS") |
There was a problem hiding this comment.
I prefer YYYY-mm-dd format
| Run counterfactual scenarios with ReefModEngine.jl and save result set to desired dir. | ||
|
|
||
| # Arguments | ||
| - `rme_path` : Path to REM folder. |
There was a problem hiding this comment.
If we move the RME option setting step outside this function as suggested below, we can remove this (and other) argument lines, but if we decide to keep it:
| - `rme_path` : Path to REM folder. | |
| - `rme_path` : Path to RME folder. |
Adds the ability to run many RME repetitions in batches. Note that the
rnd_seed, that can be set by the user, is used to generate a vector ofrnd_seedswhere each one is used for running a distinct batch.