Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Init/runoff #156

Open
wants to merge 773 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
773 commits
Select commit Hold shift + click to select a range
f8a2a0e
update create raw data
tommylees112 Nov 16, 2020
3b0d4e2
remove teherror in run_regression code
tommylees112 Nov 16, 2020
12069d9
update RAw regression code
tommylees112 Nov 16, 2020
8e59748
comment out RAW cell state regression
tommylees112 Nov 16, 2020
f5c377b
add assert False TODO in
tommylees112 Nov 16, 2020
e87b896
add cell state regression code
tommylees112 Nov 25, 2020
b070f70
update cs config raw correlations
tommylees112 Nov 25, 2020
56e9f95
update args to raw correlations
tommylees112 Nov 25, 2020
08a89b1
add upsample xarray function
tommylees112 Nov 25, 2020
5e1416c
black cpde
tommylees112 Nov 25, 2020
97c5b60
test upsampling
tommylees112 Nov 25, 2020
e3727b1
sys path
tommylees112 Nov 25, 2020
c37b0a8
get data path
tommylees112 Nov 25, 2020
f178575
open dataset
tommylees112 Nov 25, 2020
286af33
manually update datA_dir
tommylees112 Nov 25, 2020
037921a
update path
tommylees112 Nov 25, 2020
db4165b
data_dir
tommylees112 Nov 25, 2020
06ae1c9
numpy
tommylees112 Nov 25, 2020
b109783
update method to nearest
tommylees112 Nov 25, 2020
0b7c158
grid factor=1
tommylees112 Nov 25, 2020
f0d309f
grid factor 2
tommylees112 Nov 25, 2020
38182c9
update grid factor 2
tommylees112 Nov 25, 2020
7f5a00d
save HR version
tommylees112 Nov 25, 2020
96b2e52
ti netcdf
tommylees112 Nov 25, 2020
6854df1
catch errors loading lstm/ealstm
tommylees112 Nov 27, 2020
22bb10b
update read sm data
tommylees112 Nov 27, 2020
fe7c3ea
fix arg call to func
tommylees112 Nov 27, 2020
52d2831
don't upsample too
tommylees112 Nov 27, 2020
3c58dd3
update where data is saved
tommylees112 Nov 27, 2020
ac767df
update bool statement
tommylees112 Nov 27, 2020
5a1fbf3
update remapping of station ids
tommylees112 Nov 27, 2020
563c718
update updeta
tommylees112 Nov 27, 2020
cdee177
impove boolean logic
tommylees112 Nov 27, 2020
09d8c2a
error cacthinc
tommylees112 Nov 27, 2020
7d27274
add saee output option
tommylees112 Nov 27, 2020
603b1aa
split into sub functions
tommylees112 Nov 27, 2020
637b33c
add comments
tommylees112 Nov 28, 2020
073c7d4
add finetuning analysis scripts
tommylees112 Nov 30, 2020
510d29b
run finetuning for 8 top conceptual models
tommylees112 Nov 30, 2020
ff547a3
finetune models from 2:8
tommylees112 Nov 30, 2020
14b1e89
from 2:8 ....
tommylees112 Nov 30, 2020
fc033d2
remove pdb
tommylees112 Nov 30, 2020
da2a14b
move the assert statement AFTER create .txt
tommylees112 Nov 30, 2020
c09479b
run dir exists capture
tommylees112 Nov 30, 2020
6126710
change epochs to 10 and learning rate update
tommylees112 Nov 30, 2020
ca70b7e
add incldeude lstm code
tommylees112 Nov 30, 2020
5b24dd3
include paper ALL draft notebooks e.g. cell state; water balance; pet
tommylees112 Dec 8, 2020
9e5c2c5
add gr4J export and abs_pct_bias
tommylees112 Dec 8, 2020
9a231b3
Merge branch 'init/runoff' of https://github.com/esowc/ml_drought int…
tommylees112 Dec 8, 2020
ced19cd
add hydroError as dependency
tommylees112 Dec 8, 2020
7e2b936
calculate error tables using hydrostats package (script)
tommylees112 Dec 8, 2020
4b34b9d
remove dealing with inverse inside KGE func
tommylees112 Dec 8, 2020
6cd5df0
add sptial mape and spatial
tommylees112 Dec 8, 2020
f844e2a
add spatial extra efficiency metrics
tommylees112 Dec 8, 2020
f138bea
remove the error scores
tommylees112 Dec 8, 2020
6b615cb
add epsilon param
tommylees112 Dec 8, 2020
a3d950e
use the original calculate error metric functions
tommylees112 Dec 8, 2020
699b5a5
update index etc.
tommylees112 Dec 8, 2020
3319bfb
calculation of xr_mam30_ape
tommylees112 Dec 8, 2020
6f0b322
fix mam30ape
tommylees112 Dec 8, 2020
7ab1764
add error metrics for station id
tommylees112 Dec 8, 2020
5d14cb7
all_fuse_Errors to calculate_all_data_errors
tommylees112 Dec 8, 2020
370fb57
spatial flexibility oto create shape aligned cli
tommylees112 Dec 8, 2020
91de492
download gr4j scrip
tommylees112 Dec 10, 2020
13bcb3e
add ability to call individual kratzert fnctions
tommylees112 Dec 10, 2020
811a2b8
fix the metric checks
tommylees112 Dec 10, 2020
dca5f6d
update notebooks
tommylees112 Dec 10, 2020
257daf6
fix error
tommylees112 Dec 10, 2020
f361d37
Merge branch 'init/runoff' of https://github.com/esowc/ml_drought int…
tommylees112 Dec 10, 2020
55cf057
water closure levels test
tommylees112 Dec 10, 2020
3ad2a21
update data
tommylees112 Dec 10, 2020
2da1284
get condition_sids
tommylees112 Dec 14, 2020
f670b8b
write io functions
tommylees112 Dec 14, 2020
f424153
update io script
tommylees112 Dec 14, 2020
51bc534
update io results
tommylees112 Dec 14, 2020
4b8a6a3
update fuse updates
tommylees112 Dec 14, 2020
cf6d2be
update io script
tommylees112 Dec 14, 2020
2527655
update main.py
tommylees112 Dec 14, 2020
7faabfd
match station ids i/o
tommylees112 Dec 14, 2020
232e531
import nump
tommylees112 Dec 14, 2020
69a005d
update time calcs
tommylees112 Dec 14, 2020
9bd945e
update delta error
tommylees112 Dec 14, 2020
529eafc
delta errors including benchmarks
tommylees112 Dec 14, 2020
7c06168
update iuo results main
tommylees112 Dec 14, 2020
d17c099
black .
tommylees112 Dec 14, 2020
bca928a
add default
tommylees112 Dec 14, 2020
20d04b4
get only overlapping stations in df
tommylees112 Dec 15, 2020
aec02cc
update all errors
tommylees112 Dec 15, 2020
cf05268
cdf and budyko curve
tommylees112 Dec 15, 2020
a78d57b
add path to ml_drought repo
tommylees112 Dec 15, 2020
782dc7a
black files
tommylees112 Dec 15, 2020
6e09289
update figure label in plots.py main
tommylees112 Dec 15, 2020
58a9902
decompose kge
tommylees112 Dec 15, 2020
3c5b18b
test kge decomposition
tommylees112 Dec 15, 2020
2aadd8c
fix concatenating decompose_kge metrics
tommylees112 Dec 15, 2020
7566d04
update kge calculations
tommylees112 Dec 15, 2020
7e05662
add inverse kge decomposiiton too
tommylees112 Dec 15, 2020
6e56d06
include epsilon to kge decompositions
tommylees112 Dec 15, 2020
1c702a0
add comments
tommylees112 Dec 15, 2020
0d61657
update skill score to calculate for all metrics in list
tommylees112 Dec 15, 2020
72303a7
typing
tommylees112 Dec 15, 2020
3244324
update assertion
tommylees112 Dec 15, 2020
66b9813
calculate for all benchmarks
tommylees112 Dec 15, 2020
1d7f470
fix errors
tommylees112 Dec 15, 2020
3b81f98
update skill score main
tommylees112 Dec 15, 2020
87f1167
read in the stations that exist FUSE
tommylees112 Dec 15, 2020
c20078b
update default delta calculations
tommylees112 Dec 15, 2020
e220b25
include more skill scores
tommylees112 Dec 16, 2020
5f0838b
add percentage bias
tommylees112 Dec 16, 2020
958e214
include pbias in calculations
tommylees112 Dec 16, 2020
bbe1b82
save the metrics for reuse in other scripts
tommylees112 Dec 16, 2020
4601b62
sqrt kge calculations
tommylees112 Dec 16, 2020
c8da137
tests for string
tommylees112 Dec 16, 2020
a365cc4
get metrics
tommylees112 Dec 16, 2020
2065cbb
test skill scores
tommylees112 Dec 16, 2020
f897185
add save to IO script
tommylees112 Dec 16, 2020
f989304
more checks for final metrics
tommylees112 Dec 16, 2020
55cc89b
add new bias and variance error metrics
tommylees112 Dec 16, 2020
fb8c3dc
update calculate errors
tommylees112 Dec 16, 2020
2473922
update notebooks
tommylees112 Jan 6, 2021
3571b9a
update kge decomposition
tommylees112 Jan 8, 2021
7f5e0ea
Merge branch 'init/runoff' of https://github.com/esowc/ml_drought int…
tommylees112 Jan 8, 2021
2e85ca5
remove kge decomposed -1
tommylees112 Jan 8, 2021
6be9e77
options for summary line and legend
tommylees112 Jan 9, 2021
6d5d1c1
update typing
tommylees112 Jan 9, 2021
37db51a
add optimum line
tommylees112 Jan 9, 2021
2dcbed9
update linewidth
tommylees112 Jan 9, 2021
7224861
UPDATE LW
tommylees112 Jan 9, 2021
03e92d0
update ls
tommylees112 Jan 9, 2021
765fed1
update line label
tommylees112 Jan 9, 2021
bfe5807
move functions out of Processor
tommylees112 Jan 9, 2021
cf370a5
simple check to calculate_all_delta_dfs
tommylees112 Jan 9, 2021
c5375d8
update bias calc
tommylees112 Jan 9, 2021
dfa2dcd
remove asset
tommylees112 Jan 9, 2021
d4a795f
add seasonal_errors funct
tommylees112 Jan 9, 2021
567128d
black
tommylees112 Jan 9, 2021
96c7fdd
add typing
tommylees112 Jan 9, 2021
1bba553
add yilmaz errors
tommylees112 Jan 11, 2021
2e0a2e8
add yilmaz partial
tommylees112 Jan 11, 2021
01c7d78
fms to fmv
tommylees112 Jan 12, 2021
95e3623
update all
tommylees112 Jan 12, 2021
8b2e807
update delta_dfs and move class deltaError out
tommylees112 Jan 12, 2021
321bf7d
update io results
tommylees112 Jan 12, 2021
6a6e67d
remove references to DeltaError
tommylees112 Jan 12, 2021
776ceec
update kwargs dict
tommylees112 Jan 12, 2021
1de9cab
update config
tommylees112 Jan 12, 2021
1230390
remove excess clips
tommylees112 Jan 12, 2021
170b00c
move clipping
tommylees112 Jan 12, 2021
c4d4ac2
kwarg dict
tommylees112 Jan 12, 2021
440fc28
add statistical_significance
tommylees112 Jan 12, 2021
abd5e22
add tsts
tommylees112 Jan 12, 2021
c16b6f5
save seasonal metrics
tommylees112 Jan 12, 2021
2b4107e
update rgion analysis
tommylees112 Jan 13, 2021
a8bc925
update region analysis
tommylees112 Jan 13, 2021
f4f4425
update all
tommylees112 Jan 13, 2021
695e159
black
tommylees112 Jan 13, 2021
83b9c87
update run test to compare any model
tommylees112 Jan 14, 2021
a5ccd70
update budyko curve
tommylees112 Jan 14, 2021
f04c947
plots run
tommylees112 Jan 14, 2021
ce681e7
update plots
tommylees112 Jan 14, 2021
6347751
only test budyko
tommylees112 Jan 14, 2021
e6decb8
one central definition of water balance error
tommylees112 Jan 14, 2021
44884de
latlon addition
tommylees112 Jan 14, 2021
7142f75
latlon
tommylees112 Jan 14, 2021
40de3d5
fix fucking error
tommylees112 Jan 14, 2021
3152278
black
tommylees112 Jan 14, 2021
d671477
fix gepoandas
tommylees112 Jan 14, 2021
f3fbcb2
fix order of gepandas
tommylees112 Jan 14, 2021
0ad6f1e
false some code
tommylees112 Jan 14, 2021
5738399
delta errpor
tommylees112 Jan 14, 2021
3ea6580
delta erors
tommylees112 Jan 14, 2021
3853431
fix assert order
tommylees112 Jan 14, 2021
da96589
remove r
tommylees112 Jan 14, 2021
cc208c7
fix timing
tommylees112 Jan 14, 2021
b110e78
update kratzert errors
tommylees112 Jan 14, 2021
5ea97ad
add io results
tommylees112 Jan 14, 2021
2d94464
FIX THE BIG BUG
tommylees112 Jan 14, 2021
ae9d061
test model/metric
tommylees112 Jan 15, 2021
e275660
run seasonal metrics
tommylees112 Jan 15, 2021
efd9fe2
fix test
tommylees112 Jan 15, 2021
ec76cbf
fix get dataframes function
tommylees112 Jan 15, 2021
749e644
load data
tommylees112 Jan 15, 2021
06ad252
add conversion to xarray
tommylees112 Jan 15, 2021
c8bcba6
update notebooks
tommylees112 Jan 15, 2021
da6412e
add assign_region_coord xarray object
tommylees112 Jan 18, 2021
70d86b1
Merge branch 'init/runoff' of https://github.com/esowc/ml_drought int…
tommylees112 Jan 18, 2021
38bef46
notebooks from dl
tommylees112 Jan 18, 2021
729a7c2
conflict resolve
tommylees112 Jan 18, 2021
98e7a31
update notebooks
tommylees112 Jan 19, 2021
c180fdf
add calculate benchmark function
tommylees112 Jan 19, 2021
2a92057
Merge branch 'init/runoff' of https://github.com/esowc/ml_drought int…
tommylees112 Jan 19, 2021
89fd4a5
add another signficance test
tommylees112 Jan 19, 2021
bc0b887
add _plot_geospatial
tommylees112 Jan 19, 2021
804696e
edfault plot size
tommylees112 Jan 19, 2021
6a23586
patlnv
tommylees112 Jan 19, 2021
9273e24
black
tommylees112 Jan 20, 2021
9012b72
script to andreas
tommylees112 Jan 20, 2021
59c7f4b
update notebooks for paer
tommylees112 Jan 20, 2021
e8ea9a5
Merge branch 'init/runoff' of https://github.com/esowc/ml_drought int…
tommylees112 Jan 20, 2021
3dce6cd
remove HydrErr
tommylees112 Jan 27, 2021
7ee34b3
check
tommylees112 Jan 27, 2021
7e5c163
update notebooks
tommylees112 Jan 27, 2021
58baeb7
black all
tommylees112 Feb 3, 2021
87f3df9
move geopandas to optional import
tommylees112 Feb 3, 2021
f38696a
hide base init ???
tommylees112 Feb 3, 2021
13995dc
update gpd import
tommylees112 Feb 3, 2021
ed18562
io results
tommylees112 Feb 7, 2021
78c3877
invert errors dict
tommylees112 Feb 7, 2021
6cecac7
read nh results
tommylees112 Feb 7, 2021
9c1102d
update non ensemble dataset loader
tommylees112 Feb 9, 2021
ad30b58
metric csv is optional
tommylees112 Feb 9, 2021
1555fc0
update read csv results
tommylees112 Feb 9, 2021
261b569
update print statement
tommylees112 Feb 9, 2021
9bd1131
remove use of deltaerror
tommylees112 Feb 10, 2021
7eeae83
black all .
tommylees112 Feb 10, 2021
c0cc59a
update benchmark calculations
tommylees112 Feb 10, 2021
edb5de0
update cmd line invocations
tommylees112 Feb 10, 2021
493a099
update read file
tommylees112 Feb 15, 2021
f9ebd76
preprocess on aws machiens
tommylees112 Feb 21, 2021
1b53d91
hydrograph pltos
tommylees112 Feb 22, 2021
f53cdfa
update hydrographs
tommylees112 Feb 22, 2021
0d67a4c
update hgraphs
tommylees112 Feb 22, 2021
5c8a2a3
updata hgraph black mypy
tommylees112 Feb 22, 2021
db98155
fix function
tommylees112 Feb 22, 2021
a6b7f02
fix 2ndary bar
tommylees112 Feb 22, 2021
eb8261d
fix
tommylees112 Feb 22, 2021
b1c0e48
fig
tommylees112 Feb 22, 2021
36548b7
environments
tommylees112 Feb 25, 2021
df03a95
update lookup regions
tommylees112 Mar 4, 2021
af27511
update nbs
tommylees112 Mar 4, 2021
ea546b5
update read all members
tommylees112 Mar 7, 2021
aebaa27
Merge branch 'init/runoff' of https://github.com/esowc/ml_drought int…
tommylees112 Mar 7, 2021
53a2450
add coord to variable function
tommylees112 Mar 7, 2021
479e49e
fix function
tommylees112 Mar 7, 2021
3d57e96
fix function
tommylees112 Mar 7, 2021
25f5c4e
update plot cdf
tommylees112 Mar 7, 2021
3cc712b
update notebooks
tommylees112 Mar 7, 2021
fa2f01b
update cell state
tommylees112 Mar 10, 2021
b054360
update hydrograph plotting to be more general
tommylees112 Mar 11, 2021
4897ab5
update plots
tommylees112 Mar 11, 2021
1a974af
update
tommylees112 Mar 11, 2021
774b3d1
update plot function
tommylees112 Mar 11, 2021
334f231
generalsie hydrograph plotting
tommylees112 Mar 11, 2021
4ccc9d7
generalise
tommylees112 Mar 11, 2021
228c113
update noteboos
tommylees112 Mar 12, 2021
fc3a885
update cell state regression to work with dict
tommylees112 Mar 17, 2021
ff63b9e
no dict just tensor
tommylees112 Mar 17, 2021
7e81eb5
update scripts
tommylees112 May 28, 2021
96eb903
add mutual info
tommylees112 Jul 22, 2021
f80623c
UPDATE PLOT HGS
tommylees112 Jul 26, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ data/*.zip*
*.idx
scripts/*.png
scripts/variables.txt
data/*
data*

.ipynb_checkpoints

Expand All @@ -30,15 +32,26 @@ data/analysis/*
data/interim/*
data/features/*
data/models/*
data/gcloud*

Code
scripts/variables*.txt

scripts/drafts/cookies.py
switch_data_dirs.sh
pyrightconfig.json

# pytest coverage files
.coverage*

# random python stuff
core.python*
core.ipython*

# vscode settings
.vscode
# vscode options
settings.json

# lightning
events.*
lightning_logs
5 changes: 0 additions & 5 deletions data/README.md

This file was deleted.

Loading