Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dev/xarray base #56

Merged
merged 65 commits into from
Oct 15, 2024
Merged

Dev/xarray base #56

merged 65 commits into from
Oct 15, 2024

Conversation

kysolvik
Copy link
Collaborator

@kysolvik kysolvik commented Oct 3, 2024

  • Refactor to use XArray for inputs/outputs/backends

To-Do before merging:

  • Update README
  • Fix tests

@kysolvik
Data().generate() now returns an Xarray 'state vector'
5ecbef2
@kysolvik
Remove vector class import for now, replacing vector objects with sim…
f5f1bf4 
…ple xarrays
@kysolvik
Update sqgturb to work with xarray. Returns in real space, not spectral.
8f60436
@kysolvik
Updated observer with xarray object, but only basic functionality is …
9c33a8c 
…working
@kysolvik
3DVar working with xarray:
8264485
@kysolvik
ETKF working with xarray, initial commit
b728551
@kysolvik
Sort observed locations
4ccbeba
@kysolvik
Data times are stored as base numpy arrays, since xarray coords canno…
e41e60d 
…t be jax
@kysolvik
Reinserting netcdf utils into data class
4e8b970
@kysolvik
gcp with xarray
99b1b3c
@kysolvik
ETKF can handle irregular obs now, but doesn't have proper indices info
9eddcf1
@kysolvik
Use system_index variable for H
bffcfbc
@kysolvik
Observer adds integer indices for easier H calculation
0cbd940
@kysolvik
Allow datavars to not match observed vars
e023257
@kysolvik
Updated system index and remove sort from the observer
6e9de1a
@kysolvik
Making some dacycler methods part of parent class for ease of mainten…
2f87b45 
…ance
@kysolvik
Initial version of xarray var4dBP, needs testing and cleaning
ccf7e03
@kysolvik
Cleaned var4dbp using xarray
3c09ef3
@kysolvik
All dacyclers working with xarray, but possibly some accuracy issues …
cbc308f 
…with 4dvar and 4dvarBP
@kysolvik
State Vec has delta_t attribute and M is provided as xarray
1690992
@kysolvik
Working RC Model with xarray
8b81d55
@kysolvik
Fixed generator extra step rounding error
e3cd320
@kysolvik
Observer can accept random_time_density now
19572e3
@kysolvik
Add permissible xarray jax to pyproject toml
e5bfe0e
@kysolvik
Remove all vector module imports
789019a
@kysolvik
Rename i to index for toy data generators
f0f982d
@kysolvik
GCP system dim now specified
e8bd897
@kysolvik
Observer can accept list of error_sds, and now samples with replaceme…
17cab8f 
…nt when there is more than 1 dimension to sample along
@kysolvik
Fixed issue with missing time offset for 4dvar and 4dvarBP
43c9d2b
@kysolvik
Reassign coords to match input state within dacycler
5a916f5
@kysolvik
Add option for data split in fraction
5f506ab
@kysolvik
Updated load netcdf with filtering and data base test
143d922
@kysolvik
Updated LE calcs for xarray
45d0988
@kysolvik
QGS with xarray output
ebe0154
@kysolvik
All off-line data tests working with xarray
0a36581
@kysolvik
Partially working DA tests (3dVar working)
d7db346
@kysolvik
Store error sd in obs vec
5151378
@kysolvik
ETKF and var4dBP tests passing
8cb35fc
@kysolvik
Updated vals for 4dvar testing (baed on previous stable vrsion, were …
4385a5d 
…out of date)
@kysolvik
Updated presaved 4dvar test vals with consistent model delta_t (previ…
4fd0f74 
…ously was 0.01 for nature run, 0.05 for forecast model)
@kysolvik
4dvar method need different calc_default_R method (single observation…
06cab3d 
… timestep instead of full flattened observations)
@kysolvik
Test without specifying R
7bfc9e0
@kysolvik
Updated enso_indices (and tests) for xarray
d055622
@kysolvik
Removing vector class tests
0068c7f
@kysolvik
Updated gcp tests
34cd96c
@kysolvik
Smaller GCP tests to speed up downloads
9317353
@kysolvik
Smaller GCP tests, all passing
5ac3737
@kysolvik
Updated model base test for xarray, passing
107744b
@kysolvik
Observer tests updated and sqgturb system dim set in real space
502d549
@kysolvik
Skipping obsop tests (temporary, not used in any examples)
d7f2196
@kysolvik
Update README with xarray, added observer example
a542566
@kysolvik
Correct system_dim in sqgturb test
d87581c
@kysolvik kysolvik marked this pull request as ready for review October 8, 2024 13:48
@kysolvik
Copy link
Collaborator Author

kysolvik commented Oct 8, 2024

@StevePny Tests and readme are updated. The main things still to do for the xarray refactor are:

  1. Work on updating docstrings
  2. Fix/standardize notation between da cyclers, which you flagged above.

I think we could merge this before doing that work, just so that all the xarray functionality is all in main for us to pull from. But if you'd rather we wait until they're done, I could get to these two things some time this week.

@StevePny StevePny mentioned this pull request Oct 15, 2024
Open
@StevePny StevePny merged commit 0ec0b80 into main Oct 15, 2024
1 check passed
This was referenced Oct 15, 2024
Closed
Closed
@kysolvik kysolvik deleted the dev/xarray_base branch October 18, 2024 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@StevePny StevePny StevePny left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kysolvik @StevePny