Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling dask array with unknown dimensions #174

Open
saimani5 opened this issue May 17, 2023 · 0 comments
Open

Handling dask array with unknown dimensions #174

saimani5 opened this issue May 17, 2023 · 0 comments

Comments

@saimani5
Copy link
Collaborator

When the output shape of an operation is unknown, the output is still a dask array whose shape is treated as nan (not a number). When we try to convert this array of unknown shape into a sidpy dataset, it raises an error.

For example,
dset = sid.Dataset.from_array(np.random.rand(4,5))
new_dset = dset[dset<0.5] # The shape of new_dset is unknown until we use .compute() on it.

The shape of new_dset is (nan,) and dset.like_data(new_dset) does not work. This is important when modifying getitem() to always return a sidpy dataset instead of a dask array.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant