Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming multiple arrays to file #312

Open
kjmiller944 opened this issue May 5, 2022 · 1 comment
Open

Streaming multiple arrays to file #312

kjmiller944 opened this issue May 5, 2022 · 1 comment

Comments

@kjmiller944
Copy link

I would like to know how to stream multiple arrays to the same ASDF file. I modified the example to include two different streams in the tree dictionary. How do I write both to the same file?

from asdf import AsdfFile, Stream
import numpy as np

tree = {
    # Each "row" of data will have 128 entries.
    'my_stream_0': Stream([128], np.float64)
    'my_stream_1': Stream([128], np.float64)
}

ff = AsdfFile(tree)
with open('test.asdf', 'wb') as fd:
    ff.write_to(fd)
    # Write 100 rows of data, one row at a time.  ``write``
    # expects the raw binary bytes, not an array, so we use
    # ``tobytes()``.
    for i in range(100):
        fd.write(np.array([i] * 128, np.float64).tobytes())
        fd.write(np.array([i] * 128, np.float64).tobytes())
@perrygreenfield
Copy link
Contributor

I'm curious about the motivations for this, but before delving into this, are you aware of other formats that support this kind of feature? That might help us understand the issues better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants