Move ndarray conversion to a Converter#1537

Merged

braingram merged 154 commits intoasdf-format:mainfrom braingram:immutable_block_manager

Sep 8, 2023

+5,400-3,063

Commits on Aug 18, 2023

move asdf.block to asdf._block
braingram
committed
remove deprecated AsdfFile.blocks
braingram
committed
add components to break apart block manager
braingram
committed
add read/write block index
braingram
committed
add block reader/writer
braingram
committed
add _block.LinearStore for storing read blocks
braingram
committed
fix _block.store.LinearStore.__len__
braingram
committed
break update
braingram
committed
messy write support
braingram
committed
fix resolve_and_inline
braingram
committed
stream blocks working
braingram
committed
external blocks working
braingram
committed
add block data caching for ndarraytype
braingram
committed
convert ndarray to using callback instead of raw block access
braingram
committed
remove cludges added to new block manager
braingram
committed
reduce usage of asdffile reference in ndarray
braingram
committed
associate objects with write blocks
braingram
committed
associate streamed block with object
braingram
committed
Revert "break update"
braingram
committed
update working
braingram
committed
output compression extension list working
braingram
committed
fix block converter tests
braingram
committed
rebased, broke 2 tests
braingram
committed
add tests for issues
braingram
committed
prevent modification of tree or blocks during write/validate
braingram
committed
minor reorganization of 1013 test
braingram
committed
external block fixes
braingram
committed
remove old block manager
braingram
committed
increasing _block test coverage
braingram
committed
cleaning up _blocks usage in AsdfFile.write_to
braingram
committed
drop asdffile from __init__ args to NDArrayType
braingram
committed
cleanup some comments
braingram
committed
enable block only files
braingram
committed
clear _write_fd during block manager _clear_write
braingram
committed
make serialization context aware of current object
braingram
committed
add _block/callback tests
braingram
committed
add ndarray converter (external blocks broken)
braingram
committed
external blocks working again
braingram
committed
update diff to not use NDArrayType directly
braingram
committed
clean up NDArrayType, xfail 1530 test
braingram
committed
fix failing test from rebase
braingram
committed
remove Block usage in pytest plugin
braingram
committed
Allow converter to handle subclasses
braingram
committed
fixes for remote_data tests
braingram
committed
restrict subclass conversion to ndarray
braingram
committed
fix loading of empty inline arrays
braingram
committed
fix config doctests
braingram
committed
catch old FutureWarning from numpy in tests
braingram
committed
seek to 0 before update shortcuts
braingram
committed
allow file truncation on windows
braingram
committed
update array equality testing in tests
braingram
committed
reset file position after memmapping on windows
braingram
committed
attempt to fix array equality for older numpy
braingram
committed
fix reading for inline structured arrays
braingram
committed
add xfailed test for 1539
braingram
committed
increase number of tries to get reused memory
braingram
committed
fix validation of checksums on read
braingram
committed
add test for and fix asdf.stream deprecation
braingram
committed
remove fd from generate_write_header args
braingram
committed
compute len of block magic from value instead of hard coding
braingram
committed
fix write for non-seekable files
braingram
committed
mock id for _block.store testing
braingram
committed
add _Operations for SerializationContext
braingram
committed
don't assign blocks if conversion raised error
braingram
committed
add multi-block example update converter docs
braingram
committed
fix multi-block converter example in docs
braingram
committed
add try/finally to update and copy blocks in block_size increments
braingram
committed
skip 1542 test on windows
braingram
committed
add eq and copy to _block.Key
braingram
committed
move block reading to Manager.read
braingram
committed
store uri in _block.Manager instead of AsdfFile
braingram
committed
reduce code duplication in AsdfFile.update
braingram
committed
move bulk of AsdfFile.update into _blocks.Manager.update
braingram
committed
rename _streamed_block to _streamed_write_block
braingram
committed
restore AsdfFile.version after write_to
braingram
committed
change _block.key.Key matches to matches_object
braingram
committed
fix handling of io_block_size
braingram
committed
skip checksum validation for streamed blocks
braingram
committed
add unit tests for _block.external
braingram
committed
move external block uri resolution to _block.external
braingram
committed
fix missing uri check for external blocks
braingram
committed
remove reserve_blocks
braingram
committed
simplify ndarray shape loading and fix error message
braingram
committed
move SerializationContext tests
braingram
committed
update SerializationContext tests
braingram
committed
allow >1 stream if the same block
braingram
committed
fix external cache test for windows
braingram
committed
adjust key assignment during deserialization
braingram
committed
add _block.manager unit tests
braingram
committed
make DataCallback.reassign private
braingram
committed
update docs off _block submodule
braingram
committed
make all block Key methods private
braingram
committed
support 'input' compression after overwriting compression
braingram
committed
update comment about input compression
braingram
committed
simplify ReadBlocks to a subclass of UserList
braingram
committed
update _block.manager docstrings
braingram
committed
update docs
braingram
committed
add docs for subclass conversion config setting
braingram
committed
update changelog
braingram
committed
update changes
braingram
committed
index write blocks by data
braingram
committed
remove _serialization_context._Operation context manager feature
braingram
committed
add external block UseInternalType
braingram
committed
remove unused config_context in block options
braingram
committed
change key copy to match stype of options copy
braingram
committed
remove out-of-date block options comments
braingram
committed
remove leftover name and types from Stream
braingram
committed
check for valid key on Store.assign_object
braingram
committed
avoid fast_forwarding using header values for streamed blocks
braingram
committed
remove NDArrayType subclassing from Stream
braingram
committed
change generate_write_block_header to return dict instead of packed header
braingram
committed
rename and clarify _block.external.relative_uri_to_index
braingram
committed
test for and handle files will odd 'tell' results
braingram
committed
remove LinearStore
braingram
committed
add unit test for Store.keys_for_value
braingram
committed
remove unused WriteBlocks functions add docs
braingram
committed
add custom exception BlockIndexError for block index reading/parsing
braingram
committed
get version string from serialization context
braingram
committed
select SerializationContext operation at creation
braingram
committed
remove unneeded code
braingram
committed
add warning when <4 non-null bytes after blocks
braingram
committed
preserve block index during asdftool edit
braingram
committed
re-add SerializationContext to asdf.asdf for weldx
braingram
committed
allow external block memory mapping
braingram
committed
re-add block manager close
braingram
committed
clear SerializationContext object after block assignment
braingram
committed
fix external block loading over http
braingram
committed
fix file url parsing for external blocks on windows
braingram
committed
remove use of assert_roundtrip_tree in ndarray tests
braingram
committed
attempt to fix FutureWarning for array comparsion with older numpy
braingram
committed
rename some local variables to avoid confusion
braingram
committed
fix stream deprecation test
braingram
committed
fix reference uri resolution
braingram
committed
run pre-commit locally
braingram
committed
ignore hypothesis files
braingram
committed
move SerializationContext out of asdf.extension
braingram
committed
keep SerializationContext exposed at asdf.asdf
braingram
committed
temporarily use dev sphinx-asdf
braingram
committed
move SerializationContext back into asdf.extension
braingram
committed
deprecate import of asdf.asdf.SerializationContext
braingram
committed
move _issues tests to _regtests and rename tests
braingram
committed
typo prevented dev sphinx_asdf install
braingram
committed
move write_to version reset into finally
braingram
committed
add missing BlockAccess docstring
braingram
committed
add asdf.asdf.SerializationContext import deprecation to docs
braingram
committed
add parametrization to 1525 regression test
braingram
committed
remove unneeded line
braingram
committed
simplify ndarray subclass handling
braingram
committed
remove unnecessary assign_object(None)
braingram
committed
add warnings to failed block index reading
braingram
committed
remove unnecessary config context
braingram
committed
add test_update_compressed_blocks
braingram
committed
add AsdfBlockIndexWarning
braingram
committed

Commits on Sep 5, 2023

remove sphinx-asdf dev version requirement
braingram
committed