Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update reference files for new asdf standard versions #409

Merged
merged 4 commits into from
May 13, 2024
+2,568 −172
Merged

update reference files for new asdf standard versions #409

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3a850f7
update reference files for new asdf standard versions
braingram Nov 10, 2023
e79b16e
stop pre-commit from changing reference files
braingram Nov 10, 2023
5fa3b57
use asdf 3.0.1
braingram Nov 20, 2023
ad0ea12
add changelog
braingram Nov 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
exclude: "reference_files/.*"
repos:

- repo: https://github.com/pre-commit/pre-commit-hooks
Expand Down
2 changes: 1 addition & 1 deletion CHANGES.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
1.1.2 (unreleased)
------------------

-
- Update reference files [#409]

1.1.1 (2024-03-06)
------------------
Expand Down
Binary file modified reference_files/1.0.0/ascii.asdf
View file
Open in desktop
Binary file not shown.
4 changes: 2 additions & 2 deletions reference_files/1.0.0/ascii.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
data: !core/ndarray-1.0.0
data: ['', ascii]
datatype: [ascii, 5]
Expand Down
Binary file modified reference_files/1.0.0/basic.asdf
View file
Open in desktop
Binary file not shown.
4 changes: 2 additions & 2 deletions reference_files/1.0.0/basic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
data: !core/ndarray-1.0.0
data: [0, 1, 2, 3, 4, 5, 6, 7]
datatype: int64
Expand Down
Binary file modified reference_files/1.0.0/complex.asdf
View file
Open in desktop
Binary file not shown.
284 changes: 142 additions & 142 deletions reference_files/1.0.0/complex.yaml
View file
Open in desktop

Large diffs are not rendered by default.

Binary file modified reference_files/1.0.0/compressed.asdf
View file
Open in desktop
Binary file not shown.
4 changes: 2 additions & 2 deletions reference_files/1.0.0/compressed.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
bzp2: !core/ndarray-1.0.0
data: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
Expand Down
4 changes: 2 additions & 2 deletions reference_files/1.0.0/exploded.asdf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
data: !core/ndarray-1.0.0
source: exploded0000.asdf
datatype: int64
Expand Down
4 changes: 2 additions & 2 deletions reference_files/1.0.0/exploded.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
data: !core/ndarray-1.0.0
data: [0, 1, 2, 3, 4, 5, 6, 7]
datatype: int64
Expand Down
Binary file modified reference_files/1.0.0/exploded0000.asdf
View file
Open in desktop
Binary file not shown.
Binary file modified reference_files/1.0.0/float.asdf
View file
Open in desktop
Binary file not shown.
4 changes: 2 additions & 2 deletions reference_files/1.0.0/float.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
datatype<f4: !core/ndarray-1.0.0
data: [0.0, -0.0, .nan, .inf, -.inf, -3.4028234663852886e+38, 3.4028234663852886e+38,
1.1920928955078125e-07, 5.960464477539063e-08, 1.1754943508222875e-38]
Expand Down
Binary file modified reference_files/1.0.0/int.asdf
View file
Open in desktop
Binary file not shown.
4 changes: 2 additions & 2 deletions reference_files/1.0.0/int.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
datatype<i1: !core/ndarray-1.0.0
data: [127, -128, 0]
datatype: int8
Expand Down
Binary file modified reference_files/1.0.0/shared.asdf
View file
Open in desktop
Binary file not shown.
4 changes: 2 additions & 2 deletions reference_files/1.0.0/shared.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
data: !core/ndarray-1.0.0
data: [0, 1, 2, 3, 4, 5, 6, 7]
datatype: int64
Expand Down
Binary file modified reference_files/1.0.0/stream.asdf
View file
Open in desktop
Binary file not shown.
4 changes: 2 additions & 2 deletions reference_files/1.0.0/stream.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
my_stream: !core/ndarray-1.0.0
data:
- [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
Expand Down
Binary file modified reference_files/1.0.0/unicode_bmp.asdf
View file
Open in desktop
Binary file not shown.
4 changes: 2 additions & 2 deletions reference_files/1.0.0/unicode_bmp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
datatype<U: !core/ndarray-1.0.0
data: ['', Æʩ]
datatype: [ucs4, 2]
Expand Down
Binary file modified reference_files/1.0.0/unicode_spp.asdf
View file
Open in desktop
Binary file not shown.
8 changes: 4 additions & 4 deletions reference_files/1.0.0/unicode_spp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
name: asdf, version: 1.0.0}
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
datatype<U: !core/ndarray-1.0.0
data: ['', "\U00010020"]
datatype: [ucs4, 2]
datatype: [ucs4, 1]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is unclear to me what unicode_spp is supposed to be illustrating. My closest guess is that the character 𐀠 is U+10020 is part of plane 1 the Supplementary Multilingual Plane so perhaps this was suppored to be smp not spp. On my system this is stored in 4 byte unicode which explains the [ucs4, 1] change in this file. I'm not sure what system or version of numpy was using 8 bytes to store this character.

>> np.unicode_('𐀠').dtype.itemsize
4

The generate.py file could be updated to use U2 instead of U to generate a file that uses 8 bytes:

>> np.array(['𐀠'], dtype='U2').dtype.itemsize
8

At this point I suggest we hold off making this change to generate.py and instead include the updated file in this PR as it offers a slightly different example for unicode in ASDF compared to the bmp file (which has a combined character which results in [ucs4, 2]).

shape: [2]
datatype>U: !core/ndarray-1.0.0
data: ['', "\U00010020"]
datatype: [ucs4, 2]
datatype: [ucs4, 1]
shape: [2]
...
Binary file added reference_files/1.1.0/ascii.asdf
View file
Open in desktop
Binary file not shown.
12 changes: 12 additions & 0 deletions reference_files/1.1.0/ascii.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#ASDF 1.0.0
#ASDF_STANDARD 1.1.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
data: !core/ndarray-1.0.0
data: ['', ascii]
datatype: [ascii, 5]
shape: [2]
...
Binary file added reference_files/1.1.0/basic.asdf
View file
Open in desktop
Binary file not shown.
12 changes: 12 additions & 0 deletions reference_files/1.1.0/basic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#ASDF 1.0.0
#ASDF_STANDARD 1.1.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.0.0
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf',
name: asdf, version: 3.0.1}
data: !core/ndarray-1.0.0
data: [0, 1, 2, 3, 4, 5, 6, 7]
datatype: int64
shape: [8]
...
Binary file added reference_files/1.1.0/complex.asdf
View file
Open in desktop
Binary file not shown.
Loading
Loading