Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update core for ASDF standard 1.6.0 #422

Merged
merged 13 commits into from
Mar 5, 2024
Merged

Conversation

braingram
Copy link
Contributor

@braingram braingram commented Feb 7, 2024

Removes core/column and core/table (replacing them with table/column and table/table).
Removes all non-core tags from the core manifest, putting them in a astronomy manifest.
Removes subclass_metadata from 1.6.0.
Update asdf-schema metaschema version to 1.1.0 to pick up ndarray-1.1.0 datatypes.

@@ -1,14 +1,14 @@
%YAML 1.1
---
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"
id: "http://stsci.edu/schemas/asdf/core/table-1.1.0"
id: "http://stsci.edu/schemas/asdf/table/table-1.1.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're modifying this anyway, should we convert it to an asdf://asdf-format.org URI?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm all for the change but would rather change (or attempt to change) all the tags at once. Perhaps that's a good target for 1.7.0?

can be cast to the given datatype without data loss. For
exact datatype matching, set `exact_datatype` to `true`.
allOf:
- $ref: "http://stsci.edu/schemas/asdf/core/ndarray-1.1.0#/definitions/datatype"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah bummer. Such is life :)

Copy link
Contributor

@nden nden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
One question, for my understanding - Is the effect of removing the "astronomy" tags from version_map-1.6.0 only that they are not included in the 1.6.0 standard? Does it affect in any way the ability to version them, e.g. read a file with an old astronomy tag and write it out using the current version?

@braingram
Copy link
Contributor Author

Thanks for the review!

The version maps have been largely replaced by the extension manifests. They are currently only used to determine the YAML (1.1) and file format (1.0) versions. asdf-format/asdf#1745 is work towards removing them (by instead defining these versions as constants). The tags listed in the version maps are unused in asdf (their usage was removed when the type index was removed in asdf 3.0).

The tag selection for reading/writing is now entirely defined by the extensions (and manifests). To use core/column-1.0.0 as an example, let's say a file was written with asdf-standard 1.5 (the current latest 'stable' version) and that file contains a core/column-1.0.0 tag. When the file is read, asdf will look for an extension that supports core/column-1.0.0, it will find the core-1.5.0 extension (provided by asdf-astropy and defined from the 1.5.0 manifest which defines core/column-1.0.0). The tag will be read in as an astropy.table.column.Column. If this file is then rewritten asdf will look for an extension that support astropy.Column and what occurs depends on the asdf standard version provided to write_to. By default the latest stable version (1.5.0) will be used. asdf will find the core-1.5.0 extension (which requires asdf-standard version 1.5.0) and write out the Column using the core/column-1.0.0 tag. If instead, asdf standard 1.6.0 is defined, the core-1.5.0 extension will be skipped (the standard requirement isn't met) and instead asdf will use the new astronomy manifest (with support for this extension being added to asdf-astropy in this PR: astropy/asdf-astropy#219) and the Column will be written with the new table/column-1.0.0 tag. Let me know if more details would be helpful.

@braingram braingram merged commit 33ea4d3 into asdf-format:main Mar 5, 2024
14 checks passed
@braingram braingram deleted the split_core branch March 5, 2024 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants