Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added consolidated metadata to spec #309

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docs/v3/core/v3.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -803,6 +803,43 @@ Optional keys:
pairs, where the key must be a string and the value can be an arbitrary
JSON literal. Intended to allow storage of arbitrary user metadata.

``consolidated_metadata``
^^^^^^^^^^^^^^^^^^^^^^^^^

An object consolidating all the Array and Group metadata of members below
the root node in a hierarchy.

=============== ========================= ===================================================================================================
Field Type Description
=============== ========================= ===================================================================================================
metadata ``Map<string, Metadata>`` A mapping from node path to Group or Array ``Metadata`` object.
kind const ``'inline'`` The string literal ``'inline'``. Reserved for future use.
must_understand const ``False`` The boolean literal ``False``. Indicates that the field is not required to load the Zarr hierarchy.
=============== ========================= ===================================================================================================

Note that *all* children Arrays and Groups should be included in
consolidated metadata, not just the nodes immediately below the root Group.
Children nested inside other groups should be included too.
TomAugspurger marked this conversation as resolved.
Show resolved Hide resolved

Consolidated metadata can help reduce the time needed to load the metadata
for an entire hierarchy, especially when the metadata is being served over a
network. Without consolidated metadata, opening an entire hierarchy over the
network requires an HTTP request per node. Consolidated metadata enables
loading the metadata for every node in a hierarchy with a single HTTP
request.

Consolidated Metadata is optional. If present, then readers should use the
consolidated metadata when . When not present, readers should use the
TomAugspurger marked this conversation as resolved.
Show resolved Hide resolved
non-consolidated metadata located in the Store to load the data.
TomAugspurger marked this conversation as resolved.
Show resolved Hide resolved

The ``kind`` field indicates that consolidated metadata is stored inline in
the root ``zarr.json`` object. At this time, ``'inline'`` is the only
supported value for ``kind``. Future versions of the specification may allow
for consolidated metadata in other locations.

Example Group Metadata
----------------------

For example, the JSON document below defines a group::

{
Expand Down