Skip to content

Should consolidated metadata always be executed at the store root? #2920

Open
@jhamman

Description

@jhamman

Zarr version

v3.0.5

Numcodecs version

v0.15.1

Python Version

Python 3.11

Operating System

Mac

Installation

conda/pip

Description

Zarr-Python 2's consolidate_metadata included a path argument. However, regardless of the value provided to path, the .zmetadata JSON object was always placed at the root of the Store. This behavior changed in Zarr-Python 3. Was this intentional and if so, is it really what we want?

I know @TomAugspurger gave this some thought when implementing consolidated metadata for zarr-python 3. So perhaps we just need to document that this change was indeed intentional?

Steps to reproduce

# zarr-python 3
store = {}

root = zarr.group(store=store, zarr_format=2)
zarr.consolidate_metadata(store)
root.create_group("foo")
root.create_group("foo/spam")
zarr.consolidate_metadata(store, path="foo")
list(store)

# ['.zgroup',
#  '.zattrs',
#  '.zmetadata',
#  'foo/.zgroup',
#  'foo/.zattrs',
#  'foo/spam/.zgroup',
#  'foo/spam/.zattrs',
#  'foo/.zmetadata']  # <- from final consolidation
# zarr-python 2
store = {}

root = zarr.group(store=store)
zarr.consolidate_metadata(store)
root.create_group("foo")
root.create_group("foo/spam")
zarr.consolidate_metadata(store, path="foo")
list(store)
# ['.zgroup', '.zmetadata', 'foo/.zgroup', 'foo/spam/.zgroup']  # <- no foo/.zmetadata

Additional output

xref: pydata/xarray#10020

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugPotential issues with the zarr-python library

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions