Skip to content

TypeError when passing old numcodecs to zarr v3 #2964

Open
@TomNicholas

Description

@TomNicholas

Zarr version

v3.0.6

Numcodecs version

v0.16.0

Python Version

3.13

Operating System

mac

Installation

uv

Description

Passing the old stype of numcodecs codec to zarr raises a TypeError, when this scenario could be detected and upcast into the zarr-v3-compatible version of that codec instead.

This has been reported by a lot of xarray users (pydata/xarray#10032) as well as here #2710 (comment).

Traceback (most recent call last):
  File "/Users/tom/Documents/Work/Code/experimentation/bugs/blosc/pure_zarr_mve.py", line 25, in <module>
    za = zarr.create_array(
        store,
    ...<4 lines>...
        compressors=compressors,
    )
  File "/Users/tom/.cache/uv/environments-v2/pure-zarr-mve-2145b34a8fc90dca/lib/python3.13/site-packages/zarr/api/synchronous.py", line 879, in create_array
    sync(
    ~~~~^
        zarr.core.array.create_array(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<19 lines>...
        )
        ^
    )
    ^
  File "/Users/tom/.cache/uv/environments-v2/pure-zarr-mve-2145b34a8fc90dca/lib/python3.13/site-packages/zarr/core/sync.py", line 163, in sync
    raise return_result
  File "/Users/tom/.cache/uv/environments-v2/pure-zarr-mve-2145b34a8fc90dca/lib/python3.13/site-packages/zarr/core/sync.py", line 119, in _runner
    return await coro
           ^^^^^^^^^^
  File "/Users/tom/.cache/uv/environments-v2/pure-zarr-mve-2145b34a8fc90dca/lib/python3.13/site-packages/zarr/core/array.py", line 4146, in create_array
    result = await init_array(
             ^^^^^^^^^^^^^^^^^
    ...<16 lines>...
    )
    ^
  File "/Users/tom/.cache/uv/environments-v2/pure-zarr-mve-2145b34a8fc90dca/lib/python3.13/site-packages/zarr/core/array.py", line 3961, in init_array
    array_array, array_bytes, bytes_bytes = _parse_chunk_encoding_v3(
                                            ~~~~~~~~~~~~~~~~~~~~~~~~^
        compressors=compressors,
        ^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        dtype=dtype_parsed,
        ^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/tom/.cache/uv/environments-v2/pure-zarr-mve-2145b34a8fc90dca/lib/python3.13/site-packages/zarr/core/array.py", line 4330, in _parse_chunk_encoding_v3
    out_bytes_bytes = tuple(_parse_bytes_bytes_codec(c) for c in maybe_bytes_bytes)
  File "/Users/tom/.cache/uv/environments-v2/pure-zarr-mve-2145b34a8fc90dca/lib/python3.13/site-packages/zarr/core/array.py", line 4330, in <genexpr>
    out_bytes_bytes = tuple(_parse_bytes_bytes_codec(c) for c in maybe_bytes_bytes)
                            ~~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "/Users/tom/.cache/uv/environments-v2/pure-zarr-mve-2145b34a8fc90dca/lib/python3.13/site-packages/zarr/registry.py", line 184, in _parse_bytes_bytes_codec
    raise TypeError(f"Expected a BytesBytesCodec. Got {type(data)} instead.")
TypeError: Expected a BytesBytesCodec. Got <class 'numcodecs.blosc.Blosc'> instead.

Steps to reproduce

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "numpy",
#     "zarr>=3",
# ]
# ///
import numpy as np
import zarr
import numcodecs

print(zarr.__version__)
print(numcodecs.__version__)

store = "/tmp/foo.zarr"
shape = (1024 * 1024 * 1024,)
chunks = (1024 * 1024 * 16,)
dtype = np.float64
fill_value = np.nan

# cname = "blosclz"
cname = "lz4"
compressors = [numcodecs.Blosc(cname="lz4")]

za = zarr.create_array(
    store,
    shape=shape,
    chunks=chunks,
    dtype=dtype,
    fill_value=fill_value,
    compressors=compressors,
)

Additional output

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugPotential issues with the zarr-python library

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions