TypeError: Expected a BytesBytesCodec. Got <class 'numcodecs.blosc.Blosc'> instead.

[leoniewgnr]

This code runs without any problems with zarr2, but give the following error when running with zarr3:

import pandas as pd
import numpy as np
import xarray as xr
from numcodecs.blosc import Blosc

ds = xr.Dataset(
    {"foo": (("x", "y"), np.random.rand(4, 5))},
    coords={
        "x": [10, 20, 30, 40],
        "y": pd.date_range("2000-01-01", periods=5),
        "z": ("x", list("abcd")),
    },
)

tmp_path = 'tmp.zarr'

# this works
ds.to_zarr(tmp_path, mode="w")
print('Saved to tmp.zarr')

# this does not work 
compressor = Blosc(cname="zstd", clevel=3, shuffle=2)
ds.to_zarr(tmp_path, encoding={"foo": {"compressor": compressor}}, mode="w")
print('Saved to tmp.zarr')

The error message is: TypeError: Expected a BytesBytesCodec. Got <class 'numcodecs.blosc.Blosc'> instead.
The same error occurs in the documentation: https://docs.xarray.dev/en/stable/user-guide/io.html#zarr-compressors-and-filters

[welcome]

Thanks for opening your first issue here at xarray! Be sure to follow the issue template!
If you have an idea for a solution, we would really welcome a Pull Request with proposed changes.
See the Contributing Guide for more.
It may take us a while to respond here, but we really value your contribution. Contributors like you help make xarray better.
Thank you!

[TomNicholas]

Thanks for raising this @leoniewgnr ! We're still hunting down all the bugs that the move to zarr 3 created.

The same error occurs in the documentation:

That's particularly weird - errors in the documentation examples are supposed to lead to errors in the CI...

[keewis]

See also #9987

[FedeMPouzols]

I think this example needs to be updated for zarr-python 3. Something like this works for me:

diff --git a/doc/user-guide/io.rst b/doc/user-guide/io.rst
index 986d43ce..7f5d6e2b 100644
--- a/doc/user-guide/io.rst
+++ b/doc/user-guide/io.rst
@@ -829,10 +829,10 @@ For example:
     :okwarning:
 
     import zarr
-    from numcodecs.blosc import Blosc
+    from zarr.codecs import BloscCodec
 
-    compressor = Blosc(cname="zstd", clevel=3, shuffle=2)
-    ds.to_zarr("foo.zarr", encoding={"foo": {"compressor": compressor}})
+    compressor = BloscCodec(cname="zstd", clevel=3, shuffle="shuffle")
+    ds.to_zarr("foo.zarr", encoding={"foo": {"compressors": (compressor,)}})
 
 .. note::

(this is my best guess based on what I see in the backend tests some Zarr v3 related PRs. In this particular case, {"compressor": compressor} (without tuple) seems to also work.).

Perhaps @d-v-b can confirm this is now the proper way to specify encoders/help with this?

[d-v-b]

that looks right, although I'm not too familiar with what ds.to_zarr is doing under the hood. The basic idea in zarr v3 is that there can be multiple codecs that transform an array after it has been flattened to a byte stream (alternately called "compressors" or "BytesBytesCodec"), hence the tuple. but we also accept a single codec, which we will wrap in a tuple.

[fowlerovski]

My situation with numcodecs 0.15.1 and Zarr 3.0.3 mirrors this: BytesBytesCodec is unavailable in numcodecs.abc, and even numcodecs.Blosc is rejected with TypeError: Expected a BytesBytesCodec.

[roansong]

I'm running into this as well, even when using numcodecs.zarr3.Blosc or zarr.codecs.BloscCodec.

[aurelgriesser]

@FedeMPouzols, when I tried your suggested {"compressors": (compressor,)} form (with tuple value and now plural key "compressors" instead of the older singular form), I still get the "TypeError: Expected a BytesBytesCodec" of leoniewgnr.
Ta.

[rsignell]

Didn't work for me either -- here's a reproducible example notebook: https://nbviewer.org/gist/rsignell/066cc39664a0c8b7fe70be1fd7d7e0cb

[jensdebruijn]

Edited, because much simpler solution below.

It actually seems that the error is not with the compressed data array but with the coords. Xarray's default BloscCodec (used when no compressor is specified) inherits from numcodecs.abc.Codec, while it should inherit from zarr.abc.codec.BytesBytesCodec for zarr v3 to pass the isinstance assert. It also seems that xarray compresses the coords as well by default, thus using the default compressor that incorrectly inherits from the numcodecs.abc.Codec.

The (temporary) solution I found is to use zarr.codecs.BloscCodec for the data var like you would expect, and explicitly tell xarray not to compress the coordinates like so:

from zarr.codecs import BloscCodec

encoding = {
    "data": {
        "compressor": BloscCodec(
            cname="zstd",
            clevel=6,
        ),
    }
}
for coord in da.coords:
    encoding[coord] = {"compressor": None}

If you do want to compress the coords specifying the compressor explicitly from zarr.codecs should also likely work (not tested).