Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Its possible to write zarr 2 format zarr's using zarr3 that can't be read by zarr v2 #2773

Open
benritchie opened this issue Jan 27, 2025 · 0 comments
Labels
bug Potential issues with the zarr-python library

Comments

@benritchie
Copy link

Zarr version

v3.0.1

Numcodecs version

15.0

Python Version

3.12.8

Operating System

MacOs

Installation

conda

Description

Hi

I'm Loving Zarr v3 :) - thanks.

I'm trying ot use zarr 3 to create zarr 2 format Zarr's, via xarrray.

i'm using a numcodecs compressor (PCodec), and left the code in "zarr 3 style".

using this setup, zarr allows me to write the array in zarr 2 format, but when I try to read it, I get an error:

ImportError: zarr 3.0.0 or later is required to use the numcodecs zarr integration

I think zarr 3 should have some firewalling in to ensure that it only allows zarr 2 compatable compressors if writing a zarr2 array.

Steps to reproduce

zarr creation code below (run using zarr 3.0.1:

import zarr
import xarray
import numcodecs.zarr3 as numcodecsz3

#Open the zarr (any zarr should work here - just set "test_band_ below to a band from the zarr you are using)
ds = xarray.open_zarr('./hls_20200101_20201231_99cc_50q', consolidated=True)

# We want to write the DataSet with different encodings than the source, therefore
# clear all encoding config first.  
ds = ds.drop_encoding()

encoding = {}
encoding['test_band'] = {"serializer": numcodecsz3.PCodec(level=5)}

ds.to_zarr('./test2', mode="w", encoding = encoding, zarr_format=2)

read code:
(using latest version of zarr 2)

import zarr
import xarray
import numcodecs.zarr3 as numcodecsz3

# Open file
z2 = zarr.open('test2', mode='r')

print(z2['ndvi'][:])

Additional output

No response

@benritchie benritchie added the bug Potential issues with the zarr-python library label Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Potential issues with the zarr-python library
Projects
None yet
Development

No branches or pull requests

1 participant