Hi @ClementCaporal ,

awesome idea! We already have a test case for loading zarr files: https://github.com/haesleinhuepf/human-eval-bia/blob/7e9712670168b71bea44e17e7389dfc120dcc96e/test_cases/open_zarr.ipynb

Do you think you could formulate a similar task for saving ngff files?

Thanks for the suggestion!

Hello,

Sorry for the delay, I am discovering how frustrating it can be to create a good prompt for a "simple" task.
I am still working on a clever way to "ask" for a good ngff files...

Meanwhile I did this notebook that just check if it is able to save a zarr file: https://gist.github.com/ClementCaporal/02a9401877c2b8b49fe6edbca5816962
The check is permissive between saving a group or directly the array in the zarr. (Llama tends to save it with a name and GPT-3.5 directly as an array)

Hi @ClementCaporal ,

awesome, the test-case looks good to me! Consider sending a pull-request. You can also later add a more complicated test-case for zarr files.

And please do not make any decisions on how to change the test case depending on how individual LLMs solve the test. Ideallly you write a test-case without promoting any LLM. Otherwise you would bias the test 😉