New test cases suggestions

[rmassei]

Hi!
I was wondering if the following test cases might be of interest:

• Reading/adding metadata using with bioio or tifffile
• Including a couple of test cases with the library mahotas
• Convert ROIs coordinates or metadata to data structures that can be used by the library ezomero for import in OMERO (i.e convert tabular files to dictionary/lists/tupples). Mostly data wrangling with pandas.

[tischi]

+1 for bioio

[rmassei]

ome-types is another options, since I found myself to often use it to compile ome.xml with standardize metadata

[rmassei]

PR in #142

[haesleinhuepf]

Love it, thanks for the proposals @rmassei ! Regarding mahotas and omero: I'd say these test-cases would be interesting, if you formulare the prompts in a way, that mahotas / omero are not mentioned as library. So far, our test-cases typically formulate a problem, and do not ask for specific libraries (with the exception of numpy). In my opinion (others may have other opinions), we should not have test-cases like "Register the images using ITK", but better "register the images" and with this, give the LLM the freedom to also use libraries, we did not think of. You see in this figure of the current paper, that the LLMs chose to use openCV (cv2) in many cases, and we did not think of this.

Hence, do you know any use-cases that can be solved with mahotas, which were not on our list yet?

[rmassei]

Thanks for the explanation @haesleinhuepf!
I did not add mahotas cases in the PR since, after looking at the existing cases, inspiration did not knock at my door and I could not find anything new to add, at least as a snippet.
I will check the documentation and see if there are some tasks which can be interesting to solve with mahotas, but I will then probably open another PR.

[haesleinhuepf]

I dived into mahotas earlier, and found the seeded watershed implementation nice; much easier to use than the one from scikit-image. Maybe that's an inspiration. And if not that's ok too. With the use-cases you sent, you will already be the 2nd or 3rd most-contributing contributor ;-)

[tischi]

I generally agree on not asking about solutions implemented in specific packages.

However, don't we implicitly only allow for specific packages because our testing environment does not have everything? I forgot: Are the available libraries something that we communicate to the LLM for writing the code?

[haesleinhuepf]

The readme explains how to deal with missing dependencies (we add them to the requirements.txt) and has a link to a notebook for detecting missing stuff:
https://github.com/haesleinhuepf/human-eval-bia/blob/main/demo/detect_missing_requirements.ipynb