Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New Test-Case - Metadata Reading plus some basics image processing #142

Merged
merged 13 commits into from
Nov 21, 2024
Merged

New Test-Case - Metadata Reading plus some basics image processing #142

merged 13 commits into from
Nov 21, 2024

Conversation

rmassei
Copy link

@rmassei rmassei commented Oct 17, 2024 •
edited
Loading

This PR contains:

  • a new test-case for the benchmark
    • I hereby confirm that NO LLM-based technology (such as github copilot) was used while writing this benchmark
  • new dependencies in requirements.txt
    • The environment.yml file was updated using the command conda env export > environment.yml
  • new generator-functions allowing to sample from other LLMs
  • new samples (sample_....jsonl files)
  • new benchmarking results (..._results.jsonl files)
  • documentation update
  • bug fixes

Related github issue (if relevant): closes #0

Short description:
The present PR add the following test-case:

Metadata Extraction and Reading using different libraries

  • test_cases/image_proprieties_reader.ipynb
  • test_cases/read_ome_xml_from_ome_tiff.ipynb
  • test_cases/read_ome_xml_from_xml.ipynb
  • test_cases/tiff_metadata_reading.ipynb
  • test_cases/tiff_metadata_reading_2.ipynb

Data Wrangling

  • test_cases/convert_rois_coordinates.ipynb
  • test_cases/dataframe_column_rename.ipynb
  • test_cases/reshape_array_omero.ipynb

Array Reshaping:

  • test_cases/stack_and_merge.ipynb
  • test_cases/rescale_image.ipynb

How do you think will this influence the benchmark results?

  • These tasks should not be difficult to solve, especially the data wrangling. It is interesting to see the results of the metadata reading

Why do you think it makes sense to merge this PR?

  • This PR increase the diversity of the cases studies, especially the ones regarding the metadata reading

closes #141

@rmassei rmassei mentioned this pull request Oct 17, 2024
Open
@haesleinhuepf
Copy link
Owner

I will take some time for reviewing those @rmassei , but thanks so much for preparing those!

@haesleinhuepf haesleinhuepf mentioned this pull request Oct 18, 2024
Merged
5 tasks
@rmassei
Copy link
Author

rmassei commented Oct 18, 2024

Just realize that I made a mistake, I need to change the prompting for all the test-cases... I am on it!

@haesleinhuepf
Copy link
Owner

no hurry. thanks again!

@haesleinhuepf haesleinhuepf changed the base branch from main to development-collecting-new-test-cases November 21, 2024 11:28
@haesleinhuepf
Copy link
Owner

Hey Riccardo @rmassei ,

big thanks again for this! These test-cases look great!

I'm now going to merge these into a dev-branch, where we can collect new test cases. Just explaining: If we merged new test-cases now, we would have to re-run all the benchmarks as the new test-cases haven't been tested with the 20+ models we benchmarked so far.

If you are interested in becoming co-author on the manuscript, please let us know your full name and affiliation so that we can add you. Also let us know if you have any conflicts of interest that should be reported when submitting the paper.

Thanks again!

Best,
Robert

@haesleinhuepf haesleinhuepf mentioned this pull request Nov 21, 2024
Open
@haesleinhuepf haesleinhuepf mentioned this pull request Nov 21, 2024
Open
@haesleinhuepf haesleinhuepf mentioned this pull request Nov 21, 2024
Open
@haesleinhuepf haesleinhuepf mentioned this pull request Nov 21, 2024
Open
@haesleinhuepf
Copy link
Owner

I removed some test-cases (see issues linked above) and updated names and docstrings of the other ones.

Thanks again for your efforts @rmassei !

@haesleinhuepf haesleinhuepf merged commit e0c2df7 into haesleinhuepf:development-collecting-new-test-cases Nov 21, 2024
@haesleinhuepf haesleinhuepf mentioned this pull request Nov 21, 2024
Open
9 tasks
@rmassei
Copy link
Author

rmassei commented Nov 21, 2024 •
edited
Loading

Hej @haesleinhuepf ,
thanks a lot for looking into this and sorry if you had to go trough some intensive clean-up :)
I will check the the test cases causing problems an open a new PR.

Regarding my name and affiliation:
Riccardo Massei
Department of Monitoring and Exploration Technologies - Research Data Management Team (RDM)
Department of Ecotoxicology - Group Integrative Toxicology (iTox)

@rmassei rmassei mentioned this pull request Nov 22, 2024
Open
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

New test cases suggestions
2 participants
@rmassei @haesleinhuepf