Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specifying numpy dtype in test functions #122

Closed

Conversation

ian-coccimiglio
Copy link
Contributor

This PR contains:

  • a new test-case for the benchmark
    • I hereby confirm that NO LLM-based technology (such as github copilot) was used while writing this benchmark
  • new dependencies in requirements.txt
    • The environment.yml file was updated using the command conda env export > environment.yml
  • new generator-functions allowing to sample from other LLMs
  • new samples (sample_....jsonl files)
  • new benchmarking results (..._results.jsonl files)
  • documentation update
  • bug fixes

Related github issue (if relevant): would close #115

Short description:

  • I think it'd be best to specify a plausible data-type for images/arrays in the test-check.

How do you think will this influence the benchmark results?

  • For Otsu's threshold + positive pixel counting, overall model pass-rate goes from 10/230 to 75/230.
  • I haven't done other tasks yet. I would assume it would have a generally positive effect in cases where OpenCV is type-sensitive (applies to at least 2-3 more tests as far as I've seen).

Why do you think it makes sense to merge this PR?

  • I'm waiting to see whether we decide to do this method or go for prompt-editing (or both). Not to be merged yet.
  • Also it fixes the spelling of the test-case which I'm not sure we want to merge as I can't tell what it might break.

@ian-coccimiglio
Copy link
Contributor Author

My bad, Github and I don't always cooperate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Should we represent images using default numpy.asarray()?
1 participant