Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Differentiate suboptimal_program from a program that fails to execute #72

Open
ReubenJ opened this issue Mar 21, 2024 · 1 comment
Open

Comments

@ReubenJ
Copy link
Member

ReubenJ commented Mar 21, 2024

Currently, when searching using allow_evaluation_errors and all of the programs error out on evaluation (interpret errors out), the return value of synth is marked as a suboptimal_program. This is somewhat confusing because it seems to say that the returned program executed but didn't pass evaluation for some IO examples.

We can either:

  • add a third enum value to SynthResult like failing_program
    @enum SynthResult optimal_program=1 suboptimal_program=2
  • warn that allow_evaluation_errors might mean that the returned suboptimal_program might have thrown an error
@sebdumancic
Copy link
Member

This is a good idea!

Especially because if we have a program failing to run, we might be able to deduce additional constraints to remove programs that would fail for the same reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants