-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[NOT FOR MERGE] Adjust prompt to use view command #5506
base: main
Are you sure you want to change the base?
Conversation
There we go:
|
I ran 13 instances that are unresolved (0/13) in the CodeAct 2.2 results. They're all on django, and all part of the intersection of Lite with Verified. CodeAct2.2: 0/13 Too little to matter, but FWIW! @xingyaoww |
I'm thinking about whether we should still make this change in the prompt, as encouraging the agent to use |
Running evaluation on the PR. Once eval is done, the results will be posted. |
Evaluation results: ## Summary
Empty patches were from the litellm proxy error:
|
Haven't automated this part yet so here ya go: |
@openhands-agent Your last attempt to fix the conflicts didn't work. Please do this again: pull main into this branch and fix the conflicts. |
Give a summary of what the PR does, explaining any non-trivial design decisions
This is the prompt adjustment I used, with the purpose that the LLM uses more the
view
tool for directories, which is part of itsfile_editor
tool, than other options (ls -R /workspace
orls -la /workspace
).This is used for the initial exploration of the repository a lot.
I think it would be interesting to eval this after Ryan's fix is merged in
main
.Edited to add: in the event stream of CodeAct 2.2 swe-bench run, the agent uses
ls -R /workspace
a lot, andls -la
sometimes.To run this PR locally, use the following command: