Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PRO-812] manual input #18

Merged
merged 5 commits into from
Sep 30, 2024
Merged

[PRO-812] manual input #18

merged 5 commits into from
Sep 30, 2024

Conversation

toshok
Copy link
Collaborator

@toshok toshok commented Sep 26, 2024

oops, ignore the branch name.

enabled by setting MANUAL_TDD_INPUT_DIRECTORY to the base storage dir.

Removes the tdd result from the initial instructions and replaces it with:

         # HOW TO BEGIN
         To start things off, run the 'tdd_repro' command to reproduce the issue.

That way the flow is substantially easier to reason about (and the directory structure makes quite a bit more sense), as only tdd_repro usage needs to be considered.

when you run the run-instance.sh script you can now run it in two ways:

Starting a new conversation tree

./run_instance.sh django__django-12419

This will run the instance with the id django__django-12419 and will create the directory $BASE/django__django-12419/root-$conversation_hash where $conversation_hash is a hash of the conversation.json file contained within it.

This will result in the following directory structure:

$BASE/
    django__django-12419/
        root-$root_hash/
            conversation.json
            patch.diff

Testing new manual input (adding a new branch to the conversation)

First, create a .md file inside a conversation dir (sibling to the conversation.json file)

Then

./run_instance.sh django__django-12419 $CONVERSATION_PATH $CONTINUATION_LABEL

This will compute the directory $BASE/django__django-12419/$CONVERSATION_PATH, then load the following files from that directory:

  1. conversation.json
  2. patch.diff
  3. $CONTINUATION_LABEL.md

The environment will be initialized with the normal repo contents + the patch applied, and the model will be initialized with the contents of conversation.json. This conversation always ends with a tdd_repro tool_use. The agent then acts as if the model has responded with this tdd_repro tool_use, runs the tool, and augments the tool response with the contents of the file $CONTINUATION_LABEL.md. Execution proceeds normally until the next tdd_repro tool use, at which point the conversation and new patch will be saved to the subdir $CONTINUATION_LABEL-$conversation_hash/ (again where conversation_hash is a hash of the conversation.json file contained within that subdir).

As an example, the command line:

$ ./run_instance django__django-12419 root-$root_hash/my_first_label-$my_first_label_hash my_second_label

When run in a directory structure like this:

$BASE/
    django__django-12419/
        root-$root_hash/
            conversation.json
            my_first_label.md
            patch.diff

            my_first_label-$my_first_label_hash/
                conversation.json
                my_second_label.md
                patch.diff

will result in a directory structure like this:

$BASE/
    django__django-12419/
        root-$root_hash/
            conversation.json
            my_first_label.md
            patch.diff

            my_first_label-$my_first_label_hash/
                conversation.json
                my_second_label.md
                patch.diff

                my_second_label-$my_second_label_hash/   # this directory and contents will be created.
                    conversation.json
                    patch.diff

…ry + patch to a directory. add flags to run.py to continue a conversation with manual input
@toshok toshok force-pushed the toshok/pro-813-manual-input branch 2 times, most recently from ad61682 to 11c2881 Compare September 26, 2024 19:49
@toshok toshok changed the title [PRO-812] manual input [PRO-812] WIP manual input Sep 26, 2024
…ens within the container now, so it shows up properly in the tool result content
@toshok toshok changed the title [PRO-812] WIP manual input [PRO-812] manual input Sep 26, 2024
Copy link

@bhackett1024 bhackett1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Copy link

@Domiii Domiii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

  1. Can you update the runbook here?
  2. Looking forward to seeing a demo of this in action 🥳

config/default_with_tools.yaml Outdated Show resolved Hide resolved
sweagent/agent/agents.py Outdated Show resolved Hide resolved
sweagent/agent/agents.py Outdated Show resolved Hide resolved
…o), remove the line about the last reproduction (which the model didn't seem to have a problem with) and remove the tdd_results code since it's not used anymore
@toshok toshok merged commit 8d8906d into main Sep 30, 2024
2 of 4 checks passed
@toshok toshok deleted the toshok/pro-813-manual-input branch September 30, 2024 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants