feat: Enable basic sandboxed tool run functionality #1938
Conversation
letta/agent.py
Outdated
| elif len(execution.results) == 0: | ||
| function_response = "" | ||
| else: | ||
| function_response = execution.results[0].text |
There was a problem hiding this comment.
is there any issue with typing here - e.g. if the function returns a list? does this assume the function is always returning a string?
There was a problem hiding this comment.
e2b will return lists as a string and store it in the text field! i.e.
python3
>>> from e2b_code_interpreter import Sandbox
>>> sbx = Sandbox(api_key=<API_KEY>)
>>> e = sbx.run_code("""
... def func(a):
... return [1] * a
...
... func(5)
... """)
>>> e
Execution(Results: [Result([1, 1, 1, 1, 1])], Logs: Logs(stdout: [], stderr: []), Error: None)
>>> e.results[0].text
'[1, 1, 1, 1, 1]'
It seems like the primitive types will show up here, but there are more complex types that have their own field. Should be straightforward to extend as needed in the future.
There was a problem hiding this comment.
Fixed my trailing todos and added some helpers!
I got the api key working as well locally with a really hacky set up so will clean that up next (specifically, calling client.create_tool using the exec(Tool.get_composio_tool(...).source_code) way makes the lookup for inspect.getsource fail) or just hope that @mattzh72's refactor with the add_tool magically makes things work!
letta/utils.py
Outdated
|
|
||
|
|
||
| def is_foreign_tool(tool: Tool): | ||
| return "foreign" in tool.tags |
There was a problem hiding this comment.
Just adding this function as a placeholder for now - is there a good way to programmatically determine whether a tool is unvetted right now?
In the long term, we proposed having a column in the tools table that is a signature created by letta server so that it can be generated for imported tools from trusted sources
letta/agent.py
Outdated
| @@ -5,6 +5,7 @@ | |||
| from abc import ABC, abstractmethod | |||
| from typing import List, Literal, Optional, Tuple, Union | |||
|
|
|||
| from e2b_code_interpreter import Sandbox | |||
There was a problem hiding this comment.
Do you know what file I'd need to modify if I wanted to ensure pip install e2b-code-interpreter gets run during the poetry install step?
tests/test_tools.py
Outdated
|
|
||
| # create agent with tool | ||
| agent_state = client.create_agent(tools=[tool.name]) | ||
| response = client.user_message(agent_id=agent_state.id, message="hi please use the tool called print_message") |
There was a problem hiding this comment.
Right now, on an individual test run I can manually verify that the function is running in the sandboxed environment using the e2b debug logs:
DEBUG:httpcore.connection:close.started
DEBUG:httpcore.connection:close.complete
DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG:httpx:load_verify_locations cafile='/Users/carenthomas/Library/Caches/pypoetry/virtualenvs/letta-2EtcMsTd-py3.12/lib/python3.12/site-packages/certifi/cacert.pem'
DEBUG:e2b_code_interpreter.code_interpreter_sync:Executing code def print_hello_world():
"""
Returns:
str: A static string "Hello world".
"""
print("hello world")
return "hello world"
print_hello_world()
INFO:e2b.sandbox_sync.main:Request: POST https://49999-iq6tl0803m03zmbsfyu5j-dc35dfcb.e2b.dev/execute
Not ideal because if the function doesn't run on the sandbox for whatever reason the test suite will still consider this as passed. One option is leaving the sandbox running with a timeout so that I can still interact with it during the test after user_message returns, but I'd prefer if we consistently kill the server after execution to prevent future bugs so looking into other e2b suggested options!
cpacker
changed the title
mattzh72
had a problem deploying
to
Deployment
GitHub Actions
Failure
|
Duplicated by #2040 |
Please describe the purpose of this pull request.
Basic functionality to support running user-specified tools in sandbox. Submitting v1 for feedback on approach!
How to test
How can we test your PR during review? What commands should we run? What outcomes should we expect?
Have you tested this PR?
Have you tested the latest commit on the PR? If so please provide outputs from your tests.
Related issues or PRs
Please link any related GitHub issues or PRs.
Is your PR over 500 lines of code?
If so, please break up your PR into multiple smaller PRs so that we can review them quickly, or provide justification for its length.
Additional context
Add any other context or screenshots about the PR here.