[Feat] Exploring AgentDelegation through Supervisor Agent

[AlexCuadron]

End-user friendly description of the problem this fixes or functionality that this introduces
Exploring if agents can achieve better outputs if they have supervision. This is a draft PR.

• Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

---

Give a summary of what the PR does, explaining any non-trivial design decisions
We propose a hierarchical agent framework that introduces a "supervisor" agent responsible for overseeing the big picture of a given goal. This supervisor agent holds the overall objective and devises a plan comprising sequential steps to achieve it. Subordinate agents are assigned independent tasks, allowing them to focus on specific subtasks without being burdened by the overall complexity. The supervisor coordinates these agents, ensuring that their outputs align with the overarching goal.

Supervisor Agent

• Big Picture Oversight: Maintains the overall goal and the plan to achieve it.
• Task Delegation: Assigns independent tasks to subordinate agents.
• Output Coordination: Determines which outputs are needed from agents to progress towards the goal.
• Communication Hub: Acts as an intermediary for agents to pass outputs to the next agent.
• Guidance and Intervention:
  • Provides clarifications and additional information when agents request it.
  • Monitors task progress and can interrupt agents if they take too long.
  • Offers guidance based on its comprehensive understanding and outputs from other agents.

Subordinate Agents

• Focused Task Execution: Concentrate solely on their assigned tasks.
• Independent Operation: Perform tasks without needing to manage the overall plan.
• Output Submission: Provide results to the supervisor for coordination.
• Feedback Loop: Can request clarifications or more information from the supervisor.

Communication Flow

Agents communicate with the supervisor for:

• Task assignments.
• Submitting outputs.
• Requesting guidance or clarification.

The supervisor ensures outputs are appropriately routed to subsequent agents or integrated into the overall plan.

Asymmetric Model Scenarios

• Advanced Supervisors:

  • Utilize sophisticated models (e.g., o1-preview, Claude 3.5 Sonnet) as supervisors.
  • Advantages: Better at solving complex issues and managing tasks.
  • Consideration: Higher operational costs due to model complexity.

• Economical Subordinate Agents:

  • Employ smaller, cost-effective models (e.g., DeepSeek, o1-mini) for subtasks.
  • Advantages: Reduced costs while maintaining efficiency in task execution.

---

Link of any specific issues this addresses

[ngduyanhece]

Overall Feedback:

The introduction of the Supervisor Agent is a promising enhancement to the agent framework, providing a structured approach to task delegation and coordination. The design effectively outlines the roles of both the supervisor and subordinate agents, ensuring clarity in their interactions and responsibilities. The implementation appears to be well thought out, with a focus on maintaining the overall goal while allowing subordinate agents to operate independently.

Score: 90/100

Code Suggestions:

1. File: openhands/agenthub/supervisor_agent/prompt.py
   • Language: Python
   • Suggestion: In the adjust_milestones function, consider adding validation to ensure that the subtask dictionary contains the expected keys (task, suggested_approach, important_details) before attempting to access them. This can prevent potential KeyError exceptions if the dictionary is malformed.
   • Relevant Line: + milestone_task = subtask['task']

The Supervisor Agent framework is a well-structured addition, enhancing task delegation and coordination. Here are some suggestions:

1. In adjust_milestones, add validation to ensure the subtask dictionary contains the expected keys to prevent potential KeyError exceptions.

[mamoodi]

@AlexCuadron anything that needs to be done here to move it forward? Are you looking for reviews? Or is this still in progress? There's been a month of no activity so just want to understand the status.

[AlexCuadron]

@AlexCuadron anything that needs to be done here to move it forward? Are you looking for reviews? Or is this still in progress? There's been a month of no activity so just want to understand the status.

It is still in progress, I work closely with @xingyaoww on this one, I have many changes on local. I need to make the PR more "ready" then I can update the PR.

[enyst]

This is an exciting work! 🎉

[enyst]

Merge issue? 😅

[AlexCuadron]

Good catch & so fast! I'm impressed hahaha

[AlexCuadron]

Actually, the system should be done by ~15th Jan. Ill ping you directly when that happens if you are interested ^^

[enyst]

Definitely, thank you!