Discussion: New AutoGen Architecture Preview

[jackgerrits]

Please use this thread to discuss PR #3600

[elmerm]

It's great to see so many improvements on the AutoGen architecture. Congrats to you, @jackgerrits and the rest of the team on this huge milestone. 🍾🥂

Are you planning to update AutoGen for .NET as part of the 0.4 release?

[rysweet]

Yes we are, the libs in the current Autogen.* namespace will be ported over to leverage the new runtime and event driven model in the Microsoft.Autogen namespace, it will take us some time, cannot commit to dates right now, but excited to work with the community.

[Li-Yanzhi]

In real-world business applications, there are many roles involved, and the business processes are relatively fixed. We need to utilize Agents to better define and implement these processes. Currently, the round-robin and random chat sequences in Autogen do not adequately meet these business needs. We require a framework that allows flexible definition of the agent chat order.

[rysweet]

I hope that you will see that the new architecture allows you to construct many different types of process automation and agent interactions. Please try it out and give us feedback!

[afourney]

@Li-Yanzhi Absolutely. We are in the process of porting over other orchestration strategies -- including the LLM-based GroupChatManager-like strategies. The round-robin is simply the first, and it establishes the API we hope to use. In 0.4 there actually is an example of a rather sophisticated orchestrator in the magentic-one folder, but it does not yet conform to the agentchat API.

[kirk-marple]

Caveat, I haven't had a chance to read through the PR code yet, only the blog post.

I'm curious how the actor model maps to existing deployment approaches like Durable Entities or a non-Microsoft solution like Temporal or Cloudflare Durable Objects.

Is deployment based on an orchestration framework like this, or is this a new abstraction?

We currently use Durable Entities/.NET with which we've been prototyping some agent solutions that follow the actor model, so this is pretty exciting if it could be integrated for scalable agent processing.

[kostapetan]

Good question, we are introducing the concept of a runtime and our primary distributed implementation is going to be with Orleans. For local development, we are building local runtimes, but as runtimes are meant to be swappable, you can build one that leverages Durable Entities or other (even not actor based) implementation.

[kirk-marple]

Nice, that's the one big thing missing from solutions like CrewAI, that I can see. Having deployment at scale architected into the framework is super important.

Orleans makes a lot of sense, so that's good to hear as well.

[surfkansas]

Super excited for this PR! Great job team!

One question I couldn't discern from the design docs - how is the team planning on handling identity propagation? Use case is an agent listening on a topic should respect the policies in place around the human actor that initiated the request. Think of it as the Ian Malcom problem - the AI Agents were so focused on whether or not they could, they didn't stop the think if they should.

This is relatively easy in a non-distributed agentic system, but could get messy without clear guidelines about how an agent will reliably know the "who" behind the request.

And if this is in place already, can someone point me towards the code that is handling this?

[jackgerrits]

Thanks for the question! In 0.4 there are the concepts of topics and subscriptions (where an agent subscribes to a topic). Currently, when building an application the developer should be aware of the subscriptions that are in place to enforce the desired access policies. Once the PR is merged, the docs around topics and subscriptions will be easier to consume on the website too.

While not currently built, there are future directions that can make access policies more robust and built in.

• There is the concept of an intervention_handler on the in process Python runtime that allows for access policies to be implemented.
• In general the runtime based approach allows for enforced policies

We'd love community feedback around identity and permission as we work towards 0.4.

[zbram101]

Hey Team, Super excited to see this come to life! We had proposed this idea during a hackathon with Msft and really needed this change so we can make our agent pipelines more stable and extensible. Need more notebooks to help people get this setup!

[afourney]

Yes, we wanted to share things a bit early, and in preview, but notebooks, documentation, and some elements of autogen_agentchat, are all still very-much in development.

[ekzhu]

Hey @zbram101 great to see you here again. That was great feedback.

The documentation site is not live but you can build it locally following the instructions in: https://github.com/microsoft/autogen/tree/staging/python. The notebook sources are in https://github.com/microsoft/autogen/tree/staging/python/packages/autogen-core/docs/src/core-user-guide

[ScottWilliamAnderson]

Very happy to see the improvements to debugging & event-driven workflows! I haven't seen any update on the AutoGen Studio side, have they been in touch to test out the new features in the GUI? #737

[victordibia]

Hi,
We are looking to support the new architecture in Autogen studio but do not have an ETA at the moment.

[rysweet]

Contribs welcome also

[rog555]

Hi - this looks great. If we were to implement a serverless based distributed agent runtime (eg based on Cosmos DB, Pub/Sub + Azure functions), presumably this would involve a varient of WorkerAgentRuntime/WorkerAgentRuntimeHost etc? Any rough guidence appreciated

[ekzhu]

Yes, that's the idea. You can take a look at the protos directoy for the proto definition of the gPRC protocol. Though the API of distributed agents is still very much fast moving. For example, we may split the API for registering agent type and subscriptions into separate unary APIs from the message API, which is bi-directional streaming.

[rysweet]

Hi @rog555 - we are working on this - in the .NET runtime, which uses Orleans, we will be adding support to run each python agent on an Orleans grain, effectively enabling distributed clustering for the python agents as well. This is part of our goal for finishing he 0.4 release. Hopefully a draft PR for this sometime soon.

[alexbrand]

Hey @rysweet, are there additional details (e.g., a design doc) regarding the runtime atop Orleans somewhere? Would love to learn more. My use case involves building a large-scale, distributed platform to support agents running in the background, across machines, getting work done. Thanks!

[goudjr]

I just started using autogen recently and have found the latest version to be very easy to work with. My only question is will there be updates to the contribution agents like web surfer?

[ekzhu]

Yes. It is on our roadmap: #3688

[devspacenine]

How will the RAG pattern described here change with v0.4?

[ksachdeva]

Based on reading the code and seeing the examples -

The agentchat package uses some types & utils from agentcore package but does not leverage the virtual actor model. Am I missing something here?

[ekzhu]

It is using Core agents under the hood in the implementation.

It's going to support virtual actors model once we get started on exposing agent runtime through agentchat.