autogen without OPENAI - how to fork and create an openai free version?

[squatchydev9000]

I don't know where to begin... but can someone please walk me through how we could fork this thing and eliminate ALL of the openai and azure stuff...?

I HAVE BEEN STUCK WITH CONFIG_LIST NONESENSE FOR WEEKS... everything funnels through openai -> how can we trust that openai and microsoft are not using this to control our open-source attempts?

Anyone else want to form a coalition to create a langchain/autogen hybrid that doesnt go near openai and microsoft?

[rickyloynd-microsoft]

Have you checked our discord server? There's a lot of discussion there among users running non-openai LLMs with autogen.

[squatchydev9000]

got a link to a discussion that doesnt involve wrappers around the openai api... or the llms being hosted locally?

I cannot find anything related to remote hosted open-sourced llms...

[iplayfast]

Why does autogen need an OPENAI_API_KEY exported before it can do anything. This seems very monopolistic. Should be able to add the key if it's not there (and needed) or not add the key if it's not needed, ie running local llm.

[ekzhu]

Use local models and use https://docs.vllm.ai/en/latest/ which supports an OpenAI compatible API.

[squatchydev9000]

I do not want to use local... every tutorial and even autogen is assuming we all have M3 macs... its very frustrating.

Most of us have to use a remote server to get GPU usage....

[ekzhu]

When I say local models just mean models you control. Can you try running the models using vllm on a remote server and starting a service to support your application?

[squatchydev9000]

how would I integrate that service with autogen? Wouldn't it still have to 'pretend' to be an openai api? or be inside a wrapper?

[sonichi]

#831 made by @olgavrou adds custom client.
A new PR is probably going to be made for it. Are you interested in reviewing it?

[CharlyWNot]

Analysis of AutoGen, Ollama, and Open-Source AI Frameworks
Current Situation

AutoGen has introduced a dependency on OpenAI in recent updates.
This change is causing issues for users who want to use purely open-source solutions like Ollama.
The integration of OpenAI into AutoGen raises concerns about Microsoft's influence and the true "openness" of the framework.

Potential Solutions and Their Implications
A. Drop AutoGen and Create Own Agents
Pros:

Complete control over the codebase
No unexpected dependencies
Can be tailored specifically for Ollama

Cons:

Significant development effort required
Potential loss of community support and updates
May need to recreate features already present in AutoGen

B. Find an Older Version of AutoGen
Pros:

Quick solution that allows continued use of AutoGen
Avoids OpenAI dependency

Cons:

Miss out on new features and bug fixes
Potential security risks from using outdated software
May become incompatible with other dependencies over time

C. Customize AutoGen from Source
Pros:

Retain AutoGen's core functionality while removing unwanted dependencies
Potential to contribute back to the open-source community

Cons:

Requires deep understanding of AutoGen's codebase
Ongoing maintenance effort to keep up with upstream changes
May create a fork that diverges significantly from the main project

D. Seek Alternative Open-Source Frameworks
Pros:

Potentially find a framework more aligned with open-source principles
Opportunity to explore newer, more flexible solutions

Cons:

Learning curve for new framework
Uncertain long-term support and development

Long-term Considerations

Open-Source Integrity: The incident highlights the importance of truly open-source solutions that don't have hidden dependencies on proprietary services.
Vendor Lock-in: Relying on frameworks tied to specific companies (like Microsoft) can lead to unexpected changes that don't align with users' needs or values.
Community-Driven Development: Supporting and contributing to genuinely open-source projects can help ensure that the AI community remains independent and diverse.
Transparency in Dependencies: Projects should be clear about their dependencies and the implications of those dependencies for users.
Modular Design: Frameworks that allow easy swapping of components (e.g., switching between different LLM backends) provide more flexibility and resilience against unwanted changes.

Recommended Long-term Strategy

Immediate Solution: Use an older version of AutoGen or create a custom fork that removes the OpenAI dependency. This allows continued development while a long-term solution is developed.
Mid-term Plan: Investigate alternative open-source frameworks or begin development of custom agents tailored specifically for Ollama integration.
Long-term Vision: Contribute to or initiate an open-source project that provides agent frameworks similar to AutoGen but with a strong commitment to open-source principles and flexible backend support.
Community Engagement: Raise awareness about the importance of truly open AI frameworks and gather support from like-minded developers and researchers.
Continuous Evaluation: Regularly assess the AI framework landscape to ensure that chosen solutions align with open-source values and technical requirements.

By taking these steps, you can work towards a solution that not only meets your immediate needs but also contributes to a more open and flexible AI development ecosystem.