Generative AI Studio v1.1 Release Notes

OPEA Release Notes v1.1

We are pleased to announce the release of OPEA version 1.1, which includes significant contributions from the open-source community. This release addresses over 470 pull requests.

More information about how to get started with OPEA v1.1 can be found at Getting Started page. All project source code is maintained in the repository. To pull Docker images, please access the Docker Hub. For instructions on deploying Helm Charts, please refer to the guide.

What's New in OPEA v1.1

This release introduces more scenarios with general availability, including:

• Newly supported Generative AI capabilities: Image-to-Video, Text-to-Image, Text-to-SQL and Avatar Animation.
• Generative AI Studio that offers a no-code alternative to create enterprise Generative AI applications.
• Expands the portfolio of supported hardware to include Intel® Arc™ GPUs and AMD® GPUs.
• Enhanced monitoring support, providing real-time insights into runtime status and system resource utilization for CPU and Intel® Gaudi® AI Accelerator, as well as Horizontal Pod Autoscaling (HPA).
• Helm Chart support for 7 new GenAIExamples and their microservices.
• Benchmark tools for long-context language models (LCLMs) such as LongBench and HELMET.

Highlights

New GenAI Examples

• AvatarChatbot: a chatbot that combines a virtual "avatar" that can run on either Intel Gaudi 2 AI Accelerator or Intel Xeon Scalable Processors.
• DBQnA: for seamless translation of natural language queries into SQL and deliver real-time database results.
• EdgeCraftRAG: a customizable and tunable RAG example for edge solutions on Intel® Arc™ GPUs.
• GraphRAG: a Graph RAG-based approach to summarization.
• Text2Image: an application that generates images based on text prompts.
• WorkflowExecAgent: a workflow executor example to handle data/AI workflow operations via LangChain agents to execute custom-defined workflow-based tools.

Enhanced GenAI Examples

• Multi-media support: DocSum, MultimodalQnA
• Multi-language support: AudioQnA, DocSum

New GenAI Components

• Text-to-Image: add Stable Diffusion microservice
• Image-to-Video: add Stable Video Diffusion microservice
• Text-to-SQL: add Text-to-SQL microservice
• Text-to-Speech: add GPT-SoVITS microservice
• Avatar Animation: add Animation microservice
• RAG: add GraphRAG with llama-index microservice

Enhanced GenAI Components

• Asynchronous support for microservices (28672956, 9df4b3c0, f3746dc8)
• Add vLLM backends for summarization, FAQ generation, code generation, and Agents
• Multimedia support (29ef6426, baafa402)

GenAIStudio

GenAI Studio, a new project of OPEA, streamlines the creation of enterprise Generative AI applications by providing an alternative UI-based processes to create end-to-end solutions. It supports GenAI application definition, evaluation, performance benchmarking, and deployment. The GenAI Studio empowers developers to effortlessly build, test, optimize their LLM solutions, and create a deployment package. Its intuitive no-code/low-code interface accelerates innovation, enabling rapid development and deployment of cutting-edge AI applications with unparalleled efficiency and precision.

Enhanced Observability

Observability offers real-time insights into component performance and system resource utilization. We enhanced this capability by monitoring key system metrics, including CPU, host memory, storage, network, and accelerators (such as Intel Gaudi), as well as tracking OPEA application scaling.

Helm Charts Support

OPEA examples and microservices support Helm Charts as the packaging format on Kubernetes (k8s). The newly supported examples include AgentQnA, AudioQnA, FaqGen, VisualQnA. The newly supported microservices include chathistory, mongodb, prompt, and Milvus for data-prep and retriever. Helm Charts have now option to get Prometheus metrics from the applications.

Long-context Benchmark Support

We added the following two benchmark kits to response to the community's requirements of long-context language models.

• HELMET: a comprehensive benchmark for long-context language models covering seven diverse categories of tasks. The datasets are application-centric and are designed to evaluate models at different lengths and levels of complexity.
• LongBench: a benchmark tool for bilingual, multitask, and comprehensive assessment of long context understanding capabilities of large language models.

Newly Supported Models

• llama-3.2 (1B/3B/11B/90B)
• glm-4-9b-chat
• Qwen2/2.5 (7B/32B/72B)

Newly Supported Hardware

• Intel® Arc™ GPU: vLLM powered by OpenVINO can perform optimal model serving on Intel® Arc™ GPU.
• AMD® GPU: deploy GenAI examples on AMD® GPUs using AMD® ROCm™: CodeTrans, CodeGen, FaqGen, DocSum, ChatQnA.

Notable Changes

GenAIExamples

• Functionalities
  • New GenAI Examples
    • [AvatarChatbot] Initiate "AvatarChatbot" (audio) example (cfffb4c, 960805a)
    • [DBQnA] Adding DBQnA example in GenAIExamples (c0643b7, 6b9a27d)
    • [EdgeCraftRag] Add EdgeCraftRag as a GenAIExample (c9088eb, 7949045, 096a37a)
    • [GraphRAG] Add GraphRAG example a65640b
    • [Text2Image]: Add example for text2image 085d859
    • [WorkflowExecAgent] Add Workflow Executor Example bf5c391
  • Enhanced GenAI Examples
    • [AudioQnA] Add multi-language AudioQnA on Xeon 658867f
    • [AgentQnA] Update AgentQnA example for v1.1 release 5eb3d28
    • [ChatQnA] Enable vLLM Profiling for ChatQnA ([00d9bb6](https://github.com/opea-project...