From 52877471f1d00154ac80f8cc64012398811cd737 Mon Sep 17 00:00:00 2001 From: Gad Markovits Date: Tue, 22 Oct 2024 10:54:57 +0300 Subject: [PATCH 1/2] Haystack integration rfc Signed-off-by: Gad Markovits --- .../24-10-20-OPEA-001-Haystack-Integration.md | 59 +++++++++++++++++++ 1 file changed, 59 insertions(+) create mode 100644 community/rfcs/24-10-20-OPEA-001-Haystack-Integration.md diff --git a/community/rfcs/24-10-20-OPEA-001-Haystack-Integration.md b/community/rfcs/24-10-20-OPEA-001-Haystack-Integration.md new file mode 100644 index 00000000..b68aae83 --- /dev/null +++ b/community/rfcs/24-10-20-OPEA-001-Haystack-Integration.md @@ -0,0 +1,59 @@ +# 24-10-20-OPEA-001-Haystack-Integration + +## Author + +[gadmarkovits](https://github.com/gadmarkovits) + +## Status + +Under Review + +## Objective + +Create a Haystack integration for OPEA that will enable the use of OPEA components within a Haystack pipeline. + +## Motivation + +Haystack is a production-ready open source AI framework that is used by many AI practitioners. It has over 70 integrations with various GenAI components such as document stores, model providers and evaluation frameworks from companies such as Amazon, Microsoft, Nvidia and more. Creating an integration for OPEA will allow Haystack customers to use OPEA components in their pipelines. This RFC is used to present a high-level overview of the Haystack integration. + +## Design Proposal + +The idea is to create thin wrappers for OPEA components that will enable communicating with them using the existing REST API. The wrappers will match Haystack's API so that they could be used within Haystack pipelines. This will allow developers to seamlessly use OPEA components alongside other Haystack components. + +The integration will be implemented as a Python package (similar to other Haystack integrations). The source code will be hosted in OPEA's GenAIComps repo under a new directory called Integrations. The package itself will be uploaded to [PyPi](https://pypi.org/) to allow for easy installation. + +Following a discussion with Haystack's technical team, it was agreed that a ChatQnA example, using this OPEA integration, would be a good way to showcase its capabilities. To support this, several component wrappers need to be implemented in the first version of the integration (other wrappers will be added gradually): + +1. OPEA Document Embedder + + This component will receive a Haystack Document and embed it using an OPEA embedding microservice. + +2. OPEA Text Embedder + + This component will receive text input and embed it using an OPEA embedding microservice. + +3. OPEA Generator + + This component will receive a text prompt and generate a reponse using an OPEA LLM microservice. + +4. OPEA Retriever + + This component will receive an embedding and retrieve documents with similar emebddings using an OPEA retrieval microservice. + +4. GenAIEval + + The evaluation, benchmark, and scorecard suite for OPEA, targeting for performance on throughput and latency, accuracy on popular evaluation harness, safety, and hallucination. + +## Alternatives Considered + +n/a + +## Compatibility + +n/a + +## Miscs + +Once implemented, the Haystack team list the OPEA integration on their [integrations page](https://haystack.deepset.ai/integrations) which will allow for easier discovery. Haystack, in collaboration with Intel, will also publish a technical blog post showcasing a ChatQnA example using this integration (similar to this [NVidia NIM post](https://haystack.deepset.ai/blog/haystack-nvidia-nim-rag-guide)). + + From 826a0adc8ce1431ced952e4ca3017ddc3b67fd69 Mon Sep 17 00:00:00 2001 From: Gad Markovits Date: Sun, 27 Oct 2024 09:02:12 +0200 Subject: [PATCH 2/2] Removed extraneous item from components list Signed-off-by: Gad Markovits --- community/rfcs/24-10-20-OPEA-001-Haystack-Integration.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/community/rfcs/24-10-20-OPEA-001-Haystack-Integration.md b/community/rfcs/24-10-20-OPEA-001-Haystack-Integration.md index b68aae83..18b0a829 100644 --- a/community/rfcs/24-10-20-OPEA-001-Haystack-Integration.md +++ b/community/rfcs/24-10-20-OPEA-001-Haystack-Integration.md @@ -40,10 +40,6 @@ Following a discussion with Haystack's technical team, it was agreed that a Chat This component will receive an embedding and retrieve documents with similar emebddings using an OPEA retrieval microservice. -4. GenAIEval - - The evaluation, benchmark, and scorecard suite for OPEA, targeting for performance on throughput and latency, accuracy on popular evaluation harness, safety, and hallucination. - ## Alternatives Considered n/a