Litserve <> Haystack

[srini047]

Notebook

Serve Haystack pipelines using Litserve.

Reason

With its core reason to use being to serve AI/LLM models. I would like to showcase the capabilities of Haystack with Litserve.

Prework

Eg:

import litserve as ls
from haystack import Pipeline, PredefinedPipeline


class HaystackPredefinedLitAPI(ls.LitAPI):
    def setup(self):
        self.pipeline = Pipeline.from_template(PredefinedPipeline.GENERATIVE_QA)

    def decode_request(self, request):
        return request["query"]

    def predict(self, question):
        return self.pipeline.run({"prompt_builder": {"question": question}})

    def encode_response(self, output):
        return {"output": output}


if __name__ == "__main__":
    server = ls.LitServer(HaystackPredefinedLitAPI(), accelerator="auto")
    server.run(port=8000)

I would like to show how custom pipelines can be built and run.

[TuanaCelik]

@srini047 if you created an example/demo we would be happy to share it/showcase it

[srini047]

@srini047 if you created an example/demo we would be happy to share it/showcase it

Sure, I shall work on the code with demo.