API Request Parameters and Reusing Vector Store Without Regenerating Embeddings

[abrahimzaman360]

1. What parameter to send as GET/POST on API?
2. How to use Vector Store without creating a new one, every time. because it always run generate embeddings even though vector store already there?

[abrahimzaman360]

For Example:
I want to get response on the query: "hello assistant", which method should I use.

[SeeknnDestroy]

Hi @abrahimzaman360,

Thank you for your questions. Here's quick guide:

1. API Usage: To make a query like "hello assistant", use a POST request to the /query endpoint with a JSON payload. For detailed parameters and usage, visit the /docs page. Start the application with Uvicorn, and the /docs page should be available at the root URL of your server (e.g., http://your-server-address:port/docs).

2. Vector Store Usage: To use an existing vector store without creating a new one, initialize AutoQueryEngine without passing the 'documents' parameter. This approach uses the existing store, bypassing new embedding generation:

   from autollm import AutoQueryEngine
   
   # Initialize query engine without 'documents'
   query_engine = AutoQueryEngine.from_defaults()
   
   # Example query
   response = query_engine.query("Your query here")
   print(response.response)

Hope this helps! Feel free to reach out if you have further questions.

Best,
Talha

[abrahimzaman360]

Thanks

[abrahimzaman360]

One more thing:
What is llm_apibase? where can I find this? for example, there are many llms on hugging faces.
I used to use transformers pipelines for TG tasks.

[fcakyon]

If your model is deployed on HF, it should be in the format of: https://my-endpoint.huggingface.cloud

If local, then you should provide you local TGI deployment url.