[bug] embedding microservice almost openai compatible but not fully

[rbrugaro]

We claim OPENAI compatible but OPEA still returns some objects as null that should be populated.
OPEA TEI: "usage":null, "model":null,
Bare TEI: "model":"BAAI/bge-large-en-v1.5","usage":{"prompt_tokens":12,"total_tokens":12}

This issue of milling objects applies to other microservices too. Might be good to systematically correct for all.

[logan-markewich]

I think run-llama/llama_index#16666 is also blocked by this. You cannot send a list of texts to an embedding endpoint

[rbrugaro]

@logan-markewich I retested run-llama/llama_index#16666 and is working with list of text input. It was fixed on PR: #892
I think the only piece missing for full openai compatibility are the usage and model values.

[lkk12014402]

@logan-markewich I retested run-llama/llama_index#16666 and is working with list of text input. It was fixed on PR: #892 I think the only piece missing for full openai compatibility are the usage and model values.

hi, we follow this doc https://platform.openai.com/docs/api-reference/embeddings to define the input/output format https://github.com/opea-project/GenAIComps/blob/main/comps/cores/proto/api_protocol.py#L83C1-L107C1. It's true that some values ​​are null because we don't need them.

[rbrugaro]

@lkk12014402 , by "we don't need them" you mean OPEA reference examples don't make use of those values at the moment? since we are integrating OPEA microservices in llama-index, and llama-index users expect openai compatible for diverse use cases and to easily swap with other cloud/prem alternatives in their deployments it would still be good to address the gaps.

[ftian1]

Rita and Logan's concern are valid, we will fix it asap. @lkk12014402 @logan-markewich @rbrugaro

[lkk12014402]

@lkk12014402 , by "we don't need them" you mean OPEA reference examples don't make use of those values at the moment? since we are integrating OPEA microservices in llama-index, and llama-index users expect openai compatible for diverse use cases and to easily swap with other cloud/prem alternatives in their deployments it would still be good to address the gaps.

hi, @rbrugaro sorry, we use the wrong tei endpoint /embed, with which we can't get the informations of model and usage. And I will fix it by using /v1/embeddings

[lkk12014402]

We claim OPENAI compatible but OPEA still returns some objects as null that should be populated. OPEA TEI: "usage":null, "model":null, Bare TEI: "model":"BAAI/bge-large-en-v1.5","usage":{"prompt_tokens":12,"total_tokens":12}

This issue of milling objects applies to other microservices too. Might be good to systematically correct for all.

hi, @rbrugaro I fix the issue with this pr #1035

[ftian1]

@logan-markewich FYI. pls be aware of this fix made by Kaokao.