You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is other inference APIs, response_format={"type": "json_object"} restricts the model output to be a valid JSON object without enforcing a schema.
Right now this is not supported:
Failed to deserialize the JSON body into the target type: response_format: missing field `value` at line 1 column 168
I ended up with this error while using lotus-ai which uses the litellm library with response_format={ "type": "json_object" }
To reproduce:
fromhuggingface_hubimportInferenceClientc=InferenceClient("meta-llama/Llama-3.3-70B-Instruct")
c.chat_completion([{"role": "user", "content": "Give me a dummy json of a person"}], response_format={"type": "json_object"})
Is other inference APIs,
response_format={"type": "json_object"}
restricts the model output to be a valid JSON object without enforcing a schema.Right now this is not supported:
I ended up with this error while using
lotus-ai
which uses thelitellm
library withresponse_format={ "type": "json_object" }
To reproduce:
Original issue: huggingface/huggingface_hub#2744
The text was updated successfully, but these errors were encountered: