Make a new API in huggingface_hub.hf_api for recommended deployment config

[weigary]

Hi,

Right now we have a http call to https://huggingface.co/api/integrations/tgi/v1/provider/gcp/recommend" to get the recommended deployment configs. We need to make a http request and parse the http response to json. It would be better if we can have a python API, for example in the huggingface_hub.hf_api for this purpose.

Is it a reasonable feature request?

[alvarobartt]

Hi here @weigary! I believe that @Wauplin may be able to respond this issue better, but I think that the recommendation API is not integrated within the huggingface_hub neither planned, but I'll let @Wauplin confirm.

[Wauplin]

Hey there, thanks for the ping :)

For better context, what call is made to this API and what does it return? If I GET https://huggingface.co/api/integrations/tgi/v1/provider/gcp/recommend, I receive a 500 internal error so I assume this is not the intended use 😄 I am not against adding id officially to huggingface_hub but 2 things I want to ensure first:

1. is the API stable or is it subject to change in the near future to improve HF <> GCP integration? (talking about inputs/outputs here). If we add an endpoint in huggingface_hub, we will have to ensure backward compatibility on the long run, making it less flexible to updates.
2. is the API only meant for the HF <> GCP integration or is it something that end users could benefit from? If it's only for this integration to work, I would be a bit more reluctant to add it given it will add maintenance cost while not providing much benefit.

[Wauplin]

Just FYI, if you want to use huggingface_hub to make authenticated http calls using the same config as the rest of the lib, you can do:

from huggingface_hub import constants
from huggingface_hub.utils import build_hf_headers,  get_session, hf_raise_for_status

url = f"{constants.ENDPOINT}/api/integrations/tgi/v1/provider/gcp/recommend"
response = get_session().get(url, headers=build_hf_headers())
hf_raise_for_status(response)
payload = response.json()

This is very similar to response = requests.get(url); response.raise_for_status(); response.json(), with a few things for better error handling + token retrieval + shared session.

[alvarobartt]

Hi here @Wauplin thanks for the prompt response and the suggestion! 🤗

For better context, what call is made to this API and what does it return?

Here's an example API call:

curl https://huggingface.co/api/integrations/tgi/v1/provider/gcp/recommend?model_id=google/gemma-7b-it&gpu_memory=80&num_gpus=2

Hope that helps, and @weigary feel free to add any other detail if needed!

[weigary]

thanks! We will try it out!