[Feature]: Support for SageMaker-required endpoints

[nathan-az]

🚀 The feature, motivation and pitch

This was discussed before and was not supported due to AWS needing to manage the images.

I'm wondering if there is interest in at least including routing sourcecode for the required SageMaker endpoints (/invocations and /ping) to the vLLM source.

The main benefit would be the standard openai vLLM image should be automatically compatible with SageMaker endpoints. Currently, interested users have to do so through LMI, or fork vLLM and add these.

If there is interest and support from vLLM maintainers, I'm happy to contribute this to the openai entrypoints:

• a ping endpoint rerouting to /health
• an invocations endpoint that routes to the expected existing endpoint (or with an additional parameter so the user dictates the target)

My understanding is that these are the only two requirements for SageMaker support.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.