Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support ModelMetadata endpoint #26

Merged
merged 2 commits into from
Sep 26, 2023

Conversation

rafvasq
Copy link
Member

@rafvasq rafvasq commented Aug 25, 2023

Motivation

KServe V2 API's ModelMetadata endpoint is not yet supported in the rest-proxy grpc-gateway.

Modifications

Extend grpc-gateway to include it in addition to the Infer call already implemented.

Result

Resolves #14 and kserve/modelmesh-serving#418

@rafvasq rafvasq requested review from ckadner and Jooho August 25, 2023 18:33
@rafvasq rafvasq marked this pull request as ready for review August 29, 2023 19:35
@Jooho
Copy link

Jooho commented Aug 29, 2023

simply tested this and I confirm that this work well

  1. Without version
 curl -k https://example-onnx-mnist-modelmesh-serving.apps.jlee-test.oylv.p1.openshiftapps.com/v2/models/example-onnx-mnist
{"name":"example-onnx-mnist__isvc-b29c3d91f3", "versions":["1"], "platform":"OpenVINO", "inputs":[{"name":"Input3", "datatype":"FP32", "shape":["1", "1", "28", "28"]}], "outputs":[{"name":"Plus214_Output_0", "datatype":"FP32", "shape":["1", "10"]}]}  support-rest-endpoints  ~/temp/20230822_Test_repo/rest-proxy       
  1. With version
curl -k https://example-onnx-mnist-modelmesh-serving.apps.jlee-test.oylv.p1.openshiftapps.com/v2/models/example-onnx-mnist/versions/1
{"name":"example-onnx-mnist__isvc-b29c3d91f3", "versions":["1"], "platform":"OpenVINO", "inputs":[{"name":"Input3", "datatype":"FP32", "shape":["1", "1", "28", "28"]}], "outputs":[{"name":"Plus214_Output_0", "datatype":"FP32", "shape":["1", "10"]}]}

@rafvasq rafvasq added the enhancement New feature or request label Sep 13, 2023
Copy link
Member

@ckadner ckadner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rafvasq -- need a few updates for the go.mod dependencies. I get different generated results, without these updates

go.mod Outdated Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
Signed-off-by: Rafael Vasquez <[email protected]>
@ckadner
Copy link
Member

ckadner commented Sep 26, 2023

Thanks @rafvasq -- did you also regenerate the protos?

With google.golang.org/grpc v1.51.0 and `` I do get slightly different generated code

https://github.com/kserve/rest-proxy/pull/26/files#diff-2858165aafcdd2fe79ab516a84060fb8645cd00e44a92f623f3a57fe4b3b569cR446

image

Note: "Your version" is my version using grpc v1.51.0 :-)


UPDATE:

I did not notice this earlier, but I actually had different versions of grpc-gateway when installing into the developer image:

[root@957b1461dbb7 app]# go get github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway
go: downloading github.com/grpc-ecosystem/grpc-gateway/v2 v2.18.0
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.16.0
go: downloading google.golang.org/grpc v1.58.0

Although DialContext may be what we want, it should not make a difference based on the code I see in grpc-go/clientconn.go

// Dial creates a client connection to the given target.
func Dial(target string, opts ...DialOption) (*ClientConn, error) {
	return DialContext(context.Background(), target, opts...)
}

Copy link
Member

@ckadner ckadner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@ckadner ckadner merged commit ffa9d5e into kserve:main Sep 26, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Data Plane API Error - "Not Found" when Retrieving Model Metadata Support ModelMetadata endpoint
3 participants