Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to load model from .tar.gz/.tgz? #226

Open
Thijsvandepoll opened this issue Sep 6, 2022 · 4 comments
Open

How to load model from .tar.gz/.tgz? #226

Thijsvandepoll opened this issue Sep 6, 2022 · 4 comments

Comments

@Thijsvandepoll
Copy link

Hi all, I am new to KServe and Modelmesh Serving and try to get to know the software. I am trying to combine Kubeflow with Modelmesh Serving by using the following pipeline in Kubeflow:

  1. Preprocess data
  2. Train model
  3. Deploy model on Modelmesh

With Kubeflow the output of each of the steps is a .tgz file having the model data that Modelmesh requires. The problem is that serving the model is not working if you point directly to the .tgz file. I am guessing that Modelmesh cannot handle that format. The problem, however, is that I have a full repository of models already in s3 which are compressed in tar.gz.

It would be very useful for me if Modelmesh can handle these extensions. I am not sure if I am missing something, and that it should actually work. Does anyone know if this is possible currently?

Note: I can see in KServe's documentation (https://kserve.github.io/website/0.8/modelserving/storage/uri/uri/#train-and-freeze-the-model_1) that it would be possible to do that.

@njhill
Copy link
Member

njhill commented Sep 12, 2022

Hi @Thijsvandepoll, this is actually something we had intended to add support for, there is a related in-progress PR here.

In general the plan is to unify the storage handling between modelmesh-serving and single-model kserve. Have a common library and configuration shared by both.

@Thijsvandepoll
Copy link
Author

Ah that's great @njhill! I will be watching the PR then. Any idea on when this will be available?

@njhill
Copy link
Member

njhill commented Sep 13, 2022

@Thijsvandepoll there isn't any ETA that I'm aware of at the moment, any help would be welcome though!

@Thijsvandepoll
Copy link
Author

Yeah I would be happy to help! I need to get to know the project a bit better though as I am new to it. I will spend some time on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants