Base Docker images and parallel models

[jajimer]

Hi, everyone.
First of all congratulations for the project! I am trying (and enjoying!) the examples and now looking forward to use it in a "real-world" problem. I have several questions so let me give you some context about my work.

I am currently working on a project for analysing depression using audio data. The solution has several modules, and the workflow is roughly speaking the following:

1. Given an audio from a subject, first we use a model developed in C++ (and wrapped into a Python script) which analyses it and produce an scoring (speech scoring).
2. Then, we use an SpeechToText engine to transcript the speech into text.
3. We create some features from that text and apply a model developed in sklearn to produce another score (text scoring).
4. Finally, we combine the scorings from speech and text to produce the final score.

I would like to deploy this solution into production using bodywork, but I have several questions to make it work:

• Can I use another base Docker image? The speech model uses several C++ specific libraries, so I wonder if I could specify in the bodywork.yaml a different image or directly a custom Dockerfile. In both the docs and the examples only the bodyworkml/bodywork-core:latest is used.
• Is it possible to serve different models and create a workflow that executes them in parallel? Ideally the speech model and the Speech2Text could be executed in parallel (each on its container), and then combine their results. My current solution is serving them independently (similar to this discussion) and having another process that handles this parallelism, but it would be great if bodywork could support that.
• Is it planned to include some monitoring tasks in the DAG? An standard DAG like train >> serve is fully supported but it would be great to include some monitoring stage for measuring model's results, and maybe if current performance is below some threshold then train the model again. This question is related somehow to the "IF-THEN" workflow from here.

Actually my only deal-breaker point is the first one, about using custom Docker images, but I would like to know your vision on the other two as well. Any suggestions or advices are more than welcome!
Again congratulations for the project :)
Thank you very much!

[AlexIoannides]

Hi,

Thanks for getting in touch and taking an interest in Bodywork.

Answers to your questions, in order:

• Yes - you can use custom Docker images. You will need to build the image, upload it to Docker Hub and then configure project.docker_image within bodywork.yaml, to reference it. I recommend that you extend the current Bodywork Dockerfile.
• Yes - you can serve several different models at once. If I understand you correctly, you’d like to route a request to Model A and Model B simultaneously, and then have them forward their results to Model C, that will aggregate them into the final response? You can use Bodywork to deploy this system, but you’d also need to develop the individual components and another for handling the routing - i.e. Bodywork cannot handle this out-of-the-box. What we have planned for the future, however, is to develop a library of re-usable components that can be used to assemble pipelines, such as this one. Given this, we might be able to help you out here with some of the development, if you need it.
• You can go beyond a basic train >> serve pipeline - e.g. see the example DAG from the User Guide, or the 4-stage MLOps simulation here. For your particular example, you could develop and deploy a service that monitors your model and when your threshold is tripped, have it execute Bodywork from within the service, to trigger the pipeline. Also, we’re currently developing a REST API for Bodywork that will allow you execute pipelines via a web hook, which would make this process easier.

Many thanks,

Alex