.png)
This folder contains the following examples for MPT-7B models:
| File | Description | Model used | GPU minimum requirement |
|---|---|---|---|
| 01_load_inference | Environment setup and suggested configurations when using MPT models for inference on Databricks. | MPT-7b-instruct |
1xV100-16GB |
| 02_mlflow_logging_inference | Save, register, and load MPT models with MLflow, and create a Databricks model serving endpoint. | MPT-7b-instruct |
1xV100-16GB |
| 03_serve_driver_proxy | Serve MPT models on the cluster driver node with Flask. | MPT-7b-instruct |
1xV100-16GB |
| 04_langchain | Wrap a serving endpoint or cluster driver proxy app with LangChain and query it. | N/A | N/A |
| 05_fine_tune_deepspeed | Fine-tune MPT base models with DeepSpeed. | MPT-7b |
4xA10 or 2xA100-80GB |