9__Model_and_Data_Versioning

Setup and Installation

• If you are using Windows, it is recommended that you either:
  • Work through Git Bash in your VS Code Terminal.
  • Install the Ubuntu 20.04 LTS as Linux Distribution. You can find instructions here: Install WSL.
• Install Anaconda or Miniconda on your laptop. Then create an environment for this project.

conda env create --file conda.yml
conda activate insurtech

To understand how the conda env createcommand works, run conda env create --help.

• Create a Weights & Biases account
• Get the API key for your Weights & Biases account here then login

wandb login <your_API_Key>

Cookie Cutter

Cookiecutter allows us to create template steps that can then be edited. It standardizes projects and allows faster development by allowing engineers focus on what is important. We have created a template step in pipeline-step-cookie. To create a new step:

$ cookiecutter pipeline-step-cookie -o src

Cleaning out mlflow environments

If there's an issue with any of the environments mlflow creates, you can get a list by executing:

conda env list | grep mlflow | cut -f 1 -d " "

You can remove all environments with names that start with mlflow as follows:

for e in $(conda env list | grep mlflow | cut -f 1 -d " "); do conda uninstall --name $e --all -y; done

Be careful with this command!

Resources

1. How we build MLflow projects and rapidly iterate over them
2. Project templates and Cookiecutter
3. Target Encoding Vs. One-hot Encoding with Simple Examples