Skip to content

Commit

Permalink
Freshness updates to deploy-model.ipynb (#3036)
Browse files Browse the repository at this point in the history
  • Loading branch information
@msakande
msakande authored Mar 13, 2024
1 parent fb98c45 commit db660f0
Show file tree
Hide file tree
Showing 3 changed files with 37 additions and 35 deletions.
72 changes: 37 additions & 35 deletions tutorials/get-started-notebooks/deploy-model.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@
"\n",
"Learn to deploy a model to an online endpoint, using Azure Machine Learning Python SDK v2.\n",
"\n",
"In this tutorial, we use a model trained to predict the likelihood of defaulting on a credit card payment. The goal is to deploy this model and show its use.\n",
"In this tutorial, you deploy and use a model that predicts the likelihood of a customer defaulting on a credit card payment.\n",
"\n",
"The steps you'll take are:\n",
"The steps you take are:\n",
"\n",
"> * Register your model\n",
"> * Create an endpoint and a first deployment\n",
Expand All @@ -44,7 +44,7 @@
"source": [
"## Prerequisites\n",
"\n",
"1. Open in studio and select a compute instance.\n",
"1. Open in the studio and select a compute instance.\n",
" * If you opened this notebook from Azure Machine Learning studio, you need a compute instance to run the code. If you don't have a compute instance, select **Create compute** on the toolbar to first create one. You can use all the default settings. \n",
" \n",
" ![Create compute](./media/create-compute.png)\n",
Expand Down Expand Up @@ -76,12 +76,12 @@
"source": [
"## Create handle to workspace\n",
"\n",
"Before we dive in the code, you need a way to reference your workspace. You'll create `ml_client` for a handle to the workspace. You'll then use `ml_client` to manage resources and jobs.\n",
"Before you dive in the code, you need a way to reference your workspace. Create `ml_client` for a handle to the workspace and use the `ml_client` to manage resources and jobs.\n",
"\n",
"In the next cell, enter your Subscription ID, Resource Group name and Workspace name. To find these values:\n",
"In the next cell, enter your Subscription ID, Resource Group name, and Workspace name. To find these values:\n",
"\n",
"1. In the upper right Azure Machine Learning studio toolbar, select your workspace name.\n",
"1. Copy the value for workspace, resource group and subscription ID into the code.\n",
"1. Copy the value for workspace, resource group, and subscription ID into the code.\n",
"1. You'll need to copy one value, close the area and paste, then come back for the next one."
]
},
Expand Down Expand Up @@ -116,7 +116,7 @@
"metadata": {},
"source": [
"> [!NOTE]\n",
"> Creating `MLClient` will not connect to the workspace. The client initialization is lazy and will wait for the first time it needs to make a call (this will happen in the next code cell).\n"
"> Creating `MLClient` won't connect to the workspace. The client initialization is lazy and waits for the first time it needs to make a call (this happens in the next code cell).\n"
]
},
{
Expand All @@ -126,11 +126,11 @@
"source": [
"## Register the model\n",
"\n",
"If you already completed the earlier training tutorial, [Train a model](https://learn.microsoft.com/en-us/azure/machine-learning/tutorial-train-model), you've registered an MLflow model as part of the training script and can skip to the next section. \n",
"If you already completed the earlier training tutorial, [Train a model](https://learn.microsoft.com/en-us/azure/machine-learning/tutorial-train-model), you registered an MLflow model as part of the training script and can skip to the next section.\n",
"\n",
"If you didn't complete the training tutorial, you'll need to register the model. Registering your model before deployment is a recommended best practice.\n",
"If you didn't complete the training tutorial, you need to register the model. Registering your model before deployment is a recommended best practice.\n",
"\n",
"In this example, we specify the `path` (where to upload files from) inline. If you [cloned the tutorials folder](https://learn.microsoft.com/en-us/azure/machine-learning/quickstart-create-resources##learn-from-sample-notebooks), then run the following code as-is. Otherwise, [download the files and metadata for the model to deploy](https://azuremlexampledata.blob.core.windows.net/datasets/credit_defaults_model.zip). Update the path to the location on your local computer where you've unzipped the model's files. \n",
"The following code specifies the `path` (where to upload files from) inline. If you [cloned the tutorials folder](https://learn.microsoft.com/en-us/azure/machine-learning/quickstart-create-resources##learn-from-sample-notebooks), then run the following code as-is. Otherwise, [download the files and metadata for the model to deploy](https://azuremlexampledata.blob.core.windows.net/datasets/credit_defaults_model.zip) and unzip the files. Update the path to the location of the unzipped files on your local computer.\n",
"\n",
"The SDK automatically uploads the files and registers the model. \n",
"\n",
Expand Down Expand Up @@ -170,11 +170,11 @@
"source": [
"## Confirm that the model is registered\n",
"\n",
"You can check the **Models** page in [Azure Machine Learning studio](https://ml.azure.com/) to identify the latest version of your registered model.\n",
"You can check the **Models** page in [Azure Machine Learning studio](https://ml.azure.com/) to identify the latest version of the registered model.\n",
"\n",
"![View model](./media/registered-model-in-studio.png)\n",
"\n",
"Alternatively, the code below will retrieve the latest version number for you to use."
"Alternatively, the following code retrieves the latest version number for you to use."
]
},
{
Expand All @@ -197,7 +197,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that you have a registered model, you can create an endpoint and deployment. The next section will briefly cover some key details about these topics."
"Now that you have a registered model, you can create an endpoint and deployment. The next section briefly covers some key details about these topics."
]
},
{
Expand All @@ -208,7 +208,7 @@
"\n",
"After you train a machine learning model, you need to deploy it so that others can use it for inferencing. For this purpose, Azure Machine Learning allows you to create **endpoints** and add **deployments** to them.\n",
"\n",
"An **endpoint**, in this context, is an HTTPS path that provides an interface for clients to send requests (input data) to a trained model and receive the inferencing (scoring) results back from the model. An endpoint provides:\n",
"An **endpoint**, in this context, is an HTTPS path that provides an interface for clients to send requests (input data) to a trained model and receive the inferencing (scoring) results from the model. An endpoint provides:\n",
"\n",
"- Authentication using \"key or token\" based auth \n",
"- [TLS(SSL)](https://simple.wikipedia.org/wiki/Transport_Layer_Security) termination\n",
Expand All @@ -221,7 +221,7 @@
"\n",
"Azure Machine Learning allows you to implement [online endpoints](https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints#what-are-online-endpoints) for real-time inferencing on client data, and [batch endpoints](https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints#what-are-batch-endpoints) for inferencing on large volumes of data over a period of time. \n",
"\n",
"In this tutorial, we'll walk you through the steps of implementing a _managed online endpoint_. Managed online endpoints work with powerful CPU and GPU machines in Azure in a scalable, fully managed way that frees you from the overhead of setting up and managing the underlying deployment infrastructure."
"In this tutorial, you go through the steps of implementing a _managed online endpoint_. Managed online endpoints work with powerful CPU and GPU machines in Azure in a scalable, fully managed way that frees you from the overhead of setting up and managing the underlying deployment infrastructure."
]
},
{
Expand All @@ -236,7 +236,7 @@
"source": [
"## Create an online endpoint\n",
"\n",
"Now that you have a registered model, it's time to create your online endpoint. The endpoint name needs to be unique in the entire Azure region. For this tutorial, you'll create a unique name using a universally unique identifier [`UUID`](https://en.wikipedia.org/wiki/Universally_unique_identifier). For more information on the endpoint naming rules, see [managed online endpoint limits](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-quotas#azure-machine-learning-managed-online-endpoints)."
"Now that you have a registered model, it's time to create your online endpoint. The endpoint name needs to be unique in the entire Azure region. For this tutorial, you create a unique name using a universally unique identifier [`UUID`](https://en.wikipedia.org/wiki/Universally_unique_identifier). For more information on the endpoint naming rules, see [managed online endpoint limits](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-quotas#azure-machine-learning-managed-online-endpoints)."
]
},
{
Expand Down Expand Up @@ -265,12 +265,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"First, we'll define the endpoint, using the `ManagedOnlineEndpoint` class.\n",
"First, define the endpoint, using the `ManagedOnlineEndpoint` class.\n",
"\n",
"\n",
"\n",
"> [!TIP]\n",
"> * `auth_mode` : Use `key` for key-based authentication. Use `aml_token` for Azure Machine Learning token-based authentication. A `key` doesn't expire, but `aml_token` does expire. For more information on authenticating, see [Authenticate to an online endpoint](https://learn.microsoft.com/azure/machine-learning/how-to-authenticate-online-endpoint).\n",
"> * `auth_mode` : Use `key` for key-based authentication. Use `aml_token` for Azure Machine Learning token-based authentication. A `key` doesn't expire, but `aml_token` does expire. For more information on authenticating, see [Authenticate clients for online endpoints](https://learn.microsoft.com/azure/machine-learning/how-to-authenticate-online-endpoint).\n",
"> * Optionally, you can add a description and tags to your endpoint."
]
},
Expand Down Expand Up @@ -307,7 +307,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the `MLClient` created earlier, we'll now create the endpoint in the workspace. This command will start the endpoint creation and return a confirmation response while the endpoint creation continues.\n",
"Using the `MLClient` created earlier, we'll now create the endpoint in the workspace. This command starts the endpoint creation and returns a confirmation response while the endpoint creation continues.\n",
"\n",
"> [!NOTE]\n",
"> Expect the endpoint creation to take approximately 2 minutes."
Expand Down Expand Up @@ -345,7 +345,7 @@
}
},
"source": [
"Once you've created the endpoint, you can retrieve it as follows:"
"Once you create the endpoint, you can retrieve it as follows:"
]
},
{
Expand Down Expand Up @@ -395,7 +395,7 @@
"Azure Machine Learning supports no-code deployment of a model created and logged with MLflow. This means that you don't have to provide a scoring script or an environment during model deployment, as the scoring script and environment are automatically generated when training an MLflow model. If you were using a custom model, though, you'd have to specify the environment and scoring script during deployment.\n",
"\n",
"> [!IMPORTANT]\n",
"> If you typically deploy models using scoring scripts and custom environments and want to achieve the same functionality using MLflow models, we recommend reading [Using MLflow models for no-code deployment](https://learn.microsoft.com/azure/machine-learning/how-to-deploy-mlflow-models)."
"> If you typically deploy models using scoring scripts and custom environments and want to achieve the same functionality using MLflow models, we recommend reading [Guidelines for deploying MLflow models](https://learn.microsoft.com/azure/machine-learning/how-to-deploy-mlflow-models)."
]
},
{
Expand All @@ -410,7 +410,7 @@
"source": [
"## Deploy the model to the endpoint\n",
"\n",
"You'll begin by creating a single deployment that handles 100% of the incoming traffic. We've chosen an arbitrary color name (*blue*) for the deployment. To create the deployment for our endpoint, we'll use the `ManagedOnlineDeployment` class.\n",
"Begin by creating a single deployment that handles 100% of the incoming traffic. Choose an arbitrary color name (*blue*) for the deployment. To create the deployment for the endpoint, use the `ManagedOnlineDeployment` class.\n",
"\n",
"> [!NOTE]\n",
"> No need to specify an environment or scoring script as the model to deploy is an MLflow model."
Expand All @@ -434,7 +434,7 @@
"source": [
"from azure.ai.ml.entities import ManagedOnlineDeployment\n",
"\n",
"# Choose the latest version of our registered model for deployment\n",
"# Choose the latest version of the registered model for deployment\n",
"model = ml_client.models.get(name=registered_model_name, version=latest_model_version)\n",
"\n",
"# define an online deployment\n",
Expand All @@ -453,7 +453,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the `MLClient` created earlier, we'll now create the deployment in the workspace. This command will start the deployment creation and return a confirmation response while the deployment creation continues."
"Using the `MLClient` created earlier, now create the deployment in the workspace. This command starts the deployment creation and returns a confirmation response while the deployment creation continues."
]
},
{
Expand Down Expand Up @@ -525,7 +525,7 @@
"source": [
"## Test the endpoint with sample data\n",
"\n",
"Now that the model is deployed to the endpoint, you can run inference with it. Let's create a sample request file following the design expected in the run method in the scoring script."
"Now that the model is deployed to the endpoint, you can run inference with it. Begin by creating a sample request file that follows the design expected in the run method found in the scoring script."
]
},
{
Expand All @@ -545,7 +545,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, create the file in the deploy directory. The cell below uses IPython magic to write the file into the directory you just created."
"Now, create the file in the deploy directory. The following code cell uses IPython magic to write the file into the directory you just created."
]
},
{
Expand All @@ -571,13 +571,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the `MLClient` created earlier, we'll get a handle to the endpoint. The endpoint can be invoked using the `invoke` command with the following parameters:\n",
"Using the `MLClient` created earlier, we'll get a handle to the endpoint. You can invoke the endpoint by using the `invoke` command with the following parameters:\n",
"\n",
"* `endpoint_name` - Name of the endpoint\n",
"* `request_file` - File with request data\n",
"* `deployment_name` - Name of the specific deployment to test in an endpoint\n",
"\n",
"We'll test the blue deployment with the sample data."
"Test the blue deployment with the sample data."
]
},
{
Expand All @@ -600,7 +600,7 @@
"metadata": {},
"source": [
"## Get logs of the deployment\n",
"Check the logs to see whether the endpoint/deployment were invoked successfully\n",
"Check the logs to see whether the endpoint/deployment were invoked successfully.\n",
"If you face errors, see [Troubleshooting online endpoints deployment](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-troubleshoot-online-endpoints?tabs=cli)."
]
},
Expand All @@ -621,7 +621,9 @@
"metadata": {},
"source": [
"## Create a second deployment \n",
"Deploy the model as a second deployment called `green`. In practice, you can create several deployments and compare their performance. These deployments could use a different version of the same model, a completely different model, or a more powerful compute instance. In our example, you'll deploy the same model version using a more powerful compute instance that could potentially improve performance."
"Deploy the model as a second deployment called `green`. In practice, you can create several deployments and compare their performance. These deployments could use a different version of the same model, a completely different model, or a more powerful compute instance.\n",
"\n",
"In this example, you deploy the same model version, using a more powerful compute instance that could potentially improve performance."
]
},
{
Expand All @@ -630,7 +632,7 @@
"metadata": {},
"outputs": [],
"source": [
"# picking the model to deploy. Here we use the latest version of our registered model\n",
"# pick the model to deploy. Here you use the latest version of the registered model\n",
"model = ml_client.models.get(name=registered_model_name, version=latest_model_version)\n",
"\n",
"# define an online deployment using a more powerful instance type\n",
Expand All @@ -657,9 +659,9 @@
"source": [
"## Scale deployment to handle more traffic\n",
"\n",
"Using the `MLClient` created earlier, we'll get a handle to the `green` deployment. The deployment can be scaled by increasing or decreasing the `instance_count`.\n",
"Using the `MLClient` created earlier, you can get a handle to the `green` deployment. You can then scale it by increasing or decreasing the `instance_count`.\n",
"\n",
"In the following code, you'll increase the VM instance manually. However, note that it is also possible to autoscale online endpoints. Autoscale automatically runs the right amount of resources to handle the load on your application. Managed online endpoints support autoscaling through integration with the Azure monitor autoscale feature. To configure autoscaling, see [autoscale online endpoints](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-autoscale-endpoints?tabs=python)."
"In the following code, you increase the VM instance manually. However, it's also possible to autoscale online endpoints. Autoscale automatically runs the right amount of resources to handle the load on your application. Managed online endpoints support autoscaling through integration with the Azure monitor autoscale feature. To configure autoscaling, see [Autoscale online endpoints](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-autoscale-endpoints?tabs=python)."
]
},
{
Expand Down Expand Up @@ -698,7 +700,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"You can test traffic allocation by invoking the endpoint several times:"
"Test traffic allocation by invoking the endpoint several times:"
]
},
{
Expand Down Expand Up @@ -739,7 +741,7 @@
"metadata": {},
"source": [
"## View metrics using Azure Monitor\n",
"You can view various metrics (request numbers, request latency, network bytes, CPU/GPU/Disk/Memory utilization, and more) for an online endpoint and its deployments by following links from the endpoint's **Details** page in the studio. Following these links will take you to the exact metrics page in the Azure portal for the endpoint or deployment.\n",
"You can view various metrics (request numbers, request latency, network bytes, CPU/GPU/Disk/Memory utilization, and more) for an online endpoint and its deployments by following links from the endpoint's **Details** page in the studio. Following any of these links takes you to the exact metrics page in the Azure portal for the endpoint or deployment.\n",
"\n",
"![metrics page 1](./media/deployment-metrics-from-endpoint-details-page.png)\n",
"\n",
Expand Down
Binary file modified ...s/get-started-notebooks/media/deployment-metrics-from-endpoint-details-page.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tutorials/get-started-notebooks/media/registered-model-in-studio.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit db660f0

Please sign in to comment.