Initial commit of sagemaker notebooks

doc/tutorials/sagemaker/sme_deploy_model.ipynb

@@ -0,0 +1,331 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "65088350-a826-4103-b57f-26377bc967b8",
+   "metadata": {},
+   "source": [
+    "# Model deployment\n",
+    "\n",
+    "In this notebook we will deploy the binary classification model created in the [previous notebook](train_model.ipynb) to a real-time AWS SageMaker endpoint. We will then use the model to make predictions on a test dataset. Please refer to the SageMaker Extension <a href=\"https://github.com/exasol/sagemaker-extension/blob/main/doc/user_guide/user_guide.md#prediction-on-aws-sagemaker-endpoint\" target=\"_blank\" rel=\"noopener\">User Guide</a> for detailed description of this process.\n",
+    "\n",
+    "<b>Important! Please make sure you perform the last step - deletion of the endpoint. Leaving the endpoint in the cloud will incur continuous charges by AWS.</b>\n",
+    "\n",
+    "We will be running SQL queries using <a href=\"https://jupysql.ploomber.io/en/latest/quick-start.html\" target=\"_blank\" rel=\"noopener\"> JupySQL</a> SQL Magic.\n",
+    "\n",
+    "## Prerequisites\n",
+    "\n",
+    "Prior to using this notebook the following steps need to be completed:\n",
+    "1. [Configure the sandbox](../sendbox_config.ipynb).\n",
+    "2. [Initialize the SageMaker Extension](sme_init.ipynb).\n",
+    "3. [Load the MAGIC Gamma Telescope data](../data/data_telescope.ipynb).\n",
+    "4. [Train the model using SageMaker Autopilot](sme_train_model.ipynb).\n",
+    "\n",
+    "## Setup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "383998b4-bf20-4dac-b1d3-e412be58cc9e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from collections import UserDict\n",
+    "\n",
+    "class Secrets(UserDict):\n",
+    "    \"\"\"This class mimics the Secret Store we will start using soon.\"\"\"\n",
+    "\n",
+    "    def save(self, key: str, value: str) -> \"Secrets\":\n",
+    "        self[key] = value\n",
+    "        return self\n",
+    "\n",
+    "def get_value_as_attribute(self, key):\n",
+    "    val = self.get(key)\n",
+    "    if val is None:\n",
+    "        raise AttributeError(f'{key} value is not defined')\n",
+    "    return val\n",
+    "\n",
+    "Secrets.__getattr__ = get_value_as_attribute\n",
+    "\n",
+    "# For now just hardcode the configuration.\n",
+    "sb_config = Secrets({    \n",
+    "    'EXTERNAL_HOST_NAME': '192.168.124.93',\n",
+    "    'HOST_PORT': '8888',\n",
+    "    'USER': 'sys',\n",
+    "    'PASSWORD': 'exasol',\n",
+    "    'BUCKETFS_PORT': '6666',\n",
+    "    'BUCKETFS_USER': 'w',\n",
+    "    'BUCKETFS_PASSWORD': 'write',\n",
+    "    'BUCKETFS_USE_HTTPS': 'False',\n",
+    "    'BUCKETFS_SERVICE': 'bfsdefault',\n",
+    "    'BUCKETFS_BUCKET': 'default',\n",
+    "    'SCRIPT_LANGUAGE_NAME': 'PYTHON3_SME',\n",
+    "    'UDF_FLAVOR': 'python3-ds-EXASOL-6.0.0',\n",
+    "    'UDF_RELEASE': '20190116',\n",
+    "    'UDF_CLIENT': 'exaudfclient_py3',\n",
+    "    'SCHEMA': 'IDA',\n",
+    "    'AWS_KEY_ID': 'AKIASNN2LAKN3EYP2Y45',\n",
+    "    'AWS_ACCESS_KEY': 'ezgUx1qb1jaPZFyL4DyNXfdnd67a1r31zuZBRkvA',\n",
+    "    'AWS_REGION': 'eu-central-1',\n",
+    "    'AWS_ROLE': 'arn:aws:iam::166283903643:role/sagemaker-role',\n",
+    "    'AWS_BUCKET': 'ida-dataset-bucket',\n",
+    "    'AWS_CONN': 'MyAWSConn',\n",
+    "    'JOB_NAME': 'CLS20231102081841'\n",
+    "})\n",
+    "\n",
+    "EXTERNAL_HOST = f\"{sb_config.EXTERNAL_HOST_NAME}:{sb_config.HOST_PORT}\"\n",
+    "WEBSOCKET_URL = f\"exa+websocket://{sb_config.USER}:{sb_config.PASSWORD}\" \\\n",
+    "    f\"@{EXTERNAL_HOST}/{sb_config.SCHEMA}?SSLCertificate=SSL_VERIFY_NONE\"\n",
+    "\n",
+    "S3_BUCKET_URI=f\"s3://{sb_config.AWS_BUCKET}\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bdce84ae-7702-4f3e-b196-a7affcc01182",
+   "metadata": {},
+   "source": [
+    "Let's bring up JupySQL and connect to the database via SQLAlchemy. Please refer to the documentation in the sqlalchemy-exasol for details on how to connect to the database using Exasol SQLAlchemy driver."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "53a337ec-4f54-4d60-bb13-cb0c1221221c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from sqlalchemy import create_engine\n",
+    "\n",
+    "engine = create_engine(WEBSOCKET_URL)\n",
+    "\n",
+    "%load_ext sql\n",
+    "%sql engine"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4399dbb0-d2f0-471a-96d6-a02cdd7744ff",
+   "metadata": {},
+   "source": [
+    "## Deploy model to a SageMaker endpoint\n",
+    "\n",
+    "The script below deploys the best candidate model of the trained Autopilot job to an endpoint with specified name. The deployment SQL command additionally generates the prediction UDF script with the same name. This UDF can be used for making predictions in an SQL statement.\n",
+    "\n",
+    "<img src=\"./sme_deployment.png\"/>\n",
+    "<center>Model deployment</center>\n",
+    "\n",
+    "Let's define some variables."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f69e00ed-2cb2-4b7c-8eb0-a1a79f95e552",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Endpoint name, also the name of the generated UDF script.\n",
+    "ENDPOINT_NAME = \"APSPredictor\"\n",
+    "\n",
+    "# The EC2 instance type of the endpoint to deploy the Autopilot model to.\n",
+    "INSTANCE_TYPE = \"ml.m5.large\"\n",
+    "\n",
+    "# The initial number of instances to run the endpoint on.\n",
+    "INSTANCE_COUNT = 1\n",
+    "\n",
+    "# Name of the table with the test data\n",
+    "TEST_TABLE_NAME = \"TELESCOPE_TEST\"\n",
+    "\n",
+    "# Name of the column in the test table which is the prediction target.\n",
+    "TARGET_COLUMN = \"CLASS\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "028a6784-26bc-4028-9009-32725c716e06",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%sql\n",
+    "EXECUTE SCRIPT {{sb_config.SCHEMA}}.\"SME_DEPLOY_SAGEMAKER_AUTOPILOT_ENDPOINT\"(\n",
+    "    '{{sb_config.JOB_NAME}}', \n",
+    "    '{{ENDPOINT_NAME}}', \n",
+    "    '{{sb_config.SCHEMA}}',\n",
+    "    '{{INSTANCE_TYPE}}',  \n",
+    "    {{INSTANCE_COUNT}}, \n",
+    "    '{{sb_config.AWS_CONN}}', \n",
+    "    '{{sb_config.AWS_REGION}}'\n",
+    ");"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "42578a73-8049-4751-9d6f-eca60579f6e1",
+   "metadata": {},
+   "source": [
+    "Let's check if the script has been created. We should be able to see an entry with the same name as the endpoint in the list of UDF scripts."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9bde0b81-eb3c-4cde-8cdc-4384d976386d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%sql\n",
+    "SELECT SCRIPT_NAME, SCRIPT_TYPE \n",
+    "FROM SYS.EXA_ALL_SCRIPTS\n",
+    "WHERE SCRIPT_SCHEMA='{{sb_config.SCHEMA}}' AND SCRIPT_TYPE = 'UDF'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "26afbc93-e6c1-45f0-84e9-3f036ca9eabb",
+   "metadata": {},
+   "source": [
+    "## Make predictions via SageMaker endpoint\n",
+    "\n",
+    "Let's use the generated UDF script for making predictions on the test data.\n",
+    "\n",
+    "First we need to get a list of features to be passed to the UDF."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b939a986-3ddb-4e0a-a465-ec5c3167ced6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%sql column_names <<\n",
+    "SELECT COLUMN_NAME\n",
+    "FROM SYS.EXA_ALL_COLUMNS\n",
+    "WHERE COLUMN_SCHEMA = '{{sb_config.SCHEMA}}' AND COLUMN_TABLE='{{TEST_TABLE_NAME}}' AND COLUMN_NAME <> UPPER('{{TARGET_COLUMN}}');"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "639757ff-ecc0-46ad-8e96-a936af5770a2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "column_names = ', '.join(f'[{name[0]}]' for name in column_names)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "966b1c84-6078-4923-ab79-4e214690261a",
+   "metadata": {},
+   "source": [
+    "Let's predict classes for the first 10 rows of the test table, just to see how the output of the UDF looks like. Remember that the first column in the input is reserved for the sample ID. Here we can just set it to zero."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "61ba6f6c-6244-46a9-b3ee-4196c2b7b9ca",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%sql\n",
+    "SELECT \"{{sb_config.SCHEMA}}\".\"{{ENDPOINT_NAME}}\"(0, {{column_names}})\n",
+    "FROM \"{{sb_config.SCHEMA}}\".\"{{TEST_TABLE_NAME}}\"\n",
+    "LIMIT 10"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8370e769-b10b-4375-9b3f-ec0d13b372f1",
+   "metadata": {},
+   "source": [
+    "Now we will compute the confusion matrix for the test data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "422ece50-3706-4bcb-a2be-1736dbacd67f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%sql\n",
+    "WITH TEST_DATA AS\n",
+    "(\n",
+    "    -- We take data from the test table and add the row number calling it SAMPLE_ID.\n",
+    "    SELECT ROW_NUMBER() OVER () AS SAMPLE_ID, {{column_names}}, [{{TARGET_COLUMN}}] FROM \"{{sb_config.SCHEMA}}\".\"{{TEST_TABLE_NAME}}\"\n",
+    ")\n",
+    "WITH MODEL_OUTPUT AS\n",
+    "(\n",
+    "    -- Make predictions. We will pass the SAMPLE_ID that sould be returned back unchanged.\n",
+    "    SELECT \"{{sb_config.SCHEMA}}\".\"{{ENDPOINT_NAME}}\"(SAMPLE_ID, {{column_names}})\n",
+    "    FROM TEST_DATA\n",
+    ")\n",
+    "-- Finally, compute the confusion matrix.\n",
+    "SELECT predictions, [{{TARGET_COLUMN}}], COUNT(*) as count\n",
+    "FROM MODEL_OUTPUT INNER JOIN TEST_DATA ON MODEL_OUTPUT.SAMPLE_ID = TEST_DATA.SAMPLE_ID\n",
+    "GROUP BY 1, 2;"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ab428be0-c90c-440c-ab78-05816998e222",
+   "metadata": {},
+   "source": [
+    "## Delete endpoint\n",
+    "\n",
+    "It is important to delete the endpoint once we finished working with it, to avoid unnecessary charges. The following script does that."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a12eebc0-f74c-4556-9be0-264d6d225abe",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%sql\n",
+    "EXECUTE SCRIPT SME_DELETE_SAGEMAKER_AUTOPILOT_ENDPOINT(\n",
+    "    '{{ENDPOINT_NAME}}', \n",
+    "    '{{sb_config.AWS_CONN}}', \n",
+    "    '{{sb_config.AWS_REGION}}'\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2cf063c3-7afa-4853-9e61-915fcfc17d21",
+   "metadata": {},
+   "source": [
+    "## Conclusion\n",
+    "\n",
+    "In this set of notebooks, we went through the steps required to train, deploy and use models based on the SageMaker Autopilot with the help of the Exasol SageMaker-Extension. The advantages the SageMaker-Extension provides include simple and fast uploading of training data into S3 buckets and getting predictions with SQL queries."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}