4.1.multiple_linear_regression_prediction_v2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DL0110EN-SkillsNetwork/Template/module%201/images/IDSNlogo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\"  />\n",
    "</center>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Multiple Linear Regression</h1>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Objective</h2><ul><li> How to make the prediction for multiple inputs.</li><li> How to use linear class to build more complex models.</li><li> How to build a custom module.</li></ul> \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Table of Contents</h2>\n",
    "<p>In this lab, you will review how to make a prediction in several different ways by using PyTorch.</p>\n",
    "\n",
    "<ul>\n",
    "    <li><a href=\"#Prediction\">Prediction</a></li>\n",
    "    <li><a href=\"#Linear\">Class Linear</a></li>\n",
    "    <li><a href=\"#Cust\">Build Custom Modules</a></li>\n",
    "</ul>\n",
    "\n",
    "<p>Estimated Time Needed: <strong>15 min</strong></p>\n",
    "\n",
    "<hr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Preparation</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the libraries and set the random seed.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x7f2c3151e8d0>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import the libraries and set the random seed\n",
    "\n",
    "from torch import nn\n",
    "import torch\n",
    "torch.manual_seed(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Prediction\">Prediction</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set weight and bias.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the weight and bias\n",
    "\n",
    "w = torch.tensor([[2.0], [3.0]], requires_grad=True)\n",
    "b = torch.tensor([[1.0]], requires_grad=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the parameters. <code>torch.mm</code> uses matrix multiplication instead of scaler multiplication.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define Prediction Function\n",
    "\n",
    "def forward(x):\n",
    "    yhat = torch.mm(x, w) + b\n",
    "    return yhat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function <code>forward</code> implements the following equation:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter2/2.6.1_matrix_eq.png\" width=\"600\" alt=\"Matrix Linear Regression\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we input a <i>1x2</i> tensor, because we have a <i>2x1</i> tensor as <code>w</code>, we will get a <i>1x1</i> tensor: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The result:  tensor([[9.]], grad_fn=<AddBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# Calculate yhat\n",
    "\n",
    "x = torch.tensor([[1.0, 2.0]])\n",
    "yhat = forward(x)\n",
    "print(\"The result: \", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src = \"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter2/2.6.1example.png\" width = \"300\" alt=\"Linear Regression Matrix Sample One\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Each row of the following tensor represents a sample:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sample tensor X\n",
    "\n",
    "X = torch.tensor([[1.0, 1.0], [1.0, 2.0], [1.0, 3.0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The result:  tensor([[ 6.],\n",
      "        [ 9.],\n",
      "        [12.]], grad_fn=<AddBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# Make the prediction of X \n",
    "\n",
    "yhat = forward(X)\n",
    "print(\"The result: \", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Linear\">Class Linear</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the linear class to make a prediction. You'll also use the linear class to build more complex models.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us create a model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make a linear regression model using build-in function\n",
    "\n",
    "model = nn.Linear(2, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a prediction with the first sample:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The result:  tensor([[-0.3969]], grad_fn=<AddmmBackward>)\n"
     ]
    }
   ],
   "source": [
    "# Make a prediction of x\n",
    "\n",
    "yhat = model(x)\n",
    "print(\"The result: \", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Predict with multiple samples <code>X</code>: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The result:  tensor([[-0.0848],\n",
      "        [-0.3969],\n",
      "        [-0.7090]], grad_fn=<AddmmBackward>)\n"
     ]
    }
   ],
   "source": [
    "# Make a prediction of X\n",
    "\n",
    "yhat = model(X)\n",
    "print(\"The result: \", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function performs matrix multiplication as shown in this image:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src = \"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter2/2.6.1multi_sample_example.png\" width = \"600\" alt=\"Linear Regression Matrix Sample One\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Cust\">Build Custom Modules </h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, you'll build a custom module. You can make more complex models by using this method later. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create linear_regression Class\n",
    "\n",
    "class linear_regression(nn.Module):\n",
    "    \n",
    "    # Constructor\n",
    "    def __init__(self, input_size, output_size):\n",
    "        super(linear_regression, self).__init__()\n",
    "        self.linear = nn.Linear(input_size, output_size)\n",
    "    \n",
    "    # Prediction function\n",
    "    def forward(self, x):\n",
    "        yhat = self.linear(x)\n",
    "        return yhat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Build a linear regression object. The input feature size is two. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = linear_regression(2, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This will input the following equation:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter2/2.6.1_matrix_eq.png\" width=\"600\" alt=\"Matrix Linear Regression\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the randomly initialized parameters by using the <code>parameters()</code> method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The parameters:  [Parameter containing:\n",
      "tensor([[ 0.3319, -0.6657]], requires_grad=True), Parameter containing:\n",
      "tensor([0.4241], requires_grad=True)]\n"
     ]
    }
   ],
   "source": [
    "# Print model parameters\n",
    "\n",
    "print(\"The parameters: \", list(model.parameters()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also see the parameters by using the <code>state_dict()</code> method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The parameters:  OrderedDict([('linear.weight', tensor([[ 0.3319, -0.6657]])), ('linear.bias', tensor([0.4241]))])\n"
     ]
    }
   ],
   "source": [
    "# Print model parameters\n",
    "\n",
    "print(\"The parameters: \", model.state_dict())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we input a 1x2 tensor, and we will get a 1x1 tensor.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make a prediction of x\n",
    "\n",
    "yhat = model(x)\n",
    "print(\"The result: \", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The shape of the output is shown in the following image: \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter2/2.6.1_matrix_eq.png\" width=\"600\" alt=\"Matrix Linear Regression\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a prediction for multiple samples:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The result:  tensor([[ 0.0903],\n",
      "        [-0.5754],\n",
      "        [-1.2411]], grad_fn=<AddmmBackward>)\n"
     ]
    }
   ],
   "source": [
    "# Make a prediction of X\n",
    "\n",
    "yhat = model(X)\n",
    "print(\"The result: \", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The shape is shown in the following image: \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter2/2.6.1Multi_sample.png\" width=\"600\" alt=\"Multiple Samples Linear Regression\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Practice</h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Build a model or object of type <code>linear_regression</code>. Using the <code>linear_regression</code> object will predict the following tensor: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-4.4789],\n",
      "        [ 6.1731]], grad_fn=<AddmmBackward>)\n"
     ]
    }
   ],
   "source": [
    "# Practice: Build a model to predict the follow tensor.\n",
    "\n",
    "X = torch.tensor([[11.0, 112.0, 13, 14], [11, 12, 13, 4]])\n",
    "\n",
    "from torch import nn \n",
    "\n",
    "class MyModel(nn.Module):\n",
    "    def __init__(self, in_dim, out_dim): \n",
    "        super(MyModel, self).__init__() \n",
    "        self.linear = nn.Linear(in_dim, out_dim) \n",
    "        \n",
    "    def forward(self, X):\n",
    "        out = self.linear(X)\n",
    "        return out \n",
    "\n",
    "model = MyModel(4,1)\n",
    "yhat = model(X)\n",
    "print(yhat)\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- Your answer is below:\n",
    "model = linear_regression(4, 1)\n",
    "yhat = model(X)\n",
    "print(\"The result: \", yhat)\n",
    "-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://dataplatform.cloud.ibm.com/registration/stepone?context=cpdaas&apps=data_science_experience,watson_machine_learning\"><img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DL0110EN-SkillsNetwork/Template/module%201/images/Watson_Studio.png\"/></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>About the Authors:</h2> \n",
    "\n",
    "<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other contributors: <a href=\"https://www.linkedin.com/in/michelleccarey/\">Michelle Carey</a>, <a href=\"www.linkedin.com/in/jiahui-mavis-zhou-a4537814a\">Mavis Zhou</a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Change Log\n",
    "\n",
    "| Date (YYYY-MM-DD) | Version | Changed By | Change Description                                          |\n",
    "| ----------------- | ------- | ---------- | ----------------------------------------------------------- |\n",
    "| 2020-09-23        | 2.0     | Shubham    | Migrated Lab to Markdown and added to course repo in GitLab |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}