5.1logistic_regression_prediction_v2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DL0110EN-SkillsNetwork/Template/module%201/images/IDSNlogo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\"  />\n",
    "</center>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Logistic Regression</h1>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Objective</h2><ul><li> How to create a logistic regression object with the nn.Sequential model.</li></ul> \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Table of Contents</h2>\n",
    "<p>In this lab, we will cover logistic regression using PyTorch.</p>\n",
    "\n",
    "<ul>\n",
    "    <li><a href=\"#Log\">Logistic Function</a></li>\n",
    "    <li><a href=\"#Seq\">Build a Logistic Regression Using nn.Sequential</a></li>\n",
    "    <li><a href=\"#Model\">Build Custom Modules</a></li>\n",
    "</ul>\n",
    "<p>Estimated Time Needed: <strong>15 min</strong></p>\n",
    "\n",
    "<hr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Preparation</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll need the following libraries:  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the libraries we need for this lab\n",
    "\n",
    "import torch.nn as nn\n",
    "import torch\n",
    "import matplotlib.pyplot as plt "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the random seed:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x7f42640a7c30>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Set the random seed\n",
    "\n",
    "torch.manual_seed(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Log\">Logistic Function</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a tensor ranging from -100 to 100:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The tensor:  tensor([[-100.0000],\n",
      "        [ -99.9000],\n",
      "        [ -99.8000],\n",
      "        ...,\n",
      "        [  99.7000],\n",
      "        [  99.8000],\n",
      "        [  99.9000]])\n"
     ]
    }
   ],
   "source": [
    "z = torch.arange(-100, 100, 0.1).view(-1, 1)\n",
    "print(\"The tensor: \", z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a sigmoid object: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create sigmoid object\n",
    "\n",
    "sig = nn.Sigmoid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apply the element-wise function Sigmoid with the object:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use sigmoid object to calculate the \n",
    "\n",
    "yhat = sig(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the results:\n",
    "Notice here that the derivative of this sigmoid function is nearly zero for very large positive and negative values. \n",
    "This is called gradient explosion. It is the case where the sigmoid function is satiated. \n",
    "So you want your logits (Z) values to be around 0 for sigmoid activation to work properly (pass along a strong signal.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'yhat')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAYtUlEQVR4nO3dfXAc933f8feHAAk+i5RJWRQfRFqm5NBT25JQxUnjh4zjiFQSsXaaVkomst1kOGqt1J5OG6t166T1tDO260zrWDHLOBzbmTTytHIaWqWjNLEjTSZxLcqhKVI0JYiSTYqUCD2RFIEDcMC3f9yCPoIHEMDt7w4/3uc1g8Hdbxd3X+7t8XPf3dtdRQRmZta55rW7ADMzay8HgZlZh3MQmJl1OAeBmVmHcxCYmXW47nYXMFOrVq2KjRs3trsMM7OsPPbYYy9GxOpG07ILgo0bN7Jv3752l2FmlhVJP5hsmjcNmZl1OAeBmVmHcxCYmXU4B4GZWYdzEJiZdbhkQSBpt6RTkg5OMl2SPiepT9IBSTelqsXMzCaXsiP4ErB1iunbgM3Fzw7gCwlrMTOzSSQ7jiAiHpG0cYpZtgNfidp5sL8taYWkNRFxMlVNZu1WHR3j+8+f5ZkXz3F6cISB4SrD1THGAiIgiOI3tQGzOr0br+Sd1zc8Jqwp7TygbC1wrO7+8WLsoiCQtINa18CGDRtaUpxZmaqjY/z3R46y+6+f4aVzw9P+OylhUZadu9913WUXBI1W8YYfgSJiF7ALoLe31x+TLCtjY8FHv7qfBw+c5D1vuorb33YNN1y9jCsXL2BxTzcLuuYxTyAJUfvPX04Aa6F2BsFxYH3d/XXAiTbVYpbMV/cd48EDJ/nNrTfwz9/9xnaXY3aRdn59dA9wV/HtobcDp71/wC431dExfvcvn6L32pX8s3dd1+5yzBpK1hFI+mPg3cAqSceB3wLmA0TETmAvcBvQBwwAH0pVi1m7PPxkPydOV/it29/szT02Z6X81tCdl5gewIdTPb/ZXPAXh0+xtKebn77hqnaXYjYpH1lsltAjT/bzU29cxYJuv9Vs7vLaaZbIqbMVnnt1kN6NK9tditmUHARmiTx+/DQAb12/or2FmF2Cg8AskYPPnUGCLWuWt7sUsyk5CMwSeebF17jmikUs6cnuirDWYRwEZok8+9IAG1ctbncZZpfkIDBL5NmXzrHxdUvaXYbZJTkIzBI4PTDCqwMjDgLLgoPALIETpwcBuGbFojZXYnZpDgKzBPrPDgFw1fKeNldidmkOArMETo0HwTIHgc19DgKzBE6drQCw2kFgGXAQmCVw6swQS3u6WbzAxxDY3OcgMEug/7UhbxaybDgIzBLoPzPEKgeBZcJBYJaAOwLLiYPALIFXBoa5csmCdpdhNi0OArOSjY0FZwZHWL5wfrtLMZsWB4FZyV4brjIWcMUiB4HlwUFgVrLTAyOAg8Dy4SAwK9mZSi0IljsILBMOArOSnR50R2B5cRCYlezM4HhH4KOKLQ8OArOSuSOw3DgIzEp2ZrAKOAgsHw4Cs5KdHhxhnmCpL1pvmXAQmJXs9OAIyxfNR1K7SzGbFgeBWcnOVHxUseXFQWBWsnNDo94sZFlxEJiVbGC4ypKernaXYTZtDgKzkp0bHvWVySwrSYNA0lZJRyT1Sbq3wfQrJH1d0vckHZL0oZT1mLXCwJA7AstLsiCQ1AXcB2wDtgB3StoyYbYPA09ExFuBdwOfleSTuFvWBtwRWGZSdgS3AH0RcTQihoH7ge0T5glgmWrfs1sKvAxUE9Zklty54SpLFrgjsHykDIK1wLG6+8eLsXqfB34MOAE8DnwkIsYmPpCkHZL2SdrX39+fql6zUgwMjbLY3xqyjKQMgkZH08SE+7cC+4FrgLcBn5e0/KI/itgVEb0R0bt69eqy6zQrzXB1jOHRMXcElpWUQXAcWF93fx21T/71PgR8LWr6gGeANyWsySypweFRAO8jsKykDIJHgc2SNhU7gO8A9kyY54fAewAkvR64ATiasCazpM4N13Zx+VtDlpNkH1sioirpHuAhoAvYHRGHJN1dTN8JfBL4kqTHqW1K+lhEvJiqJrPUBoogcEdgOUm6tkbEXmDvhLGddbdPAD+bsgazVjo3VNs05I7AcuIji81KdM4dgWXIQWBWooHxjsBBYBlxEJiV6HxH4E1DlhEHgVmJBs5/fdRBYPlwEJiVaPw4gkXzHQSWDweBWYmGqrUzpPR0OwgsHw4CsxINVWsdQU+331qWD6+tZiWqjIyxoGse8+b5wvWWDweBWYmGqqPuBiw7XmPNSlQZGaPHO4otMw4CsxK5I7AceY01K9HQyBgL5/ttZXnxGmtWolpH4E1DlhcHgVmJKu4ILENeY81K5I7AcuQgMCuROwLLkddYsxK5I7AcOQjMSuSOwHLkNdasRO4ILEcOArMSuSOwHHmNNSvRUHXUp5iw7DgIzEoSEbWOwKeYsMx4jTUryfBocVEadwSWGQeBWUkqI+NXJ/PbyvLiNdasJOevTuaOwDLjIDAryVDREXgfgeXGa6xZSdwRWK4cBGYlqbgjsEx5jTUriTsCy5WDwKwk7ggsV0nXWElbJR2R1Cfp3knmebek/ZIOSXo4ZT1mKbkjsFx1p3pgSV3AfcB7gePAo5L2RMQTdfOsAH4P2BoRP5R0Vap6zFI73xH4XEOWmZRr7C1AX0QcjYhh4H5g+4R5fhn4WkT8ECAiTiWsxyyp8x2Bzz5qmUkZBGuBY3X3jxdj9a4HVkr6K0mPSbqr0QNJ2iFpn6R9/f39ico1a86QOwLLVMo1Vg3GYsL9buBm4OeAW4F/L+n6i/4oYldE9EZE7+rVq8uv1KwElRF3BJanZPsIqHUA6+vurwNONJjnxYg4B5yT9AjwVuDJhHWZJTFU9bmGLE8p19hHgc2SNklaANwB7Jkwz58C75DULWkx8OPA4YQ1mSXjk85ZrpJ1BBFRlXQP8BDQBeyOiEOS7i6m74yIw5L+DDgAjAFfjIiDqWoyS2moOkr3PNHd5SCwvKTcNERE7AX2ThjbOeH+Z4DPpKzDrBVql6n0/gHLjz+6mJWkduF6v6UsP15rzUrijsBy5SAwK4k7AsuV11qzklRGxnyeIcuSg8CsJO4ILFeXXGsl9UxnzKzTDY2M+fQSlqXprLV/O80xs45W6wi8acjyM+lxBJKupnaSuEWSbuRH5w5aDixuQW1mWam4I7BMTXVA2a3AB6mdI+h36sbPAv82YU1mWXJHYLmaNAgi4svAlyX9YkQ80MKazLLkjsBydclTTETEA5J+DngzsLBu/D+mLMwsN+4ILFfT+dbQTuCfAL9BbT/BLwHXJq7LLDvuCCxX01lrfzIi7gJeiYj/APwEF15nwKzjRYQ7AsvWdIJgsPg9IOkaYATYlK4ks/yMjAZj4ctUWp6mcxrqByWtoHaq6O9Su9zkF1MWZZYbX7jecjadncWfLG4+IOlBYGFEnE5bllleKr5wvWVsWhemkfSTwMbx+SUREV9JWJdZVtwRWM4uGQSS/hC4DtgPjBbDATgIzArnr1fsjsAyNJ2OoBfYEhGRuhizXLkjsJxN5+PLQeDq1IWY5Wyo6n0Elq+pTjr3dWqbgJYBT0j6DjA0Pj0ibk9fnlkeKiPuCCxfU20a+i/F77cAnwNeTl+OWZ7cEVjOpjrp3MMAkt4LfITaMQS7gYe8v8DsQkPuCCxjl/z4EhH/DtgM/AG101I/Jek/S7oucW1m2RjvCPytIcvRtNbaogN4vvipAiuB/yXp0wlrM8vG+D6Chb54vWVoOscR/AvgA8CL1E4t8a8jYkTSPOAp4DfTlmg2953vCHzxesvQdI4jWAW8PyJ+UD8YEWOSfj5NWWZ5cUdgOZvOuYY+McW0w+WWY5anoRF3BJYvr7VmJahUR+maJ+Z3+S1l+fFaa1aCoZExdwOWLa+5ZiWoVEe9f8CylTQIJG2VdERSn6R7p5jv70salfSPUtZjloo7AstZsjVXUhdwH7AN2ALcKWnLJPN9CngoVS1mqVWqY+4ILFspP8LcAvRFxNGIGAbuB7Y3mO83gAeAUwlrMUtqaGTUHYFlK+WauxY4Vnf/eDF2nqS1wPuAnVM9kKQdkvZJ2tff3196oWbNqlTH6HFHYJlKGQRqMDbxZHX/FfhYRIw2mPdHfxSxKyJ6I6J39erVZdVnVprKyCgL3RFYpqZ1zeJZOg6sr7u/DjgxYZ5e4H5JUDuC+TZJ1Yj43wnrMivd0MgoKxYvaHcZZrOSMggeBTZL2gQ8B9wB/HL9DBGxafy2pC8BDzoELEeVkTFfi8CylSwIIqIq6R5q3wbqAnZHxCFJdxfTp9wvYJYTH0dgOUvZERARe4G9E8YaBkBEfDBlLWYp1fYROAgsT+5lzUrgTUOWM6+5ZiWojHjTkOXLQWDWpIhgqOpTTFi+vOaaNelH1yt2R2B5chCYNclXJ7PcOQjMmlQprk7mncWWK6+5Zk063xH466OWKQeBWZMqVW8asrw5CMya5E1DljuvuWZN8s5iy52DwKxJ418fdUdgufKaa9ak8Y6gxzuLLVMOArMmedOQ5c5BYNakIe8stsx5zTVrkr8+arlzEJg1yZuGLHcOArMmnT+OwGcftUx5zTVrUmVklO55orvLbyfLk9dcsybVrk7mzUKWLweBWZMq1VFflMay5rXXrEmDw6MsWuCOwPLlIDBr0sBwlSULuttdhtmsOQjMmjTgjsAy5yAwa9LA8ChLehwEli8HgVmTBoZHWTTfm4YsXw4CsyYNDldZ7E1DljEHgVmTznnTkGXOQWDWpEFvGrLMOQjMmhARta+PuiOwjDkIzJowVB1jLPDXRy1rSYNA0lZJRyT1Sbq3wfRfkXSg+PkbSW9NWY9Z2QaGa6egXuxzDVnGkgWBpC7gPmAbsAW4U9KWCbM9A7wrIt4CfBLYlaoesxQGhqsALO7xPgLLV8qO4BagLyKORsQwcD+wvX6GiPibiHiluPttYF3CesxKd74j8KYhy1jKIFgLHKu7f7wYm8yvAd9oNEHSDkn7JO3r7+8vsUSz5owHgc81ZDlLGQRqMBYNZ5R+mloQfKzR9IjYFRG9EdG7evXqEks0a87AUG3TkHcWW85Sfow5Dqyvu78OODFxJklvAb4IbIuIlxLWY1Y6bxqyy0HKjuBRYLOkTZIWAHcAe+pnkLQB+BrwqxHxZMJazJIYGHEQWP6SdQQRUZV0D/AQ0AXsjohDku4upu8EPgG8Dvg9SQDViOhNVZNZ2V6r1DYNLe2Z3+ZKzGYv6R6uiNgL7J0wtrPu9q8Dv56yBrOUzlZGAFi20DuLLV8+stisCWcrVbrmyZuGLGsOArMmnKmMsLSnm2LTplmWHARmTThbqXqzkGXPQWDWhLOVEZYt9I5iy5uDwKwJZypVlrsjsMw5CMyaUNs05I7A8uYgMGvC2cqIOwLLnoPArAneWWyXAweB2SyNjUWtI1jkTUOWNweB2SydHhxhLODKJQvaXYpZUxwEZrP08sAw4CCw/DkIzGbp5XO1IFi52EFgeXMQmM3SeBC4I7DcOQjMZumV8Y7AQWCZcxCYzdJL4x2BNw1Z5hwEZrP0yrlhFs6f5+sVW/YcBGazdOrsEFctW9juMsya5iAwm6XnT1e4+goHgeXPQWA2SyfPDLLGQWCXAQeB2SyMjQUvnB5yR2CXBQeB2Sy8PDDM8OgY11yxqN2lmDXNQWA2C8+9MgjgjsAuCw4Cs1l45sVzALxh1ZI2V2LWPAeB2Sw83f8aXfPEhtctbncpZk1zEJjNwtP9r7F+5SJ6un0wmeXPQWA2C0eeP8sbr1ra7jLMSuEgMJuhVweGebr/HDduWNnuUsxK4SAwm6G/O/YqADduWNHWOszK4iAwm6GHj/TT0z2Pt61f0e5SzErhIDCbgdGx4M8OPs87r1/N4gXd7S7HrBQOArMZ+MbBkzx/psL7b1zb7lLMSpM0CCRtlXREUp+kextMl6TPFdMPSLopZT1mzTh1tsJ/+j+Huf71S7n1zVe3uxyz0iTrbSV1AfcB7wWOA49K2hMRT9TNtg3YXPz8OPCF4rfZnDA6Fpw8PchfP/Uiv/vNPl4dGGHXr/Yyb57aXZpZaVJu5LwF6IuIowCS7ge2A/VBsB34SkQE8G1JKyStiYiTZRfz8JP9fPLBJy4Yqz3thS4eaTzYaL5pPx7QYFaiwdwN55vsQWdZT1O1TPPxGs05/ceb/b9jJn/faLBSHWVktDbhTVcv475fuYm/t+6Kxk9klqmUQbAWOFZ3/zgXf9pvNM9a4IIgkLQD2AGwYcOGWRWztKebG16/7OIJDT7YNfqsJ1082ni+xs/fzGM2rrHB307739LE402zwJbU0qgUGi/XxvNd+rl75s9j/crFvGXdFbz5muXTfmyznKQMgkbvmImfuaYzDxGxC9gF0NvbO83Pwxe6+dqV3HytDwAyM5so5c7i48D6uvvrgBOzmMfMzBJKGQSPApslbZK0ALgD2DNhnj3AXcW3h94OnE6xf8DMzCaXbNNQRFQl3QM8BHQBuyPikKS7i+k7gb3AbUAfMAB8KFU9ZmbWWNJDIyNiL7X/7OvHdtbdDuDDKWswM7Op+chiM7MO5yAwM+twDgIzsw7nIDAz63BqdPj9XCapH/jBLP98FfBiieWUZa7WBXO3Ntc1M65rZi7Huq6NiNWNJmQXBM2QtC8iettdx0RztS6Yu7W5rplxXTPTaXV505CZWYdzEJiZdbhOC4Jd7S5gEnO1Lpi7tbmumXFdM9NRdXXUPgIzM7tYp3UEZmY2gYPAzKzDXbZBIOmXJB2SNCapd8K0fyOpT9IRSbfWjd8s6fFi2ueU+HJUkr4qaX/x86yk/cX4RkmDddN2XuKhyq7rtyU9V/f8t9VNa7jsWlTXZyR9X9IBSX8iaUUx3tblVdSwtVgmfZLubfXz19WxXtK3JB0u1v+PFOOTvqYtrO3Z4v21X9K+YuxKSf9X0lPF75ZePUrSDXXLZL+kM5I+2o7lJWm3pFOSDtaNTbp8Sn0vRsRl+QP8GHAD8FdAb934FuB7QA+wCXga6CqmfQf4CWpXTvsGsK2F9X4W+ERxeyNwsI3L7reBf9VgfNJl16K6fhboLm5/CvjUHFleXcWyeAOwoFhGW9pUyxrgpuL2MuDJ4nVr+Jq2uLZngVUTxj4N3Fvcvnf8NW3j6/g8cG07lhfwTuCm+nV5suVT9nvxsu0IIuJwRBxpMGk7cH9EDEXEM9SuhXCLpDXA8oj426gt6a8A/7AVtRadxz8G/rgVz9eEhsuuVU8eEX8eEdXi7repXdFuLrgF6IuIoxExDNxPbVm1XEScjIjvFrfPAoepXQd8rtoOfLm4/WVa9J6bxHuApyNitmcuaEpEPAK8PGF4suVT6nvxsg2CKawFjtXdP16MrS1uTxxvhXcAL0TEU3VjmyT9naSHJb2jRXXUu6fYBLO7rh2dbNm1wz+l1rWNa+fymkvL5TxJG4Ebgf9XDDV6TVspgD+X9JikHcXY66O4KmHx+6o21DXuDi78MNbu5QWTL59S17msg0DSX0g62OBnqk9jjbb7xxTjrajxTi5cAU8CGyLiRuBfAv9D0vJma5lBXV8ArgPeVtTy2fE/a/BQpX7/eDrLS9LHgSrwR8VQ8uV1qbIbjLX1e9mSlgIPAB+NiDNM/pq20j+IiJuAbcCHJb2zDTU0pNrldG8H/mcxNBeW11RKXeeSXqEstYj4mVn82XFgfd39dcCJYnxdg/GmXKpGSd3A+4Gb6/5mCBgqbj8m6WngemBfs/VMt666+n4feLC4O9myK800ltcHgJ8H3lNswmvJ8rqE5MtlJiTNpxYCfxQRXwOIiBfqpte/pi0TESeK36ck/Qm1TRkvSFoTESeLzbOnWl1XYRvw3fHlNBeWV2Gy5VPqOpd1RzBLe4A7JPVI2gRsBr5TtF1nJb292GZ/F/CnLajnZ4DvR8T5zVKSVkvqKm6/oajxaAtqGX/+NXV33weMf4uh4bJrYV1bgY8Bt0fEQN14W5cX8CiwWdKm4pPlHdSWVcsV6+4fAIcj4nfqxid7TVtV1xJJy8ZvU9vxf5DacvpAMdsHaM17rpELuvJ2L686ky2fct+L7dpD34I98O+jlppDwAvAQ3XTPk5tL/sR6r4ZBPRSe8GfBj5PceR14jq/BNw9YewXgUPUvhXwXeAXWrzs/hB4HDhQrHBrLrXsWlRXH7XtovuLn51zYXkVNdxG7Rs6TwMfb/Xz19XxU9Q2ERyoW063TfWatqiuNxSvz/eK1+rjxfjrgL8Enip+X9mGZbYYeAm4om6s5cuLWhCdBEaK/7t+barlU+Z70aeYMDPrcJ24acjMzOo4CMzMOpyDwMyswzkIzMw6nIPAzKzDOQjMzDqcg8DMrMM5CMyaJOnuuvPWPyPpW+2uyWwmfECZWUmKc/x8E/h0RHy93fWYTZc7ArPy/Dfgmw4By03WZx81myskfZDala3uaXMpZjPmTUNmTZJ0M7WrR70jIl5pdz1mM+VNQ2bNuwe4EvhWscP4i+0uyGwm3BGYmXU4dwRmZh3OQWBm1uEcBGZmHc5BYGbW4RwEZmYdzkFgZtbhHARmZh3u/wNVDgQzdmUqzQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(z.numpy(), yhat.numpy())\n",
    "plt.xlabel('z')\n",
    "plt.ylabel('yhat')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apply the element-wise Sigmoid from the function module and plot the results:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f41706f5320>]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAWYklEQVR4nO3de4xc53nf8e/DXXLFqyiZlC3xIsq2JEdGbUvaKi4aXwInsaS2Vp1eICWIXTeBoNYqYhRFrcJoEsDoH67honCsmGVTwnGRREYQp1FVOkqTNhKC1I0oR5ZEy5SoSyyKlLi6kRSXe5ndp3/MWXY0OrM7JGdn9t39foAF51x25tGZd3569p0zcyIzkSSVb9WgC5Ak9YaBLknLhIEuScuEgS5Jy4SBLknLxPCgHnjLli25a9euQT28JBXp4Ycffjkzt9ZtG1ig79q1i/379w/q4SWpSBHx1522OeUiScuEgS5Jy4SBLknLhIEuScuEgS5Jy8SCgR4ReyPiWEQ83mF7RMRXI+JQRDwaEdf1vkxJ0kK66dC/Adw4z/abgCurn9uBr59/WZKks7XgeeiZ+WBE7Jpnl1uAb2bze3i/GxGbI+LSzDzaqyKlpaYxM8sPXzzJsy+f4vjpacanGkw1ZplNyIQkq39prpBajO66mA9fVfvZoPPSiw8WbQOeb1k+XK17S6BHxO00u3h27tzZg4eW+qsxM8t/evAZ9v75s7xyaqrr34tYxKJUnDs+8q4lG+h1Q7W2JcnMPcAegNHRUdsWFWV2Nvnctx7hvkeP8rH3XMInPnAZV79jIxevW8O6kWHWDK1iVUBEEDRDPExy9VEvAv0wsKNleTtwpAf3Ky0p39r/PPc9epR/fePV/POPvnvQ5Uhv0YvTFu8FPlWd7fJB4Ljz51puGjOz/PqfPsXo5Rfxzz7yrkGXI9VasEOPiN8FPgpsiYjDwK8CqwEyczewD7gZOASMA59ZrGKlQXngyTGOHJ/gVz/xXqdRtGR1c5bLbQtsT+CzPatIWoL+5IljbBgZ5ievvmTQpUgd+UlRqQsPPjnGT7x7C2uGfclo6XJ0Sgs4dnKCF14/zeiuiwZdijQvA11awGOHjwPw/h2bB1uItAADXVrA4y+cIAKuuXTToEuR5mWgSwt49uU3uOzCtawfGdgVG6WuGOjSAp57ZZxdW9YNugxpQQa6tIDnXjnFrretH3QZ0oIMdGkex8eneX182kBXEQx0aR5Hjp8G4LLNawdcibQwA12ax9jJSQAu2TQy4EqkhRno0jyOzQX6RgNdS5+BLs3j2MkJALYa6CqAgS7N49iJSTaMDLNujeega+kz0KV5jL0x6XSLimGgS/MYOzHJFgNdhTDQpXnYoaskBro0j9fGp7h4/ZpBlyF1xUCXOpidTU6cnmbTBasHXYrUFQNd6uCNqQazCReuNdBVBgNd6uD4+DRgoKscBrrUwYmJZqBvMtBVCANd6uD4aTt0lcVAlzo4cXquQ/dToiqDgS51YIeu0hjoUgcnTjcAA13lMNClDo6fnmZVwAYvDq1CGOhSB8dPT7Np7WoiYtClSF0x0KUOTkz4KVGVxUCXOjg1OeN0i4pioEsdjE81WD8yNOgypK4Z6FIHp6ZmvFKRitJVoEfEjRFxMCIORcRdNdsvjIj/HhHfj4gDEfGZ3pcq9df4pB26yrJgoEfEEHA3cBNwDXBbRFzTtttngR9k5vuBjwJfiQi/RFpFG7dDV2G66dBvAA5l5jOZOQXcA9zStk8CG6N5ftcG4FWg0dNKpT47NdVg/Ro7dJWjm0DfBjzfsny4Wtfqa8CPAUeAx4BfzszZ9juKiNsjYn9E7B8bGzvHkqX+GJ+cYZ1nuagg3QR63acqsm3548AjwGXAB4CvRcSmt/xS5p7MHM3M0a1bt55lqVL/TDVmmZqZtUNXUboJ9MPAjpbl7TQ78VafAb6dTYeAZ4H39KZEqf9OT80AOIeuonQT6A8BV0bEFdUbnbcC97bt8yPgYwAR8XbgauCZXhYq9dOpqeZbQJ7lopIs2H5kZiMi7gTuB4aAvZl5ICLuqLbvBr4IfCMiHqM5RfP5zHx5EeuWFtV4Feh26CpJV6M1M/cB+9rW7W65fQT4md6WJg3OqcnmlIsdukriJ0WlGqfs0FUgA12qMT7XoRvoKoiBLtU406E75aKCGOhSjfEzpy0a6CqHgS7VmDsPfe1qA13lMNClGpON5jdXjAwb6CqHgS7VmGw0O/SRYV8iKoejVaoxMT3LmqFVrFrlBaJVDgNdqjHZmLE7V3EcsVKNielZRnxDVIUx0KUadugqkSNWqjE5PcsFq315qCyOWKlGs0N3ykVlMdClGhN26CqQI1aqYYeuEhnoUg07dJXIESvVsENXiQx0qYYdukrkiJVq2KGrRAa6VMMOXSVyxEo1JhszfvRfxTHQpTaZ2ezQ/ei/CuOIldpMzVQXt7BDV2EMdKnNxPTc1Yp8eagsjlipzZmrFdmhqzAGutRmsurQnUNXaRyxUhs7dJXKQJfaTNihq1COWKmNHbpKZaBLbezQVaquRmxE3BgRByPiUETc1WGfj0bEIxFxICIe6G2ZUv/YoatUwwvtEBFDwN3ATwOHgYci4t7M/EHLPpuB3wBuzMwfRcQli1SvtOjOdOh+l4sK082IvQE4lJnPZOYUcA9wS9s+Pwd8OzN/BJCZx3pbptQ/Zzp0v21Rhekm0LcBz7csH67WtboKuCgi/iwiHo6IT9XdUUTcHhH7I2L/2NjYuVUsLbJJO3QVqpsRGzXrsm15GLge+DvAx4F/GxFXveWXMvdk5mhmjm7duvWsi5X6YWLaDl1lWnAOnWZHvqNleTtwpGaflzPzFHAqIh4E3g882ZMqpT6abPhdLipTNyP2IeDKiLgiItYAtwL3tu3zh8CHImI4ItYBPw480dtSpf7wy7lUqgU79MxsRMSdwP3AELA3Mw9ExB3V9t2Z+URE/BHwKDAL/GZmPr6YhUuLZbIxw/CqYHjIQFdZuplyITP3Afva1u1uW/4y8OXelSYNRvPyc86fqzy2IFKb5gWifWmoPI5aqY0dukploEtt7NBVKket1GZietbvcVGRDHSpjR26SuWoldpMTs/6sX8VyVErtWl26E65qDwGutRmwg5dhXLUSm3s0FUqA11qY4euUjlqpTZ26CqVgS61sUNXqRy1UovMtENXsQx0qcX0TDKbXn5OZXLUSi28QLRKZqBLLSa8QLQK5qiVWtihq2QGutTizPVE7dBVIEet1MIOXSUz0KUWkw3n0FUuR63UYmLaDl3lMtClFnboKpmjVmoxaYeughnoUou5Dt2zXFQiR63UYm4O/QIvEq0CGehSizMduheJVoEctVILO3SVzECXWkxO26GrXI5aqcVEY4ahVcHqIV8aKo+jVmoxOT1rd65iOXKlFhONGefPVayuAj0iboyIgxFxKCLumme/vxkRMxHxD3tXotQ/dugq2YIjNyKGgLuBm4BrgNsi4poO+30JuL/XRUr9MtGYtUNXsbppRW4ADmXmM5k5BdwD3FKz378Afh841sP6pL6anJ6xQ1exuhm524DnW5YPV+vOiIhtwCeB3fPdUUTcHhH7I2L/2NjY2dYqLbqJxiwjdugqVDeBHjXrsm35PwKfz8yZ+e4oM/dk5mhmjm7durXLEqX+mZie4QI7dBVquIt9DgM7Wpa3A0fa9hkF7okIgC3AzRHRyMz/1osipX6ZnJ5h87o1gy5DOifdBPpDwJURcQXwAnAr8HOtO2TmFXO3I+IbwH2GuUo0MT3rd6GrWAsGemY2IuJOmmevDAF7M/NARNxRbZ933lwqieehq2TddOhk5j5gX9u62iDPzH9y/mVJg9GcQzfQVSb/tpRaOOWikjlypRYT0065qFwGulTJTCYbfvRf5XLkSpX/fz1RO3SVyUCXKl6tSKUz0KXKRHW1It8UVakcuVLlTIfuaYsqlIEuVSYaTrmobAa6VHHKRaVz5EoV3xRV6Qx0qTJ32qIdukrlyJUqcx36iG+KqlAGulRxykWlM9ClyqRviqpwjlyp4mmLKp2BLlWcclHpDHSpcuY8dL9tUYVy5EqViekZhlcFw0O+LFQmR65UaV6tyOkWlctAlyoTjRkvbqGiOXqlyumpGdausUNXuQx0qTI+1WD9muFBlyGdMwNdqozboatwBrpUGZ+aYf2Iga5yGehSZXxqhrWrnXJRuQx0qXJ6qsE6p1xUMANdqpxyykWFM9ClymmnXFQ4A10CMrN52qIdugpmoEs0Lz83m3jaoorWVaBHxI0RcTAiDkXEXTXbfz4iHq1+/iIi3t/7UqXFMz7V/OrcdX6Xiwq2YKBHxBBwN3ATcA1wW0Rc07bbs8BHMvN9wBeBPb0uVFpM41MNANaNOIeucnXTod8AHMrMZzJzCrgHuKV1h8z8i8x8rVr8LrC9t2VKi+tMh+6UiwrWTaBvA55vWT5crevkF4Hv1G2IiNsjYn9E7B8bG+u+SmmRzQW63+WiknUT6FGzLmt3jPhJmoH++brtmbknM0czc3Tr1q3dVyktsvHJ5pSLb4qqZN20I4eBHS3L24Ej7TtFxPuA3wRuysxXelOe1B9OuWg56KZDfwi4MiKuiIg1wK3Ava07RMRO4NvAL2Tmk70vU1pc49MGusq3YIeemY2IuBO4HxgC9mbmgYi4o9q+G/gV4G3Ab0QEQCMzRxevbKm33phoTrlsGFk94Eqkc9fVO0CZuQ/Y17Zud8vtXwJ+qbelSf1zcmIagI0X+KaoyuUnRSXg5ESDoVXhlIuKZqBLwImJaTaMDFNNGUpFMtAlmh260y0qnYEu0ZxD33iBb4iqbAa6BJyYaLDJDl2FM9Al5qZc7NBVNgNdojnlYoeu0hnoEr4pquXBQNeKNzubzQ59rVMuKpuBrhXv+OlpZhMuXr9m0KVI58VA14r36vgUYKCrfAa6VrxXTzUD/aJ1BrrKZqBrxZsLdDt0lc5A14r32lyHbqCrcAa6VrxX5jp0p1xUOANdK95rp6a4YPUqryeq4hnoWvGOnZzkko0XDLoM6bwZ6FrxXjw+wTsuNNBVPgNdK97RE6e51EDXMmCga0WbnU1eOj5ph65lwUDXivbq+BRTM7NcduHaQZcinTcDXSvaC6+dBrBD17JgoGtFe/blUwC8c8v6AVcinT8DXSva02NvMLQq2Pm2dYMuRTpvBrpWtKfH3mDHRWsZGfZDRSqfga4V7eCLJ3n3JRsGXYbUEwa6VqzXx6d4euwU1+68aNClSD1hoGvF+qvnXwfg2p2bB1qH1CsGulasBw6OMTK8ig/s2DzoUqSeMNC1Is3MJn/0+It8+KqtrFszPOhypJ4w0LUifefxo7x4YoKfvXbboEuReqarQI+IGyPiYEQcioi7arZHRHy12v5oRFzX+1Kl3jh2coJ/9z+e4Kq3b+Dj733HoMuRembBvzUjYgi4G/hp4DDwUETcm5k/aNntJuDK6ufHga9X/0pLwsxscvT4af78qZf59f91iNfHp9nzC6OsWhWDLk3qmW4mD28ADmXmMwARcQ9wC9Aa6LcA38zMBL4bEZsj4tLMPNrrgh94cowv3veDN61rPuybvXVN/cq6/bq+P6BmV7Jm79r9Ot3pOdZzXrV0eX91e3Z/f+f+33E2v1+3cqIxw/RMc8N73rGRu3/+Ov7G9gvrH0gqVDeBvg14vmX5MG/tvuv22Qa8KdAj4nbgdoCdO3eeba0AbBgZ5uq3b3zrhppGq673injr2vr96h//fO6zvsaa3+36v+U87q/LAvtSS10p1B/X+v0WfuyR1avYcdE63rf9Qt572aau71sqSTeBXjfy23ugbvYhM/cAewBGR0e77E/f7PrLL+L6y/0giCS16+ZN0cPAjpbl7cCRc9hHkrSIugn0h4ArI+KKiFgD3Arc27bPvcCnqrNdPggcX4z5c0lSZwtOuWRmIyLuBO4HhoC9mXkgIu6otu8G9gE3A4eAceAzi1eyJKlOVx+Ry8x9NEO7dd3ultsJfLa3pUmSzoafFJWkZcJAl6RlwkCXpGXCQJekZSLqPk7dlweOGAP++hx/fQvwcg/L6ZWlWhcs3dqs6+xY19lZjnVdnplb6zYMLNDPR0Tsz8zRQdfRbqnWBUu3Nus6O9Z1dlZaXU65SNIyYaBL0jJRaqDvGXQBHSzVumDp1mZdZ8e6zs6KqqvIOXRJ0luV2qFLktoY6JK0TCz5QI+IfxQRByJiNiJG27b9m+rC1Acj4uMt66+PiMeqbV+NRb48TUR8KyIeqX6ei4hHqvW7IuJ0y7bdC9xVr+v6tYh4oeXxb27ZVnvs+lTXlyPih9UFxf8gIjZX6wd6vKoa5r0geh/r2BER/zsinqjG/y9X6zs+p32s7bnq9fVIROyv1l0cEf8zIp6q/u3rVWgi4uqWY/JIRJyIiM8N4nhFxN6IOBYRj7es63h8evpazMwl/QP8GHA18GfAaMv6a4DvAyPAFcDTwFC17S+Bv0XzSkrfAW7qY71fAX6lur0LeHyAx+7XgH9Vs77jsetTXT8DDFe3vwR8aYkcr6HqWLwTWFMdo2sGVMulwHXV7Y3Ak9XzVvuc9rm254Atbev+PXBXdfuuued0gM/ji8DlgzhewIeB61rHcqfj0+vX4pLv0DPzicw8WLPpFuCezJzMzGdpfhf7DRFxKbApM/9PNo/YN4G/349aq78E/jHwu/14vPNQe+z69eCZ+ceZ2agWv0vzCldLwZkLomfmFDB3QfS+y8yjmfm96vZJ4Ama1+ldqm4Bfqu6/Vv06TXXwceApzPzXD+Jfl4y80Hg1bbVnY5PT1+LSz7Q59HpwtTbqtvt6/vhQ8BLmflUy7orIuKvIuKBiPhQn+podWc1tbG35c+8TsduEP4pzb+i5gzyeC2l43JGROwCrgX+b7Wq7jntpwT+OCIejuaF3wHentVVyqp/LxlAXXNu5c1N1aCPF3Q+Pj0dc0si0CPiTyLi8Zqf+bqjThem7uqC1YtU4228eSAdBXZm5rXAvwR+JyI2nW8tZ1HX14F3AR+oavnK3K/V3FVPz1/t5nhFxBeABvDb1apFP14LlV2zbqDn9UbEBuD3gc9l5gk6P6f99Lcz8zrgJuCzEfHhAdRQK5qXyfwE8HvVqqVwvObT0zHX1RWLFltm/tQ5/FqnC1Mf5s1/wvfkgtUL1RgRw8DPAte3/M4kMFndfjgingauAvafbz3d1tVS338G7qsWF/2i3l0cr08Dfxf4WDU11pfjtYAldbHziFhNM8x/OzO/DZCZL7Vsb31O+yYzj1T/HouIP6A5RfBSRFyamUerac9j/a6rchPwvbnjtBSOV6XT8enpmFsSHfo5uhe4NSJGIuIK4ErgL6s/Z05GxAerOe1PAX/Yh3p+CvhhZp6Z7omIrRExVN1+Z1XjM32oZe7xL21Z/CQw96577bHrY103Ap8HPpGZ4y3rB3q86O6C6H1Rjd3/AjyRmf+hZX2n57Rfda2PiI1zt2m+wf04zeP06Wq3T9Of11ydN/2VPOjj1aLT8enta3FQ70SfxTvGn6T5f7FJ4CXg/pZtX6D5rvBBWs5kAUZpPnFPA1+j+kTsItf5DeCOtnX/ADhA813s7wF/r8/H7r8CjwGPVgPn0oWOXZ/qOkRz3vCR6mf3UjheVQ030zyj5GngC/1+/JY6foLmn96Pthynm+d7TvtU1zur5+f71XP1hWr924A/BZ6q/r14AMdsHfAKcGHLur4fL5r/QzkKTFfZ9YvzHZ9evhb96L8kLRMlT7lIkloY6JK0TBjokrRMGOiStEwY6JK0TBjokrRMGOiStEz8P3pTGhMjx4xnAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "yhat = torch.sigmoid(z)\n",
    "plt.plot(z.numpy(), yhat.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Seq\">Build a Logistic Regression with <code>nn.Sequential</code></h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a 1x1 tensor where x represents one data sample with one dimension, and 2x1 tensor X represents two data samples of one dimension:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x =  tensor([[1.]])\n",
      "X =  tensor([[  1.],\n",
      "        [100.]])\n"
     ]
    }
   ],
   "source": [
    "# Create x and X tensor\n",
    "\n",
    "x = torch.tensor([[1.0]])\n",
    "X = torch.tensor([[1.0], [100]])\n",
    "print('x = ', x)\n",
    "print('X = ', X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a logistic regression object with the <code>nn.Sequential</code> model with a one-dimensional input:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use sequential function to create model\n",
    "\n",
    "model = nn.Sequential(\n",
    "    nn.Linear(1,1), \n",
    "    nn.Sigmoid()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The object is represented in the following diagram: \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src = \"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter3/3.1.1_logistic_regression_block_diagram.png\" width = 800, align = \"center\" alt=\"logistic regression block diagram\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the parameters are randomly initialized. You can view them the following ways:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "list(model.parameters()):\n",
      "  [Parameter containing:\n",
      "tensor([[-0.2199]], requires_grad=True), Parameter containing:\n",
      "tensor([0.8177], requires_grad=True)]\n",
      "\n",
      "model.state_dict():\n",
      "  OrderedDict([('0.weight', tensor([[-0.2199]])), ('0.bias', tensor([0.8177]))])\n"
     ]
    }
   ],
   "source": [
    "# Print the parameters\n",
    "\n",
    "print(\"list(model.parameters()):\\n \", list(model.parameters()))\n",
    "print(\"\\nmodel.state_dict():\\n \", model.state_dict())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a prediction with one sample:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The prediction:  tensor([[0.6452]], grad_fn=<SigmoidBackward>)\n"
     ]
    }
   ],
   "source": [
    "# The prediction for x\n",
    "\n",
    "yhat = model(x)\n",
    "print(\"The prediction: \", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calling the object with tensor <code>X</code> performed the following operation <b>(code values may not be the same as the diagrams value  depending on the version of PyTorch) </b>:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter3/3.1.1_logistic_functio_example%20.png\" width=\"400\" alt=\"Logistic Example\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a prediction with multiple samples:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[6.4515e-01],\n",
       "        [6.3835e-10]], grad_fn=<SigmoidBackward>)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The prediction for X\n",
    "\n",
    "yhat = model(X)\n",
    "yhat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calling the object performed the following operation: \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a 1x2 tensor where x represents one data sample with one dimension, and 2x3 tensor X represents one data sample of two dimensions:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x =  tensor([[1., 1.]])\n",
      "X =  tensor([[1., 1.],\n",
      "        [1., 2.],\n",
      "        [1., 3.]])\n"
     ]
    }
   ],
   "source": [
    "# Create and print samples\n",
    "\n",
    "x = torch.tensor([[1.0, 1.0]])\n",
    "X = torch.tensor([[1.0, 1.0], [1.0, 2.0], [1.0, 3.0]])\n",
    "print('x = ', x)\n",
    "print('X = ', X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a logistic regression object with the <code>nn.Sequential</code> model with a two-dimensional input: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create new model using nn.sequential()\n",
    "\n",
    "model = nn.Sequential(nn.Linear(2, 1), nn.Sigmoid())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The object will apply the Sigmoid function to the output of the linear function as shown in the following diagram:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter3/3.1.1logistic_output.png\" width=\"800\" alt=\"The structure of nn.sequential\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the parameters are randomly initialized. You can view them the following ways:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "list(model.parameters()):\n",
      "  [Parameter containing:\n",
      "tensor([[0.0472, 0.2932]], requires_grad=True), Parameter containing:\n",
      "tensor([0.2992], requires_grad=True)]\n",
      "\n",
      "model.state_dict():\n",
      "  OrderedDict([('0.weight', tensor([[0.0472, 0.2932]])), ('0.bias', tensor([0.2992]))])\n"
     ]
    }
   ],
   "source": [
    "# Print the parameters\n",
    "\n",
    "print(\"list(model.parameters()):\\n \", list(model.parameters()))\n",
    "print(\"\\nmodel.state_dict():\\n \", model.state_dict())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a prediction with one sample:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The prediction:  tensor([[0.6547]], grad_fn=<SigmoidBackward>)\n"
     ]
    }
   ],
   "source": [
    "# Make the prediction of x\n",
    "\n",
    "yhat = model(x)\n",
    "print(\"The prediction: \", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The operation is represented in the following diagram:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter3/3.3.1.logisticwithouptut.png\" width=\"500\" alt=\"Sequential Example\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a prediction with multiple samples:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The prediction:  tensor([[0.6547],\n",
      "        [0.7177],\n",
      "        [0.7731]], grad_fn=<SigmoidBackward>)\n"
     ]
    }
   ],
   "source": [
    "# The prediction of X\n",
    "\n",
    "yhat = model(X)\n",
    "print(\"The prediction: \", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The operation is represented in the following diagram: \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter3/3.1.1_logistic_with_outputs2.png\" width=\"800\" alt=\"Sequential Example\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Model\">Build Custom Modules</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section, you will build a custom Module or class. The model or object function is identical to using <code>nn.Sequential</code>.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a logistic regression custom module:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create logistic_regression custom class\n",
    "\n",
    "class logistic_regression(nn.Module):\n",
    "    \n",
    "    # Constructor\n",
    "    def __init__(self, n_inputs):\n",
    "        super(logistic_regression, self).__init__()\n",
    "        self.linear = nn.Linear(n_inputs, 1)\n",
    "    \n",
    "    # Prediction\n",
    "    def forward(self, x):\n",
    "        yhat = torch.sigmoid(self.linear(x))\n",
    "        return yhat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a 1x1 tensor where x represents one data sample with one dimension, and 3x1 tensor where $X$ represents one data sample of one dimension:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x =  tensor([[1.]])\n",
      "X =  tensor([[-100.],\n",
      "        [   0.],\n",
      "        [ 100.]])\n"
     ]
    }
   ],
   "source": [
    "# Create x and X tensor\n",
    "\n",
    "x = torch.tensor([[1.0]])\n",
    "X = torch.tensor([[-100], [0], [100.0]])\n",
    "print('x = ', x)\n",
    "print('X = ', X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a model to predict one dimension: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create logistic regression model\n",
    "\n",
    "model = logistic_regression(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the parameters are randomly initialized. You can view them the following ways:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "list(model.parameters()):\n",
      "  [Parameter containing:\n",
      "tensor([[0.9617]], requires_grad=True), Parameter containing:\n",
      "tensor([-0.9795], requires_grad=True)]\n",
      "\n",
      "model.state_dict():\n",
      "  OrderedDict([('linear.weight', tensor([[0.9617]])), ('linear.bias', tensor([-0.9795]))])\n"
     ]
    }
   ],
   "source": [
    "# Print parameters \n",
    "\n",
    "print(\"list(model.parameters()):\\n \", list(model.parameters()))\n",
    "print(\"\\nmodel.state_dict():\\n \", model.state_dict())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a prediction with one sample:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The prediction result: \n",
      " tensor([[0.4956]], grad_fn=<SigmoidBackward>)\n"
     ]
    }
   ],
   "source": [
    "# Make the prediction of x\n",
    "\n",
    "yhat = model(x)\n",
    "print(\"The prediction result: \\n\", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a prediction with multiple samples:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The prediction result: \n",
      " tensor([[0.0000],\n",
      "        [0.2730],\n",
      "        [1.0000]], grad_fn=<SigmoidBackward>)\n"
     ]
    }
   ],
   "source": [
    "# Make the prediction of X\n",
    "\n",
    "yhat = model(X)\n",
    "print(\"The prediction result: \\n\", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a logistic regression object with a function with two inputs: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create logistic regression model\n",
    "\n",
    "model = logistic_regression(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a 1x2 tensor where x represents one data sample with one dimension, and 3x2 tensor X represents one data sample of one dimension:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x =  tensor([[1., 2.]])\n",
      "X =  tensor([[ 100., -100.],\n",
      "        [   0.,    0.],\n",
      "        [-100.,  100.]])\n"
     ]
    }
   ],
   "source": [
    "# Create x and X tensor\n",
    "\n",
    "x = torch.tensor([[1.0, 2.0]])\n",
    "X = torch.tensor([[100, -100], [0.0, 0.0], [-100, 100]])\n",
    "print('x = ', x)\n",
    "print('X = ', X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a prediction with one sample:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The prediction result: \n",
      " tensor([[0.5845]], grad_fn=<SigmoidBackward>)\n"
     ]
    }
   ],
   "source": [
    "# Make the prediction of x\n",
    "\n",
    "yhat = model(x)\n",
    "print(\"The prediction result: \\n\", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a prediction with multiple samples: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The prediction result: \n",
      " tensor([[0.7863],\n",
      "        [0.6228],\n",
      "        [0.4256]], grad_fn=<SigmoidBackward>)\n"
     ]
    }
   ],
   "source": [
    "# Make the prediction of X\n",
    "\n",
    "yhat = model(X)\n",
    "print(\"The prediction result: \\n\", yhat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Practice</h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make your own model <code>my_model</code> as applying linear regression first and then logistic regression using <code>nn.Sequential()</code>. Print out your prediction.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([0.0065], grad_fn=<SigmoidBackward>)\n",
      "<generator object Module.parameters at 0x7f416c6b5e08>\n",
      "OrderedDict([('0.weight', tensor([[ 0.5128, -0.0091,  0.1915]])), ('0.bias', tensor([-0.5416]))])\n",
      "tensor([0.3330], grad_fn=<SigmoidBackward>)\n",
      "<generator object Module.parameters at 0x7f416c6b5e08>\n",
      "OrderedDict([('linear.weight', tensor([[-0.1841,  0.2815, -0.5259]])), ('linear.bias', tensor([0.5030]))])\n"
     ]
    }
   ],
   "source": [
    "# Practice: Make your model and make the prediction\n",
    "\n",
    "#Instead, I will write my own custom logistic regression module.\n",
    "\n",
    "X = torch.tensor([-10.0, -5.0, 3.1])\n",
    "\n",
    "myModel = nn.Sequential(\n",
    "    nn.Linear(3,1), \n",
    "    nn.Sigmoid()\n",
    ")\n",
    "\n",
    "\n",
    "class CustomLogisticRegression(nn.Module):\n",
    "    \n",
    "    #Constructor, takes in the dimension of the single input sample.\n",
    "    def __init__(self, in_dim):\n",
    "        super(CustomLogisticRegression, self).__init__()\n",
    "        self.linear = nn.Linear(in_dim, 1)\n",
    "    \n",
    "    #defines a custom forward pass on the input tensor X\n",
    "    def forward(self, X):\n",
    "        return torch.sigmoid(self.linear(X))\n",
    "\n",
    "\n",
    "yhat = myModel(X)\n",
    "print(yhat)\n",
    "print(myModel.parameters())\n",
    "print(myModel.state_dict())\n",
    "\n",
    "\n",
    "customModel = CustomLogisticRegression(3)\n",
    "yhat_custom = customModel(X)\n",
    "\n",
    "print(yhat_custom)\n",
    "print(customModel.parameters())\n",
    "print(customModel.state_dict())\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- \n",
    "my_model = nn.Sequential(nn.Linear(1, 1),nn.Sigmoid())\n",
    "yhat = my_model(X)\n",
    "print(\"The prediction: \", yhat)\n",
    "-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://dataplatform.cloud.ibm.com/registration/stepone?context=cpdaas&apps=data_science_experience,watson_machine_learning\"><img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DL0110EN-SkillsNetwork/Template/module%201/images/Watson_Studio.png\"/></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>About the Authors:</h2> \n",
    "\n",
    "<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other contributors: <a href=\"https://www.linkedin.com/in/michelleccarey/\">Michelle Carey</a>, <a href=\"www.linkedin.com/in/jiahui-mavis-zhou-a4537814a\">Mavis Zhou</a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Change Log\n",
    "\n",
    "| Date (YYYY-MM-DD) | Version | Changed By | Change Description                                          |\n",
    "| ----------------- | ------- | ---------- | ----------------------------------------------------------- |\n",
    "| 2020-09-23        | 2.0     | Shubham    | Migrated Lab to Markdown and added to course repo in GitLab |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}