26_linear_regression_terminology.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "54b445b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "np.set_printoptions(suppress=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97ebcc9d",
   "metadata": {},
   "source": [
    "# Introduction to Simple Linear Regression\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6393e9b2",
   "metadata": {},
   "source": [
    "### Goal of Regression Analysis\n",
    "- A common goal in statistics is to answer the question: \"Is the variable X (or more likely, X1, ..., Xp) associated with a variable Y, and if so, what is the relationship and can we use it to predict Y?\"\n",
    "- This process of training a model on data where the outcome is known, for subsequent application to data where the outcome is not known, is termed supervised learning.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4babd70a",
   "metadata": {},
   "source": [
    "### Simple Linear Regression\n",
    "- Simple linear regression provides a model of the relationship between the magnitude of one variable and that of a second.\n",
    "  - For example, as X increases, Y also increases, or as X increases, Y decreases.\n",
    "- Correlation is another way to measure how two variables are related.\n",
    "- The difference is that while correlation measures the strength of an association between two variables, regression quantifies the nature of the relationship.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83574599",
   "metadata": {},
   "source": [
    "### Key Terms\n",
    "\n",
    "- Response: The variable we are trying to predict.\n",
    "  - Synonyms: dependent variable, Y variable, target, outcome\n",
    "- Independent variable: The variable used to predict the response.\n",
    "  - Synonyms: X variable, feature, attribute, predictor\n",
    "- Intercept: The intercept of the regression line—that is, the predicted value when X = 0.\n",
    "  - Synonyms: b0, β0\n",
    "- Regression coefficient: The slope of the regression line.\n",
    "  - Synonyms: slope, b1, β1\n",
    "  - b0 and b1 are often called parameter estimates, weights\n",
    "- Fitted values: The estimates Yi obtained from the regression line.\n",
    "  - Synonym: predicted values\n",
    "- Residuals: The difference between the observed values and the fitted values.\n",
    "  - Synonym: errors\n",
    "- Least squares: The method of fitting a regression by minimizing the sum of squared residuals.\n",
    "  - Synonyms: ordinary least squares, OLS"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b13f4211",
   "metadata": {},
   "source": [
    "### The Regression Equation\n",
    "\n",
    "- Simple linear regression estimates how much Y will change when X changes by a certain amount.\n",
    "- With the correlation coefficient, the variables X and Y are interchangeable.\n",
    "- With regression, we are trying to predict the Y variable from X using a linear relationship (i.e., a line):\n",
    "  - $Y = b_0 + b_1X$\n",
    "- We read this as \"Y equals b1 times X, plus a constant b0.\"\n",
    "  - The symbol b0 is known as the intercept (or constant), and the symbol b1 as the slope for X.\n",
    "- The Y variable is known as the response or dependent variable since it depends on X.\n",
    "- The X variable is known as the predictor or independent variable.\n",
    "- The machine learning community tends to use other terms, calling Y the target and X a feature vector.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "57d52bd9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19]\n",
      "[ 5.   6.4  7.8  9.2 10.6 12.  13.4 14.8 16.2 17.6 19.  20.4 21.8 23.2\n",
      " 24.6 26.  27.4 28.8 30.2 31.6]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "b0 = 5\n",
    "b1 = 1.4 \n",
    "\n",
    "x = np.arange(0,20,1)\n",
    "error = np.random.normal(0, 6, 20)\n",
    "y = b0 + b1 * x \n",
    "\n",
    "y_with_error = b0 + b1 * x  + error\n",
    "\n",
    "print(x)\n",
    "print(y)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "5e1d7838",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x12724eeb0>]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABAkUlEQVR4nO3de3iU9Z3//9ed0ySQZCBAMpMSMSoeYtRfUTlYK2oXCtumWt2trdXFdreXRXTLl/by+OsF7F4FtVu37bqlrWupXtTFa9dD5WuL0p+CdikFBbdArKUaFMsMAUJmcpwkM5/fHx+TIeRAJplzno/rmqven7ln8p7ewLzyuT8HxxhjBAAAkCQ5qS4AAACML4QPAACQVIQPAACQVIQPAACQVIQPAACQVIQPAACQVIQPAACQVIQPAACQVHmpLuBUkUhEhw8fVklJiRzHSXU5AABgBIwxamlpUWVlpXJyhu/bSLvwcfjwYVVVVaW6DAAAMAqHDh3S9OnThz0n7cJHSUmJJFt8aWlpiqsBAAAjEQwGVVVV1fc9Ppy0Cx+9t1pKS0sJHwAAZJiRDJlgwCkAAEgqwgcAAEgqwgcAAEgqwgcAAEgqwgcAAEgqwgcAAEgqwgcAAEgqwgcAAEiqtFtkDACAdBWOGO1saFJjS6fKSwo1u7pMuTnsQxYrwgcAACOweZ9PqzfVyxfo7Gvzugu1sq5Gi2q9Kaws83DbBQCA09i8z6elG3b3Cx6S5A90aumG3dq8z5eiyjLTmMLH2rVr5TiOli9f3tdmjNGqVatUWVmpoqIiXX311dq/f/9Y6wQAICXCEaPVm+plBnmut231pnqFI4OdgcGMOnzs2rVLP/3pT3XxxRf3a3/44Yf1yCOP6NFHH9WuXbvk8Xi0YMECtbS0jLlYAACSbWdD04Aej5MZSb5Ap3Y2NCWvqAw3qvDR2tqqL3/5y3rsscc0efLkvnZjjL7//e/rgQce0A033KDa2lo98cQTam9v11NPPRW3ogEASJbGlqGDx2jOwyjDx7Jly/SZz3xGf/VXf9WvvaGhQX6/XwsXLuxrc7lcmj9/vrZv3z7oe4VCIQWDwX4PAADSRXlJYVzPwyjCx8aNG7V7926tXbt2wHN+v1+SVFFR0a+9oqKi77lTrV27Vm63u+9RVVUVa0kAACTM7Ooyed2FGmpCrSM762V2dVkyy8poMYWPQ4cO6Rvf+IY2bNigwsKhE57j9L9ExpgBbb3uu+8+BQKBvsehQ4diKQkAgITKzXG0sq5GkgYEkN7jlXU1rPcRg5jCx5tvvqnGxkZdeumlysvLU15enrZt26Yf/vCHysvL6+vxOLWXo7GxcUBvSC+Xy6XS0tJ+DwAA0smiWq/W3TJLHnf/X7w97kKtu2UW63zEKKZFxj71qU9p7969/dq+8pWv6Pzzz9c999yjs846Sx6PR1u2bNHHP/5xSVJXV5e2bdumhx56KH5VAwCQZItqvVpQ42GF0ziIKXyUlJSotra2X9vEiRM1ZcqUvvbly5drzZo1mjlzpmbOnKk1a9ZowoQJuvnmm+NXNQAAKZCb42je2VNSXUbGi/vy6nfffbc6Ojp0xx136MSJE5ozZ45efvlllZSUxPtHAQCADOQYY9JqSbZgMCi3261AIMD4DwAAMkQs39/s7QIAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJIqL9UFAADGl3DEaGdDkxpbOlVeUqjZ1WXKzXFSXRaSKKaej3Xr1uniiy9WaWmpSktLNW/ePP3617/ue/62226T4zj9HnPnzo170QCAzLR5n09XPvSKvvTYDn1j41v60mM7dOVDr2jzPl+qS0MSxRQ+pk+frgcffFBvvPGG3njjDV177bW67rrrtH///r5zFi1aJJ/P1/f41a9+FfeiAQCZZ/M+n5Zu2C1foLNfuz/QqaUbdhNAxpGYbrvU1dX1O/7Od76jdevWaceOHbrwwgslSS6XSx6PJ34VAgAyXjhitHpTvcwgzxlJjqTVm+q1oMbDLZhxYNQDTsPhsDZu3Ki2tjbNmzevr33r1q0qLy/Xueeeq6997WtqbGwc9n1CoZCCwWC/BwAgu+xsaBrQ43EyI8kX6NTOhqbkFYWUiTl87N27V8XFxXK5XPr617+u5557TjU1NZKkxYsX6xe/+IVeeeUVfe9739OuXbt07bXXKhQKDfl+a9euldvt7ntUVVWN/tMAANJSY8vQwWM05yGzOcaYwXrBhtTV1aUPPvhAzc3NeuaZZ/Qf//Ef2rZtW18AOZnP59OMGTO0ceNG3XDDDYO+XygU6hdOgsGgqqqqFAgEVFpaGuPHAQCko9+9e1xfemzHac/7z6/N1byzp4zpZzGbJjWCwaDcbveIvr9jnmpbUFCgc845R5J02WWXadeuXfrBD36gn/zkJwPO9Xq9mjFjhg4cODDk+7lcLrlcrljLAABkkNnVZfK6C+UPdA467sOR5HHboDAWm/f5tHpTfb9bPF53oVbW1WhRrXdM7434GfMiY8aYIW+rHD9+XIcOHZLXywUHgPEsN8fRyjrbQ35qH0Tv8cq6mjH1UDCbJnPEFD7uv/9+vf766zp48KD27t2rBx54QFu3btWXv/xltba26lvf+pZ+97vf6eDBg9q6davq6uo0depUff7zn09U/QCADLGo1qt1t8ySx13Yr93jLtS6W2aNqWfidLNpJDubJhyJaaQBEiSm2y5HjhzRrbfeKp/PJ7fbrYsvvlibN2/WggUL1NHRob179+rJJ59Uc3OzvF6vrrnmGj399NMqKSlJVP0AgAyyqNarBTWeuI/JiGU2zVjHlGDsYgofjz/++JDPFRUV6aWXXhpzQQCA7Jab48Q9ADCbJrOwsRwAIOOVlxSe/qQYzkNiET4AABmvdzbNUDdvHNlZL2OdTYP4IHwAADJeMmbTIH4IHwCArJDI2TSIr5gXGQMAIF0lajYN4ovwAQDIKomYTYP44rYLAABIKsIHAABIKsIHAABIKsIHAABIKsIHAABIKsIHAABIKsIHAABIKsIHAABIKsIHAABIKsIHAABIKsIHAABIKsIHAABIKsIHAABIKsIHAADjSWenFAymtIS8lP50AACQHB0dkt8vNTVJ+flSba2Uk5o+CMIHAADZrL1d8vmk5uZo24QJUjhM+AAAAHHU1mZDRyAQbZs0SfJ6bfhIIcIHAADZpLXVho6Tx3WUlUkej1RUlLq6TkL4AAAgG7S02NDR0mKPHScaOgoLU1vbKQgfAABkskDAho62NnvsONKUKTZ0uFyprW0IhA8AADJRc7MNHe3t9thxpGnTpIoKqaAgpaWdDuEDAIBMYUw0dHR02LacnGjoyM9PaXkjRfgAACDdGSOdOGFDR2enbcvNjYaOvMz6Os+sagEACReOGO1saFJjS6fKSwo1u7pMuTlOqssan4yRjh+3i4OFQrYtN1cqL7ePDAsdvWJaXWTdunW6+OKLVVpaqtLSUs2bN0+//vWv+543xmjVqlWqrKxUUVGRrr76au3fvz/uRQMAEmPzPp+ufOgVfemxHfrGxrf0pcd26MqHXtHmfb5Ulza+GCMdPSrt2ye9/74NHnl50sc+Jl10kVRZmbHBQ4oxfEyfPl0PPvig3njjDb3xxhu69tprdd111/UFjIcffliPPPKIHn30Ue3atUsej0cLFixQS++0HwBA2tq8z6elG3bLF+js1+4PdGrpht0EkGSIRKTGRmnvXumDD6SuLjuOY/p0Gzo8HtvzkeEcY4wZyxuUlZXpu9/9rr761a+qsrJSy5cv1z333CNJCoVCqqio0EMPPaTbb799RO8XDAbldrsVCARUWlo6ltIAACMUjhhd+dArA4JHL0eSx12o395zLbdgEiESsT0dfr/U02Pb8vNt2Jg6NWXLoMcilu/vUX+acDisjRs3qq2tTfPmzVNDQ4P8fr8WLlzYd47L5dL8+fO1ffv2Id8nFAopGAz2ewAAkmtnQ9OQwUOSjCRfoFM7G5qSV9R4EA7bQaR790offmiDR0GBdMYZtqejvDwjgkesYr5htHfvXs2bN0+dnZ0qLi7Wc889p5qamr6AUVFR0e/8iooKvf/++0O+39q1a7V69epYywAAxFFjy9DBYzTn4TR6euztlcZGG0AkuyCY12tXJXWyu3cp5vBx3nnn6a233lJzc7OeeeYZLVmyRNu2bet73jnl/zBjzIC2k913331asWJF33EwGFRVVVWsZQEAxqC8ZGTLb4/0PAyhp0c6csSGjkjEthUW2tAxeXLWh45eMYePgoICnXPOOZKkyy67TLt27dIPfvCDvnEefr9fXq+37/zGxsYBvSEnc7lccqXp8q8AMF7Mri6T110of6BTgw0E7B3zMbu6LNmlZYfubjue49ixaOgoKoqGjnFmzDeSjDEKhUKqrq6Wx+PRli1b+p7r6urStm3bdMUVV4z1xwAAEig3x9HKuhpJNmicrPd4ZV0Ng01j1dVlZ63s3Rvt7Zg4UTrnHKmmZlwGDynGno/7779fixcvVlVVlVpaWrRx40Zt3bpVmzdvluM4Wr58udasWaOZM2dq5syZWrNmjSZMmKCbb745UfUDAOJkUa1X626ZpdWb6vsNPvW4C7WyrkaLar3DvBr9hEK2p+P4cbtmhyQVF9ueDmZyxhY+jhw5oltvvVU+n09ut1sXX3yxNm/erAULFkiS7r77bnV0dOiOO+7QiRMnNGfOHL388ssqKSlJSPEAgPhaVOvVghoPK5yOVmenDR1NTdHQUVJiQwffhX3GvM5HvLHOBwAg43R02CmzJ05E20pLbegoLk5dXUkUy/d35q7NCgBAqrW329DR3BxtmzTJLg42cWKqqkp7hA8AAGLV1mZDRyAQbZs82fZ0FBWlrq4MQfgAAGCkWltt6Dh5Ne6yMhs6ClkDZaQIHwAAnE4waENHa6s9dpxo6GCtqpgRPgAAGEogYENHW5s9dhy70ZvHY/dgwagQPgAAOFVzsw0d7e32OCfHho6KioSGjnDEjItpzoQPAAAkuy5Hb+jo6LBtOTnStGk2dOTnJ/THb97nG7DAmzdLF3gjfAAAxjdj7KJgfr9dJEyScnOjoSMv8V+Vm/f5tHTD7gH76vgDnVq6YbfW3TIrqwII4QMAMD4ZY5c/9/vtcuiSDR0VFVJ5uf3vJAhHjFZvqh90Qz8ju7fO6k31WlDjyZpbMIQPAMD4EolEQ0dXl23Ly7OhY9q0pIWOXjsbmvrdajmVkeQLdGpnQ5PmnT0leYUlEOEDADA+RCLS0aPSkSN2i3vJjuPoDR05Y97ofVQaW4YOHqM5LxMQPgAA2S0cjoaOnh7bVlBgp8tOmZKy0NGrvGRki5ON9LxMQPgAAGSncFhqbLSP3tDhckVDh5Me4ydmV5fJ6y6UP9A56LgPR5LHbafdZovUxj0AAOKtp0c6fFjau9f+b0+PXfr8zDOlCy+063WkSfCQpNwcRyvraiTZoHGy3uOVdTVZM9hUInwAALJFd7f04Yc2dPh8tuejqEiqrpZqatKqt+NUi2q9WnfLLHnc/W+teNyFWTfNVuK2CwAg03V325krx47ZQaWSNGGC3Xdl0qSUlhaLRbVeLajxsMIpAABpq6srGjrMR6MlJk60ocPtTm1to5Sb42TNdNrhED4AAJklFLKh4/jxaOgoLraho7Q0tbVhRAgfAIDM0Nlpx3I0NUXbSktt6CguTl1diBnhAwCQ3jo6bOg4cSLa5nbb0DFxYurqykDpsmsu4QMAkJ7a223oaG6Otk2aZEPHhAmpqipjpdOuuYQPAEB6aW21oSMYjLZNnmxDR1FR6urKYOm2ay7hAwCQHlpabOhoabHHjiOVldkVSQuzZ2nxZEvHXXMJHwCA1AoGbehobbXHjmMXBPN47HLoGJN03DWX8AEASI1AwIaOtjZ77Dh26XOPx278hrhIx11zCR8AgOQ6ccKu09Hebo9zcuyW9hUVdot7xFU67ppL+AAAJJ4xNnT4fHa9DsmGjvJyGzry+DpKlHTcNZeN5QAAiWOMXYl0/36pocEGj9xcO3Plooukj32M4JFg6bhrLuEDABB/xtg9V/btkw4etEui5+VJlZU2dFRWEjqSKN12zeXKAwDiJxKxoePIEbvxm2RDhsdjx3Xk8DtvqqTTrrkx/SlYu3atLr/8cpWUlKi8vFzXX3+93nnnnX7n3HbbbXIcp99j7ty5cS0aAJBmIhEbOPbtkw4dssEjP1+qqrI9HRUVBI800Ltr7nX/z8c07+wpKQkeUow9H9u2bdOyZct0+eWXq6enRw888IAWLlyo+vp6TTxpff1FixZp/fr1fccFTJkCgOwUDktHj9rg0dNj2woKbE/H1Kl2+ixwipjCx+bNm/sdr1+/XuXl5XrzzTd11VVX9bW7XC55PJ74VAgASD/hsA0cjY32vyW7IJjHYxcII3RgGGMa8xEIBCRJZWX9p+ds3bpV5eXlmjRpkubPn6/vfOc7Ki8vH/Q9QqGQQqFQ33Hw5LX8AQDppafHho6jR6Oho7DQzl6ZPJnQgRFxjDGDTfs9LWOMrrvuOp04cUKvv/56X/vTTz+t4uJizZgxQw0NDfr2t7+tnp4evfnmm3INskzuqlWrtHr16gHtgUBApaWloykNABBv3d3R0BGJ2LaiIhs6Jk0idEDBYFBut3tE39+jDh/Lli3Tiy++qN/+9reaPn36kOf5fD7NmDFDGzdu1A033DDg+cF6PqqqqggfAJAOurqioaP362LChGjoAD4SS/gY1W2Xu+66Sy+88IJee+21YYOHJHm9Xs2YMUMHDhwY9HmXyzVojwgAIIVCIbsE+vHj0dAxcaINHW53amtDxospfBhjdNddd+m5557T1q1bVV1dfdrXHD9+XIcOHZLXm9wFTAAAo9DZaUNHU1M0dJSU2NBRUpLa2pA1Ygofy5Yt01NPPaVf/vKXKikpkd/vlyS53W4VFRWptbVVq1at0o033iiv16uDBw/q/vvv19SpU/X5z38+IR8AABAHHR3R0NGrtNSGjuLi1NWFrBTTmA9niAFF69ev12233aaOjg5df/312rNnj5qbm+X1enXNNdfon//5n1VVVTWinxHLPSMAwBi1t9vN3pqbo21utw0dJ63fBJxOwsZ8nC6nFBUV6aWXXorlLQEAqdDWZkPHR0smSLJTZT0eO6AUSCD2dgGA8aS11YaOk9dUKiuzoaOoKHV1YVwhfADAeNDSYkNHS4s9dpxo6CgsHP61QJwRPgAgmwUCNnS0tdljx7HLn3s8djl0IAUIHwCQjZqbbehob7fHOTl2o7eKCrvxG5BChA8AyBbGRENHR4dty8mRpk2zoSM/P6XlAb0IHwCQAOGI0c6GJjW2dKq8pFCzq8uUm5Og/U+MkU6csKGjs9O25eZGQ0ce/9QjvfAnEgDibPM+n1Zvqpcv0NnX5nUXamVdjRbVxnG1Z2Ps8ud+v10OXbKho6JCKi+3/w2kIcIHEi6pvwECKbZ5n09LN+zWqasi+QOdWrpht9bdMmvsAcQY6dgxGzq6umxbXp4NHdOmETqQ9ggfSKik/QYIpIFwxGj1pvoBwUOSjCRH0upN9VpQ4xldAI9EoqGju9u25edHQ0dOzhiqB5KHP6lImN7fAE8OHlL0N8DN+3wpqgxIjJ0NTQP+vJ/MSPIFOrWzoWnIcwYVidht7ffulQ4dssGjoECqqpJqa234IHggg9DzgYRI+G+AQBpqbBk6eIzmPIXDUmOjffT02LaCArvvypQpds0OIAMRPpAQsfwGOO/sKckrDEig8pKRrRR62vN6eqKhIxy2bS6XDR1lZYQOZDzCBxIi7r8BAhlgdnWZvO5C+QOdg/b6OZI8bjvoelA9Pfb2SmOjvdUi2aXPvV676RuhA1mCm4RIiLj9BghkkNwcRyvraiTZoHGy3uOVdTUDbzV2d9uxHHv32sGkkYjdWfass6QLL6S3A1mH8IGE6P0NcKh/Lh3ZWS9D/gYIZKhFtV6tu2WWPO7+wdrjLhw4zbarS/rgAxs6ens7Jk6UzjlHuuAC29sBZCFuuyAhen8DXLphtxypXxf0sL8BAllgUa1XC2o8Q69vEwrZHo7jx+2aHZJUXGxvr5SWpq5wIEkcY8xgtyZTJhgMyu12KxAIqJS/hBmPdT6Ak3R22tDR1BQNHSUlNnSUlKS2NmCMYvn+pucDCXXa3wCB8aCjw+67cuJEtM3tttvaFxenri4gRQgfSLjcHIfptBif2ttt6GhujrZNmmR7OiZMSFVVQMoRPgAg3trabOgIBKJtkyfb0FFUlLq6gDRB+ACAeGlttaEjGIy2lZXZ0FHItHKgF+EDAMYqGLSho7XVHjuOXf7c47ErkwLoh/ABAKMVCNjQ0dZmjx1HmjrVho6CgtTWBqQxwgcAxKq52YaO9nZ7nJMTDR35+SktDcgEhA8AGAlj7FRZv99OnZVs6Jg2zW5pT+gARozwAQDDMcYuCub320XCJCk3Vyovt488/hkFYsXfGgAYjDF2+XO/3y6HLtnQUVFhQ0dubmrrAzIY4QMAThaJRENHV5dty8uzoWPaNEIHEAeEDwCQbOg4elQ6csRucS/ZcRy9oSOHTcCBeCF8ABjfwuFo6OjpsW0FBXbmypQphA4gAQgfAMancFhqbLShIxy2bS5XNHQ4bH4IJEpMkX7t2rW6/PLLVVJSovLycl1//fV65513+p1jjNGqVatUWVmpoqIiXX311dq/f39ciwaAUevpkf7yF2nvXunwYRs8CgulM8+ULrzQrtdB8AASKqbwsW3bNi1btkw7duzQli1b1NPTo4ULF6qtd3U/SQ8//LAeeeQRPfroo9q1a5c8Ho8WLFiglpaWuBcPACPW3S19+KENHX6/DR1FRdJZZ0k1NfR2AEnkGGPMaF989OhRlZeXa9u2bbrqqqtkjFFlZaWWL1+ue+65R5IUCoVUUVGhhx56SLfffvtp3zMYDMrtdisQCKi0tHS0pQGA1d1tw8axY3ZQqWS3s/d67fb2AOIilu/vMY35CHy0XXRZWZkkqaGhQX6/XwsXLuw7x+Vyaf78+dq+ffug4SMUCinUO4f+o+IBYMy6uqKho/d3rIkTbehwu1NbGzDOjTp8GGO0YsUKXXnllaqtrZUk+f1+SVJFRUW/cysqKvT+++8P+j5r167V6tWrR1sGAPQXCtl9V5qaoqGjuNiGDnpTgbQw6vBx55136g9/+IN++9vfDnjOOeW+qTFmQFuv++67TytWrOg7DgaDqqqqGm1ZAMarzs5o6OhVWmpDR3Fx6uoCMMCowsddd92lF154Qa+99pqmT5/e1+7xeCTZHhCv19vX3tjYOKA3pJfL5ZLL5RpNGQBgN3nz+eymb73cbhs6Jk5MXV0AhhTTbBdjjO688049++yzeuWVV1RdXd3v+erqank8Hm3ZsqWvraurS9u2bdMVV1wRn4oBQLLb2b/7rlRfHw0ekyZJF1wgnXMOwQNIYzH1fCxbtkxPPfWUfvnLX6qkpKRvjIfb7VZRUZEcx9Hy5cu1Zs0azZw5UzNnztSaNWs0YcIE3XzzzQn5AADGmdZW29Nx8uD0yZNtT0dRUerqAjBiMYWPdevWSZKuvvrqfu3r16/XbbfdJkm6++671dHRoTvuuEMnTpzQnDlz9PLLL6ukpCQuBQMYp1pabOjoXTPIcaSyMrsiaWFhamsDEJMxrfORCKzzAUCSwhGjnQ1NavIdVWVHsy6elKfcHMeGjilTbOhgvBiQNpK2zgcAJMLmfT79y3/tlDnsU1G3XQdoSrFLX7nuUl09/xK78RuAjEX4AJBWfvM/b+t767eqsKdLkmQcR8eL3HqnaJJe+/8atc57XItqvad5FwDpjPABIPWMkU6cUPgvh7XhP1+Xq6dLxnF0bIJbxyZOVjgnV5LkSFq9qV4Lajz2FgyAjET4AJA6xthFwXw+KRTS/g+b1djWo2MTJ+v4xEl9oaPvdEm+QKd2NjRp3tlTUlMzgDEjfABIPmPsnit+v92DRZLy8nR4wmT9cdoMRU4JHadqbOlMQpEAEoXwASB5IhEbOo4ciYaO/HypokKaNk3uhhOK5PhO+zblJUytBTIZ4QNA4kUi0tGjNnR0d9u2/Hw7XXbqVCnHLrY8u7pMXneh/IFODbYGgCPJ4y7U7OqypJUOIP5iWl4dAGISDttbK3v3Sh9+aINHQYF0xhnSRRdJ5eV9wUOScnMcrayrkWSDxsl6j1fW1TDYFMhwhA8A8RcOS4cP29Dxl79IPT12QbAzz5Rqa6Vp0+xiYYNYVOvVultmyePuf2vF4y7UultmMc0WyALcdgEQPz099tbK0aM2gEh26XOv1+6/MkTgONWiWq8W1Hi0s6FJjS2dKi+xt1ro8QCyA+EDwNh1d0dDRyRi24qKoqFjFHJzHKbTAlmK8AFg9Lq6oqGjd5uoCRNs6Jg0KaWlAUhfhA8AsQuF7EDS48ejoaO42IYONoQEcBqEDwAj19lpQ0dTUzR0lJTY0FFSktraAGQMwgeA0+voiIaOXqWlNnQUF6euLgAZifABYGjt7XbflebmaJvbbUPHxIkpKysewhHDbBogRQgfAAZqa7OhIxCItk2ebFcknTAhdXXFyeZ9Pq3eVC9fILpHjNddqJV1NawjAiQB4QNAVGurDR3BYLStrMyGjqKi1NUVR5v3+bR0w+4By7f7A51aumE3C5kBSUD4ACC1tNjQ0dJijx0nGjoKs2cTt3DEaPWm+kH3jTGyS7iv3lSvBTUebsEACUT4AMazQMCGjrY2e+w40pQpNnS4XKmtLQF2NjT1u9VyKiPJF+jUzoYmFjgDEojwAYxHzc02dLS322PHsfutVFTYjd+yVGPL0MFjNOcBGB3CBzIesxZGyJho6OjosG05OdHQkZ+f0vKSobxkZLeQRnoegNEhfCCjMWthBIyRTpywoaPzo/+fcnOjoSNv/PwzMLu6TF53ofyBzkHHfTiyu+fOri5LdmnAuJKT6gKA0eqdtXDqPfzeWQub9/lSVFmaMEY6dkzav19qaLDBIzfXrtFRWyt97GPjKnhIdrO6lXU1kmzQOFnv8cq6GnrOgAQjfCAjnW7WgmRnLYQjg52R5YyxG73t2ye9/77dhyUvz4aNiy6SKivHXeg42aJar9bdMksed/9bKx53IdNsgSQZv/8CIaMxa2EQkYjt6fD77Rb3ksK5edoTKtBfnGKVt+VrdnmOcuP4IzN1vM2iWq8W1HgysnYgGxA+kJGYtXCSSMT2dPj9Uk+PbcvP1ysnHP2/v/XrcEtX36nxHA+T6eNtcnOc8RNMgTTDbRdkJGYtSAqH7SDSvXulDz+0waOgQJoxQ5udqfr7X3/QL3hI8RsPw3gbAGNB+EBG6p21MFQnuSP7W3hWzlro6ZEOH7ah4/Bhe+xySWeeKdXWKlw2Rav/79sJGw/DeBsAY0X4QEYal7MWenqkv/zFhg6fz/Z8FBZK1dXShRfalUkdJ6bxMKOR6PcHkP1iDh+vvfaa6urqVFlZKcdx9Pzzz/d7/rbbbpPjOP0ec+fOjVe9QJ9xM2uhu1s6dMiGDr/fjvEoKpLOOsuGjrIyu0LpRxI9HobxNgDGKuYBp21tbbrkkkv0la98RTfeeOOg5yxatEjr16/vOy7I4uWakVpZPWuhq8uGjWPH7PRZSZo40a7T4XYP+bJEj4dhvA2AsYo5fCxevFiLFy8e9hyXyyWPxzPqooBYZN2shVDIho7jx6Oho7jYho7S0tO+PNGreLJKKICxSsiYj61bt6q8vFznnnuuvva1r6mxsXHIc0OhkILBYL8HMC51dkoHD9oVSXt7O0pKpHPPlc47b0TBQ0r8eJhxOd4GQFzFPXwsXrxYv/jFL/TKK6/oe9/7nnbt2qVrr71WoVBo0PPXrl0rt9vd96iqqop3SUB66+iQ3nvPho7e3o7SUhs4zj3XBpAYJXo8zLgZbwMgIRxjzKjnwzmOo+eee07XX3/9kOf4fD7NmDFDGzdu1A033DDg+VAo1C+YBINBVVVVKRAIqHSEv+kBGam93c5aaW6Otk2aJHk8dmxHHCR6BdJMXeEUQPwFg0G53e4RfX8nfIVTr9erGTNm6MCBA4M+73K55HK5El0GkD7a2mzoCASibZMn2zEdRUVx/VGJHg+TdeNtACRFwsPH8ePHdejQIXm9dMNinGtpsaGjpSXaVlZmQ0chM0MAjB8xh4/W1lb9+c9/7jtuaGjQW2+9pbKyMpWVlWnVqlW68cYb5fV6dfDgQd1///2aOnWqPv/5z8e1cCBjBIM2dLS22mPHsQuCeTx2ZVIAGGdiDh9vvPGGrrnmmr7jFStWSJKWLFmidevWae/evXryySfV3Nwsr9era665Rk8//bRKRjFoDshogYANHW1t9thxpKlTbehg7RsA49iYBpwmQiwDVoC01NxsQ0d7uz3OybGho6KC0AEga6XVgFNgXDBGOnHCLg7W0WHbcnKkadNs6MjPT219AJBGCB/AWBgjNTXZ0NH50V4mublSebl95PFXDABOxb+MwGgYYxcE8/vtcuiSDR0VFTZ05Oamtj4ASGOEDyAWkUg0dHR12ba8PBs6pk0jdADACBA+gJGIRKSjR6UjR+wW95Idx9EbOnISsk0SAGQlwgcwnHA4Gjp6emxbQYGdLjtlyphDB8uTAxiPCB/AYMJhqbHRho5w2La5XNHQ4Yw9IGze59PqTfXyBTr72rzuQq2sq2FjNgBZjfABnKynx4aOxsZo6CgstKGjrCwuoUOywWPpht06dZEdf6BTSzfsZmdYnBa9ZshkhA9AsuM4jhyxt1giEdtWVGT3XZk0KW6hQ7JfGqs31Q8IHpJkJDmSVm+q14IaD18mGBS9Zsh0jJLD+NbdLR06JO3bZ8NHJCJNmCCdfbZUU2N3m41j8JCknQ1N/b40TmUk+QKd2tnQFNefi+zQ22t26p+h3l6zzft8KaoMGDl6PjA+dXXZ6bLHjtk1OyRp4kTb0+F2J/RHN7YMHTxGcx7GD3rNkC0IHxhfQiG770pTUzR0FBfb0JGkvYTKSwrjeh7Gj1h6zeadPSV5hQExInxgfOjsjIaOXqWlNnQUFye1lNnVZfK6C+UPdA76G6wjyeO2AwiBk9FrhmzBmA9kt44O6b33pP37o8HD7ZbOP1+aOTPpwUOScnMcrayrkWSDxsl6j1fW1dBtjgHoNUO2IHwgO7W3S+++K9XX291mJTtr5YILpHPOseM7UmhRrVfrbpklj7v/l4THXcg0Wwypt9dsqFjqyM56odcM6Y7bLsgura329kowGG2bPNneXikqSl1dg1hU69WCGg9rNWDEenvNlm7YLUfqd9uOXjNkEscYM9ht55QJBoNyu90KBAIqTdIAQGSBlhYbOlpa7LHj2EXBPB67SBiQRVjnA+kolu9vej6Q2YJBGzpaW+2x49jlzz0euxw6kIXoNUOmI3wgMwUCNnS0tdljx5GmTrWho6AgtbUBSZCb4zCdFhmL8IHMcuKEDR0dHfY4J8duaV9RYbe4BwCkPcIH0p8x0dDR+dE97pwcqbzcho48/hgDQCbhX22kL2Ps2hw+n12ZVJJyc23oKC8ndABAhuJfb6QfY+yeK36/3YNFskGjN3Tk5qa2PgDAmBA+kD4iERs6jhzpHzo8HjuuI4c18QAgGxA+kHqRiHT0qA0d3d22LT/fho6pUwkdAJBlCB9InXA4Gjp6emxbQUE0dDipX7MgHDGspQAAcUb4QPKFwzZwNDba/5bsgmAej10gLA1Ch8QqkgCQKIQPJE9Pjw0dR49GQ0dhod13ZfLktAkdkg0eSzfsHrDlvT/QqaUbdrP5GwCMAeEDidfdHQ0dkYhtKyqyoWPSpLQKHZK91bJ6U/2A4CHZjbwcSas31WtBjYdbMAAwCoQPJE5XVzR09O5fOGFCNHSkqZ0NTf1utZzKSPIFOrWzoYnlrQFgFGKeRvDaa6+prq5OlZWVchxHzz//fL/njTFatWqVKisrVVRUpKuvvlr79++PV73IBKGQ9P770r59dlyHMdLEidI550gXXJDWwUOSGluGDh6jOQ8A0F/M4aOtrU2XXHKJHn300UGff/jhh/XII4/o0Ucf1a5du+TxeLRgwQK19G51juzV2SkdPCjt32/X6zBGKimRzj1XOv98ye1OdYUjUl5SGNfzAAD9xXzbZfHixVq8ePGgzxlj9P3vf18PPPCAbrjhBknSE088oYqKCj311FO6/fbbx1Yt0lNHh12NtKkp2lZaam+vFBenrq5Rml1dJq+7UP5A56DjPhxJHreddgsAiF1cV29qaGiQ3+/XwoUL+9pcLpfmz5+v7du3D/qaUCikYDDY74EM0d4uvfuuVF8fDR5ut+3lmDkzI4OHZLcqX1lXI8kGjZP1Hq+sq2GwKQCMUlzDh9/vlyRVVFT0a6+oqOh77lRr166V2+3ue1RVVcWzJCRCW5v05z9Lb78tNTfbtsmT7XiOc86x4zsy3KJar9bdMksed/9bKx53IdNsAWCMEjLbxTll6qQxZkBbr/vuu08rVqzoOw4GgwSQdNXaaneYPbl3qqzMLg5WVJS6uhJkUa1XC2o8rHAKAHEW1/Dh8Xgk2R4Qrzf6m2FjY+OA3pBeLpdLLpcrnmUg3lpabOjoHTTsONHQUZjdgy5zcxym0wJAnMX1tkt1dbU8Ho+2bNnS19bV1aVt27bpiiuuiOePQjIEAtIf/yj96U82eDiO3XPlwgulM8/M+uABAEiMmHs+Wltb9ec//7nvuKGhQW+99ZbKysp0xhlnaPny5VqzZo1mzpypmTNnas2aNZowYYJuvvnmuBaOBGputj0d7e322HHslvYVFXbjNwAAxiDm8PHGG2/ommuu6TvuHa+xZMkS/fznP9fdd9+tjo4O3XHHHTpx4oTmzJmjl19+WSUlJfGrGvFnTDR0dHTYtpycaOjIz09peQCA7OEYYwZbyiBlgsGg3G63AoGASktLU11O9jNGOnHCho7Oj1bszM2Nho48VuAHAJxeLN/ffLOMV8ZIx4/bxcFCIduWmyuVl9sHoQMAkCB8w4w3kUg0dHR12ba8PNvLMW2aDSAAACQQ4WO8iETsfit+v93iXrLjOHpDR05cJz4BADAkwke2C4ftlvZHjkg9PbYtP9+u0TF1qpSTo3DEaOe7x1lICwCQFISPbBUO2+3sGxujoaOgwG72NmWKnT4rafM+n1ZvqpcvEN0e3usu1Mq6GpYQBwAkBOEj2/T0RENHOGzbXC4bOsrK+kKHZIPH0g27B+zc6g90aumG3exhAgBICMJHtujpsbdWGhvt+A7JrkDq9dpN307ZWyccMVq9qX7QLeON7O6tqzfVa0GNh1swAIC4Inxkuu5uGzqOHo2GjqKiaOgYws6Gpn63Wk5lJPkCndrZ0MTeJgCAuCJ8ZKquLjtz5dgxu2aHZLey93olt/u0L29sGTp4jOY8AABGivARJ+GISc7W66GQDR3Hj0dDR3GxDR0xrAhbXjKyTeFGeh4AACNF+IiDpMwY6ey0oaOpKRo6Skps6BjFvjmzq8vkdRfKH+gcdNyHI8njtiEKAIB4YmWpMeqdMXLq+IneGSOb9/nG9gM6OqT33pP274/2drjd0nnnSeeeO6rgIUm5OY5W1tVIskHjZL3HK+tqGGwKAIg7wscYnG7GiGRnjIQjo9i7r71devddqb7ebvwmSZMmSRdcIJ1zjr3VMkaLar1ad8ssedz9b6143IVMswUAJAy3XcYgITNG2trsDrOBQLRt8mR7e6WoaGwFD2JRrVcLajzJGa8CAIAIH2MS1xkjLS02dLS0RNvKymzoKEzsoM/cHIfptACApCF8jEFcZowEgzZ0tLbaY8exy597PHZlUgAAsgzhYwzGNGMkELCho63to5Mdu9Gbx2P3YAEAIEsx4HQMRjVjpLlZevtt6c9/tsEjJ0cqL5cuukg64wyCBwAg6xE+xmhEM0aMsetz1NfbGSzt7TZ0VFRItbVSVZXd5h4AgHGA2y5xMOSMEUd2bQ6/3y4SJkm5ubano7xcyuP/fgDA+MO3X5z0mzFiTDR0hEIfnZBrezrKy+1/AwAwThE+4ikSiYaOri7blpdnQ8e0aYQOAABE+IiPSMRuaX/kiN3iXrJjOHpDRw5DawAA6EX4GItwOBo6enpsW0GBnS47ZQqhAwCAQRA+RiMclhobbegIh22byxUNHQ5LkwMAMBTCRyx6emzgOHo0GjoKC23oKCsjdAAAMAKEj5Ho7o6GjkjEthUV2X1XJk0idAAAEAPCx3C6u+3MlWPHoqFjwoRo6AAAADEjfAymqysaOsxHu7ZMnGhDh9ud2toAAMhwhI+ThUJ2s7empmjoKC62oaO0NLW1AQCQJQgfkl36vDd09CottaGjuDh1dQEAkIXivhDFqlWr5DhOv4fH44n3j4mPjg7pvfek/fujwcPtls4/X5o5k+ABAEACJKTn48ILL9RvfvObvuPcdFtWvL3d9nQ0N0fbJk2yPR0TJqSqKgAAxoWEhI+8vLz07O1obbWhIxiMtk2ebENHUVHq6gIAYBxJSPg4cOCAKisr5XK5NGfOHK1Zs0ZnnXXWoOeGQiGFend+lRQ8ORjEU0eH9M479r8dxy4K5vHYRcIAAEDSxH3Mx5w5c/Tkk0/qpZde0mOPPSa/368rrrhCx48fH/T8tWvXyu129z2qqqriXZJVVGQHkU6dKl14oXTmmQQPAABSwDGmd05pYrS1tenss8/W3XffrRUrVgx4frCej6qqKgUCAZUyvRUAgIwQDAbldrtH9P2d8Km2EydO1EUXXaQDBw4M+rzL5ZLL5Up0GQAAIE0kfM/3UCikt99+W16vN9E/CgAAZIC4h49vfetb2rZtmxoaGvT73/9ef/M3f6NgMKglS5bE+0cBAIAMFPfbLh9++KG+9KUv6dixY5o2bZrmzp2rHTt2aMaMGfH+UQAAIAPFPXxs3Lgx3m8JAACySMLHfAAAAJyM8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJKK8AEAAJIqYeHjRz/6kaqrq1VYWKhLL71Ur7/+eqJ+FAAAyCAJCR9PP/20li9frgceeEB79uzRJz/5SS1evFgffPBBIn4cAADIII4xxsT7TefMmaNZs2Zp3bp1fW0XXHCBrr/+eq1du3bY1waDQbndbgUCAZWWlsatpnDEaGdDkxpbOlVeUqjZ1WXKzXHi9v6JlMm1AwDGh1i+v/Pi/cO7urr05ptv6t577+3XvnDhQm3fvn3A+aFQSKFQqO84GAzGuyRt3ufT6k318gU6+9q87kKtrKvRolpv3H9ePGVy7QAADCbut12OHTumcDisioqKfu0VFRXy+/0Dzl+7dq3cbnffo6qqKq71bN7n09INu/t9eUuSP9CppRt2a/M+X1x/Xjxlcu0AAAwlYQNOHaf/bQFjzIA2SbrvvvsUCAT6HocOHYpbDeGI0epN9RrsvlJv2+pN9QpH4n7nacwyuXYAAIYT9/AxdepU5ebmDujlaGxsHNAbIkkul0ulpaX9HvGys6FpQK/ByYwkX6BTOxua4vYz4yWTawcAYDhxDx8FBQW69NJLtWXLln7tW7Zs0RVXXBHvHzesxpahv7xHc14yZXLtAAAMJ+4DTiVpxYoVuvXWW3XZZZdp3rx5+ulPf6oPPvhAX//61xPx44ZUXlIY1/OSKZNrBwBgOAkJHzfddJOOHz+uf/qnf5LP51Ntba1+9atfacaMGYn4cUOaXV0mr7tQ/kDnoGMnHEket526mm4yuXYAAIaTsAGnd9xxhw4ePKhQKKQ333xTV111VaJ+1JBycxytrKuRZL+sT9Z7vLKuJi3XzMjk2gEAGE7W7+2yqNardbfMksfd//aEx12odbfMSuu1MjK5dgAAhpKQFU7HghVOB8rk2gEA40NKVzhNV7k5juadPSXVZYxKJtcOAMCpsv62CwAASC+EDwAAkFSEDwAAkFSEDwAAkFSEDwAAkFSEDwAAkFSEDwAAkFSEDwAAkFSEDwAAkFRpt8Jp72rvwWAwxZUAAICR6v3eHsmuLWkXPlpaWiRJVVVVKa4EAADEqqWlRW63e9hz0m5juUgkosOHD6ukpESOE9/N04LBoKqqqnTo0KG4blqXjsbTZ5XG1+fls2av8fR5+azZxxijlpYWVVZWKidn+FEdadfzkZOTo+nTpyf0Z5SWlmb1H4CTjafPKo2vz8tnzV7j6fPyWbPL6Xo8ejHgFAAAJBXhAwAAJNW4Ch8ul0srV66Uy+VKdSkJN54+qzS+Pi+fNXuNp8/LZx3f0m7AKQAAyG7jqucDAACkHuEDAAAkFeEDAAAkFeEDAAAkVdaFjx/96Eeqrq5WYWGhLr30Ur3++uvDnr9t2zZdeumlKiws1FlnnaUf//jHSap09NauXavLL79cJSUlKi8v1/XXX6933nln2Nds3bpVjuMMePzxj39MUtWjt2rVqgF1ezyeYV+TiddVks4888xBr9OyZcsGPT+Trutrr72muro6VVZWynEcPf/88/2eN8Zo1apVqqysVFFRka6++mrt37//tO/7zDPPqKamRi6XSzU1NXruuecS9AliM9zn7e7u1j333KOLLrpIEydOVGVlpf7u7/5Ohw8fHvY9f/7znw96vTs7OxP8aYZ3umt72223Dah57ty5p33fdLy2p/usg10fx3H03e9+d8j3TNfrmkhZFT6efvppLV++XA888ID27NmjT37yk1q8eLE++OCDQc9vaGjQX//1X+uTn/yk9uzZo/vvv1//+I//qGeeeSbJlcdm27ZtWrZsmXbs2KEtW7aop6dHCxcuVFtb22lf+84778jn8/U9Zs6cmYSKx+7CCy/sV/fevXuHPDdTr6sk7dq1q9/n3LJliyTpb//2b4d9XSZc17a2Nl1yySV69NFHB33+4Ycf1iOPPKJHH31Uu3btksfj0YIFC/r2exrM7373O91000269dZb9b//+7+69dZb9YUvfEG///3vE/UxRmy4z9ve3q7du3fr29/+tnbv3q1nn31Wf/rTn/S5z33utO9bWlra71r7fD4VFhYm4iOM2OmurSQtWrSoX82/+tWvhn3PdL22p/usp16bn/3sZ3IcRzfeeOOw75uO1zWhTBaZPXu2+frXv96v7fzzzzf33nvvoOfffffd5vzzz+/Xdvvtt5u5c+cmrMZEaGxsNJLMtm3bhjzn1VdfNZLMiRMnkldYnKxcudJccsklIz4/W66rMcZ84xvfMGeffbaJRCKDPp+p11WSee655/qOI5GI8Xg85sEHH+xr6+zsNG632/z4xz8e8n2+8IUvmEWLFvVr+/SnP22++MUvxr3msTj18w5m586dRpJ5//33hzxn/fr1xu12x7e4OBvssy5ZssRcd911Mb1PJlzbkVzX6667zlx77bXDnpMJ1zXesqbno6urS2+++aYWLlzYr33hwoXavn37oK/53e9+N+D8T3/603rjjTfU3d2dsFrjLRAISJLKyspOe+7HP/5xeb1efepTn9Krr76a6NLi5sCBA6qsrFR1dbW++MUv6r333hvy3Gy5rl1dXdqwYYO++tWvnnaTxUy9rr0aGhrk9/v7XTeXy6X58+cP+fdXGvpaD/eadBUIBOQ4jiZNmjTsea2trZoxY4amT5+uz372s9qzZ09yChyjrVu3qry8XOeee66+9rWvqbGxcdjzs+HaHjlyRC+++KL+/u///rTnZup1Ha2sCR/Hjh1TOBxWRUVFv/aKigr5/f5BX+P3+wc9v6enR8eOHUtYrfFkjNGKFSt05ZVXqra2dsjzvF6vfvrTn+qZZ57Rs88+q/POO0+f+tSn9NprryWx2tGZM2eOnnzySb300kt67LHH5Pf7dcUVV+j48eODnp8N11WSnn/+eTU3N+u2224b8pxMvq4n6/07Gsvf397XxfqadNTZ2al7771XN99887Abj51//vn6+c9/rhdeeEH/+Z//qcLCQn3iE5/QgQMHklht7BYvXqxf/OIXeuWVV/S9731Pu3bt0rXXXqtQKDTka7Lh2j7xxBMqKSnRDTfcMOx5mXpdxyLtdrUdq1N/QzTGDPtb42DnD9aeru6880794Q9/0G9/+9thzzvvvPN03nnn9R3PmzdPhw4d0r/8y7/oqquuSnSZY7J48eK+/77ooos0b948nX322XriiSe0YsWKQV+T6ddVkh5//HEtXrxYlZWVQ56Tydd1MLH+/R3ta9JJd3e3vvjFLyoSiehHP/rRsOfOnTu330DNT3ziE5o1a5b+7d/+TT/84Q8TXeqo3XTTTX3/XVtbq8suu0wzZszQiy++OOwXc6Zf25/97Gf68pe/fNqxG5l6Xccia3o+pk6dqtzc3AGpuLGxcUB67uXxeAY9Py8vT1OmTElYrfFy11136YUXXtCrr76q6dOnx/z6uXPnZmSynjhxoi666KIha8/06ypJ77//vn7zm9/oH/7hH2J+bSZe197ZS7H8/e19XayvSSfd3d36whe+oIaGBm3ZsiXm7dZzcnJ0+eWXZ9z19nq9mjFjxrB1Z/q1ff311/XOO++M6u9wpl7XWGRN+CgoKNCll17aNzug15YtW3TFFVcM+pp58+YNOP/ll1/WZZddpvz8/ITVOlbGGN1555169tln9corr6i6unpU77Nnzx55vd44V5d4oVBIb7/99pC1Z+p1Pdn69etVXl6uz3zmMzG/NhOva3V1tTweT7/r1tXVpW3btg3591ca+loP95p00Rs8Dhw4oN/85jejCsbGGL311lsZd72PHz+uQ4cODVt3Jl9byfZcXnrppbrkkktifm2mXteYpGqkayJs3LjR5Ofnm8cff9zU19eb5cuXm4kTJ5qDBw8aY4y59957za233tp3/nvvvWcmTJhg/s//+T+mvr7ePP744yY/P9/893//d6o+wogsXbrUuN1us3XrVuPz+foe7e3tfeec+ln/9V//1Tz33HPmT3/6k9m3b5+59957jSTzzDPPpOIjxOSb3/ym2bp1q3nvvffMjh07zGc/+1lTUlKSdde1VzgcNmeccYa55557BjyXyde1paXF7Nmzx+zZs8dIMo888ojZs2dP3+yOBx980LjdbvPss8+avXv3mi996UvG6/WaYDDY9x633nprv9lr//M//2Nyc3PNgw8+aN5++23z4IMPmry8PLNjx46kf75TDfd5u7u7zec+9zkzffp089Zbb/X7exwKhfre49TPu2rVKrN582bz7rvvmj179pivfOUrJi8vz/z+979PxUfsM9xnbWlpMd/85jfN9u3bTUNDg3n11VfNvHnzzMc+9rGMvLan+3NsjDGBQMBMmDDBrFu3btD3yJTrmkhZFT6MMebf//3fzYwZM0xBQYGZNWtWv+mnS5YsMfPnz+93/tatW83HP/5xU1BQYM4888wh/7CkE0mDPtavX993zqmf9aGHHjJnn322KSwsNJMnTzZXXnmlefHFF5Nf/CjcdNNNxuv1mvz8fFNZWWluuOEGs3///r7ns+W69nrppZeMJPPOO+8MeC6Tr2vvtOBTH0uWLDHG2Om2K1euNB6Px7hcLnPVVVeZvXv39nuP+fPn953f67/+67/MeeedZ/Lz883555+fNsFruM/b0NAw5N/jV199te89Tv28y5cvN2eccYYpKCgw06ZNMwsXLjTbt29P/oc7xXCftb293SxcuNBMmzbN5OfnmzPOOMMsWbLEfPDBB/3eI1Ou7en+HBtjzE9+8hNTVFRkmpubB32PTLmuieQY89FIPAAAgCTImjEfAAAgMxA+AABAUhE+AABAUhE+AABAUhE+AABAUhE+AABAUhE+AABAUhE+AABAUhE+AABAUhE+AABAUhE+AABAUhE+AABAUv3/gNaSIGLerysAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(x,y_with_error)\n",
    "plt.plot(x,y, alpha=0.2, color ='red')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "f9fa1576",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.23677158, -0.28675654, -0.65790874, -1.27214874,  0.19245299,\n",
       "        1.01465445,  0.29305414,  0.43046277,  1.84568287, -0.21933376,\n",
       "        1.04222732, -0.31529624,  0.103163  , -0.05807558, -0.68186429,\n",
       "       -0.19595686,  0.46954276,  1.78625156,  0.37763657,  0.37791268])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "8873bafd",
   "metadata": {},
   "source": [
    "### Fitted Values and Residuals\n",
    "\n",
    "- Important concepts in regression analysis are the fitted values (the predictions) and residuals (prediction errors).\n",
    "- In general, the data doesn't fall exactly on a line, so the regression equation should include an explicit error term ei:\n",
    "  - $Y_i = b_0 + b_1X_i + e_i$\n",
    "- The fitted values, also referred to as the predicted values, are typically denoted by $\\hat{Y}_i$ (Y-hat). These are given by:\n",
    "  - $\\hat{Y}_i = b_0 + b_1X_i$\n",
    "- The notation $\\hat{b}_0$ and $\\hat{b}_1$ indicates that the coefficients are estimated versus known.\n",
    "  - The \"hat\" notation is used to differentiate between estimates and known values. So the symbol $\\hat{b}$ (\"b-hat\") is an estimate of the unknown parameter b. Why do statisticians differentiate between the estimate and the true value? The estimate has uncertainty, whereas the true value is fixed.\n",
    "- We compute the residuals $e_i$ by subtracting the predicted values from the original data:\n",
    "  - $\\hat{e}_i = Y_i - \\hat{Y}_i$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48961496",
   "metadata": {},
   "source": [
    "### Historical Use of Regression\n",
    "- Historically, a primary use of regression was to illuminate a supposed linear relationship between predictor variables and an outcome variable.\n",
    "- The primary focus in science and certain disciplines is on the estimated slope of the regression equation, b.\n",
    "  - The model is explainable (as opposed to a black box).\n",
    "  - Economists want to know the relationship between consumer spending and GDP growth.\n",
    "  - Management of ABC stores want to know whether contributing to promoting tourism translates to increased sales.\n",
    "- In such cases, the focus is not on predicting individual cases but rather on understanding the overall relationship among variables.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "810fba8f",
   "metadata": {},
   "source": [
    "### Machine Learning and Other Disciplines\n",
    "- In Machine Learning and other disciplines, regression is used to form a model to predict individual outcomes for new data.\n",
    "- In this instance, the main items of interest are the fitted values $\\hat{Y}$.\n",
    "  - In marketing, regression can be used to predict the change in revenue in response to the size of an ad campaign.\n",
    "  - Universities use regression to predict students' GPA based on their SAT scores.\n",
    "- See the paper [To Explain or to Predict?](https://projecteuclid.org/journals/statistical-science/volume-25/issue-3/To-Explain-or-to-Predict/10.1214/10-STS330.full)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfa6a475",
   "metadata": {},
   "source": [
    "\n",
    "### Simple Linear Regression: Key Ideas\n",
    "\n",
    "- The regression equation models the relationship between a response variable $Y$ and a predictor variable $X$ as a line.\n",
    "- A regression model yields fitted values and residuals—predictions of the response and the errors of the predictions.\n",
    "- Regression models are typically fit by the method of least squares.\n",
    "- Regression is used both for prediction and explanation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "258e3cb3",
   "metadata": {},
   "source": [
    "\n",
    "## Multiple Linear Regression\n",
    "\n",
    "- When there are multiple predictors, the equation is simply extended to accommodate them:\n",
    "\n",
    "$Y = b_0 + b_1X_1 + b_2X_2 + ... + b_pX_p + e$\n",
    "\n",
    "- Instead of a line, we now have a linear model—the relationship between each coefficient and its variable (feature) is linear."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8436781d",
   "metadata": {},
   "source": [
    "### Performance Metrics\n",
    "\n",
    "- Root mean squared error (RMSE): The square root of the average squared error of the regression (the most widely used metric to compare regression models).\n",
    "- Residual standard error (RSE): The same as the root mean squared error, but adjusted for degrees of freedom.\n",
    "- R-squared (R²): The proportion of variance explained by the model, from 0 to 1. Also known as the coefficient of determination.\n",
    "- t-statistic: The coefficient for a predictor, divided by the standard error of the coefficient, giving a metric to compare the importance of variables in the model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49d50e34",
   "metadata": {},
   "source": [
    "## Extending Simple Linear Regression Concepts\n",
    "\n",
    "- All of the other concepts in simple linear regression, such as fitting by least squares and the definition of fitted values and residuals, extend to the multiple linear regression setting. \n",
    "\n",
    "- Here, the model is a hyperplane, instead of a line\n",
    "\n",
    "- For example, the fitted values are given by:\n",
    "\n",
    "$\\hat{Y}_i = b_0 + b_1X_{1, i} + b_2X_{2, i} + ... + b_pX_{p, i}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eba9397f",
   "metadata": {},
   "source": [
    "\n",
    "## Example: King County Housing Data\n",
    "\n",
    "- An example of using multiple linear regression is in estimating the value of houses. \n",
    "  - County assessors must estimate the value of a house for the purposes of assessing taxes. \n",
    "  - Real estate professionals and home buyers consult popular websites such as Zillow to ascertain a fair price.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd36873c",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c7bea79c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>id</th>\n",
       "      <th>date</th>\n",
       "      <th>price</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>sqft_living</th>\n",
       "      <th>sqft_lot</th>\n",
       "      <th>floors</th>\n",
       "      <th>waterfront</th>\n",
       "      <th>...</th>\n",
       "      <th>grade</th>\n",
       "      <th>sqft_above</th>\n",
       "      <th>sqft_basement</th>\n",
       "      <th>yr_built</th>\n",
       "      <th>yr_renovated</th>\n",
       "      <th>zipcode</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "      <th>sqft_living15</th>\n",
       "      <th>sqft_lot15</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>7129300520</td>\n",
       "      <td>20141013T000000</td>\n",
       "      <td>221900.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1180</td>\n",
       "      <td>5650</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>7</td>\n",
       "      <td>1180</td>\n",
       "      <td>0</td>\n",
       "      <td>1955</td>\n",
       "      <td>0</td>\n",
       "      <td>98178</td>\n",
       "      <td>47.5112</td>\n",
       "      <td>-122.257</td>\n",
       "      <td>1340</td>\n",
       "      <td>5650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>6414100192</td>\n",
       "      <td>20141209T000000</td>\n",
       "      <td>538000.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.25</td>\n",
       "      <td>2570</td>\n",
       "      <td>7242</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>7</td>\n",
       "      <td>2170</td>\n",
       "      <td>400</td>\n",
       "      <td>1951</td>\n",
       "      <td>1991</td>\n",
       "      <td>98125</td>\n",
       "      <td>47.7210</td>\n",
       "      <td>-122.319</td>\n",
       "      <td>1690</td>\n",
       "      <td>7639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>5631500400</td>\n",
       "      <td>20150225T000000</td>\n",
       "      <td>180000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>770</td>\n",
       "      <td>10000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>6</td>\n",
       "      <td>770</td>\n",
       "      <td>0</td>\n",
       "      <td>1933</td>\n",
       "      <td>0</td>\n",
       "      <td>98028</td>\n",
       "      <td>47.7379</td>\n",
       "      <td>-122.233</td>\n",
       "      <td>2720</td>\n",
       "      <td>8062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>2487200875</td>\n",
       "      <td>20141209T000000</td>\n",
       "      <td>604000.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.00</td>\n",
       "      <td>1960</td>\n",
       "      <td>5000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>7</td>\n",
       "      <td>1050</td>\n",
       "      <td>910</td>\n",
       "      <td>1965</td>\n",
       "      <td>0</td>\n",
       "      <td>98136</td>\n",
       "      <td>47.5208</td>\n",
       "      <td>-122.393</td>\n",
       "      <td>1360</td>\n",
       "      <td>5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>1954400510</td>\n",
       "      <td>20150218T000000</td>\n",
       "      <td>510000.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.00</td>\n",
       "      <td>1680</td>\n",
       "      <td>8080</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>8</td>\n",
       "      <td>1680</td>\n",
       "      <td>0</td>\n",
       "      <td>1987</td>\n",
       "      <td>0</td>\n",
       "      <td>98074</td>\n",
       "      <td>47.6168</td>\n",
       "      <td>-122.045</td>\n",
       "      <td>1800</td>\n",
       "      <td>7503</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0          id             date     price  bedrooms  bathrooms   \n",
       "0           0  7129300520  20141013T000000  221900.0       3.0       1.00  \\\n",
       "1           1  6414100192  20141209T000000  538000.0       3.0       2.25   \n",
       "2           2  5631500400  20150225T000000  180000.0       2.0       1.00   \n",
       "3           3  2487200875  20141209T000000  604000.0       4.0       3.00   \n",
       "4           4  1954400510  20150218T000000  510000.0       3.0       2.00   \n",
       "\n",
       "   sqft_living  sqft_lot  floors  waterfront  ...  grade  sqft_above   \n",
       "0         1180      5650     1.0           0  ...      7        1180  \\\n",
       "1         2570      7242     2.0           0  ...      7        2170   \n",
       "2          770     10000     1.0           0  ...      6         770   \n",
       "3         1960      5000     1.0           0  ...      7        1050   \n",
       "4         1680      8080     1.0           0  ...      8        1680   \n",
       "\n",
       "   sqft_basement  yr_built  yr_renovated  zipcode      lat     long   \n",
       "0              0      1955             0    98178  47.5112 -122.257  \\\n",
       "1            400      1951          1991    98125  47.7210 -122.319   \n",
       "2              0      1933             0    98028  47.7379 -122.233   \n",
       "3            910      1965             0    98136  47.5208 -122.393   \n",
       "4              0      1987             0    98074  47.6168 -122.045   \n",
       "\n",
       "   sqft_living15  sqft_lot15  \n",
       "0           1340        5650  \n",
       "1           1690        7639  \n",
       "2           2720        8062  \n",
       "3           1360        5000  \n",
       "4           1800        7503  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "\n",
    "    \n",
    "    \n",
    "file_name='https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DA0101EN/coursera/project/kc_house_data_NaN.csv'\n",
    "df=pd.read_csv(file_name)\n",
    "df = df.dropna()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5396b742",
   "metadata": {},
   "source": [
    "### King County Housing Data\n",
    "\n",
    "| Column        | Description                                                                                                                     |\n",
    "|---------------|---------------------------------------------------------------------------------------------------------------------------------|\n",
    "| id            | A notation for a house                                                                                                          |\n",
    "| date          | Date house was sold                                                                                                             |\n",
    "| price         | Price is prediction target                                                                                                      |\n",
    "| bedrooms      | Number of bedrooms                                                                                                              |\n",
    "| bathrooms     | Number of bathrooms                                                                                                             |\n",
    "| sqft_living   | Square footage of the home                                                                                                      |\n",
    "| sqft_lot      | Square footage of the lot                                                                                                       |\n",
    "| floors        | Total floors (levels) in house                                                                                                  |\n",
    "| waterfront    | House which has a view to a waterfront                                                                                          |\n",
    "| view          | Has been viewed                                                                                                                 |\n",
    "| condition     | How good the condition is overall                                                                                               |\n",
    "| grade         | overall grade given to the housing unit, based on King County grading system                                                    |\n",
    "| sqft_above    | Square footage of house apart from basement                                                                                     |\n",
    "| sqft_basement | Square footage of the basement                                                                                                  |\n",
    "| yr_built      | Built Year                                                                                                                      |\n",
    "| yr_renovated  | Year when house was renovated                                                                                                   |\n",
    "| zipcode       | Zip code                                                                                                                        |\n",
    "| lat           | Latitude coordinate                                                                                                             |\n",
    "| long          | Longitude coordinate                                                                                                            |\n",
    "| sqft_living15 | Living room area in 2015(implies-- some renovations) This might or might not have affected the lotsize area                     |\n",
    "| sqft_lot15    | LotSize area in 2015(implies-- some renovations)                                                                                |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "496b4514",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-5 {color: black;}#sk-container-id-5 pre{padding: 0;}#sk-container-id-5 div.sk-toggleable {background-color: white;}#sk-container-id-5 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-5 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-5 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-5 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-5 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-5 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-5 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-5 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-5 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-5 div.sk-item {position: relative;z-index: 1;}#sk-container-id-5 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-5 div.sk-item::before, #sk-container-id-5 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-5 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-5 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-5 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-5 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-5 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-5 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-5 div.sk-label-container {text-align: center;}#sk-container-id-5 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-5 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-5\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearRegression(fit_intercept=False)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" checked><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearRegression</label><div class=\"sk-toggleable__content\"><pre>LinearRegression(fit_intercept=False)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "LinearRegression(fit_intercept=False)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "predictors = ['sqft_living15', 'sqft_lot15', 'bathrooms', 'bedrooms', 'grade']\n",
    "outcome = 'price'\n",
    "house_lm = LinearRegression(fit_intercept=False)\n",
    "house_lm.fit(df[predictors], df[outcome])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "86b04548",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearRegression(fit_intercept=False)\n"
     ]
    }
   ],
   "source": [
    "print(house_lm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "6be6884e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "house_lm.intercept_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "cbd379ad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([   187.67630277,     -0.31098733, 122782.88040016, -45991.44743304,\n",
       "        10594.42932037])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "house_lm.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83027b40",
   "metadata": {},
   "source": [
    "The interpretation of the coefficients is as with simple linear regression: the predicted value $Y$ changes by the coefficient $b_j$ for each unit change in $X_j$ assuming all the other variables, $X_k$ for $k \\neq j$, remain the same. For example, adding an extra finished square foot to a house increases the estimated value by roughly $\\$187.67$. This adds 1,000 finished square feet implies the value will increase by \\$187,670.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d86bd57b",
   "metadata": {},
   "source": [
    "### Root Mean Squared Error (RMSE)\n",
    "\n",
    "The most important performance metric from a data science perspective is root mean squared error, or RMSE. RMSE is the square root of the average squared error in the predicted $\\hat{y}_i$ values:\n",
    "\n",
    "$RMSE = \\sqrt{\\frac{\\sum_{i=1}^n (y_i - \\hat{y}_i)^2}{n}}$\n",
    "\n",
    "This measures the overall accuracy of the model and is a basis for comparing it to other models (including models fit using machine learning techniques).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9415755d",
   "metadata": {},
   "source": [
    "### Residual Standard Error (RSE)\n",
    "\n",
    "Similar to RMSE is the residual standard error, or RSE. In this case we have $p$ predictors, and the RSE is given by:\n",
    "\n",
    "$RSE = \\sqrt{\\frac{\\sum_{i=1}^n (y_i - \\hat{y}_i)^2}{n - p - 1}}$\n",
    "\n",
    "The only difference is that the denominator is the degrees of freedom, as opposed to the number of records.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "726825ac",
   "metadata": {},
   "source": [
    "## Coefficient of Determination (R²)\n",
    "\n",
    "- Another useful metric is the coefficient of determination, also called the R-squared statistic or $R^2$. \n",
    "  - Ranges from 0 to 1 and measures the proportion of variation in the data that is accounted for in the model. The formula for $R^2$ is:\n",
    "\n",
    "$R^2 = 1 - \\frac{\\sum_{i=1}^n (y_i - \\hat{y}_i)^2}{\\sum_{i=1}^n (y_i - \\bar{y})^2}$\n",
    "\n",
    "- The denominator is proportional to the variance of $Y$. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8b9a0c4",
   "metadata": {},
   "source": [
    "### t-Statistic and p-Value\n",
    "\n",
    "* The t-statistic—and its mirror image, the p-value—measures the extent to which a coefficient is \"statistically\n",
    "\n",
    "- Note: Data scientists do not generally get too involved with the interpretation of these statistics, nor with the issue of statistical significance. \n",
    "  * primarily focus on the t-statistic as a useful guide for whether to include a predictor in a model or not. \n",
    "    * High t-statistics (which go with p-values near 0) indicate a predictor should be retained in a model\n",
    "    * Very low t-statistics indicate a predictor could be dropped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3cd5c1d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import statsmodels.api as sm\n",
    "model = sm.OLS(df[outcome], df[predictors])\n",
    "results = model.fit()\n",
    "results.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "d92e6625",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>          <td>price</td>      <th>  R-squared (uncentered):</th>       <td>   0.805</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared (uncentered):</th>  <td>   0.805</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th>           <td>1.784e+04</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Wed, 27 Mar 2024</td> <th>  Prob (F-statistic):</th>            <td>  0.00</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>11:23:18</td>     <th>  Log-Likelihood:    </th>          <td>-3.0216e+05</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td> 21597</td>      <th>  AIC:               </th>           <td>6.043e+05</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td> 21592</td>      <th>  BIC:               </th>           <td>6.044e+05</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     5</td>      <th>                     </th>               <td> </td>     \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>               <td> </td>     \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "        <td></td>           <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>sqft_living15</th> <td>  187.6763</td> <td>    4.018</td> <td>   46.712</td> <td> 0.000</td> <td>  179.801</td> <td>  195.551</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>sqft_lot15</th>    <td>   -0.3110</td> <td>    0.073</td> <td>   -4.245</td> <td> 0.000</td> <td>   -0.455</td> <td>   -0.167</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>bathrooms</th>     <td> 1.228e+05</td> <td> 3527.657</td> <td>   34.806</td> <td> 0.000</td> <td> 1.16e+05</td> <td>  1.3e+05</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>bedrooms</th>      <td>-4.599e+04</td> <td> 2333.164</td> <td>  -19.712</td> <td> 0.000</td> <td>-5.06e+04</td> <td>-4.14e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>grade</th>         <td> 1.059e+04</td> <td> 1413.114</td> <td>    7.497</td> <td> 0.000</td> <td> 7824.622</td> <td> 1.34e+04</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td>20449.378</td> <th>  Durbin-Watson:     </th>  <td>   1.972</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th>  <td> 0.000</td>   <th>  Jarque-Bera (JB):  </th> <td>1907303.443</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>           <td> 4.326</td>   <th>  Prob(JB):          </th>  <td>    0.00</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>       <td>48.218</td>   <th>  Cond. No.          </th>  <td>5.59e+04</td>  \n",
       "</tr>\n",
       "</table><br/><br/>Notes:<br/>[1] R² is computed without centering (uncentered) since the model does not contain a constant.<br/>[2] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[3] The condition number is large, 5.59e+04. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/latex": [
       "\\begin{center}\n",
       "\\begin{tabular}{lclc}\n",
       "\\toprule\n",
       "\\textbf{Dep. Variable:}    &      price       & \\textbf{  R-squared (uncentered):}      &      0.805   \\\\\n",
       "\\textbf{Model:}            &       OLS        & \\textbf{  Adj. R-squared (uncentered):} &      0.805   \\\\\n",
       "\\textbf{Method:}           &  Least Squares   & \\textbf{  F-statistic:       }          &  1.784e+04   \\\\\n",
       "\\textbf{Date:}             & Wed, 27 Mar 2024 & \\textbf{  Prob (F-statistic):}          &      0.00    \\\\\n",
       "\\textbf{Time:}             &     11:23:18     & \\textbf{  Log-Likelihood:    }          & -3.0216e+05  \\\\\n",
       "\\textbf{No. Observations:} &       21597      & \\textbf{  AIC:               }          &  6.043e+05   \\\\\n",
       "\\textbf{Df Residuals:}     &       21592      & \\textbf{  BIC:               }          &  6.044e+05   \\\\\n",
       "\\textbf{Df Model:}         &           5      & \\textbf{                     }          &              \\\\\n",
       "\\textbf{Covariance Type:}  &    nonrobust     & \\textbf{                     }          &              \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "\\begin{tabular}{lcccccc}\n",
       "                        & \\textbf{coef} & \\textbf{std err} & \\textbf{t} & \\textbf{P$> |$t$|$} & \\textbf{[0.025} & \\textbf{0.975]}  \\\\\n",
       "\\midrule\n",
       "\\textbf{sqft\\_living15} &     187.6763  &        4.018     &    46.712  &         0.000        &      179.801    &      195.551     \\\\\n",
       "\\textbf{sqft\\_lot15}    &      -0.3110  &        0.073     &    -4.245  &         0.000        &       -0.455    &       -0.167     \\\\\n",
       "\\textbf{bathrooms}      &    1.228e+05  &     3527.657     &    34.806  &         0.000        &     1.16e+05    &      1.3e+05     \\\\\n",
       "\\textbf{bedrooms}       &   -4.599e+04  &     2333.164     &   -19.712  &         0.000        &    -5.06e+04    &    -4.14e+04     \\\\\n",
       "\\textbf{grade}          &    1.059e+04  &     1413.114     &     7.497  &         0.000        &     7824.622    &     1.34e+04     \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "\\begin{tabular}{lclc}\n",
       "\\textbf{Omnibus:}       & 20449.378 & \\textbf{  Durbin-Watson:     } &      1.972   \\\\\n",
       "\\textbf{Prob(Omnibus):} &    0.000  & \\textbf{  Jarque-Bera (JB):  } & 1907303.443  \\\\\n",
       "\\textbf{Skew:}          &    4.326  & \\textbf{  Prob(JB):          } &       0.00   \\\\\n",
       "\\textbf{Kurtosis:}      &   48.218  & \\textbf{  Cond. No.          } &   5.59e+04   \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "%\\caption{OLS Regression Results}\n",
       "\\end{center}\n",
       "\n",
       "Notes: \\newline\n",
       " [1] R² is computed without centering (uncentered) since the model does not contain a constant. \\newline\n",
       " [2] Standard Errors assume that the covariance matrix of the errors is correctly specified. \\newline\n",
       " [3] The condition number is large, 5.59e+04. This might indicate that there are \\newline\n",
       " strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                                 OLS Regression Results                                \n",
       "=======================================================================================\n",
       "Dep. Variable:                  price   R-squared (uncentered):                   0.805\n",
       "Model:                            OLS   Adj. R-squared (uncentered):              0.805\n",
       "Method:                 Least Squares   F-statistic:                          1.784e+04\n",
       "Date:                Wed, 27 Mar 2024   Prob (F-statistic):                        0.00\n",
       "Time:                        11:23:18   Log-Likelihood:                     -3.0216e+05\n",
       "No. Observations:               21597   AIC:                                  6.043e+05\n",
       "Df Residuals:                   21592   BIC:                                  6.044e+05\n",
       "Df Model:                           5                                                  \n",
       "Covariance Type:            nonrobust                                                  \n",
       "=================================================================================\n",
       "                    coef    std err          t      P>|t|      [0.025      0.975]\n",
       "---------------------------------------------------------------------------------\n",
       "sqft_living15   187.6763      4.018     46.712      0.000     179.801     195.551\n",
       "sqft_lot15       -0.3110      0.073     -4.245      0.000      -0.455      -0.167\n",
       "bathrooms      1.228e+05   3527.657     34.806      0.000    1.16e+05     1.3e+05\n",
       "bedrooms      -4.599e+04   2333.164    -19.712      0.000   -5.06e+04   -4.14e+04\n",
       "grade          1.059e+04   1413.114      7.497      0.000    7824.622    1.34e+04\n",
       "==============================================================================\n",
       "Omnibus:                    20449.378   Durbin-Watson:                   1.972\n",
       "Prob(Omnibus):                  0.000   Jarque-Bera (JB):          1907303.443\n",
       "Skew:                           4.326   Prob(JB):                         0.00\n",
       "Kurtosis:                      48.218   Cond. No.                     5.59e+04\n",
       "==============================================================================\n",
       "\n",
       "Notes:\n",
       "[1] R² is computed without centering (uncentered) since the model does not contain a constant.\n",
       "[2] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[3] The condition number is large, 5.59e+04. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import statsmodels.api as sm\n",
    "model = sm.OLS(df[outcome], df[predictors])\n",
    "results = model.fit()\n",
    "results.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94a3b5ec",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b2145748",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}