Chapter_Forecasting_Performance_Measures.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forecast Error( or Residual Forecast Error)\n",
    "The forecast error is calculated as the expected value minus the predicted value. This is called the residual error of the prediction.\n",
    "__forecast error = expected value - predicted value__.\n",
    "\n",
    "The units of the forecast error are the same as the units of the prediction. A forecast error of zero indicates no error, or perfect skill for that forecast.\n",
    "\n",
    "### Mean Forecast Error( or  Forecast Bias)\n",
    "It is the average of the forecast error\n",
    "__mean forecast error = mean(forecast error)__\n",
    "\n",
    "The units of the forecast bias are the same as the units of the predictions. A forecast bias of zero, or a very small number near zero, shows an unbiased model.\n",
    "\n",
    "### Mean Absolute Error( MAE)\n",
    "The mean absolute error, or MAE, is calculated as the average of the forecast error values, where all of the forecast values are forced to be positive.\n",
    "\n",
    "__mean absolute error = mean(abs(forecast error))__\n",
    "\n",
    "These error values are in the original units of the predicted values. A mean absolute error of zero indicates no error.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mean Squared Error ( MSE)\n",
    "\n",
    "The mean squared error, or MSE, is calculated as the average of the squared forecast error values. __Squaring__ the forecast error values forces them to be positive; it also has the effect of __putting more weight on large errors__. Very large or outlier forecast errors are squared, which in turn has the effect of dragging the mean of the squared forecast errors out resulting in a larger mean squared error score. In effect, the score gives worse performance to those models that make large wrong forecasts.\n",
    "\n",
    "__mean squared error = mean(forecast error^2)__\n",
    "\n",
    "The error values are in squared units of the predicted values. A mean squared error of zero indicates perfect skill, or no error.\n",
    "\n",
    "### Root Mean Squared Error (RMSE)\n",
    "\n",
    "\n",
    "The mean squared error described above is in the squared units of the predictions. It can be transformed back into the original units of the predictions by taking the square root of the mean squared error score. This is called the root mean squared error, or RMSE.\n",
    "\n",
    "__rmse = sqrt(mean_squared_error)__\n",
    "\n",
    "The RMES error values are in the same units as the predictions. As with the mean squared error, an RMSE of zero indicates no error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean forecast error :-0.026\n",
      "Mean absoulte error :0.335\n",
      "Mean squared error (MSE) :0.168\n",
      "Root Mean forecast error (RMSE) :0.409\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.metrics import mean_absolute_error,mean_squared_error\n",
    "\n",
    "observed  = np.random.random(1000)\n",
    "prediction = np.random.random(1000)\n",
    "\n",
    "resid = observed - prediction\n",
    "mean_forecast_error = np.mean(resid)\n",
    "print(\"Mean forecast error :{:.3f}\".format(mean_forecast_error))\n",
    "print(\"Mean absoulte error :{:.3f}\".format(mean_absolute_error(observed,prediction)))\n",
    "print(\"Mean squared error (MSE) :{:.3f}\".format(mean_squared_error(observed,prediction)))\n",
    "print(\"Root Mean forecast error (RMSE) :{:.3f}\".format(np.sqrt(mean_squared_error(observed,prediction))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "- How to calculate forecast residual error and how to estimate the bias in a list of forecasts.\n",
    "- How to calculate mean absolute forecast error to describe error in the same units as the predictions.\n",
    "- How to calculate the widely used mean squared error and root mean squared error for forecasts."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}