2b-birgerk-processByTPOT.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from tpot import TPOTRegressor\n",
    "import sklearn.model_selection\n",
    "import sklearn.datasets\n",
    "import sklearn.metrics\n",
    "from sklearn.preprocessing import Imputer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Process Data with TPOT\n",
    "This demo shows how to use DFS-generated features with [TPOT](https://rhiever.github.io/tpot/).\n",
    "\n",
    "Hint: Because of the features were exported to a CSV-file, there are small differences to [TPOT example](https://rhiever.github.io/tpot/examples/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "feature_matrix = pd.read_csv('./example_data.csv', index_col=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split X and y\n",
    "The imported feature-matrix contains all features we can use.\n",
    "\n",
    "We are going to predict the mean number of items per purchase which is the feature `MEAN(invoices.MEAN(item_purchases.UnitPrice))`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = feature_matrix.drop('MEAN(invoices.MEAN(item_purchases.UnitPrice))', axis=1)\n",
    "y = feature_matrix['MEAN(invoices.MEAN(item_purchases.UnitPrice))']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learn model\n",
    "Learn the model like shown in [TPOT example](https://rhiever.github.io/tpot/examples/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/anaconda3/lib/python3.6/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n",
      "Optimization Progress:  32%|███▏      | 95/300 [02:14<02:50,  1.20pipeline/s] "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generation 1 - Current best internal CV score: 1117.6338186510259\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Optimization Progress:  47%|████▋     | 141/300 [03:44<05:26,  2.05s/pipeline]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generation 2 - Current best internal CV score: 1117.6338186510259\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Optimization Progress:  61%|██████    | 183/300 [05:22<04:04,  2.09s/pipeline]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generation 3 - Current best internal CV score: 1117.6338186510259\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Optimization Progress:  75%|███████▍  | 224/300 [07:56<04:56,  3.91s/pipeline]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generation 4 - Current best internal CV score: 1117.6338186510259\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                              "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generation 5 - Current best internal CV score: 1091.2792884506362\n",
      "\n",
      "Best pipeline: LinearSVR(RobustScaler(input_matrix), LinearSVR__C=15.0, LinearSVR__dual=False, LinearSVR__epsilon=1.0, LinearSVR__loss=squared_epsilon_insensitive, LinearSVR__tol=0.0001)\n",
      "R2 score: 0.976035708232\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r"
     ]
    }
   ],
   "source": [
    "X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, random_state=1)\n",
    "\n",
    "tpot = TPOTRegressor(generations=5, population_size=50, verbosity=2, random_state=42)\n",
    "tpot.fit(X_train, y_train)\n",
    "\n",
    "y_hat = tpot.predict(X_test)\n",
    "print(\"R2 score:\", sklearn.metrics.r2_score(y_test, y_hat))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TPOT-Pipeline\n",
    "TPOT creates python-scripts which contain the ML-pipelines. The following script seems to be the optimal which TPOT found."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tpot.export('./tpot_pipeline.py')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# %load ./tpot_pipeline.py\n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.preprocessing import RobustScaler\n",
    "from sklearn.svm import LinearSVR\n",
    "\n",
    "# NOTE: Make sure that the class is labeled 'class' in the data file\n",
    "tpot_data = np.recfromcsv('PATH/TO/DATA/FILE', delimiter='COLUMN_SEPARATOR', dtype=np.float64)\n",
    "features = np.delete(tpot_data.view(np.float64).reshape(tpot_data.size, -1), tpot_data.dtype.names.index('class'), axis=1)\n",
    "training_features, testing_features, training_target, testing_target = \\\n",
    "    train_test_split(features, tpot_data['class'], random_state=42)\n",
    "\n",
    "exported_pipeline = make_pipeline(\n",
    "    RobustScaler(),\n",
    "    LinearSVR(C=15.0, dual=False, epsilon=1.0, loss=\"squared_epsilon_insensitive\", tol=0.0001)\n",
    ")\n",
    "\n",
    "exported_pipeline.fit(training_features, training_target)\n",
    "results = exported_pipeline.predict(testing_features)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analysing Prediction Result\n",
    "Because of the nearly perfect prediction result, we want to know which features \"important\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(steps=[('robustscaler', RobustScaler(copy=True, quantile_range=(25.0, 75.0), with_centering=True,\n",
       "       with_scaling=True)), ('linearsvr', LinearSVR(C=15.0, dual=False, epsilon=1.0, fit_intercept=True,\n",
       "     intercept_scaling=1.0, loss='squared_epsilon_insensitive',\n",
       "     max_iter=1000, random_state=None, tol=0.0001, verbose=0))])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.preprocessing import RobustScaler\n",
    "from sklearn.svm import LinearSVR\n",
    "\n",
    "training_features, testing_features, training_target, testing_target = \\\n",
    "    train_test_split(X_train, y_train, random_state=42)\n",
    "\n",
    "exported_pipeline = make_pipeline(\n",
    "    RobustScaler(),\n",
    "    LinearSVR(C=15.0, dual=False, epsilon=1.0, loss=\"squared_epsilon_insensitive\", tol=0.0001)\n",
    ")\n",
    "\n",
    "exported_pipeline.fit(training_features, training_target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show the features with each used coefficient:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MEAN(invoices.SUM(item_purchases.UnitPrice))                   41.589479\n",
       "Country = France                                               22.695646\n",
       "Country = United Kingdom                                       13.322339\n",
       "Country = Germany                                               8.935684\n",
       "MEAN(item_purchases.items.SUM(item_purchases.Quantity))         5.995957\n",
       "Country = Portugal                                              5.833740\n",
       "SUM(item_purchases.UnitPrice)                                   2.159102\n",
       "MEAN(item_purchases.items.SUM(item_purchases.UnitPrice))        1.395468\n",
       "MEAN(item_purchases.items.MEAN(item_purchases.Quantity))        1.236994\n",
       "MEAN(invoices.SUM(item_purchases.Quantity))                     0.582053\n",
       "MEAN(item_purchases.UnitPrice)                                  0.410761\n",
       "Country = Belgium                                               0.369748\n",
       "AVG_TIME_BETWEEN(item_purchases)                                0.064857\n",
       "SUM(item_purchases.Quantity)                                    0.053161\n",
       "MEAN(invoices.MEAN(item_purchases.Quantity))                   -0.027889\n",
       "MEAN(item_purchases.Quantity)                                  -0.194407\n",
       "AVG_TIME_BETWEEN(invoices)                                     -0.317734\n",
       "MEAN(item_purchases.items.AVG_TIME_BETWEEN(item_purchases))    -0.382567\n",
       "MEAN(invoices.AVG_TIME_BETWEEN(item_purchases))                -0.486654\n",
       "DAY(first_invoices_time)                                       -0.611945\n",
       "COUNT(item_purchases)                                          -0.614657\n",
       "MEAN(item_purchases.items.MEAN(item_purchases.UnitPrice))      -1.144358\n",
       "COUNT(invoices)                                                -1.561834\n",
       "Country = Switzerland                                          -1.578211\n",
       "Country = unknown                                              -4.502102\n",
       "Country = Spain                                                -4.712754\n",
       "Country = Finland                                              -5.452420\n",
       "Country = Norway                                               -7.533844\n",
       "MEAN(item_purchases.items.COUNT(item_purchases))               -8.076588\n",
       "Country = Italy                                               -27.377826\n",
       "MEAN(invoices.COUNT(item_purchases))                          -33.242062\n",
       "dtype: float64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "important_coefs = pd.Series(data=exported_pipeline.steps[1][1].coef_, index=X.columns)\n",
    "sorted_coef = important_coefs.sort_values(ascending=False)\n",
    "\n",
    "sorted_coef"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Seems like there are some important features. Let us focus on top 6:"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}