module-2-linear-classifier-assignment-blank.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting sentiment from product reviews\n",
    "\n",
    "\n",
    "The goal of this first notebook is to explore logistic regression and feature engineering with existing GraphLab functions.\n",
    "\n",
    "In this notebook you will use product review data from Amazon.com to predict whether the sentiments about a product (from its reviews) are positive or negative.\n",
    "\n",
    "* Use SFrames to do some feature engineering\n",
    "* Train a logistic regression model to predict the sentiment of product reviews.\n",
    "* Inspect the weights (coefficients) of a trained logistic regression model.\n",
    "* Make a prediction (both class and probability) of sentiment for a new product review.\n",
    "* Given the logistic regression weights, predictors and ground truth labels, write a function to compute the **accuracy** of the model.\n",
    "* Inspect the coefficients of the logistic regression model and interpret their meanings.\n",
    "* Compare multiple logistic regression models.\n",
    "\n",
    "Let's get started!\n",
    "    \n",
    "## Fire up GraphLab Create"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make sure you have the latest version of GraphLab Create."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from __future__ import division\n",
    "import graphlab\n",
    "import math\n",
    "import string\n",
    "import numpy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data preparation\n",
    "\n",
    "We will use a dataset consisting of baby product reviews on Amazon.com."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] graphlab.cython.cy_server: GraphLab Create v2.1 started. Logging: /tmp/graphlab_server_1501778542.log\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This non-commercial license of GraphLab Create for academic use is assigned to luv.aggarwal@nsitonline.in and will expire on March 07, 2018.\n"
     ]
    }
   ],
   "source": [
    "products = graphlab.SFrame('amazon_baby.gl/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let us see a preview of what the dataset looks like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">name</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">review</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">rating</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Planetwise Flannel Wipes</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">These flannel wipes are<br>OK, but in my opinion ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Planetwise Wipe Pouch</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">it came early and was not<br>disappointed. i love ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Annas Dream Full Quilt<br>with 2 Shams ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Very soft and comfortable<br>and warmer than it ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Stop Pacifier Sucking<br>without tears with ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">This is a product well<br>worth the purchase.  I ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Stop Pacifier Sucking<br>without tears with ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">All of my kids have cried<br>non-stop when I tried to ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Stop Pacifier Sucking<br>without tears with ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">When the Binky Fairy came<br>to our house, we didn't ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">A Tale of Baby's Days<br>with Peter Rabbit ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Lovely book, it's bound<br>tightly so you may no ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Baby Tracker&amp;reg; - Daily<br>Childcare Journal, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Perfect for new parents.<br>We were able to keep ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Baby Tracker&amp;reg; - Daily<br>Childcare Journal, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">A friend of mine pinned<br>this product on Pinte ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Baby Tracker&amp;reg; - Daily<br>Childcare Journal, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">This has been an easy way<br>for my nanny to record ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4.0</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[183531 rows x 3 columns]<br/>Note: Only the head of the SFrame is printed.<br/>You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tname\tstr\n",
       "\treview\tstr\n",
       "\trating\tfloat\n",
       "\n",
       "Rows: 183531\n",
       "\n",
       "Data:\n",
       "+-------------------------------+-------------------------------+--------+\n",
       "|              name             |             review            | rating |\n",
       "+-------------------------------+-------------------------------+--------+\n",
       "|    Planetwise Flannel Wipes   | These flannel wipes are OK... |  3.0   |\n",
       "|     Planetwise Wipe Pouch     | it came early and was not ... |  5.0   |\n",
       "| Annas Dream Full Quilt wit... | Very soft and comfortable ... |  5.0   |\n",
       "| Stop Pacifier Sucking with... | This is a product well wor... |  5.0   |\n",
       "| Stop Pacifier Sucking with... | All of my kids have cried ... |  5.0   |\n",
       "| Stop Pacifier Sucking with... | When the Binky Fairy came ... |  5.0   |\n",
       "| A Tale of Baby's Days with... | Lovely book, it's bound ti... |  4.0   |\n",
       "| Baby Tracker&reg; - Daily ... | Perfect for new parents. W... |  5.0   |\n",
       "| Baby Tracker&reg; - Daily ... | A friend of mine pinned th... |  5.0   |\n",
       "| Baby Tracker&reg; - Daily ... | This has been an easy way ... |  4.0   |\n",
       "+-------------------------------+-------------------------------+--------+\n",
       "[183531 rows x 3 columns]\n",
       "Note: Only the head of the SFrame is printed.\n",
       "You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns."
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "products"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build the word count vector for each review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us explore a specific example of a baby product.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'name': 'The First Years Massaging Action Teether',\n",
       " 'rating': 5.0,\n",
       " 'review': 'A favorite in our house!'}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "products[269]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we will perform 2 simple data transformations:\n",
    "\n",
    "1. Remove punctuation using [Python's built-in](https://docs.python.org/2/library/string.html) string functionality.\n",
    "2. Transform the reviews into word-counts.\n",
    "\n",
    "**Aside**. In this notebook, we remove all punctuations for the sake of simplicity. A smarter approach to punctuations would preserve phrases such as \"I'd\", \"would've\", \"hadn't\" and so forth. See [this page](https://www.cis.upenn.edu/~treebank/tokenization.html) for an example of smart handling of punctuations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def remove_punctuation(text):\n",
    "    import string\n",
    "    return text.translate(None, string.punctuation) \n",
    "\n",
    "review_without_punctuation = products['review'].apply(remove_punctuation)\n",
    "products['word_count'] = graphlab.text_analytics.count_words(review_without_punctuation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let us explore what the sample example above looks like after these 2 transformations. Here, each entry in the **word_count** column is a dictionary where the key is the word and the value is a count of the number of times the word occurs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 1, 'favorite': 1, 'house': 1, 'in': 1, 'our': 1}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "products[269]['word_count']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extract sentiments\n",
    "\n",
    "We will **ignore** all reviews with *rating = 3*, since they tend to have a neutral sentiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "166752"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "products = products[products['rating'] != 3]\n",
    "len(products)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we will assign reviews with a rating of 4 or higher to be *positive* reviews, while the ones with rating of 2 or lower are *negative*. For the sentiment column, we use +1 for the positive class label and -1 for the negative class label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">name</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">review</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">rating</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">word_count</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sentiment</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Planetwise Wipe Pouch</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">it came early and was not<br>disappointed. i love ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'and': 3, 'love': 1,<br>'it': 3, 'highly': 1, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Annas Dream Full Quilt<br>with 2 Shams ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Very soft and comfortable<br>and warmer than it ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'and': 2, 'quilt': 1,<br>'it': 1, 'comfortable': ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Stop Pacifier Sucking<br>without tears with ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">This is a product well<br>worth the purchase.  I ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'and': 3, 'ingenious':<br>1, 'love': 2, 'what': 1, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Stop Pacifier Sucking<br>without tears with ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">All of my kids have cried<br>non-stop when I tried to ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'and': 2, 'all': 2,<br>'help': 1, 'cried': 1, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Stop Pacifier Sucking<br>without tears with ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">When the Binky Fairy came<br>to our house, we didn't ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'and': 2, 'this': 2,<br>'her': 1, 'help': 2, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">A Tale of Baby's Days<br>with Peter Rabbit ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Lovely book, it's bound<br>tightly so you may no ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'shop': 1, 'noble': 1,<br>'is': 1, 'it': 1, 'as': ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Baby Tracker&amp;reg; - Daily<br>Childcare Journal, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Perfect for new parents.<br>We were able to keep ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'and': 2, 'all': 1,<br>'right': 1, 'had': 1, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Baby Tracker&amp;reg; - Daily<br>Childcare Journal, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">A friend of mine pinned<br>this product on Pinte ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'and': 1, 'fantastic':<br>1, 'help': 1, 'give': 1, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Baby Tracker&amp;reg; - Daily<br>Childcare Journal, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">This has been an easy way<br>for my nanny to record ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'all': 1, 'standarad':<br>1, 'another': 1, 'when': ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Baby Tracker&amp;reg; - Daily<br>Childcare Journal, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">I love this journal and<br>our nanny uses it ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'all': 2, 'nannys': 1,<br>'just': 1, 'food': 1, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[166752 rows x 5 columns]<br/>Note: Only the head of the SFrame is printed.<br/>You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tname\tstr\n",
       "\treview\tstr\n",
       "\trating\tfloat\n",
       "\tword_count\tdict\n",
       "\tsentiment\tint\n",
       "\n",
       "Rows: 166752\n",
       "\n",
       "Data:\n",
       "+-------------------------------+-------------------------------+--------+\n",
       "|              name             |             review            | rating |\n",
       "+-------------------------------+-------------------------------+--------+\n",
       "|     Planetwise Wipe Pouch     | it came early and was not ... |  5.0   |\n",
       "| Annas Dream Full Quilt wit... | Very soft and comfortable ... |  5.0   |\n",
       "| Stop Pacifier Sucking with... | This is a product well wor... |  5.0   |\n",
       "| Stop Pacifier Sucking with... | All of my kids have cried ... |  5.0   |\n",
       "| Stop Pacifier Sucking with... | When the Binky Fairy came ... |  5.0   |\n",
       "| A Tale of Baby's Days with... | Lovely book, it's bound ti... |  4.0   |\n",
       "| Baby Tracker&reg; - Daily ... | Perfect for new parents. W... |  5.0   |\n",
       "| Baby Tracker&reg; - Daily ... | A friend of mine pinned th... |  5.0   |\n",
       "| Baby Tracker&reg; - Daily ... | This has been an easy way ... |  4.0   |\n",
       "| Baby Tracker&reg; - Daily ... | I love this journal and ou... |  4.0   |\n",
       "+-------------------------------+-------------------------------+--------+\n",
       "+-------------------------------+-----------+\n",
       "|           word_count          | sentiment |\n",
       "+-------------------------------+-----------+\n",
       "| {'and': 3, 'love': 1, 'it'... |     1     |\n",
       "| {'and': 2, 'quilt': 1, 'it... |     1     |\n",
       "| {'and': 3, 'ingenious': 1,... |     1     |\n",
       "| {'and': 2, 'all': 2, 'help... |     1     |\n",
       "| {'and': 2, 'this': 2, 'her... |     1     |\n",
       "| {'shop': 1, 'noble': 1, 'i... |     1     |\n",
       "| {'and': 2, 'all': 1, 'righ... |     1     |\n",
       "| {'and': 1, 'fantastic': 1,... |     1     |\n",
       "| {'all': 1, 'standarad': 1,... |     1     |\n",
       "| {'all': 2, 'nannys': 1, 'j... |     1     |\n",
       "+-------------------------------+-----------+\n",
       "[166752 rows x 5 columns]\n",
       "Note: Only the head of the SFrame is printed.\n",
       "You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns."
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "products['sentiment'] = products['rating'].apply(lambda rating : +1 if rating > 3 else -1)\n",
    "products"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can see that the dataset contains an extra column called **sentiment** which is either positive (+1) or negative (-1)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split data into training and test sets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's perform a train/test split with 80% of the data in the training set and 20% of the data in the test set. We use `seed=1` so that everyone gets the same result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "133416\n",
      "33336\n"
     ]
    }
   ],
   "source": [
    "train_data, test_data = products.random_split(.8, seed=1)\n",
    "print len(train_data)\n",
    "print len(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train a sentiment classifier with logistic regression\n",
    "\n",
    "We will now use logistic regression to create a sentiment classifier on the training data. This model will use the column **word_count** as a feature and the column **sentiment** as the target. We will use `validation_set=None` to obtain same results as everyone else.\n",
    "\n",
    "**Note:** This line may take 1-2 minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre>Logistic regression:</pre>"
      ],
      "text/plain": [
       "Logistic regression:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 133416</pre>"
      ],
      "text/plain": [
       "Number of examples          : 133416"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of classes           : 2</pre>"
      ],
      "text/plain": [
       "Number of classes           : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of feature columns   : 1</pre>"
      ],
      "text/plain": [
       "Number of feature columns   : 1"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 121712</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 121712"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients    : 121713</pre>"
      ],
      "text/plain": [
       "Number of coefficients    : 121713"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting L-BFGS</pre>"
      ],
      "text/plain": [
       "Starting L-BFGS"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Step size | Elapsed Time | Training-accuracy |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Step size | Elapsed Time | Training-accuracy |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 5        | 0.000002  | 3.822749     | 0.840754          |</pre>"
      ],
      "text/plain": [
       "| 1         | 5        | 0.000002  | 3.822749     | 0.840754          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 2         | 9        | 3.000000  | 6.557835     | 0.931350          |</pre>"
      ],
      "text/plain": [
       "| 2         | 9        | 3.000000  | 6.557835     | 0.931350          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 3         | 10       | 3.000000  | 7.441595     | 0.882046          |</pre>"
      ],
      "text/plain": [
       "| 3         | 10       | 3.000000  | 7.441595     | 0.882046          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 4         | 11       | 3.000000  | 8.057402     | 0.954076          |</pre>"
      ],
      "text/plain": [
       "| 4         | 11       | 3.000000  | 8.057402     | 0.954076          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 5         | 12       | 3.000000  | 8.697718     | 0.960964          |</pre>"
      ],
      "text/plain": [
       "| 5         | 12       | 3.000000  | 8.697718     | 0.960964          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 6         | 13       | 3.000000  | 9.410325     | 0.975033          |</pre>"
      ],
      "text/plain": [
       "| 6         | 13       | 3.000000  | 9.410325     | 0.975033          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>TERMINATED: Terminated due to numerical difficulties.</pre>"
      ],
      "text/plain": [
       "TERMINATED: Terminated due to numerical difficulties."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>This model may not be ideal. To improve it, consider doing one of the following:\n",
       "(a) Increasing the regularization.\n",
       "(b) Standardizing the input data.\n",
       "(c) Removing highly correlated features.\n",
       "(d) Removing `inf` and `NaN` values in the training data.</pre>"
      ],
      "text/plain": [
       "This model may not be ideal. To improve it, consider doing one of the following:\n",
       "(a) Increasing the regularization.\n",
       "(b) Standardizing the input data.\n",
       "(c) Removing highly correlated features.\n",
       "(d) Removing `inf` and `NaN` values in the training data."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sentiment_model = graphlab.logistic_classifier.create(train_data,\n",
    "                                                      target = 'sentiment',\n",
    "                                                      features=['word_count'],\n",
    "                                                      validation_set=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Class                          : LogisticClassifier\n",
       "\n",
       "Schema\n",
       "------\n",
       "Number of coefficients         : 121713\n",
       "Number of examples             : 133416\n",
       "Number of classes              : 2\n",
       "Number of feature columns      : 1\n",
       "Number of unpacked features    : 121712\n",
       "\n",
       "Hyperparameters\n",
       "---------------\n",
       "L1 penalty                     : 0.0\n",
       "L2 penalty                     : 0.01\n",
       "\n",
       "Training Summary\n",
       "----------------\n",
       "Solver                         : lbfgs\n",
       "Solver iterations              : 6\n",
       "Solver status                  : TERMINATED: Terminated due to numerical difficulties.\n",
       "Training time (sec)            : 10.0205\n",
       "\n",
       "Settings\n",
       "--------\n",
       "Log-likelihood                 : inf\n",
       "\n",
       "Highest Positive Coefficients\n",
       "-----------------------------\n",
       "word_count[mobileupdate]       : 41.9847\n",
       "word_count[placeid]            : 41.7354\n",
       "word_count[labelbox]           : 41.151\n",
       "word_count[httpwwwamazoncomreviewrhgg6qp7tdnhbrefcmcrprcmtieutf8asinb00318cla0nodeid] : 40.0454\n",
       "word_count[knobskeeping]       : 36.2091\n",
       "\n",
       "Lowest Negative Coefficients\n",
       "----------------------------\n",
       "word_count[probelm]            : -44.9283\n",
       "word_count[impulsejeep]        : -43.081\n",
       "word_count[infantsyoung]       : -39.5945\n",
       "word_count[cutereditafter]     : -35.6875\n",
       "word_count[avacado]            : -35.0542"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentiment_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Aside**. You may get a warning to the effect of \"Terminated due to numerical difficulties --- this model may not be ideal\". It means that the quality metric (to be covered in Module 3) failed to improve in the last iteration of the run. The difficulty arises as the sentiment model puts too much weight on extremely rare words. A way to rectify this is to apply regularization, to be covered in Module 4. Regularization lessens the effect of extremely rare words. For the purpose of this assignment, however, please proceed with the model above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have fitted the model, we can extract the weights (coefficients) as an SFrame as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['name', 'index', 'class', 'value', 'stderr']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weights = sentiment_model.coefficients\n",
    "weights.column_names()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a total of `121713` coefficients in the model. Recall from the lecture that positive weights $w_j$ correspond to weights that cause positive sentiment, while negative weights correspond to negative sentiment. \n",
    "\n",
    "Fill in the following block of code to calculate how many *weights* are positive ( >= 0). (**Hint**: The `'value'` column in SFrame *weights* must be positive ( >= 0))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of positive weights: 68419 \n",
      "Number of negative weights: 53294 \n"
     ]
    }
   ],
   "source": [
    "num_positive_weights = len(weights[weights['value'] >= 0])\n",
    "num_negative_weights = len(weights[weights['value'] < 0])\n",
    "\n",
    "print \"Number of positive weights: %s \" % num_positive_weights\n",
    "print \"Number of negative weights: %s \" % num_negative_weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quiz Question:** How many weights are >= 0?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making predictions with logistic regression\n",
    "\n",
    "Now that a model is trained, we can make predictions on the **test data**. In this section, we will explore this in the context of 3 examples in the test dataset.  We refer to this set of 3 examples as the **sample_test_data**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5.0, 2.0, 1.0]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">name</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">review</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">rating</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">word_count</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sentiment</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Our Baby Girl Memory Book</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Absolutely love it and<br>all of the Scripture in ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'and': 2, 'all': 1,<br>'love': 1, 'purchased': ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Wall Decor Removable<br>Decal Sticker - Colorful ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Would not purchase again<br>or recommend. The decals ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'and': 1, 'would': 2,<br>'almost': 1, 'decals' ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">New Style Trailing Cherry<br>Blossom Tree Decal ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Was so excited to get<br>this product for my baby ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1.0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{'all': 1, 'money': 1,<br>'into': 1, 'back': 1, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-1</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[3 rows x 5 columns]<br/>\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tname\tstr\n",
       "\treview\tstr\n",
       "\trating\tfloat\n",
       "\tword_count\tdict\n",
       "\tsentiment\tint\n",
       "\n",
       "Rows: 3\n",
       "\n",
       "Data:\n",
       "+-------------------------------+-------------------------------+--------+\n",
       "|              name             |             review            | rating |\n",
       "+-------------------------------+-------------------------------+--------+\n",
       "|   Our Baby Girl Memory Book   | Absolutely love it and all... |  5.0   |\n",
       "| Wall Decor Removable Decal... | Would not purchase again o... |  2.0   |\n",
       "| New Style Trailing Cherry ... | Was so excited to get this... |  1.0   |\n",
       "+-------------------------------+-------------------------------+--------+\n",
       "+-------------------------------+-----------+\n",
       "|           word_count          | sentiment |\n",
       "+-------------------------------+-----------+\n",
       "| {'and': 2, 'all': 1, 'love... |     1     |\n",
       "| {'and': 1, 'would': 2, 'al... |     -1    |\n",
       "| {'all': 1, 'money': 1, 'in... |     -1    |\n",
       "+-------------------------------+-----------+\n",
       "[3 rows x 5 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_test_data = test_data[10:13]\n",
    "print sample_test_data['rating']\n",
    "sample_test_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's dig deeper into the first row of the **sample_test_data**. Here's the full review:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Absolutely love it and all of the Scripture in it.  I purchased the Baby Boy version for my grandson when he was born and my daughter-in-law was thrilled to receive the same book again.'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_test_data[0]['review']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That review seems pretty positive.\n",
    "\n",
    "Now, let's see what the next row of the **sample_test_data** looks like. As we could guess from the sentiment (-1), the review is quite negative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Would not purchase again or recommend. The decals were thick almost plastic like and were coming off the wall as I was applying them! The would NOT stick! Literally stayed stuck for about 5 minutes then started peeling off.'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_test_data[1]['review']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now make a **class** prediction for the **sample_test_data**. The `sentiment_model` should predict **+1** if the sentiment is positive and **-1** if the sentiment is negative. Recall from the lecture that the **score** (sometimes called **margin**) for the logistic regression model  is defined as:\n",
    "\n",
    "$$\n",
    "\\mbox{score}_i = \\mathbf{w}^T h(\\mathbf{x}_i)\n",
    "$$ \n",
    "\n",
    "where $h(\\mathbf{x}_i)$ represents the features for example $i$.  We will write some code to obtain the **scores** using GraphLab Create. For each row, the **score** (or margin) is a number in the range **[-inf, inf]**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[6.734619727059404, -5.734130996760414, -14.66846040446854]\n"
     ]
    }
   ],
   "source": [
    "scores = sentiment_model.predict(sample_test_data, output_type='margin')\n",
    "print scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predicting sentiment\n",
    "\n",
    "These scores can be used to make class predictions as follows:\n",
    "\n",
    "$$\n",
    "\\hat{y} = \n",
    "\\left\\{\n",
    "\\begin{array}{ll}\n",
    "      +1 & \\mathbf{w}^T h(\\mathbf{x}_i) > 0 \\\\\n",
    "      -1 & \\mathbf{w}^T h(\\mathbf{x}_i) \\leq 0 \\\\\n",
    "\\end{array} \n",
    "\\right.\n",
    "$$\n",
    "\n",
    "Using scores, write code to calculate $\\hat{y}$, the class predictions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, -1, -1]\n"
     ]
    }
   ],
   "source": [
    "print scores.apply(lambda scores : +1 if scores > 0 else -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the following code to verify that the class predictions obtained by your calculations are the same as that obtained from GraphLab Create."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Class predictions according to GraphLab Create:\n",
      "[1, -1, -1]\n"
     ]
    }
   ],
   "source": [
    "print \"Class predictions according to GraphLab Create:\" \n",
    "print sentiment_model.predict(sample_test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Checkpoint**: Make sure your class predictions match with the one obtained from GraphLab Create.\n",
    "\n",
    "### Probability predictions\n",
    "\n",
    "Recall from the lectures that we can also calculate the probability predictions from the scores using:\n",
    "$$\n",
    "P(y_i = +1 | \\mathbf{x}_i,\\mathbf{w}) = \\frac{1}{1 + \\exp(-\\mathbf{w}^T h(\\mathbf{x}_i))}.\n",
    "$$\n",
    "\n",
    "Using the variable **scores** calculated previously, write code to calculate the probability that a sentiment is positive using the above formula. For each row, the probabilities should be a number in the range **[0, 1]**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.9988123848377198, 0.0032232681818001783, 4.2615579966553296e-07]\n"
     ]
    }
   ],
   "source": [
    "print scores.apply(lambda scores : 1 / ( 1 + numpy.exp( - scores ) ) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Checkpoint**: Make sure your probability predictions match the ones obtained from GraphLab Create."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Class predictions according to GraphLab Create:\n",
      "[0.99881238483772, 0.0032232681818001787, 4.261557996655327e-07]\n"
     ]
    }
   ],
   "source": [
    "print \"Class predictions according to GraphLab Create:\" \n",
    "print sentiment_model.predict(sample_test_data, output_type='probability')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Quiz Question:** Of the three data points in **sample_test_data**, which one (first, second, or third) has the **lowest probability** of being classified as a positive review?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Find the most positive (and negative) review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now turn to examining the full test dataset, **test_data**, and use GraphLab Create to form predictions on all of the test data points for faster performance.\n",
    "\n",
    "Using the `sentiment_model`, find the 20 reviews in the entire **test_data** with the **highest probability** of being classified as a **positive review**. We refer to these as the \"most positive reviews.\"\n",
    "\n",
    "To calculate these top-20 reviews, use the following steps:\n",
    "1.  Make probability predictions on **test_data** using the `sentiment_model`. (**Hint:** When you call `.predict` to make predictions on the test data, use option `output_type='probability'` to output the probability rather than just the most likely class.)\n",
    "2.  Sort the data according to those predictions and pick the top 20. (**Hint:** You can use the `.topk` method on an SFrame to find the top k rows sorted according to the value of a specified column.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------------------------------+-------------------------------+--------+\n",
      "|              name              |             review            | rating |\n",
      "+--------------------------------+-------------------------------+--------+\n",
      "| Cozy Infant Remote Wireles...  | Pan And Tilt : Works fast ... |  5.0   |\n",
      "| Combi Kobuk Air-Thru, Licorice | So, I do need to begin wit... |  5.0   |\n",
      "| Quinny 2012 Buzz Stroller,...  | Choice - Quinny Buzz 2011 ... |  4.0   |\n",
      "| RECARO ProRIDE Convertible...  | We recently moved our son ... |  5.0   |\n",
      "| RECARO ProRIDE Convertible...  | We just got this car seat ... |  5.0   |\n",
      "| Delta Universal 6 Drawer D...  | I spent weeks trying to fi... |  5.0   |\n",
      "| Britax Pinnacle 90 Booster...  | Hands down my favorite fea... |  5.0   |\n",
      "| Gerber First Essential Cle...  | I wanted to write a review... |  4.0   |\n",
      "| Freemie Hands-Free Conceal...  | I absolutely love this pro... |  5.0   |\n",
      "|    Display Box - 30 singles    | I'm kind of OCD about show... |  5.0   |\n",
      "|  BABYBJORN Potty Chair - Red   | Our family is just startin... |  5.0   |\n",
      "| CTA Digital 2-in-1 iPotty ...  | I'll just say in advance t... |  5.0   |\n",
      "| We Sell Mats 36 Sq Ft Alph...  | They showed up in good con... |  5.0   |\n",
      "| BRICA Baby In-Sight Magica...  | First off, let me start by... |  4.0   |\n",
      "|  Fisher-Price Deluxe Jumperoo  | I had already decided that... |  5.0   |\n",
      "| Rainy Day Indoor Playgroun...  | This is the #1 played with... |  5.0   |\n",
      "| RECARO ProRIDE Convertible...  | this is super easy to inst... |  5.0   |\n",
      "| The Nesting Pillow - Organ...  | Lactation consultants told... |  5.0   |\n",
      "| Graco Pack 'n Play Element...  | My husband and I assembled... |  4.0   |\n",
      "| Orbit Baby Stroller Travel...  | My daughter was born a pre... |  5.0   |\n",
      "| We Sell Mats Anti-Fatigue ...  | We bought 10 packs of thes... |  5.0   |\n",
      "| Maxi-Cosi Priori Convertib...  | We bought the Maxi Cosi Ca... |  5.0   |\n",
      "| OXO Tot Infant On-The-Go G...  | OXO products have always p... |  5.0   |\n",
      "| Cosco Scenera Convertible ...  | Background: My husband and... |  4.0   |\n",
      "| Emily Green 6&quot; Bowl, ...  | I have 4 complete sets of ... |  5.0   |\n",
      "| Cardinal Gates Window Ward...  | We have wood / aluminum cl... |  5.0   |\n",
      "| Chicco KeyFit 22 Cortina T...  | I am greatly impressed wit... |  5.0   |\n",
      "| Fisher-Price Zen Collectio...  | We started out with a trav... |  5.0   |\n",
      "| Medela CSF Bags - Economy ...  | I've had absolutely no pro... |  4.0   |\n",
      "| Fisher-Price Cradle 'N Swi...  | I got this swing when my s... |  5.0   |\n",
      "| Regalo Extra Wide 58 Inch ...  | I read several of the revi... |  5.0   |\n",
      "| Joovy Caboose Ultralight S...  | We love the joovy caboose ... |  5.0   |\n",
      "| The First Years Hands Free...  | I have a relatively new do... |  5.0   |\n",
      "| Fisher-Price Space Saver H...  | I ordered this to take to ... |  5.0   |\n",
      "| Stokke Scoot Stroller - Li...  | We've been using the Stokk... |  5.0   |\n",
      "| Skip Hop Studio Diaper Bag...  | i love this diaper bag! sk... |  5.0   |\n",
      "| Safety 1st Complete Air Pr...  | We got this car seat for o... |  4.0   |\n",
      "| Kushies &quot;On the Go&qu...  | I gave this four stars bec... |  4.0   |\n",
      "| Philips AVENT Digital Vide...  | The Phillips AVENT is by f... |  5.0   |\n",
      "| Safe Traffic System Ride S...  | I'd been looking around fo... |  5.0   |\n",
      "| Contours Options Tandem II...  | Your journey to locate a s... |  5.0   |\n",
      "| Skip Hop Grand Central Dia...  | I have had 4 Skip Hop bags... |  4.0   |\n",
      "| RECARO Performance RIDE Co...  | This is my 3rd Recaro seat... |  5.0   |\n",
      "| North States Supergate Ext...  | I ordered two of these gat... |  5.0   |\n",
      "| The First Years Compass Pa...  | We first bought a much che... |  5.0   |\n",
      "| JJ Cole Original Infant Bu...  | Love this for the stroller... |  5.0   |\n",
      "| Baby Merlin's Magic Sleeps...  | I'm a second time mom.  I ... |  5.0   |\n",
      "| Britax Frontier 85 Combina...  | I did a LOT of research be... |  5.0   |\n",
      "| Joovy Groove Ultralight Um...  | I've put an embarrassing a... |  5.0   |\n",
      "| Graco Glider LX Gliding Sw...  | With my first two children... |  5.0   |\n",
      "| Parent Units 2 Pack Equipm...  | These straps affix to the ... |  4.0   |\n",
      "| Angel Dear Pair and a Spar...  | When my son was born, a de... |  5.0   |\n",
      "| HALO SleepSack 5-Piece Bum...  | I recently received this s... |  5.0   |\n",
      "| Britax Marathon in Ashley ...  | We used the Roundabout wit... |  5.0   |\n",
      "| Infantino Roomy Back Seat ...  | Not long after you have yo... |  4.0   |\n",
      "| Bumble Bags Sarah Shopping...  | I wanted a shopping cart c... |  5.0   |\n",
      "| Wimmer-Ferguson Double-Fea...  | After years of being off t... |  5.0   |\n",
      "| The First Years True Fit S...  | I haven't had a chance to ... |  5.0   |\n",
      "| Safety 1st Complete Air 70...  | I bought this model to hau... |  4.0   |\n",
      "| babyletto Hudson 3 in 1 Co...  | My wife and I ordered this... |  4.0   |\n",
      "| Infant Optics DXR-5 2.4 GH...  | We got this monitor a few ... |  4.0   |\n",
      "| Graco Quattro Tour Duo Str...  | I purchased this double st... |  5.0   |\n",
      "| aden + anais Rayon from Ba...  | I almost didn't buy these ... |  5.0   |\n",
      "| Summer Infant Spectra Trav...  | I was given this product b... |  5.0   |\n",
      "| BOB Revolution SE Single S...  | I spent several weeks rese... |  5.0   |\n",
      "| Britax 2013 B-Agile Stroll...  | So, when it comes to strol... |  5.0   |\n",
      "| The First Years Hands Free...  | My house was built in 1899... |  5.0   |\n",
      "| Fisher-Price Discover n' G...  | Con: Has straps to velcro ... |  5.0   |\n",
      "| BabyPlus Prenatal Educatio...  | After reading all of the g... |  2.0   |\n",
      "| Summer Infant Metal Expans...  | We are now my mother's car... |  5.0   |\n",
      "| EZ Squeezees Refillable Fo...  | I purchased them directly ... |  5.0   |\n",
      "| The First Years Compass De...  | I did a lot of research fo... |  5.0   |\n",
      "|   Infantino Squeeze Station    | I love this product - it i... |  5.0   |\n",
      "| SOHO Classic Deluxe Baby d...  | I was a bit worried about ... |  5.0   |\n",
      "| Boba 3G Baby Carrier, Mont...  | Just bought this Boba 3G c... |  5.0   |\n",
      "| Fisher-Price Rainforest Me...  | I love this baby gym, and ... |  5.0   |\n",
      "| North States Supergate Eas...  | This gate has brought a li... |  5.0   |\n",
      "| Pigeon Nail Scissor (New B...  | I am very happy with this ... |  5.0   |\n",
      "| Door Monkey, Childproof Do...  | These are the BEST safety ... |  5.0   |\n",
      "| Boon Lawn Countertop Dryin...  | I saw this and was a littl... |  5.0   |\n",
      "| The First Years Indigo Str...  | I really have no complaint... |  5.0   |\n",
      "| Bottle Snugglers Feeding T...  | With two other kids beside... |  4.0   |\n",
      "| bumGenius One-Size Hook &a...  | I have had many friends wh... |  5.0   |\n",
      "| Toy Hammock - Deluxe Jumbo...  | TheToy Hammock by Freddie ... |  5.0   |\n",
      "| Svan High Chair with Tray ...  | After living with the Svan... |  5.0   |\n",
      "| Itzbeen Pocket Nanny Baby ...  | I'm not a big fan of gadge... |  5.0   |\n",
      "| Maclaren Triumph Stroller,...  | We bought this stroller pr... |  5.0   |\n",
      "| Nuby 4 Pack Snack Cup and ...  | I bought these bowls becau... |  4.0   |\n",
      "| Comotomo Baby Bottle, Gree...  | I spent a LOOOONG time res... |  5.0   |\n",
      "| RECARO Performance BOOSTER...  | I bought 2 of these, one f... |  4.0   |\n",
      "| Angel Dear Pair and a Spar...  | The best tip I've ever got... |  5.0   |\n",
      "|    Graco Harmony Highchair     | I originally bought this h... |  4.0   |\n",
      "| BabyComfyNose Nasal Aspira...  | This is my first review fo... |  5.0   |\n",
      "| Britax B-Ready Stroller, Black | I research to extremes (as... |  5.0   |\n",
      "| Golden Asian Silky Baby Cr...  | I must say I was apprehens... |  5.0   |\n",
      "| Boon Flair Pedestal Highch...  | I looked at a number of ch... |  5.0   |\n",
      "| Britax B-Agile Double Stro...  | I absolutely LOVE this str... |  5.0   |\n",
      "| Baby Chef Artisan Yogurt Maker | I make yogurt at home ever... |  5.0   |\n",
      "| Kolcraft Contours Options ...  | This stroller is definitel... |  4.0   |\n",
      "| Summer Infant Sure &amp; S...  | My wife had a friend who h... |  5.0   |\n",
      "+--------------------------------+-------------------------------+--------+\n",
      "+-------------------------------+-----------+-------------+\n",
      "|           word_count          | sentiment | probability |\n",
      "+-------------------------------+-----------+-------------+\n",
      "| {'infant': 1, 'since': 2, ... |     1     |     1.0     |\n",
      "| {'infant': 3, 'childs': 1,... |     1     |     1.0     |\n",
      "| {'all': 2, 'go': 2, 'assem... |     1     |     1.0     |\n",
      "| {'all': 2, 'help': 1, 'jus... |     1     |     1.0     |\n",
      "| {'rating': 1, 'somewhat': ... |     1     |     1.0     |\n",
      "| {'all': 1, 'just': 2, 'mon... |     1     |     1.0     |\n",
      "| {'opinions': 1, 'all': 1, ... |     1     |     1.0     |\n",
      "| {'breastfeeding': 1, 'all'... |     1     |     1.0     |\n",
      "| {'bottom': 1, 'office': 1,... |     1     |     1.0     |\n",
      "| {'scouring': 1, 'saturated... |     1     |     1.0     |\n",
      "| {'managed': 1, 'just': 3, ... |     1     |     1.0     |\n",
      "| {'all': 4, 'out': 11, 'sle... |     1     |     1.0     |\n",
      "| {'walking': 1, 'think': 2,... |     1     |     1.0     |\n",
      "| {'tuners': 1, 'think': 1, ... |     1     |     1.0     |\n",
      "| {'all': 1, 'just': 1, 'adj... |     1     |     1.0     |\n",
      "| {'son': 2, 'infant': 2, 'o... |     1     |     1.0     |\n",
      "| {'all': 1, 'for': 3, 'inst... |     1     |     1.0     |\n",
      "| {'grabbing': 1, 'breastfee... |     1     |     1.0     |\n",
      "| {'all': 2, '45': 1, 'sleep... |     1     |     1.0     |\n",
      "| {'rocker': 1, 'all': 3, 'j... |     1     |     1.0     |\n",
      "| {'just': 1, 'less': 1, 'be... |     1     |     1.0     |\n",
      "| {'infant': 1, 'think': 1, ... |     1     |     1.0     |\n",
      "| {'all': 3, 'think': 1, 'mo... |     1     |     1.0     |\n",
      "| {'son': 2, 'think': 1, 'ba... |     1     |     1.0     |\n",
      "| {'holds': 1, 'newest': 1, ... |     1     |     1.0     |\n",
      "| {'and': 1, 'recommend': 1,... |     1     |     1.0     |\n",
      "| {'all': 3, 'shoes': 1, 'fr... |     1     |     1.0     |\n",
      "| {'cute': 1, 'being': 2, 'm... |     1     |     1.0     |\n",
      "| {'lansinoh': 1, 'all': 1, ... |     1     |     1.0     |\n",
      "| {'anything': 1, 'all': 4, ... |     1     |     1.0     |\n",
      "| {'all': 3, 'over': 2, 'dou... |     1     |     1.0     |\n",
      "| {'all': 2, 'just': 1, 'sto... |     1     |     1.0     |\n",
      "| {'gates': 1, 'taller': 2, ... |     1     |     1.0     |\n",
      "| {'cute': 1, 'help': 1, 'ju... |     1     |     1.0     |\n",
      "| {'all': 2, 'hooks': 1, 'ac... |     1     |     1.0     |\n",
      "| {'cute': 2, 'all': 3, 'whe... |     1     |     1.0     |\n",
      "| {'all': 3, 'think': 2, 'he... |     1     |     1.0     |\n",
      "| {'mom': 1, 'just': 1, 'abl... |     1     |     1.0     |\n",
      "| {'all': 2, 'consider': 1, ... |     1     |     1.0     |\n",
      "| {'infant': 2, 'just': 3, '... |     1     |     1.0     |\n",
      "| {'infant': 1, 'strollerpur... |     1     |     1.0     |\n",
      "| {'selection': 1, 'being': ... |     1     |     1.0     |\n",
      "| {'years': 2, 'industrial':... |     1     |     1.0     |\n",
      "| {'gates': 7, 'all': 4, 'ju... |     1     |     1.0     |\n",
      "| {'limited': 1, 'all': 1, '... |     1     |     1.0     |\n",
      "| {'all': 1, 'just': 2, 'les... |     1     |     1.0     |\n",
      "| {'all': 1, 'just': 2, 'bei... |     1     |     1.0     |\n",
      "| {'all': 1, 'just': 2, 'les... |     1     |     1.0     |\n",
      "| {'all': 1, 'skip': 1, 'hit... |     1     |     1.0     |\n",
      "| {'all': 3, 'packagingsince... |     1     |     1.0     |\n",
      "| {'over': 1, 'affix': 1, 'c... |     1     |     1.0     |\n",
      "| {'cute': 1, 'childs': 1, '... |     1     |     1.0     |\n",
      "| {'all': 1, 'sleep': 4, 'lo... |     1     |     1.0     |\n",
      "| {'represent': 1, 'infant':... |     1     |     1.0     |\n",
      "| {'help': 2, 'just': 2, 'ov... |     1     |     1.0     |\n",
      "| {'all': 5, 'they': 1, 'jus... |     1     |     1.0     |\n",
      "| {'being': 1, 'years': 1, '... |     1     |     1.0     |\n",
      "| {'all': 3, 'just': 3, 'swi... |     1     |     1.0     |\n",
      "| {'son': 1, 'all': 2, 'help... |     1     |     1.0     |\n",
      "| {'all': 3, 'assembling': 2... |     1     |     1.0     |\n",
      "| {'saying': 1, 'bomb': 1, '... |     1     |     1.0     |\n",
      "| {'infant': 1, 'feedback': ... |     1     |     1.0     |\n",
      "| {'all': 2, 'being': 1, 'pl... |     1     |     1.0     |\n",
      "| {'summer': 1, 'all': 1, 'o... |     1     |     1.0     |\n",
      "| {'unfolds': 1, 'money': 1,... |     1     |     1.0     |\n",
      "| {'all': 3, 'just': 2, 'ove... |     1     |     1.0     |\n",
      "| {'gates': 1, 'all': 1, 'ab... |     1     |     1.0     |\n",
      "| {'up': 1, 'and': 3, 'strap... |     1     |     1.0     |\n",
      "| {'all': 2, 'aroundso': 1, ... |     -1    |     1.0     |\n",
      "| {'gates': 1, 'all': 2, 'ch... |     1     |     1.0     |\n",
      "| {'cute': 2, 'all': 1, 'jus... |     1     |     1.0     |\n",
      "| {'just': 2, 'deployment': ... |     1     |     1.0     |\n",
      "| {'just': 2, 'when': 1, 'en... |     1     |     1.0     |\n",
      "| {'mom': 1, 'being': 1, 'no... |     1     |     1.0     |\n",
      "| {'35yr': 1, 'all': 1, 'jus... |     1     |     1.0     |\n",
      "| {'just': 2, 'dance': 1, 'o... |     1     |     1.0     |\n",
      "| {'just': 2, 'over': 1, 'in... |     1     |     1.0     |\n",
      "| {'scratch': 2, 'yellow': 1... |     1     |     1.0     |\n",
      "| {'all': 3, 'locking': 1, '... |     1     |     1.0     |\n",
      "| {'all': 1, 'just': 2, 'loo... |     1     |     1.0     |\n",
      "| {'bright': 1, 'looks': 1, ... |     1     |     1.0     |\n",
      "| {'cute': 1, 'infant': 1, '... |     1     |     1.0     |\n",
      "| {'sleeps': 1, 'hands': 1, ... |     1     |     1.0     |\n",
      "| {'all': 1, 'just': 1, 'hoo... |     1     |     1.0     |\n",
      "| {'all': 1, 'just': 1, 'bei... |     1     |     1.0     |\n",
      "| {'all': 3, 'help': 2, 'jus... |     1     |     1.0     |\n",
      "| {'shop': 1, 'all': 2, 'hap... |     1     |     1.0     |\n",
      "| {'cute': 1, 'just': 4, 'wh... |     1     |     1.0     |\n",
      "| {'all': 1, 'stores': 1, 'j... |     1     |     1.0     |\n",
      "| {'just': 1, 'both': 2, 'go... |     1     |     1.0     |\n",
      "| {'cute': 2, 'washeshaving'... |     1     |     1.0     |\n",
      "| {'all': 2, 'help': 1, 'col... |     1     |     1.0     |\n",
      "| {'all': 1, 'birth': 1, 'ju... |     1     |     1.0     |\n",
      "| {'all': 1, 'listening': 1,... |     1     |     1.0     |\n",
      "| {'gold': 2, 'satisfied': 1... |     1     |     1.0     |\n",
      "| {'all': 3, 'particularly':... |     1     |     1.0     |\n",
      "| {'all': 2, 'help': 1, 'ove... |     1     |     1.0     |\n",
      "| {'summer': 1, 'all': 1, 'j... |     1     |     1.0     |\n",
      "| {'conscons1': 1, 'opinions... |     1     |     1.0     |\n",
      "| {'gates': 1, 'over': 1, 'm... |     1     |     1.0     |\n",
      "+-------------------------------+-----------+-------------+\n",
      "+-------------------------------+\n",
      "|       word_count_subset       |\n",
      "+-------------------------------+\n",
      "|   {'little': 1, 'would': 1}   |\n",
      "| {'love': 1, 'easy': 2, 'wo... |\n",
      "| {'even': 3, 'little': 3, '... |\n",
      "| {'even': 2, 'great': 2, 'l... |\n",
      "| {'even': 2, 'great': 1, 'l... |\n",
      "| {'even': 2, 'perfect': 2, ... |\n",
      "| {'great': 1, 'love': 1, 'c... |\n",
      "| {'even': 1, 'perfect': 2, ... |\n",
      "| {'product': 1, 'love': 1, ... |\n",
      "| {'even': 1, 'little': 1, '... |\n",
      "| {'great': 2, 'love': 1, 'w... |\n",
      "| {'even': 5, 'great': 1, 'o... |\n",
      "| {'even': 1, 'little': 1, '... |\n",
      "| {'great': 1, 'little': 2, ... |\n",
      "| {'love': 1, 'little': 1, '... |\n",
      "| {'even': 3, 'perfect': 1, ... |\n",
      "| {'car': 3, 'well': 1, 'eas... |\n",
      "| {'product': 1, 'old': 2, '... |\n",
      "| {'able': 2, 'great': 3, 'o... |\n",
      "| {'great': 3, 'love': 1, 'e... |\n",
      "| {'even': 3, 'car': 1, 'wel... |\n",
      "| {'car': 3, 'great': 2, 'ab... |\n",
      "| {'great': 1, 'love': 1, 'w... |\n",
      "| {'even': 2, 'car': 6, 'lit... |\n",
      "| {'perfect': 1, 'little': 3... |\n",
      "|          {'easy': 1}          |\n",
      "| {'perfect': 1, 'product': ... |\n",
      "| {'great': 1, 'old': 1, 'wo... |\n",
      "| {'love': 1, 'well': 1, 'wo... |\n",
      "| {'great': 2, 'love': 1, 'w... |\n",
      "| {'great': 2, 'well': 1, 'w... |\n",
      "| {'car': 2, 'great': 3, 'lo... |\n",
      "| {'well': 2, 'great': 1, 'l... |\n",
      "| {'old': 1, 'little': 1, 'w... |\n",
      "| {'little': 4, 'old': 1, 'w... |\n",
      "| {'perfect': 1, 'great': 2,... |\n",
      "| {'even': 1, 'product': 1, ... |\n",
      "| {'product': 1, 'little': 1... |\n",
      "| {'even': 3, 'product': 1, ... |\n",
      "| {'even': 1, 'car': 2, 'lov... |\n",
      "| {'car': 5, 'great': 1, 'lo... |\n",
      "| {'perfect': 1, 'great': 4,... |\n",
      "| {'perfect': 1, 'car': 1, '... |\n",
      "| {'perfect': 2, 'love': 1, ... |\n",
      "| {'even': 1, 'great': 2, 'l... |\n",
      "| {'car': 2, 'well': 1, 'lov... |\n",
      "|          {'loves': 2}         |\n",
      "| {'even': 1, 'great': 1, 'o... |\n",
      "| {'even': 1, 'old': 1, 'lov... |\n",
      "| {'even': 1, 'old': 1, 'wou... |\n",
      "|   {'product': 1, 'well': 1}   |\n",
      "|    {'little': 1, 'easy': 1}   |\n",
      "| {'great': 1, 'love': 2, 'l... |\n",
      "| {'old': 1, 'little': 1, 'w... |\n",
      "| {'car': 2, 'great': 2, 'li... |\n",
      "| {'great': 3, 'little': 3, ... |\n",
      "|    {'easy': 2, 'would': 1}    |\n",
      "| {'even': 1, 'great': 1, 'l... |\n",
      "| {'even': 1, 'perfect': 1, ... |\n",
      "| {'perfect': 1, 'product': ... |\n",
      "| {'even': 1, 'product': 1, ... |\n",
      "| {'even': 2, 'perfect': 1, ... |\n",
      "| {'perfect': 1, 'product': ... |\n",
      "| {'product': 4, 'love': 4, ... |\n",
      "| {'perfect': 1, 'even': 1, ... |\n",
      "| {'even': 2, 'perfect': 1, ... |\n",
      "| {'love': 1, 'able': 2, 'ea... |\n",
      "|   {'product': 2, 'work': 2}   |\n",
      "| {'great': 1, 'old': 1, 'wo... |\n",
      "| {'even': 1, 'little': 1, '... |\n",
      "| {'even': 1, 'great': 1, 'w... |\n",
      "| {'perfect': 1, 'love': 1, ... |\n",
      "| {'even': 1, 'little': 1, '... |\n",
      "| {'perfect': 1, 'product': ... |\n",
      "| {'even': 1, 'little': 1, '... |\n",
      "| {'perfect': 1, 'great': 1,... |\n",
      "| {'little': 2, 'old': 1, 'w... |\n",
      "|     {'old': 1, 'easy': 1}     |\n",
      "| {'perfect': 1, 'able': 1, ... |\n",
      "| {'great': 1, 'love': 2, 'w... |\n",
      "| {'car': 2, 'love': 4, 'eas... |\n",
      "| {'even': 1, 'product': 2, ... |\n",
      "| {'even': 2, 'little': 1, '... |\n",
      "| {'great': 2, 'would': 1, '... |\n",
      "| {'great': 2, 'old': 1, 'wo... |\n",
      "| {'great': 1, 'little': 1, ... |\n",
      "|    {'easy': 5, 'would': 1}    |\n",
      "| {'great': 1, 'old': 1, 'wo... |\n",
      "| {'little': 1, 'work': 1, '... |\n",
      "| {'little': 1, 'old': 4, 'w... |\n",
      "| {'perfect': 2, 'car': 1, '... |\n",
      "| {'little': 2, 'love': 4, '... |\n",
      "| {'old': 1, 'little': 1, 'l... |\n",
      "| {'even': 2, 'little': 1, '... |\n",
      "| {'little': 2, 'great': 1, ... |\n",
      "|    {'would': 1, 'easy': 4}    |\n",
      "| {'even': 1, 'great': 2, 'l... |\n",
      "| {'perfect': 2, 'even': 1, ... |\n",
      "| {'great': 4, 'love': 2, 'w... |\n",
      "| {'able': 2, 'little': 1, '... |\n",
      "+-------------------------------+\n",
      "[100 rows x 7 columns]\n",
      "\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "test_data['probability'] = sentiment_model.predict(test_data, output_type='probability')\n",
    "most_positive_reviews = test_data.sort('probability',ascending = False)[100:200]\n",
    "print most_positive_reviews.print_rows(num_rows = 100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quiz Question**: Which of the following products are represented in the 20 most positive reviews? [multiple choice]\n",
    "\n",
    "\n",
    "Now, let us repeat this exercise to find the \"most negative reviews.\" Use the prediction probabilities to find the  20 reviews in the **test_data** with the **lowest probability** of being classified as a **positive review**. Repeat the same steps above but make sure you **sort in the opposite order**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------------------------------+-------------------------------+--------+\n",
      "|              name             |             review            | rating |\n",
      "+-------------------------------+-------------------------------+--------+\n",
      "| Jolly Jumper Arctic Sneak ... | I am a \"research-aholic\" i... |  5.0   |\n",
      "| Levana Safe N'See Digital ... | This is the first review I... |  1.0   |\n",
      "| Snuza Portable Baby Moveme... | I would have given the pro... |  1.0   |\n",
      "| Fisher-Price Ocean Wonders... | We have not had ANY luck w... |  2.0   |\n",
      "| VTech Communications Safe ... | This is my second video mo... |  1.0   |\n",
      "| Safety 1st High-Def Digita... | We bought this baby monito... |  1.0   |\n",
      "| Chicco Cortina KeyFit 30 T... | My wife and I have used th... |  1.0   |\n",
      "| Prince Lionheart Warmies W... | *****IMPORTANT UPDATE*****... |  1.0   |\n",
      "| Valco Baby Tri-mode Twin S... | I give one star to the dim... |  1.0   |\n",
      "| Adiri BPA Free Natural Nur... | I will try to write an obj... |  2.0   |\n",
      "| Munchkin Nursery Projector... | Updated January 3, 2014.  ... |  1.0   |\n",
      "| The First Years True Choic... | Note: we never installed b... |  1.0   |\n",
      "| Nuby Natural Touch Silicon... | I'm honestly confused by s... |  1.0   |\n",
      "| Peg-Perego Tatamia High Ch... | I ordered this high chair ... |  1.0   |\n",
      "|    Fisher-Price Royal Potty   | This was the worst potty e... |  1.0   |\n",
      "| Safety 1st Exchangeable Ti... | I thought it sounded great... |  1.0   |\n",
      "| Safety 1st Lift Lock and S... | Don't buy this product. If... |  1.0   |\n",
      "| Evenflo Take Me Too Premie... | I am absolutely disgusted ... |  1.0   |\n",
      "| Cloth Diaper Sprayer--styl... | I bought this sprayer out ... |  1.0   |\n",
      "| The First Years 3 Pack Bre... | I purchased several of the... |  1.0   |\n",
      "+-------------------------------+-------------------------------+--------+\n",
      "+-------------------------------+-----------+--------------------+\n",
      "|           word_count          | sentiment |    probability     |\n",
      "+-------------------------------+-----------+--------------------+\n",
      "| {'raining': 1, 'all': 8, '... |     1     | 7.80415068212e-100 |\n",
      "| {'all': 2, 'they': 4, 'jus... |     -1    | 6.83650885514e-25  |\n",
      "| {'contacted': 1, 'being': ... |     -1    | 2.12654510823e-24  |\n",
      "| {'fishstarfish': 1, 'infan... |     -1    | 2.24582080779e-23  |\n",
      "| {'all': 4, 'reviewers': 1,... |     -1    | 1.32962966148e-22  |\n",
      "| {'all': 3, 'being': 1, 'ov... |     -1    | 2.06872097469e-20  |\n",
      "| {'all': 4, 'wrestle': 1, '... |     -1    | 5.93881994671e-20  |\n",
      "| {'less': 1, 'move': 1, 'no... |     -1    | 6.28510016537e-20  |\n",
      "| {'limited': 2, 'forget': 1... |     -1    | 8.05528712688e-20  |\n",
      "| {'all': 2, 'forget': 1, 'g... |     -1    | 8.46521724938e-20  |\n",
      "| {'all': 2, 'just': 1, 'mon... |     -1    |  1.5285394517e-19  |\n",
      "| {'all': 3, 'go': 1, 'worke... |     -1    | 1.77901889383e-19  |\n",
      "| {'all': 2, 'just': 3, 'bei... |     -1    | 1.15227353848e-18  |\n",
      "| {'just': 2, 'food': 2, 'mo... |     -1    | 1.26175666136e-18  |\n",
      "| {'son': 1, 'old': 1, 'is':... |     -1    | 1.60282966315e-18  |\n",
      "| {'headsplittinglyannoying'... |     -1    | 7.04887411709e-18  |\n",
      "| {'all': 2, 'money': 1, 'ov... |     -1    | 9.84839237568e-18  |\n",
      "| {'chore': 1, 'managed': 1,... |     -1    | 1.00120730395e-17  |\n",
      "| {'all': 1, 'just': 1, 'rev... |     -1    |  1.169063556e-17   |\n",
      "| {'all': 1, 'just': 2, 'mon... |     -1    | 1.22003532002e-17  |\n",
      "+-------------------------------+-----------+--------------------+\n",
      "+-------------------------------+\n",
      "|       word_count_subset       |\n",
      "+-------------------------------+\n",
      "| {'even': 4, 'great': 1, 'w... |\n",
      "| {'even': 2, 'product': 1, ... |\n",
      "| {'money': 1, 'product': 2,... |\n",
      "| {'even': 2, 'product': 5, ... |\n",
      "| {'even': 2, 'product': 3, ... |\n",
      "| {'even': 2, 'little': 1, '... |\n",
      "| {'even': 2, 'product': 1, ... |\n",
      "| {'product': 3, 'well': 1, ... |\n",
      "| {'even': 3, 'easy': 1, 'wo... |\n",
      "| {'even': 1, 'product': 1, ... |\n",
      "| {'even': 1, 'money': 1, 'g... |\n",
      "| {'even': 3, 'great': 1, 'o... |\n",
      "| {'disappointed': 1, 'money... |\n",
      "| {'money': 1, 'product': 1,... |\n",
      "|     {'old': 1, 'would': 1}    |\n",
      "|    {'great': 1, 'well': 1}    |\n",
      "| {'even': 1, 'money': 1, 'p... |\n",
      "| {'even': 1, 'product': 2, ... |\n",
      "| {'disappointed': 1, 'money... |\n",
      "| {'perfect': 1, 'money': 1,... |\n",
      "+-------------------------------+\n",
      "[20 rows x 7 columns]\n",
      "\n"
     ]
    },
    {
     "ename": "TypeError",
     "evalue": "'NoneType' object has no attribute '__getitem__'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-107-91623893b586>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mmost_negative_reviews\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtest_data\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msort\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'probability'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mascending\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m20\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0;32mprint\u001b[0m \u001b[0mmost_negative_reviews\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprint_rows\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnum_rows\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m20\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'name'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: 'NoneType' object has no attribute '__getitem__'"
     ]
    }
   ],
   "source": [
    "most_negative_reviews = test_data.sort('probability',ascending = True)[0:20]\n",
    "print most_negative_reviews.print_rows(num_rows = 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Quiz Question**: Which of the following products are represented in the 20 most negative reviews?  [multiple choice]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compute accuracy of the classifier\n",
    "\n",
    "We will now evaluate the accuracy of the trained classifier. Recall that the accuracy is given by\n",
    "\n",
    "\n",
    "$$\n",
    "\\mbox{accuracy} = \\frac{\\mbox{# correctly classified examples}}{\\mbox{# total examples}}\n",
    "$$\n",
    "\n",
    "This can be computed as follows:\n",
    "\n",
    "* **Step 1:** Use the trained model to compute class predictions (**Hint:** Use the `predict` method)\n",
    "* **Step 2:** Count the number of data points when the predicted class labels match the ground truth labels (called `true_labels` below).\n",
    "* **Step 3:** Divide the total number of correct predictions by the total number of data points in the dataset.\n",
    "\n",
    "Complete the function below to compute the classification accuracy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def get_classification_accuracy(model, data, true_labels):\n",
    "    # First get the predictions\n",
    "    predictions = model.predict(data)\n",
    "    \n",
    "    # Compute the number of correctly classified examples\n",
    "    number_correct = len( predictions[ predictions == true_labels ] )\n",
    "\n",
    "    # Then compute accuracy by dividing num_correct by total number of examples\n",
    "    accuracy = number_correct / len(data)\n",
    "    \n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's compute the classification accuracy of the **sentiment_model** on the **test_data**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9145368370530358"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_classification_accuracy(sentiment_model, test_data, test_data['sentiment'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quiz Question**: What is the accuracy of the **sentiment_model** on the **test_data**? Round your answer to 2 decimal places (e.g. 0.76).\n",
    "\n",
    "**Quiz Question**: Does a higher accuracy value on the **training_data** always imply that the classifier is better?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learn another classifier with fewer words\n",
    "\n",
    "There were a lot of words in the model we trained above. We will now train a simpler logistic regression model using only a subset of words that occur in the reviews. For this assignment, we selected a 20 words to work with. These are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "significant_words = ['love', 'great', 'easy', 'old', 'little', 'perfect', 'loves', \n",
    "      'well', 'able', 'car', 'broke', 'less', 'even', 'waste', 'disappointed', \n",
    "      'work', 'product', 'money', 'would', 'return']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(significant_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each review, we will use the **word_count** column and trim out all words that are **not** in the **significant_words** list above. We will use the [SArray dictionary trim by keys functionality]( https://dato.com/products/create/docs/generated/graphlab.SArray.dict_trim_by_keys.html). Note that we are performing this on both the training and test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "train_data['word_count_subset'] = train_data['word_count'].dict_trim_by_keys(significant_words, exclude=False)\n",
    "test_data['word_count_subset'] = test_data['word_count'].dict_trim_by_keys(significant_words, exclude=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see what the first example of the dataset looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'it came early and was not disappointed. i love planet wise bags and now my wipe holder. it keps my osocozy wipes moist and does not leak. highly recommend it.'"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[0]['review']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The **word_count** column had been working with before looks like the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'and': 3, 'love': 1, 'it': 3, 'highly': 1, 'osocozy': 1, 'bags': 1, 'holder': 1, 'leak': 1, 'moist': 1, 'does': 1, 'recommend': 1, 'was': 1, 'wipes': 1, 'early': 1, 'not': 2, 'now': 1, 'disappointed': 1, 'wipe': 1, 'keps': 1, 'wise': 1, 'i': 1, 'planet': 1, 'my': 2, 'came': 1}\n"
     ]
    }
   ],
   "source": [
    "print train_data[0]['word_count']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we are only working with a subset of these words, the column **word_count_subset** is a subset of the above dictionary. In this example, only 2 `significant words` are present in this review."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'love': 1, 'disappointed': 1}\n"
     ]
    }
   ],
   "source": [
    "print train_data[0]['word_count_subset']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train a logistic regression model on a subset of data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now build a classifier with **word_count_subset** as the feature and **sentiment** as the target. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre>Logistic regression:</pre>"
      ],
      "text/plain": [
       "Logistic regression:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 133416</pre>"
      ],
      "text/plain": [
       "Number of examples          : 133416"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of classes           : 2</pre>"
      ],
      "text/plain": [
       "Number of classes           : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of feature columns   : 1</pre>"
      ],
      "text/plain": [
       "Number of feature columns   : 1"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 20</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 20"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients    : 21</pre>"
      ],
      "text/plain": [
       "Number of coefficients    : 21"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting Newton Method</pre>"
      ],
      "text/plain": [
       "Starting Newton Method"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+-------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+-------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Elapsed Time | Training-accuracy |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Elapsed Time | Training-accuracy |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+-------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+-------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 2        | 0.236733     | 0.862917          |</pre>"
      ],
      "text/plain": [
       "| 1         | 2        | 0.236733     | 0.862917          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 2         | 3        | 0.362578     | 0.865713          |</pre>"
      ],
      "text/plain": [
       "| 2         | 3        | 0.362578     | 0.865713          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 3         | 4        | 0.481209     | 0.866478          |</pre>"
      ],
      "text/plain": [
       "| 3         | 4        | 0.481209     | 0.866478          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 4         | 5        | 0.597356     | 0.866748          |</pre>"
      ],
      "text/plain": [
       "| 4         | 5        | 0.597356     | 0.866748          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 5         | 6        | 0.713253     | 0.866815          |</pre>"
      ],
      "text/plain": [
       "| 5         | 6        | 0.713253     | 0.866815          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 6         | 7        | 0.832686     | 0.866815          |</pre>"
      ],
      "text/plain": [
       "| 6         | 7        | 0.832686     | 0.866815          |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+--------------+-------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+--------------+-------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>SUCCESS: Optimal solution found.</pre>"
      ],
      "text/plain": [
       "SUCCESS: Optimal solution found."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre></pre>"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "Class                          : LogisticClassifier\n",
       "\n",
       "Schema\n",
       "------\n",
       "Number of coefficients         : 21\n",
       "Number of examples             : 133416\n",
       "Number of classes              : 2\n",
       "Number of feature columns      : 1\n",
       "Number of unpacked features    : 20\n",
       "\n",
       "Hyperparameters\n",
       "---------------\n",
       "L1 penalty                     : 0.0\n",
       "L2 penalty                     : 0.01\n",
       "\n",
       "Training Summary\n",
       "----------------\n",
       "Solver                         : newton\n",
       "Solver iterations              : 6\n",
       "Solver status                  : SUCCESS: Optimal solution found.\n",
       "Training time (sec)            : 0.8577\n",
       "\n",
       "Settings\n",
       "--------\n",
       "Log-likelihood                 : 44323.7254\n",
       "\n",
       "Highest Positive Coefficients\n",
       "-----------------------------\n",
       "word_count_subset[loves]       : 1.6773\n",
       "word_count_subset[perfect]     : 1.5145\n",
       "word_count_subset[love]        : 1.3654\n",
       "(intercept)                    : 1.2995\n",
       "word_count_subset[easy]        : 1.1937\n",
       "\n",
       "Lowest Negative Coefficients\n",
       "----------------------------\n",
       "word_count_subset[disappointed] : -2.3551\n",
       "word_count_subset[return]      : -2.1173\n",
       "word_count_subset[waste]       : -2.0428\n",
       "word_count_subset[broke]       : -1.658\n",
       "word_count_subset[money]       : -0.8979"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "simple_model = graphlab.logistic_classifier.create(train_data,\n",
    "                                                   target = 'sentiment',\n",
    "                                                   features=['word_count_subset'],\n",
    "                                                   validation_set=None)\n",
    "simple_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can compute the classification accuracy using the `get_classification_accuracy` function you implemented earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8693004559635229"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_classification_accuracy(simple_model, test_data, test_data['sentiment'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we will inspect the weights (coefficients) of the **simple_model**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">name</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">index</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">class</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">value</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">stderr</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">(intercept)</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">None</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1.2995449552</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.0120888541331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">word_count_subset</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">disappointed</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-2.35509250061</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.0504149888557</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">word_count_subset</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">love</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1.36543549368</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.0303546295109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">word_count_subset</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">little</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.520628636025</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.0214691475665</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">word_count_subset</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">loves</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1.67727145556</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.0482328275384</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">word_count_subset</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">product</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-0.320555492996</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.0154311321362</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">word_count_subset</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">well</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.504256746398</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.021381300631</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">word_count_subset</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">great</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.94469126948</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.0209509926591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">word_count_subset</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">easy</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1.19366189833</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.029288869202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">word_count_subset</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">work</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">-0.621700012425</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0.0230330597946</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[21 rows x 5 columns]<br/>Note: Only the head of the SFrame is printed.<br/>You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tname\tstr\n",
       "\tindex\tstr\n",
       "\tclass\tint\n",
       "\tvalue\tfloat\n",
       "\tstderr\tfloat\n",
       "\n",
       "Rows: 21\n",
       "\n",
       "Data:\n",
       "+-------------------+--------------+-------+-----------------+-----------------+\n",
       "|        name       |    index     | class |      value      |      stderr     |\n",
       "+-------------------+--------------+-------+-----------------+-----------------+\n",
       "|    (intercept)    |     None     |   1   |   1.2995449552  | 0.0120888541331 |\n",
       "| word_count_subset | disappointed |   1   |  -2.35509250061 | 0.0504149888557 |\n",
       "| word_count_subset |     love     |   1   |  1.36543549368  | 0.0303546295109 |\n",
       "| word_count_subset |    little    |   1   |  0.520628636025 | 0.0214691475665 |\n",
       "| word_count_subset |    loves     |   1   |  1.67727145556  | 0.0482328275384 |\n",
       "| word_count_subset |   product    |   1   | -0.320555492996 | 0.0154311321362 |\n",
       "| word_count_subset |     well     |   1   |  0.504256746398 |  0.021381300631 |\n",
       "| word_count_subset |    great     |   1   |  0.94469126948  | 0.0209509926591 |\n",
       "| word_count_subset |     easy     |   1   |  1.19366189833  |  0.029288869202 |\n",
       "| word_count_subset |     work     |   1   | -0.621700012425 | 0.0230330597946 |\n",
       "+-------------------+--------------+-------+-----------------+-----------------+\n",
       "[21 rows x 5 columns]\n",
       "Note: Only the head of the SFrame is printed.\n",
       "You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns."
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "simple_model.coefficients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's sort the coefficients (in descending order) by the **value** to obtain the coefficients with the most positive effect on the sentiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------------------+--------------+-------+-----------------+-----------------+\n",
      "|        name       |    index     | class |      value      |      stderr     |\n",
      "+-------------------+--------------+-------+-----------------+-----------------+\n",
      "| word_count_subset |    loves     |   1   |  1.67727145556  | 0.0482328275384 |\n",
      "| word_count_subset |   perfect    |   1   |  1.51448626703  |  0.049861952294 |\n",
      "| word_count_subset |     love     |   1   |  1.36543549368  | 0.0303546295109 |\n",
      "|    (intercept)    |     None     |   1   |   1.2995449552  | 0.0120888541331 |\n",
      "| word_count_subset |     easy     |   1   |  1.19366189833  |  0.029288869202 |\n",
      "| word_count_subset |    great     |   1   |  0.94469126948  | 0.0209509926591 |\n",
      "| word_count_subset |    little    |   1   |  0.520628636025 | 0.0214691475665 |\n",
      "| word_count_subset |     well     |   1   |  0.504256746398 |  0.021381300631 |\n",
      "| word_count_subset |     able     |   1   |  0.191438302295 | 0.0337581955697 |\n",
      "| word_count_subset |     old      |   1   | 0.0853961886678 | 0.0200863423025 |\n",
      "| word_count_subset |     car      |   1   |  0.058834990068 | 0.0168291532091 |\n",
      "| word_count_subset |     less     |   1   | -0.209709815216 |  0.040505735954 |\n",
      "| word_count_subset |   product    |   1   | -0.320555492996 | 0.0154311321362 |\n",
      "| word_count_subset |    would     |   1   | -0.362308947711 | 0.0127544751985 |\n",
      "| word_count_subset |     even     |   1   |  -0.51173855127 | 0.0199612760261 |\n",
      "| word_count_subset |     work     |   1   | -0.621700012425 | 0.0230330597946 |\n",
      "| word_count_subset |    money     |   1   | -0.897884155776 | 0.0339936732836 |\n",
      "| word_count_subset |    broke     |   1   |  -1.65796447838 | 0.0580878907166 |\n",
      "| word_count_subset |    waste     |   1   |   -2.042773611  | 0.0644702932444 |\n",
      "| word_count_subset |    return    |   1   |  -2.11729659718 | 0.0578650807241 |\n",
      "| word_count_subset | disappointed |   1   |  -2.35509250061 | 0.0504149888557 |\n",
      "+-------------------+--------------+-------+-----------------+-----------------+\n",
      "[21 rows x 5 columns]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "simple_model.coefficients.sort('value', ascending=False).print_rows(num_rows=21)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quiz Question**: Consider the coefficients of **simple_model**. There should be 21 of them, an intercept term + one for each word in **significant_words**. How many of the 20 coefficients (corresponding to the 20 **significant_words** and *excluding the intercept term*) are positive for the `simple_model`?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['loves', 'perfect', 'love', 'easy', 'great', 'little', 'well', 'able', 'old', 'car', ... ]\n"
     ]
    }
   ],
   "source": [
    "req_values = simple_model.coefficients.sort('value', ascending=False)[0:21]\n",
    "positive_significant_words = req_values[ req_values['value'] >= 0 ]\n",
    "positive_significant_words = positive_significant_words[ positive_significant_words['name'] != '(intercept)' ]['index']\n",
    "print positive_significant_words"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quiz Question**: Are the positive words in the **simple_model** (let us call them `positive_significant_words`) also positive words in the **sentiment_model**?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------+--------------+\n",
      "| index |    value     |\n",
      "+-------+--------------+\n",
      "| loves | 1.5664851757 |\n",
      "+-------+--------------+\n",
      "[? rows x 2 columns]\n",
      "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
      "You can use sf.materialize() to force materialization.\n",
      "+---------+---------------+\n",
      "|  index  |     value     |\n",
      "+---------+---------------+\n",
      "| perfect | 1.75190114392 |\n",
      "+---------+---------------+\n",
      "[? rows x 2 columns]\n",
      "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
      "You can use sf.materialize() to force materialization.\n",
      "+-------+---------------+\n",
      "| index |     value     |\n",
      "+-------+---------------+\n",
      "|  love | 1.43301685439 |\n",
      "+-------+---------------+\n",
      "[? rows x 2 columns]\n",
      "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
      "You can use sf.materialize() to force materialization.\n",
      "+-------+---------------+\n",
      "| index |     value     |\n",
      "+-------+---------------+\n",
      "|  easy | 1.21346937822 |\n",
      "+-------+---------------+\n",
      "[? rows x 2 columns]\n",
      "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
      "You can use sf.materialize() to force materialization.\n",
      "+-------+---------------+\n",
      "| index |     value     |\n",
      "+-------+---------------+\n",
      "| great | 1.31459245039 |\n",
      "+-------+---------------+\n",
      "[? rows x 2 columns]\n",
      "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
      "You can use sf.materialize() to force materialization.\n",
      "+--------+----------------+\n",
      "| index  |     value      |\n",
      "+--------+----------------+\n",
      "| little | 0.674162457499 |\n",
      "+--------+----------------+\n",
      "[? rows x 2 columns]\n",
      "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
      "You can use sf.materialize() to force materialization.\n",
      "+-------+----------------+\n",
      "| index |     value      |\n",
      "+-------+----------------+\n",
      "|  well | 0.627964877567 |\n",
      "+-------+----------------+\n",
      "[? rows x 2 columns]\n",
      "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
      "You can use sf.materialize() to force materialization.\n",
      "+-------+----------------+\n",
      "| index |     value      |\n",
      "+-------+----------------+\n",
      "|  able | 0.174331272552 |\n",
      "+-------+----------------+\n",
      "[? rows x 2 columns]\n",
      "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
      "You can use sf.materialize() to force materialization.\n",
      "+-------+------------------+\n",
      "| index |      value       |\n",
      "+-------+------------------+\n",
      "|  old  | 0.00912230113669 |\n",
      "+-------+------------------+\n",
      "[? rows x 2 columns]\n",
      "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
      "You can use sf.materialize() to force materialization.\n",
      "+-------+----------------+\n",
      "| index |     value      |\n",
      "+-------+----------------+\n",
      "|  car  | 0.195263670618 |\n",
      "+-------+----------------+\n",
      "[? rows x 2 columns]\n",
      "Note: Only the head of the SFrame is printed. This SFrame is lazily evaluated.\n",
      "You can use sf.materialize() to force materialization.\n"
     ]
    }
   ],
   "source": [
    "for i in range(0,len(positive_significant_words)):\n",
    "    print sentiment_model.coefficients[ sentiment_model.coefficients['index'] == positive_significant_words[i] ]['index','value']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Comparing models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now compare the accuracy of the **sentiment_model** and the **simple_model** using the `get_classification_accuracy` method you implemented above.\n",
    "\n",
    "First, compute the classification accuracy of the **sentiment_model** on the **train_data**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.979440247046831"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_classification_accuracy(sentiment_model, train_data, train_data['sentiment'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, compute the classification accuracy of the **simple_model** on the **train_data**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8668150746537147"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_classification_accuracy(simple_model, train_data, train_data['sentiment'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quiz Question**: Which model (**sentiment_model** or **simple_model**) has higher accuracy on the TRAINING set?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we will repeat this exercise on the **test_data**. Start by computing the classification accuracy of the **sentiment_model** on the **test_data**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9145368370530358"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_classification_accuracy(sentiment_model, test_data, test_data['sentiment'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will compute the classification accuracy of the **simple_model** on the **test_data**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8693004559635229"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_classification_accuracy(simple_model, test_data, test_data['sentiment'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quiz Question**: Which model (**sentiment_model** or **simple_model**) has higher accuracy on the TEST set?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Baseline: Majority class prediction\n",
    "\n",
    "It is quite common to use the **majority class classifier** as the a baseline (or reference) model for comparison with your classifier model. The majority classifier model predicts the majority class for all data points. At the very least, you should healthily beat the majority class classifier, otherwise, the model is (usually) pointless.\n",
    "\n",
    "What is the majority class in the **train_data**?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "112164\n",
      "21252\n"
     ]
    }
   ],
   "source": [
    "num_positive  = (train_data['sentiment'] == +1).sum()\n",
    "num_negative = (train_data['sentiment'] == -1).sum()\n",
    "print num_positive\n",
    "print num_negative"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now compute the accuracy of the majority class classifier on **test_data**.\n",
    "\n",
    "**Quiz Question**: Enter the accuracy of the majority class classifier model on the **test_data**. Round your answer to two decimal places (e.g. 0.76)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8693004559635229"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_classification_accuracy(simple_model, test_data, test_data['sentiment'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quiz Question**: Is the **sentiment_model** definitely better than the majority class classifier (the baseline)?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}