Applying_Basic_Text_Classification_to_IMDB_movie_reviews.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notebook Outline\n",
    "* **Data transform and storage**  I have taken data directly from the original source. All the data is present as a txt file. We will run a pre-process procedure to clean a data little bit and store it in more usable format later i.e in a `csv` file.\n",
    "* **Data cleaning** Since we have text data in hand, we are going to apply standard cleaning pipelines using various nlp tools available. \n",
    "\n",
    "\n",
    "\n",
    "DataSource <a href = \"http://ai.stanford.edu/~amaas/data/sentiment/\">here</a>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-24T08:56:07.846964Z",
     "start_time": "2020-05-24T08:56:07.843965Z"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from tqdm.notebook import tqdm\n",
    "import os\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-24T08:55:42.047898Z",
     "start_time": "2020-05-24T08:55:42.043899Z"
    }
   },
   "outputs": [],
   "source": [
    "data_source_path = \"./IMDB_raw_movie_review_data/\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-24T08:56:24.347734Z",
     "start_time": "2020-05-24T08:56:24.339733Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['cleaned_imdb_reviews.csv',\n",
       " 'imdb.vocab',\n",
       " 'imdbEr.txt',\n",
       " 'README',\n",
       " 'test',\n",
       " 'train',\n",
       " 'transformed_data_reviews.csv']"
      ]
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir(data_source_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['labeledBow.feat',\n",
       " 'neg',\n",
       " 'pos',\n",
       " 'unsup',\n",
       " 'unsupBow.feat',\n",
       " 'urls_neg.txt',\n",
       " 'urls_pos.txt',\n",
       " 'urls_unsup.txt']"
      ]
     },
     "execution_count": 162,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir(data_source_path+\"/train\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['labeledBow.feat', 'neg', 'pos', 'urls_neg.txt', 'urls_pos.txt']"
      ]
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    " os.listdir(data_source_path+\"/test\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['0_3.txt',\n",
       " '10000_4.txt',\n",
       " '10001_4.txt',\n",
       " '10002_1.txt',\n",
       " '10003_1.txt',\n",
       " '10004_3.txt',\n",
       " '10005_3.txt',\n",
       " '10006_4.txt',\n",
       " '10007_1.txt',\n",
       " '10008_2.txt']"
      ]
     },
     "execution_count": 164,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir(data_source_path+\"/train/neg\")[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['0_2.txt',\n",
       " '10000_4.txt',\n",
       " '10001_1.txt',\n",
       " '10002_3.txt',\n",
       " '10003_3.txt',\n",
       " '10004_2.txt',\n",
       " '10005_2.txt',\n",
       " '10006_2.txt',\n",
       " '10007_4.txt',\n",
       " '10008_4.txt']"
      ]
     },
     "execution_count": 165,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir(data_source_path+\"/test/neg\")[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* As we can see data is divided into `test` and `train` folders. And the `labels` are represented as folders i.e `pos` and `neg` here.\n",
    "* each review is represented as `/train/neg/0_3.txt` i.e id for this review is `0`, star rating for that review on a 1-10 scale is `3` and the label for this review is `neg` and the data belongs to `train-set`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "### Data transformation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "sets = ['train','test']\n",
    "labels = ['pos','neg']\n",
    "\n",
    "labels_dict = dict(pos = 0,neg = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "def transform_data(base_data_source_path):\n",
    "    \"\"\"\n",
    "    This function will read each review and it's corresponding label and other useful info one by one.\n",
    "    Will perform a small clean-up step i.e strip operation to get rid of extra spaced and store the review\n",
    "    and it's information in a dataFrame row.\n",
    "    At the end it will return the populated dataFrame\n",
    "    \n",
    "    parameters :\n",
    "    base_data_source_path - base path where the dataset is present in raw form\n",
    "    \n",
    "    returns :\n",
    "    a dataFrame populated with reviews and some meta-data\n",
    "    \"\"\"\n",
    "    \n",
    "    df_columns = ['id','text','rating','label']\n",
    "    \n",
    "    rows = []\n",
    "    \n",
    "    for set_ in tqdm(sets,total = len(sets),unit = \"set \"):\n",
    "        \n",
    "        for label in labels:\n",
    "            current_path = base_data_source_path+f\"{set_}/{label}\"\n",
    "            files = os.listdir(current_path)\n",
    "            for f in tqdm(files,total = len(files),unit = \"review \",leave = False):\n",
    "                file_name_tokens = f.split(\".\")[0].split(\"_\")\n",
    "                \n",
    "                review_id = file_name_tokens[0]\n",
    "                \n",
    "                review_rating = file_name_tokens[1]\n",
    "                \n",
    "                review_data = None\n",
    "                \n",
    "                with open(current_path+f\"/{f}\",\"r\", encoding=\"utf8\") as Fp:\n",
    "                    review_data = Fp.read()\n",
    "                \n",
    "                review_data = review_data.strip()\n",
    "                \n",
    "                rows.append((review_id,review_data,review_rating,label))\n",
    "        \n",
    "    return pd.DataFrame(columns = df_columns,data = rows)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {
    "hidden": true
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "50eb3370e5b248f9b51ea01dfef1c47f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=2.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=12500.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=12500.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=12500.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=12500.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "df_reviews = transform_data(data_source_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {
    "hidden": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>rating</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>21475</th>\n",
       "      <td>6829</td>\n",
       "      <td>I've written at least a half dozen scathing re...</td>\n",
       "      <td>1</td>\n",
       "      <td>neg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42188</th>\n",
       "      <td>2970</td>\n",
       "      <td>42/100. Often referred as \"Tarzan with clothes...</td>\n",
       "      <td>4</td>\n",
       "      <td>neg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16870</th>\n",
       "      <td>2684</td>\n",
       "      <td>I've found the movie offensive for Americans w...</td>\n",
       "      <td>1</td>\n",
       "      <td>neg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11298</th>\n",
       "      <td>891</td>\n",
       "      <td>What can I add that the previous comments have...</td>\n",
       "      <td>10</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22235</th>\n",
       "      <td>7512</td>\n",
       "      <td>I had to walk out of the theater. After an hou...</td>\n",
       "      <td>1</td>\n",
       "      <td>neg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22183</th>\n",
       "      <td>7466</td>\n",
       "      <td>Wow. Some movies just leave me speechless. Thi...</td>\n",
       "      <td>1</td>\n",
       "      <td>neg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32622</th>\n",
       "      <td>5610</td>\n",
       "      <td>Hartley on low-key form: Martin Donovan, born ...</td>\n",
       "      <td>7</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26548</th>\n",
       "      <td>11394</td>\n",
       "      <td>What has to change in today's attitude towards...</td>\n",
       "      <td>10</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25817</th>\n",
       "      <td>10736</td>\n",
       "      <td>I think not! I mean yeah if you compare this f...</td>\n",
       "      <td>9</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48409</th>\n",
       "      <td>856</td>\n",
       "      <td>Okay, now I am pretty sure that my summary got...</td>\n",
       "      <td>3</td>\n",
       "      <td>neg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13341</th>\n",
       "      <td>10758</td>\n",
       "      <td>This is probably the most boring, worse and us...</td>\n",
       "      <td>1</td>\n",
       "      <td>neg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2732</th>\n",
       "      <td>1245</td>\n",
       "      <td>Whenever I see most reviews it's called 'a mis...</td>\n",
       "      <td>7</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36117</th>\n",
       "      <td>8757</td>\n",
       "      <td>I had been very curious to see the original si...</td>\n",
       "      <td>10</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29434</th>\n",
       "      <td>2741</td>\n",
       "      <td>If there's one cartoon that helped to put UPA ...</td>\n",
       "      <td>10</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28395</th>\n",
       "      <td>1806</td>\n",
       "      <td>Sweet, rich valley girl develops crush on a pu...</td>\n",
       "      <td>9</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7790</th>\n",
       "      <td>5762</td>\n",
       "      <td>I have to admit to enjoying bad movies. I love...</td>\n",
       "      <td>8</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44155</th>\n",
       "      <td>4740</td>\n",
       "      <td>this movie made me watch Paul W.S. Anderson's ...</td>\n",
       "      <td>1</td>\n",
       "      <td>neg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20450</th>\n",
       "      <td>5906</td>\n",
       "      <td>This movie was terrible. The plot sucked, the ...</td>\n",
       "      <td>1</td>\n",
       "      <td>neg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6165</th>\n",
       "      <td>429</td>\n",
       "      <td>It has been about 50 years since a movie has b...</td>\n",
       "      <td>10</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16924</th>\n",
       "      <td>2732</td>\n",
       "      <td>When 'My Deja Vu, My Deja Vu' aired last seaso...</td>\n",
       "      <td>1</td>\n",
       "      <td>neg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          id                                               text rating label\n",
       "21475   6829  I've written at least a half dozen scathing re...      1   neg\n",
       "42188   2970  42/100. Often referred as \"Tarzan with clothes...      4   neg\n",
       "16870   2684  I've found the movie offensive for Americans w...      1   neg\n",
       "11298    891  What can I add that the previous comments have...     10   pos\n",
       "22235   7512  I had to walk out of the theater. After an hou...      1   neg\n",
       "22183   7466  Wow. Some movies just leave me speechless. Thi...      1   neg\n",
       "32622   5610  Hartley on low-key form: Martin Donovan, born ...      7   pos\n",
       "26548  11394  What has to change in today's attitude towards...     10   pos\n",
       "25817  10736  I think not! I mean yeah if you compare this f...      9   pos\n",
       "48409    856  Okay, now I am pretty sure that my summary got...      3   neg\n",
       "13341  10758  This is probably the most boring, worse and us...      1   neg\n",
       "2732    1245  Whenever I see most reviews it's called 'a mis...      7   pos\n",
       "36117   8757  I had been very curious to see the original si...     10   pos\n",
       "29434   2741  If there's one cartoon that helped to put UPA ...     10   pos\n",
       "28395   1806  Sweet, rich valley girl develops crush on a pu...      9   pos\n",
       "7790    5762  I have to admit to enjoying bad movies. I love...      8   pos\n",
       "44155   4740  this movie made me watch Paul W.S. Anderson's ...      1   neg\n",
       "20450   5906  This movie was terrible. The plot sucked, the ...      1   neg\n",
       "6165     429  It has been about 50 years since a movie has b...     10   pos\n",
       "16924   2732  When 'My Deja Vu, My Deja Vu' aired last seaso...      1   neg"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_reviews.sample(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "hidden": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50000, 4)"
      ]
     },
     "execution_count": 170,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_reviews.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "# save the transformed_data\n",
    "\n",
    "df_reviews.to_csv(\"./IMDB_raw_movie_review_data/transformed_data_reviews.csv\",index = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data-cleaning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [],
   "source": [
    "# read the transformed-data\n",
    "\n",
    "df_reviews = pd.read_csv(data_source_path+\"transformed_data_reviews.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>rating</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Bromwell High is a cartoon comedy. It ran at t...</td>\n",
       "      <td>9</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10000</td>\n",
       "      <td>Homelessness (or Houselessness as George Carli...</td>\n",
       "      <td>8</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10001</td>\n",
       "      <td>Brilliant over-acting by Lesley Ann Warren. Be...</td>\n",
       "      <td>10</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10002</td>\n",
       "      <td>This is easily the most underrated film inn th...</td>\n",
       "      <td>7</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10003</td>\n",
       "      <td>This is not the typical Mel Brooks film. It wa...</td>\n",
       "      <td>8</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id                                               text  rating label\n",
       "0      0  Bromwell High is a cartoon comedy. It ran at t...       9   pos\n",
       "1  10000  Homelessness (or Houselessness as George Carli...       8   pos\n",
       "2  10001  Brilliant over-acting by Lesley Ann Warren. Be...      10   pos\n",
       "3  10002  This is easily the most underrated film inn th...       7   pos\n",
       "4  10003  This is not the typical Mel Brooks film. It wa...       8   pos"
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_reviews.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [],
   "source": [
    "# nlp\n",
    "\n",
    "import spacy\n",
    "import nltk\n",
    "from bs4 import BeautifulSoup\n",
    "from sklearn.feature_extraction import stop_words\n",
    "from spacy.lemmatizer import Lemmatizer\n",
    "from spacy.tokenizer import Tokenizer\n",
    "from spacy.lookups import Lookups\n",
    "from nltk.tokenize import word_tokenize\n",
    "import re\n",
    "import string\n",
    "\n",
    "lemmatizer = Lemmatizer(Lookups())\n",
    "\n",
    "nlp = spacy.load(\"en_core_web_sm\")\n",
    "\n",
    "stop_words_sklearn = list(stop_words.ENGLISH_STOP_WORDS)\n",
    "stop_words_spacy = list(nlp.Defaults.stop_words)\n",
    "\n",
    "stop_words = list(set(stop_words_sklearn + stop_words_spacy)) # all the stop words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pre-process pipeline\n",
    "\n",
    "def cleanpunc(review):\n",
    "    nopunc = [char for char in review if char not in string.punctuation]\n",
    "    nopunc = ''.join(nopunc)\n",
    "    return nopunc\n",
    "\n",
    "def remove_html_content(review):\n",
    "    \"\"\"\n",
    "    remove html content if any using bs4\n",
    "    \"\"\"\n",
    "    return BeautifulSoup(review).get_text()\n",
    "\n",
    "def tokenization(review):\n",
    "    return word_tokenize(review)\n",
    "    \n",
    "\n",
    "def remove_stop_words(review):\n",
    "    return [word for word in review if not word in stop_words]\n",
    "    \n",
    "    \n",
    "def lemmatization(review):\n",
    "    return [lemmatizer.lookup(word) for word in review] \n",
    "\n",
    "\n",
    "def clean_data(reviews):\n",
    "    \"\"\"\n",
    "    the transform fn that will apply all the transforms\n",
    "    \"\"\"\n",
    "    transformed_reviews = []\n",
    "    \n",
    "    for review in tqdm(reviews,total = reviews.shape[0],unit = \"review \"):\n",
    "        if len(review) != 0:\n",
    "            review  = review.lower()\n",
    "            review = remove_html_content(review)\n",
    "            review = cleanpunc(review)\n",
    "            review_tok = tokenization(review)\n",
    "            review = remove_stop_words(review_tok)\n",
    "            review_lem = lemmatization(review)\n",
    "            \n",
    "            transformed_reviews.append(\" \".join(review_lem))\n",
    "            \n",
    "    return np.array(transformed_reviews)       "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3b660e4eb51c4b40810c1b992be124ac",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=50000.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# let's apply the above pipeline\n",
    "\n",
    "df_reviews.text = clean_data(df_reviews.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['bromwell high cartoon comedy ran time programs school life teachers 35 years teaching profession lead believe bromwell highs satire closer reality teachers scramble survive financially insightful students right pathetic teachers pomp pettiness situation remind schools knew students saw episode student repeatedly tried burn school immediately recalled high classic line inspector im sack teachers student welcome bromwell high expect adults age think bromwell high far fetched pity isnt',\n",
       "       'homelessness houselessness george carlin stated issue years plan help street considered human going school work vote matter people think homeless lost cause worrying things racism war iraq pressuring kids succeed technology elections inflation worrying theyll end streetsbut given bet live streets month luxuries home entertainment sets bathroom pictures wall computer treasure like homeless goddard bolts lessonmel brooks directs stars bolt plays rich man world deciding bet sissy rival jeffery tambor live streets thirty days luxuries bolt succeeds wants future project making buildings bets bolt thrown street bracelet leg monitor step sidewalk hes given nickname pepto vagrant written forehead bolt meets characters including woman molly lesley ann warren exdancer got divorce losing home pals sailor howard morris fumes teddy wilson streets theyre survivors bolt isnt hes reaching mutual agreements like rich fight flight kill killedwhile love connection molly bolt wasnt necessary plot life stinks mel brooks observant films prior comedy shows tender compared slapstick work blazing saddles young frankenstein spaceballs matter like having valuable losing day hand making stupid bet like rich people dont know money maybe homeless instead like monopoly moneyor maybe film inspire help'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 177,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_reviews.text.values[:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save the cleaned reviews\n",
    "\n",
    "df_reviews.to_csv(data_source_path+\"cleaned_imdb_reviews.csv\",index = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Some visuals and  feature eng."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.style.use(\"ggplot\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [],
   "source": [
    "# read the cleaned reviews\n",
    "\n",
    "df_reviews = pd.read_csv(data_source_path+\"cleaned_imdb_reviews.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>rating</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>bromwell high cartoon comedy ran time programs...</td>\n",
       "      <td>9</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10000</td>\n",
       "      <td>homelessness houselessness george carlin state...</td>\n",
       "      <td>8</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10001</td>\n",
       "      <td>brilliant overacting lesley ann warren best dr...</td>\n",
       "      <td>10</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10002</td>\n",
       "      <td>easily underrated film inn brooks cannon sure ...</td>\n",
       "      <td>7</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10003</td>\n",
       "      <td>typical mel brooks film slapstick movies actua...</td>\n",
       "      <td>8</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id                                               text  rating label\n",
       "0      0  bromwell high cartoon comedy ran time programs...       9   pos\n",
       "1  10000  homelessness houselessness george carlin state...       8   pos\n",
       "2  10001  brilliant overacting lesley ann warren best dr...      10   pos\n",
       "3  10002  easily underrated film inn brooks cannon sure ...       7   pos\n",
       "4  10003  typical mel brooks film slapstick movies actua...       8   pos"
      ]
     },
     "execution_count": 181,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_reviews.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "rating\n",
       "1     10122\n",
       "2      4586\n",
       "3      4961\n",
       "4      5331\n",
       "7      4803\n",
       "8      5859\n",
       "9      4607\n",
       "10     9731\n",
       "dtype: int64"
      ]
     },
     "execution_count": 182,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# group by on ratings\n",
    "\n",
    "df_reviews.groupby('rating').size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEJCAYAAAB/pOvWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X9U1HW+x/HnCIHiCA0z/gjDH4RuNxWxIMluwiq7a9va0s+9/thN007KXk2oVip3293aou4aLKm5p7p6b3bqdCrZ1sq7lyXwHLm2o4AhJeJGpkcNYfDH+AMEPvcPj5PjF38N6mC8Huf4x3z4fL/f9+fDyGu+3893ZmzGGIOIiMgpegS7ABER6XoUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKB+nShgwZwrPPPhvsMq5ov/3tb4mPj78sx6qqquLmm2+mZ8+eDBky5LIcUy4NhcMVKi0tjdmzZ1vad+3ahc1mo6Sk5PIX1QmzZ88mLS3N0u52u8nKyrr8BUlAfvWrXxEZGcnWrVtxu90d9tmzZw/Tpk1jxIgRhIaGkp6eft77/8Mf/sBtt91GZGQkNpuNXbt2ddjvxRdfZPDgwfTs2ZMxY8bwt7/9LaDxdGcKB7lgLS0tl6RvR/r27Uvv3r07tQ+5fGpra0lNTWXIkCH07du3wz7Nzc1ER0eTnZ19QcFwcts777yTp5566ox98vPzefrpp3nmmWeoqKjgBz/4AZMnT+azzz67oGN1e0auSKmpqWbWrFmW9p07dxrAfPLJJ8YYY1paWkxWVpYZOHCgCQsLMwMGDDA/+9nP/LZ56623zOjRo014eLgZPHiwycrKMl6v1+9YDz74oFm0aJEZMGCAcblcHdb0ySefGMCsWbPG3HrrrSY8PNy8/PLLxuPxmGnTppnY2FjTs2dPM3z4cPPHP/7RtLe3G2OMefrppw3g92/FihXGGGMGDx5snnnmGd8xBg8ebH7961+b+fPnG4fDYfr162ceffRR09ra6utz5MgR89BDD5nIyEhz9dVXm7lz55qcnBxz3XXX+fps2bLF/PCHPzRRUVEmIiLCXH/99ea///u/zzrnH374obnxxhtNWFiY6du3r5k7d67fPD3wwANm4sSJ5s9//rMZNGiQ6dOnj7nzzjtNfX39WfdbWFhoEhMTTa9evUxUVJRJTk425eXlvp9v377d3HvvvcbhcJhevXqZUaNGmb/+9a/GGHPOuT05v6eO3Rhj/va3v5lx48aZnj17mpiYGDNjxgzT0NBw1jp3795tfvazn5moqCjTs2dPk5qaatxutzHGmLq6Osvv8Omnnz7r/k6dswt18rm2c+dOv/b29nYTExNjnnjiCb/2pKQk88ADD1zwcbqz0CBlklwmL7/8Mu+88w6rVq0iLi6Ob775hvXr1/t+vnLlSrKysigoKODWW29l165d/Pu//zv79u3jjTfe8PV75513mDZtGn//+99pa2s76zEfffRRXnzxRUaNGsVVV11Fc3Mzo0aNIjs7G4fDwfr165kzZw7R0dHMnDmTxx57jNraWurq6nj//fcBiIqKOuuYFi5cyKeffkp5ebnvEsXMmTMBWLhwIX/5y1944403+N73vsfKlStZtmyZ3yvZKVOmMHLkSMrKyujZsyc1NTVnHddnn33GnXfeybx581i1ahVfffUVDz/8MIcOHfKbJ7fbTd++ffnwww85ePAgU6ZM4bHHHuO//uu/Otzv3r17ue+++3j22We57777OHbsGBUVFYSGhvp+Pm7cOEaNGsUHH3zANddcw5YtW+jR48RJ/7nmtiPFxcX89Kc/5YUXXmDlypXs37+fX/3qV9x1112UlpZis9ks2xhjyMjIoLm5mTVr1hAVFcWzzz7LD37wA2pra4mNjWXPnj0kJyczbdo0FixYgN1uP+N8XipfffUVu3fvZtKkSX7tkyZN4q233rrs9VzRgp1OEpjzPXOYP3+++f73v+/3SvJUgwcPNq+88opfW2lpqQGMx+PxHWvYsGGmra3trDWdfDV3rlfgJ+tKT0/3PZ41a5ZJTU3tsL7TzxwmT57s1+dHP/qR+bd/+zdjjDFer9eEhYWZ1157za/P2LFj/V49R0ZG+s5Ozsf06dNNcnKyX1thYaGx2Wzmq6++MsaceBXscrnMsWPHfH2ef/55M2DAgDPut7y83ACmrq6uw58vWrTI9O/f3+8M5VxOn9vTzxxSU1PNwoUL/bbZsWOHAUxFRUWH+ywqKjKAqa6u9rUdO3bMDBgwwPzud7/ztZ3++zqXi33msH79egOYmpoav/YlS5aYiIiICz5Od6Y1h++4mTNnUlVVRXx8PHPmzOG9997zrQPs27ePHTt2kJ2djd1u9/27/fbbAdi+fbtvPzfddJPv1eq53HzzzX6P29vbyc3NJTExEZfLhd1uZ/ny5ezYsSOgMSUmJvo9HjhwIN98842v5paWFlJSUvz63HLLLX6PH3vsMd8i+G9/+1vKy8vPeszq6mrGjx/v15aamooxhs8//9zX9i//8i+Eh4d3WFtHEhIS+NGPfsTIkSO56667+NOf/sTOnTt9P9+0aRPjxo0747pLIHPrdrvJz8/3+53fcMMNwIk1gzON3+l0+voBhIeHM3bsWKqrq894rECNGDHCV9uIESMuyj47OiOSM1M4XKHCw8M5cOCApX3//v0A9OzZEzjxh7Suro4//vGPhIWF8cgjj5CYmMjBgwdpb28H4E9/+hOVlZW+f5s3b6a2tpZRo0b59nshi8Kn9128eDHPP/888+bN43//93+prKxk9uzZAS9Wh4WF+T222Wy+sZzadja//vWv2bZtG/fffz9btmwhJSWFRYsWnXWbM+3z1PaOajNn+eDjkJAQPv74Y4qLi0lOTua9995j+PDhrFmz5rzGEsjctre3s3DhQr/feWVlJbW1tb4XBuca50nGmEvyR/ejjz7y1fXRRx+d93bXXHMNcOJy3Km++eYbBgwYcFFr/K5TOFyhrr/+ejZt2mS5Tv6Pf/yDHj16MGzYMF+b3W7nrrvuoqCggI0bN/LFF19QWlpK//79iY2Npaamhvj4eMu/kwHTWevWrWPSpEnMmjWLMWPGEB8fb3mFGhYWds61jPMRHx9PWFgY//d//+fXvmHDBkvfuLg4MjMzeffdd/n973/PK6+8csb9jhgxgtLSUr+2k9fnT301HQibzcbNN9/Mk08+ybp160hNTWXFihXAiTO29evXc/jw4Q63PZ+5PV1SUhLV1dUd/s7PtE4wYsQIGhoa/M6Smpub+cc//nHRXtmfavDgwb6aBg8efN7bDRkyhJiYGP7nf/7Hr33t2rX867/+68Uu8ztN4XCFmjNnDnv37mXmzJls2rSJf/7zn7z99ts8+eST/OIXv8DpdALwH//xH7z55ptUV1dTV1fHf/7nfxISEsLw4cOBE/eNFxQU8Oyzz7JlyxZqamooLCzk4Ycfvmi1fu9736OkpIRPPvmEbdu2sWjRIj799FO/PkOHDmXr1q1UV1fT0NBAc3NzQMfq3bs3Dz/8MIsWLWLNmjVs27aNp556ii+++ML3Ctfr9fLLX/6S4uJi6urqqKioYO3atWf9I//4449TXl5OdnY2W7duZe3atcybN49p06YxaNCggGoFKCsr45lnnuHTTz/l66+/5u9//zufffaZr5bMzEza29v56U9/yvr166mrq2PNmjV8/PHHwPnN7el+//vf85e//IWsrCwqKyv55z//ydq1a5k1axZHjx7tcJsJEyZw8803M3XqVNavX8+WLVv4xS9+wbFjx5g7d+4Fj/vkWYHH48Hr9foen8vXX39NZWWl75Ln559/7tsPnAjaxx9/nLy8PFatWsXWrVvJyclh8+bNer/MhQrymod0wubNm83kyZPNNddcYyIiIszIkSPNCy+8YJqbm319li9fbm688UbTp08f07t3b5OUlGQKCwv99rN69WqTkpJievXqZfr06WNGjx7tt8h4psXv051pkXD//v3mvvvuM3369DHR0dEmMzPTLFq0yAwePNjXp7Gx0dx+++0mMjLynLeynr7gefpi9slbWfv06WOioqLM3LlzzSOPPGJGjhxpjDHm6NGjZsqUKWbIkCEmPDzc9O3b19x///3m66+/Puv4Tr2V1eVymTlz5nR4K+up3njjDXO2/2Zbtmwxt99+u+nfv78JCwszgwYNMo899pjf77CmpsZkZGSYyMhI06tXL5OQkGA+/PDD857bjm5lXbdunZk4caKx2+2+W3kfeeQRc/z48TPWevqtrOPHj/fdynrS+S5Ic9ptryf/ncsDDzzQ4Xan31zwwgsvmNjYWBMWFmZGjx5t1q5de859iz+bMfomOPnumzBhAg6Hg/feey/YpYhcEfQ+B/nOqaqqory8nFtuuYWWlhbeeOMNPvnkkwta2BTp7hQO8p1js9l45ZVXmD9/Pu3t7Vx//fWsXr36rHfiiIg/XVYSEREL3a0kIiIWCgcREbG4otccdu/eHewSOsXlctHQ0BDsMroEzYU/zYc/zce3OjMXMTEx591XZw4iImKhcBAREQuFg4iIWCgcRETEQuEgIiIWCgcREbFQOIiIiIXCQURELM75Jrhly5ZRXl5OVFQUixcvBk58WUpeXh779u2jb9++ZGVlYbfbMcawYsUKKioqCA8PJzMzk7i4OABKSkp4//33Abj77rtJS0sD4Msvv2Tp0qW0tLQwZswYZs6cqe96FREJsnOGQ1paGpMmTWLp0qW+tsLCQkaNGkVGRgaFhYUUFhYyffp0Kioq2Lt3LwUFBdTW1vLaa6/x3HPP4fV6effdd8nNzQUgJyeHpKQk7HY7r776Kg8//DDDhg3j+eefp7KykjFjxly6EQNtD915Sfd/JiGvfhCU44qIXKhzXla64YYbLN8r63a7SU1NBSA1NRW32w3Axo0bGT9+PDabjeHDh3P48GGampqorKwkISEBu92O3W4nISGByspKmpqaOHr0KMOHD8dmszF+/HjfvkREJHgC+mylAwcO4HA4AHA4HBw8eBAAj8eDy+Xy9XM6nXg8Hjwej+87jQGio6M7bD/Z/0yKioooKioCIDc31+9YF+KbgLbqvNPrDQ0NDXgM3zWaC3+aD3+aj29drrm4qB+819FXQ5xp/cBms3XY/2zS09NJT0/3Pb7SPojr9Hr1YWLf0lz403z403x8q0t/8F5UVBRNTU0ANDU1ERkZCZx45X9q0Y2NjTgcDqKjo2lsbPS1ezweHA4HTqfTr72xsZHo6OhAShIRkYsooHBISkqitLQUgNLSUpKTk33t69atwxjDtm3biIiIwOFwkJiYyObNm/F6vXi9XjZv3kxiYiIOh4NevXqxbds2jDGsW7eOpKSkizc6EREJyDkvK+Xn5/P5559z6NAh5syZw/33309GRgZ5eXkUFxfjcrnIzs4GYMyYMZSXlzN//nzCwsLIzMwEwG63c8899/DEE08AcO+99/oWuWfPns2yZctoaWkhMTHxkt+pJCIi53ZFf4d0oF/201VuZdV11G9pLvxpPvxpPr51udYcruhvghMRCZZgvchkddllOYw+PkNERCwUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsFA4iImKhcBAREQuFg4iIWCgcRETEQuEgIiIWCgcREbFQOIiIiIXCQURELBQOIiJioXAQERELhYOIiFgoHERExELhICIiFgoHERGxUDiIiIiFwkFERCwUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsQjtzMZr1qyhuLgYm81GbGwsmZmZ7N+/n/z8fLxeL0OHDmXevHmEhoZy/PhxlixZwpdffkmfPn1YsGAB/fr1A2D16tUUFxfTo0cPZs6cSWJi4kUZnIiIBCbgMwePx8PHH39Mbm4uixcvpr29nbKyMlatWsUdd9xBQUEBvXv3pri4GIDi4mJ69+7Nyy+/zB133MGbb74JwK5duygrK+Oll17iqaee4vXXX6e9vf3ijE5ERALSqctK7e3ttLS00NbWRktLC1dffTXV1dWkpKQAkJaWhtvtBmDjxo2kpaUBkJKSwpYtWzDG4Ha7GTduHFdddRX9+vVjwIABbN++vXOjEhGRTgn4slJ0dDSTJ09m7ty5hIWFMXr0aOLi4oiIiCAkJMTXx+PxACfONJxOJwAhISFERERw6NAhPB4Pw4YN89vvyW1OV1RURFFREQC5ubm4XK6Aav8moK067/R6Q0NDAx7Dd43mwp/mw19XnI9g/R25XHMRcDh4vV7cbjdLly4lIiKCl156icrKyjP2N8ZY2mw2W4ftZ5Kenk56errvcUNDw4UVHWSn1+tyua64MVwqmgt/mg9/mo9vtba2BjwXMTEx59034MtKVVVV9OvXj8jISEJDQxk7diw1NTUcOXKEtrY24MTZQnR0NABOp5PGxkYA2traOHLkCHa73a/99G1ERCQ4Ag4Hl8tFbW0tzc3NGGOoqqri2muvZcSIEWzYsAGAkpISkpKSALjpppsoKSkBYMOGDYwYMQKbzUZSUhJlZWUcP36c+vp69uzZQ3x8fOdHJiIiAQv4stKwYcNISUlh4cKFhISEMGTIENLT07nxxhvJz8/n7bffZujQoUyYMAGACRMmsGTJEubNm4fdbmfBggUAxMbGcsstt5CdnU2PHj2YNWsWPXro7RciIsFkMxdy0b+L2b17d0DbtT1050Wu5PyEvPqB32NdR/2W5sKf5sNfV5yPYP0d6b+6rGuvOYiIyHeXwkFERCwUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsFA4iImKhcBAREQuFg4iIWCgcRETEQuEgIiIWCgcREbFQOIiIiIXCQURELBQOIiJioXAQERELhYOIiFgoHERExELhICIiFgoHERGxUDiIiIiFwkFERCwUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsQjtzMaHDx9m+fLl7Ny5E5vNxty5c4mJiSEvL499+/bRt29fsrKysNvtGGNYsWIFFRUVhIeHk5mZSVxcHAAlJSW8//77ANx9992kpaV1emAiIhK4ToXDihUrSExM5NFHH6W1tZXm5mZWr17NqFGjyMjIoLCwkMLCQqZPn05FRQV79+6loKCA2tpaXnvtNZ577jm8Xi/vvvsuubm5AOTk5JCUlITdbr8oAxQRkQsX8GWlI0eO8MUXXzBhwgQAQkND6d27N263m9TUVABSU1Nxu90AbNy4kfHjx2Oz2Rg+fDiHDx+mqamJyspKEhISsNvt2O12EhISqKysvAhDExGRQAV85lBfX09kZCTLli1jx44dxMXFMWPGDA4cOIDD4QDA4XBw8OBBADweDy6Xy7e90+nE4/Hg8XhwOp2+9ujoaDweT4fHLCoqoqioCIDc3Fy//V2IbwLaqvNOrzc0NDTgMXzXaC78aT78dcX5CNbfkcs1FwGHQ1tbG3V1dTz44IMMGzaMFStWUFhYeMb+xhhLm81m67DvmdrT09NJT0/3PW5oaLjAqoPr9HpdLtcVN4ZLRXPhT/PhT/PxrdbW1oDnIiYm5rz7BnxZyel04nQ6GTZsGAApKSnU1dURFRVFU1MTAE1NTURGRvr6nzqgxsZGHA4H0dHRNDY2+to9Ho/vzENERIIj4DOHq6++GqfTye7du4mJiaGqqoprr72Wa6+9ltLSUjIyMigtLSU5ORmApKQk1q5dy6233kptbS0RERE4HA4SExN566238Hq9AGzevJmpU6denNGJyGXR9tCdl3T/Z7qEE/LqB5f0uN1Zp+5WevDBBykoKKC1tZV+/fqRmZmJMYa8vDyKi4txuVxkZ2cDMGbMGMrLy5k/fz5hYWFkZmYCYLfbueeee3jiiScAuPfee3WnkohIkHUqHIYMGeK7BfVUv/nNbyxtNpuN2bNnd7ifCRMm+O56EhGR4NM7pEVExELhICIiFgoHERGxUDiIiIiFwkFERCwUDiIiYqFwEBERC4WDiIhYdOpNcCLncr4fq3ApPuFSH60gEjidOYiIiIXCQURELBQOIiJioXAQERELhYOIiFgoHERExEK3snYTl/qbukTku0VnDiIiYqFwEBERC4WDiIhYaM1B5DtC60pyMenMQURELBQOIiJioXAQERELrTlcRqdfE74UH1Mt0p1oneXS0ZmDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsFA4iImKhcBAREYtOvwmuvb2dnJwcoqOjycnJob6+nvz8fLxeL0OHDmXevHmEhoZy/PhxlixZwpdffkmfPn1YsGAB/fr1A2D16tUUFxfTo0cPZs6cSWJiYqcHJiIigev0mcNHH33EwIEDfY9XrVrFHXfcQUFBAb1796a4uBiA4uJievfuzcsvv8wdd9zBm2++CcCuXbsoKyvjpZde4qmnnuL111+nvb29s2WJiEgndCocGhsbKS8vZ+LEiQAYY6iuriYlJQWAtLQ03G43ABs3biQtLQ2AlJQUtmzZgjEGt9vNuHHjuOqqq+jXrx8DBgxg+/btnSlLREQ6qVOXlVauXMn06dM5evQoAIcOHSIiIoKQkBAAoqOj8Xg8AHg8HpxOJwAhISFERERw6NAhPB4Pw4YN8+3z1G1OV1RURFFREQC5ubm4XK6A6tZnGnUPgT4/uoLQ0NALrl/P6+4hkOdGQMcJdMNNmzYRFRVFXFwc1dXV5+xvjLG02Wy2DtvPJD09nfT0dN/jhoaG895Wup8r+fnhcrmu6Prl0mltbQ34uRETE3PefQMOh5qaGjZu3EhFRQUtLS0cPXqUlStXcuTIEdra2ggJCcHj8RAdHQ2A0+mksbERp9NJW1sbR44cwW63+9pPOnUbEREJjoDXHKZOncry5ctZunQpCxYsYOTIkcyfP58RI0awYcMGAEpKSkhKSgLgpptuoqSkBIANGzYwYsQIbDYbSUlJlJWVcfz4cerr69mzZw/x8fGdH5mIiATson+fw7Rp08jPz+ftt99m6NChTJgwAYAJEyawZMkS5s2bh91uZ8GCBQDExsZyyy23kJ2dTY8ePZg1axY9eujtFyIiwWQzF3LRv4vZvXt3QNvpC0K6h5BXPwh2CQELZM1Bz+vuof/qssuy5qCX6CIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsFA4iImKhcBAREQuFg4iIWCgcRETEQuEgIiIWCgcREbFQOIiIiIXCQURELBQOIiJioXAQERELhYOIiFgoHERExELhICIiFgoHERGxUDiIiIiFwkFERCwUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsQgPdsKGhgaVLl7J//35sNhvp6en8+Mc/xuv1kpeXx759++jbty9ZWVnY7XaMMaxYsYKKigrCw8PJzMwkLi4OgJKSEt5//30A7r77btLS0i7K4EREJDABh0NISAg///nPiYuL4+jRo+Tk5JCQkEBJSQmjRo0iIyODwsJCCgsLmT59OhUVFezdu5eCggJqa2t57bXXeO655/B6vbz77rvk5uYCkJOTQ1JSEna7/aINUkRELkzAl5UcDofvlX+vXr0YOHAgHo8Ht9tNamoqAKmpqbjdbgA2btzI+PHjsdlsDB8+nMOHD9PU1ERlZSUJCQnY7XbsdjsJCQlUVlZehKGJiEigAj5zOFV9fT11dXXEx8dz4MABHA4HcCJADh48CIDH48Hlcvm2cTqdeDwePB4PTqfT1x4dHY3H4+nwOEVFRRQVFQGQm5vrt78L8U1AW8mVJtDnR1cQGhp6wfXred09BPLcCOg4nd3BsWPHWLx4MTNmzCAiIuKM/YwxljabzdZh3zO1p6enk56e7nvc0NBwgdVKd3IlPz9cLtcVXb9cOq2trQE/N2JiYs67b6fuVmptbWXx4sXcdtttjB07FoCoqCiampoAaGpqIjIyEjhxpnDqgBobG3E4HERHR9PY2Ohr93g8vjMPEREJjoDDwRjD8uXLGThwID/5yU987UlJSZSWlgJQWlpKcnKyr33dunUYY9i2bRsRERE4HA4SExPZvHkzXq8Xr9fL5s2bSUxM7OSwRESkMwK+rFRTU8O6desYNGgQjz/+OABTpkwhIyODvLw8iouLcblcZGdnAzBmzBjKy8uZP38+YWFhZGZmAmC327nnnnt44oknALj33nt1p5KISJDZTEeLAVeI3bt3B7Rd20N3XuRKpCsKefWDYJcQsEDWHPS87h76ry7r+msOIiLy3aRwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsFA4iImKhcBAREQuFg4iIWCgcRETEQuEgIiIWCgcREbFQOIiIiIXCQURELBQOIiJioXAQERELhYOIiFgoHERExELhICIiFgoHERGxUDiIiIiFwkFERCwUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsQoNdwEmVlZWsWLGC9vZ2Jk6cSEZGRrBLEhHptrrEmUN7ezuvv/46Tz75JHl5eaxfv55du3YFuywRkW6rS4TD9u3bGTBgAP379yc0NJRx48bhdruDXZaISLfVJS4reTwenE6n77HT6aS2ttbSr6ioiKKiIgByc3OJiYkJ7IAfbgxsO5HL6IKf33pedxsB/+27AF3izMEYY2mz2WyWtvT0dHJzc8nNzb0cZV1yOTk5wS6hy9Bc+NN8+NN8fOtyzUWXCAen00ljY6PvcWNjIw6HI4gViYh0b10iHK677jr27NlDfX09ra2tlJWVkZSUFOyyRES6rS6x5hASEsKDDz7IH/7wB9rb2/n+979PbGxssMu65NLT04NdQpehufCn+fCn+fjW5ZoLm+nogr+IiHRrXeKykoiIdC0KBxERsegSaw7dSUNDA0uXLmX//v3YbDbS09P58Y9/HOyygq69vZ2cnByio6O79W2Lhw8fZvny5ezcuRObzcbcuXMZPnx4sMsKmjVr1lBcXIzNZiM2NpbMzEzCwsKCXdZls2zZMsrLy4mKimLx4sXBQLPBAAADOklEQVQAeL1e8vLy2LdvH3379iUrKwu73X7Rj61wuMxCQkL4+c9/TlxcHEePHiUnJ4eEhASuvfbaYJcWVB999BEDBw7k6NGjwS4lqFasWEFiYiKPPvoora2tNDc3B7ukoPF4PHz88cfk5eURFhbGSy+9RFlZGWlpacEu7bJJS0tj0qRJLF261NdWWFjIqFGjyMjIoLCwkMLCQqZPn37Rj63LSpeZw+EgLi4OgF69ejFw4EA8Hk+QqwquxsZGysvLmThxYrBLCaojR47wxRdfMGHCBABCQ0Pp3bt3kKsKrvb2dlpaWmhra6OlpaXbvf/phhtusJwVuN1uUlNTAUhNTb1kHzWkM4cgqq+vp66ujvj4+GCXElQrV65k+vTp3f6sob6+nsjISJYtW8aOHTuIi4tjxowZ9OzZM9ilBUV0dDSTJ09m7ty5hIWFMXr0aEaPHh3ssoLuwIEDvpB0OBwcPHjwkhxHZw5BcuzYMRYvXsyMGTOIiIgIdjlBs2nTJqKionxnU91ZW1sbdXV1/PCHP+TFF18kPDycwsLCYJcVNF6vF7fbzdKlS/nzn//MsWPHWLduXbDL6jYUDkHQ2trK4sWLue222xg7dmywywmqmpoaNm7cyC9/+Uvy8/PZsmULBQUFwS4rKJxOJ06nk2HDhgGQkpJCXV1dkKsKnqqqKvr160dkZCShoaGMHTuWbdu2BbusoIuKiqKpqQmApqYmIiMjL8lxdFnpMjPGsHz5cgYOHMhPfvKTYJcTdFOnTmXq1KkAVFdX89e//pX58+cHuarguPrqq3E6nezevZuYmBiqqqq69Y0KLpeL2tpampubCQsLo6qqiuuuuy7YZQVdUlISpaWlZGRkUFpaSnJy8iU5jt4hfZlt3bqV3/zmNwwaNMj3ybNTpkzhxhtvDHJlwXcyHLrzraxfffUVy5cvp7W1lX79+pGZmXlJblO8UrzzzjuUlZUREhLCkCFDmDNnDldddVWwy7ps8vPz+fzzzzl06BBRUVHcf//9JCcnk5eXR0NDAy6Xi+zs7EvyHFE4iIiIhdYcRETEQuEgIiIWCgcREbFQOIiIiIXCQURELBQOIiJioXAQERGL/wfsmgee3l+4kwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(df_reviews.rating)\n",
    "plt.title(\"User ratings on scale of 1-10\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Observation** - Most reviews are rated either 1 or 10. No review has ratings = `5` or `6`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"4\" halign=\"left\">text</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>unique</th>\n",
       "      <th>top</th>\n",
       "      <th>freq</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>label</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>neg</th>\n",
       "      <td>25000</td>\n",
       "      <td>24695</td>\n",
       "      <td>got movie free job similar movies watched low ...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>pos</th>\n",
       "      <td>25000</td>\n",
       "      <td>24883</td>\n",
       "      <td>loved todays variety solely cooking great stim...</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        text                                                               \n",
       "       count unique                                                top freq\n",
       "label                                                                      \n",
       "neg    25000  24695  got movie free job similar movies watched low ...    3\n",
       "pos    25000  24883  loved todays variety solely cooking great stim...    5"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_reviews[['text','label']].groupby(\"label\").describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pos    25000\n",
       "neg    25000\n",
       "Name: label, dtype: int64"
      ]
     },
     "execution_count": 185,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# class distribution\n",
    "\n",
    "df_reviews['label'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAO4AAAD8CAYAAABw8JiyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAG0RJREFUeJzt3XlYlOXeB/DvMwPDsA0om4ApmyCQqAmmotg57h6zzquW9eaSYq7F63XM5TXLUltMU092XvWo5cEtt1OalSWJkJkihiWCAoq7xCL7Mtvz/kHOEREcFLjve+b3uS6vmmGWL+Pz9b6fZ55FkmVZBiFEKArWAQghTUfFJURAVFxCBETFJURAVFxCBETFJURAVlfciRMnYuDAgU16jp+fH5YuXfrI7/2wryNJErZu3drg7eb01FNPITY2tsHbza25PltrY8M6AGm6mzdvwtXV1azHbt26FePGjYO5X9fv27cPNjbNv1jExsYiOzsbiYmJde5PSUmBg4NDs7+fpaPiCqhdu3bN/pparRYqlQpt27Zt9tdujIeHR6u+n6WwuqnyvU6fPo1hw4bB09MTTk5OiIqKwrffflvvcVVVVYiNjYVGo4G7uzvmzZsHo9Fo+rler8fixYvh7+8PtVqN8PBwrF+/vsl5jhw5goiICKjVakRERODIkSP1HnPvVHnjxo0IDQ2FWq2Gm5sbYmJicO3aNSQmJmLcuHGm50iShIkTJwKonQJPnjwZixYtgre3N3x9fU333zs1NhqNmD9/Ptzd3aHRaBAbG4uqqirTz+/3nKVLl8LPzw8AsHjxYmzatAlHjx415fjss88A1J8ql5WVYerUqfDw8IBarUZkZCS+++47089zc3MhSRJ27dqFp59+Gg4ODggICEB8fHwTP2mxWX1xS0tLMXbsWCQmJuL06dMYMmQIRo4ciQsXLtR53McffwwfHx+kpKRg1apVWLt2LVavXm36eWxsLPbt24f169cjIyMDb775JubNm4dNmzaZneXGjRsYMWIEevTogdOnT2PlypWIi4tr9DmpqamYNm0aFixYgPPnzyMxMRHjx48HAPTp0wdr164FUDu9vnnzJtasWWN67q5du5Cfn4+EhAT88MMPDb7Hnj17UFhYiOTkZGzbtg379+/HvHnzzP695syZgxdffBG9e/c25Xj++efv+9hJkybh0KFD2Lp1K3755RdER0djxIgRyMzMrPO4+fPnY9y4cfj111/x3HPP4eWXX0ZWVpbZmYQnW5kJEybIAwYMaPQxERER8tKlS023O3bsKPft27fOYxYsWCD7+vrKsizLFy9elCVJkjMyMuo85u2335a7du1a53WWLFnS4PsuXLhQ7tChg6zT6Uz3HThwQAYgx8fHm+67+/a+fftkjUYjl5SU3Pc14+Pj5fv9Nffv31/u1KmTbDAY6t0/efLkOrc7duwo6/V6033r16+XVSqVXF5eft/nyLIsL1myRO7YsaPp9uTJk+X+/fvXy3H3Z5KVlSUDkA8ePFjnMd27d5dffvllWZZl+dKlSzIAeeXKlaaf63Q62dHRUV63bt19PwNLZPUjbn5+PmbMmIHOnTvD1dUVTk5OSE9Px+XLl+s8rnfv3nVuR0dH4/r16ygtLcWpU6cgyzIiIyPh5ORk+vPuu+82aRQ4d+4cevbsWWfjUN++fRt9zqBBgxAQEAB/f3+MHTsWGzZsQEFBgVnv16NHDygUD14EevbsCaVSabodHR0NrVaLnJwcs97HXOfOnQMAxMTE1Lk/JiYG6enpde7r1q2b6f9tbGzg5eWFvLy8Zs3DM6vfODVx4kRcuXIFy5cvh7+/P+zt7TF27FhotdpGnyfftZX2zrruTz/9VG8LqSRJZmeRZbne4x/0fCcnJ5w6dQrHjh3D4cOHsW7dOsydOxcJCQno0aNHo891dHQ0O9u9Oe+mUCjq3afT6R7qtRt6v3s/B5VKVee2JEl1tjlYOqsfcZOSkjBjxgyMHDkSXbp0gbe3Ny5evFjvcT///HOd28ePH4ePjw80Go2pIFeuXEFQUFCdP4GBgWZnCQ8Px4kTJ2AwGEz3/fjjjw98nlKpRExMDN555x2kpqbC29sb27dvB/CfBfzu12yqlJSUOs8/fvw4VCqV6Xfz9PTEjRs36jzn9OnTdW6rVKoHZggPDwdQ+3dyt+TkZNPPSC2rL25ISAi2bduG3377DWlpaXjhhRfuu4ClpaVh8eLFuHDhArZv3441a9Zg9uzZAICgoCBMmjQJU6ZMQXx8PLKzs3HmzBls3rwZH3zwgdlZpk+fjvz8fLzyyivIyMhAQkICFi5c2OhzvvzyS6xatQqpqam4cuUKvvjiC1y9ehVhYWEAAH9/fwDA/v37kZ+fj/LycrPz3FFYWIiZM2ciIyMDBw8exKJFizBlyhTTiD1w4EAcPnwYu3btQnZ2Nt5//30kJyfXeQ1/f39kZmYiPT0dBQUFqKmpqfc+gYGBGDNmDGbMmIFDhw4hMzMTcXFxOHv2LF5//fUm57ZkVl/cTz/9FEajET179sSzzz6LoUOHIioqqt7jXn31VVy+fBmRkZGYNWsWpk+fbiouAGzYsAGzZ8/GsmXLEBYWhgEDBmDLli0ICAgwO4uvry8OHDiAkydPolu3boiLi8NHH33U6HPatGmDAwcOYOjQoQgODsbcuXPxxhtvYNKkSQCAqKgoxMXFYdq0afDy8sKsWbPMznPH6NGj4ezsjL59+2Ls2LEYPnw4li9fbvr5hAkTMHPmTMyaNQuRkZG4evUqXnvttTqvMXnyZERFRaFPnz7w8PDAjh077vteGzduxJAhQ/DSSy+ha9euOHbsGL766it07ty5ybktmSTfu3JCCOGe1Y+4hIiIikuIgKi4hAiIikuIgKi4hAiIikuIgKi4hAiIikuIgKi4hAiIikuIgKi4hAiIikuIgKi4hAiIikuIgKi4hAiIimvhcnNz0blzZ0yYMAEREREYPXo0KisrkZCQgO7du6NLly6YNGmS6YwU8+fPR1hYGCIiIjBnzhzG6UlD6EB6C5ebmwt/f3/8+OOPiI6OxqRJkxAQEID169cjISEBwcHBGD9+PJ544gmMHz8evXv3RmZmJiRJQnFxsdmXOiGti0ZcK/DYY48hOjoaAPDSSy8hISEB/v7+CA4OBlB76pmkpCRoNBqo1WrTyd3pmj78ouJaAXNPEWtjY4OTJ09i1KhR+OKLLzB06NAWTkYeFhXXCly5cgXHjx8HAOzYsQMDBw5Ebm4usrOzAQDx8fHo378/ysvLUVJSguHDh2P16tVIS0tjGZs0wupPiG4NQkNDsWXLFkydOhWdOnXCmjVr0KtXL4wZMwZ6vR5RUVGYNm0aioqK8Mwzz6C6uhqyLGPVqlWso5MG0MYpC5ebm4sRI0bg7NmzrKOQZkRTZUIERCMuIQKiEZcQAVFxCREQbVUWnLGsFIbbBTAU5sNYUQZZrweMhtr/yjKgUEBSKgGFEpKtLRTOLlC6eUDZ1gMKe9rBQlRUXM4Zy0qhzcmENuc8DPm3YCgqgKEoH4aiQhhvF0DW1r/qnbkkewco27hD2bb2j6KtO2y8fKAK6gzbgBAo1PbN+JuQ5kQbpzhiKml2BrRZGdBmZ8Bw6zqbMAoFbNr7QdUpFKqgUCozZ6i4DMk6LarPpKDqRBJqfjkB/c1rrCM1TqGA7WP+UPfoA/snY6AKjaidhpNWR8VtZYaSYlSfTEbViSRUp52AXFXJOtJDU2hcoY6Khv2TMVA/0ZvWmVsRFbcV6At+R+XRb1H181FoM38DjEbWkZqfrQrqiEjY94qBQ8wQKJycWSeyaFTcFiLLMmrSTqL84B5UnUwCDAbWkVqNZKeGQ8xgOI14DqogupJ8S6DiNjNjeRkqDh9A+Td7ob92mXUc5lTB4XD6yxg4xAyCpLJjHcdiUHGbiTYnE+Vf7Ubl0UOQa6pZx+GOQuMCx4Ej4TR8FGy827OOIzwq7iPSXc5B8ZZPUH0iiXUUMSiUcBz0NDQvvgIbd0/WaYRFxX1I+t9voWTrOlQe+doyNza1MMnODk4jnodmzEQonDWs4wiHittEhpJilH6+CeVf7wV0WtZxhCc5OkMzegKcnhkLhZ2adRxhUHHNZKyuQtm/t6FsXzzkygrWcSyO0s0Dmhdi4Tj4GUhK2hP3Qai4Zqg+k4KiNUtgyLvBOorFsw0KRdvZb0HlF8Q6CteouI0wVlWiePMaVHyzr/ZIG9I6bGzh8kIsnMdMoNG3AVTcBtAoyx6Nvg2j4t6DRlnO0Oh7X1Tcu9Sc/QWFH71FoyyHbINC4fb6Eti292MdhQtU3D+UfbUbxRtWWNU+xaKRHBzhNncZ7KP6so7CnNUXV9brcXvd8tqpMeGfQgGXCTOhGT2BdRKmrLq4hpJiFL47FzVnT7OOQprI4alhaBv3htUeuGC1xdVeykLBkr/R+qzAVMFhcH9jJZRuHqyjtDqrLG7lsR9Q9NFbkKurWEchj0jR1h3ub6yAXcjjrKO0KqsrbtmBz1G8fgV91WNBJJUd3BYuh31kNOsorcaqilu6Nx4lm9ewjkFago0t3Oa/B4feT7FO0iqsprglOzaidOs61jFIS1Iq4TZnCRxiBrNO0uKs4hIkVForYTCg8MNFqEz6jnWSFmfxxS3d+y8qrTUxGlC4YhEqjyeyTtKiLLq4Zft3omTz31nHIK3NYEDh+wtQdeoY6yQtxmKLW5H4LYo3rGQdg7Ci16Fw2VzUZJ5lnaRFWGRxtVkZuP33JfSVj5WTtTUoWDYHhsJ81lGancUV11BUgIKlcyDXPPxV7IjlMN5ZHh7hqoY8sqjiyjotCpa9DkNBHusohCPaC+ko+vtS1jGalUUVt2jte7XX5iHkHpVHvkHp3n+xjtFsLKa4Zf/ehsrDB1jHIBwr+WytxWxptojiVv/yM4rpax/yIEYjCpcvhO5aLuskj0z44hrKSlC48i3ASGeuIA8mV5Sj8MNFkA161lEeifDFLV63AsbbhaxjEIHosjNQtkfs9V2hi1t1PBGVid+wjkEEVLL9n9DmZrOO8dCELa6hrARFn7zHOgYRlV6HolVvCztlFra4NEUmj0rkKbOQxaUpMmkuok6ZhSsuTZFJsxJ0yixccUt3bKQpMmlWuuwMVHy3n3WMJhGquPq8Gyg/uId1DGKBSnf8E8aaatYxzCZUcUu2rgP0OtYxiAUyFOaj/MudrGOYTZjianOzUZn4LesYxIKV7tkCY1kp6xhmEaa4JVs+AYxG1jGIBZMrylC6+zPWMcwiRHFr0tNQfTKZdQxiBcq/+hz6gt9Zx3ggIYpb/NnHrCMQKyHX1KB0+z9Zx3gg7otblXoc2nNnWMcgVqTi+/3Q37rOOkajuC9u+YHPWUcg1sZoQPnXfH/tyHVx9Xk3UJ36E+sYxApVfL+f6xPMcV3c8q/30pZkwoSxtASVyd+zjtEgbosr67So+F6s3dCIZeF5Lz1ui1uZfBjGktusYxArpj1/FtrsTNYx7ovb4vL8rx2xHuUHd7OOcF9cFlebcx7azF9ZxyAElUcPwVhexjpGPVwWt4LOj0w4IddUozLpEOsY9XBZ3Kqfj7KOQIhJ1Ykk1hHq4a642ktZMPx+k3UMQkyqz5yCsbqKdYw6uCtu1QkabQlndFpUpx5nnaIO7opbzeG0hBDepstcFddQVABtVgbrGITUU53yI2SO9uLjqrhVJ5LoKvKES8bSYmgz+PmKkr/iEsIpnra/cFNcWa9Hza8prGMQ0iCeNlBxU1zd5WzINfweRkWI7solGKv5OIUrN8XldWduQkyMBugunmedAgBXxaWtyYR/vCyn3BRXx8kHQkhjqLh3kfV6aC+Jd8U0Yn2ouHfRXc4GdFrWMQh5IP3Vy1xsoOKiuLRhigiDkw1UVFxCmkibw3555aK4hrwbrCMQYjY9B8srH8UtKmAdgRCzGTlYXjkpbj7rCISYjYfllXlxZb0extJi1jEIMRsPM0TmxTUUF9KhfEQohqJC1hE4KG4h+2kHIU0hV1XAWFXJNAPz4vKwok9IU7GeLjMvLusPgJCHwXoDFfviltGGKSIe1htUmRcXOj3rBIQ0nZ7tcsu8uLLRwDoCIU0mG9gut8yLCwONuERAjItrw/TdAZz36YKMwTNZxyCkSXq5+KIzw/dnXtyf1R2wlQ7FJYJpa9+OaXGZT5VtFBLrCIQ0mY2CbXWYF1dJxSUCYj3gMC8u6w+AkIfBesBhXlxnNfPVbEKajPVyy7y4Ho52rCMQ0mQeTmyXW+bFdWP8ARDyMNwdVUzfn3lxWX8AhDSVnY0CzmpbphmYF5f1lIOQpuJhsGFeXJWNAhraQEUE4s7BYMO8uAAfHwQh5uJhlshHcTmYehBiLjcOllcuittOo2YdgRCz8bC8clHcEE9n1hEIMVuIpxPrCHwUt7MXFZeIg4fllYviBns6QSnRPsuEf+1d7Zl/hwtwUly1rRJ+bg6sYxDyQDyMtgAnxQWA0HZ8fCCENCaUiltXqJeGdQRCHoiXAYab4vIyBSGkMbwsp9wUN9jTifnByYQ05jFONkwBHBVXbatEV18X1jEIaVBPv7asI5hwU1wAiAl0Zx2BkAbFBPGzfPJVXI4+GELu5qhSIqpDG9YxTLgq7mNtHOBP3+cSDj3p1xa2Sn7qwk+SP/Sj6TLhEG/LJXfFpfVcwhulJKFvgBvrGHVwV9wuvi5o48DHJndCAKCLjwauDuyPwb0bd8VVSBL6BtCoS/jB40ZT7ooLAMPD27GOQAiA2isWDAnlb3nksriRHdrQ1mXChZhAd3g6sz/H1L24LC4AjOrmyzoCIRjdnc/lkNvi/iXcG/a2StYxiBXza+uAnh352c3xbtwW18nOBkNDvVjHIFbsvzie9XFbXIDfaQqxfGpbBZ5+3Jt1jAZxXdxgT2dE+NARQ6T1DQn1gpMdv1fY4Lq4APDcEzTqktY3plt71hEaxX1xB4Z40VdDpFX1D3JHCCdnumgI98VVKiRM7xvIOgaxEkpJwox+/C9v3BcXAP4U7IHHvelkcqTlDQv3QoC7I+sYDyREcQFgVgz//woSsamUCkyNDmAdwyzCFLdHhzbo7c/nl+HEMozu7svFBb3MIUxxAWBmv0DQeSBJS3BUKfFyLz/WMcwmVHFDvJwxmPamIi1gXM8OcLUX5zhwoYoLADP6BcCB9mEmzcjHRY0Xe3RgHaNJhCuuj4s9ZvWnDVWkeUgAFg0Nhb1KrMFAuOICwOhuvojs4Mo6BrEA/9XNF5EcnXbVXEIWV5IkvDEklA77I4/Ex0WN1wSdvQlZXADwdbXHq4J+6IS9O1NkBxW/BxI0RtjiAjRlJg9P1CnyHUIXl6bM5GGIPEW+Q+jiArVT5jkDOrGOQQRho5CweHiYsFPkO4QvLgCM7OKD57rzffwk4cPf/twJ3duLv3plEcUFgNl/DuLqamqEP6O6+mK0hfwDbzHFtVEo8N7Ix9He1Z51FMKhJx5ztahVKospLgC42NtixV+7wFGwvWBIy/JxUeODkY/DhqPLZD4qy/lN/hDo7oR3/hIOBR1GRAA42Cqx8q8R3F2061FZXHGB2os0TesrxgHRpOVIABYPD0OQhxPrKM3OIosLAC/38sMYOi+zVZszIBh/CvZgHaNFWGxxAeD1AcF4JoLfk1qTlvM/TwXhuScsYwvy/Vh0cSVJwv8O7oxhYXTwvTWZ3jcA/x0l1vG1TWXRxQVqL5T91rAwKq+VmBrtj0m9/VjHaHGSLMsy6xCtwSjLePe7THz5603WUUgLea1/EMb1tOyR9g6rKS4AyLKMDxMuYPcv11lHIc1IQu2GKEtep72XVRX3jk9/zsX/JV+E1f3iFsjORoE3h4Za3UkErbK4AJCUXYA3D6ajQmtgHYU8JE9nO6x4tgtC21nfVS6strgAcLGgAn/796+4VlzFOgppoi4+Gnz4bATcHC1rjyhzWXVxAaCkSocF+88i5cpt1lGImZ5+3BsLBofA1oL2PW4qqy8uAOiNRqw+ko3PT19jHYU0QilJeO2pILwY+RjrKMxRce+y/7cbWJmQhUodrffyxs1RhbeHh+FJP7p+FEDFred6cRWWHsrAqSvFrKOQPwwL88KcAcHQqMW5REhLo+LehyzL2JN2HR8fzUEVjb7MuDmqsGBwCPoHWeaBAo+CitsIGn3ZoVG2cVTcB6DRt3XRKGseKq6ZbpRU4R/JF/FdRh7tcdUCVEoFRnXzxeTefnAR6HKXrFBxm+h8Xhk+Sc7B8UtFrKNYBIUEDAtrh6nR/vB2oRP9mYuK+5BSr97GJ0dz8NvNUtZRhBUT5I4Z/QIQ6G55p5ZpaVTcR5SYlY9/JOfgUmEl6yjC6ObrglkxgehqAScmZ4WK2wwMRhmHz+dh9y/XceZ6Ces4XFJIQJ8ANzzfvT16+buxjiM8Km4zy/q9HHvSruHbc3m0BxaANg62GNnFB6O6+tA6bDOi4raQ8ho9vk6/hT1p13GpsIJ1nFYX4eOC0d19MTDE06oPBmgpVNxWkHrlNg6m38KxiwUoqtSxjtNifFzUiAlyx9OPeyPY05l1HItGxW1FRlnGbzdKkZSdj+ScAuE3aEkAwrw1iAl0R0yQu0WeeJxXVFyGrt6uRHJOAZKyC5B2vQQGI/9/FXY2CkR1bIOYQHf0C3SHu5Md60hWiYrLiWqdAVn55ci4VYaMvFJk3irDpcJKGBj+9aiUCgR5OCK0nQahXs7o7OWMQHdHi7p4lqiouBy7u8wXfi/DrdJqFFRoUVBeg5JqfbO8h4TaLb/uTnZwd7SDj4sanamk3KPiCkqrN6KgogYF5VrTf0urdTAYZRhkGXqDDKMsQ6mQav9IEmyUElztVXB3UsHd0Q4eTnZo62gLGwWVUzRUXEIERP/UEiIgKi4hAqLiEiIgKi4hAqLiEiIgKi6HcnNzERoaiilTpiA8PByDBw9GVVUVcnJyMHToUPTo0QP9+vVDZmYmACAnJwe9evVCVFQU3nzzTTg50a6Hlo6Ky6msrCzMnDkT6enpcHV1xd69e/HKK6/g448/RmpqKlasWIEZM2YAAOLi4hAXF4eUlBT4+PgwTk5aA32Py6Hc3FwMGjQIWVlZAIAPPvgAOp0Oy5YtQ0hIiOlxNTU1yMjIgJubG/Ly8mBjY4PS0lL4+PigvLycVXzSCmxYByD3Z2f3n533lUol8vLy4OrqirS0NIapCC9oqiwIjUYDf39/7N69G0Dt+Z7PnDkDAOjVqxf27t0LANi5cyezjKT1UHEFsm3bNmzatAldu3ZFeHg4vvzySwDA6tWr8dFHH6Fnz564efMmXFxcGCclLY3WcS1AZWUl7O3tIUkSdu7ciR07dphKTSwTreNagNTUVMyaNQuyLMPV1RWbN29mHYm0MBpxCREQreMSIiAqLiECouISIiAqLiECouISIiAqLiECouISIiAqLiECouISIiAqLiECouISIiAqLiECouISIiAqLiECouISIiAqLiECouISIqD/B6xkPIglp11SAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.pie(df_reviews['label'].value_counts(),labels = ['pos','neg'])\n",
    "plt.title(\"label distribution\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Observation** - Perfectly balanced class labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews = df_reviews[['text','label']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>bromwell high cartoon comedy ran time programs...</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>homelessness houselessness george carlin state...</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>brilliant overacting lesley ann warren best dr...</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>easily underrated film inn brooks cannon sure ...</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>typical mel brooks film slapstick movies actua...</td>\n",
       "      <td>pos</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                text label\n",
       "0  bromwell high cartoon comedy ran time programs...   pos\n",
       "1  homelessness houselessness george carlin state...   pos\n",
       "2  brilliant overacting lesley ann warren best dr...   pos\n",
       "3  easily underrated film inn brooks cannon sure ...   pos\n",
       "4  typical mel brooks film slapstick movies actua...   pos"
      ]
     },
     "execution_count": 188,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {},
   "outputs": [],
   "source": [
    "# add a review length feature \n",
    "\n",
    "reviews['length'] = reviews['text'].apply(len)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>label</th>\n",
       "      <th>length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>bromwell high cartoon comedy ran time programs...</td>\n",
       "      <td>pos</td>\n",
       "      <td>487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>homelessness houselessness george carlin state...</td>\n",
       "      <td>pos</td>\n",
       "      <td>1290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>brilliant overacting lesley ann warren best dr...</td>\n",
       "      <td>pos</td>\n",
       "      <td>517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>easily underrated film inn brooks cannon sure ...</td>\n",
       "      <td>pos</td>\n",
       "      <td>361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>typical mel brooks film slapstick movies actua...</td>\n",
       "      <td>pos</td>\n",
       "      <td>307</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                text label  length\n",
       "0  bromwell high cartoon comedy ran time programs...   pos     487\n",
       "1  homelessness houselessness george carlin state...   pos    1290\n",
       "2  brilliant overacting lesley ann warren best dr...   pos     517\n",
       "3  easily underrated film inn brooks cannon sure ...   pos     361\n",
       "4  typical mel brooks film slapstick movies actua...   pos     307"
      ]
     },
     "execution_count": 190,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(min,max review lenghts) -  (17, 8625)\n"
     ]
    }
   ],
   "source": [
    "print(\"(min,max review lenghts) - \",(reviews.length.min(),reviews.length.max()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEJCAYAAAB/pOvWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XtcVPW+//HXyHjDUZwZRINIxcveZSGcsNB2gsjjdNrW3m5zd7VSu1MaWj202951KuPsHckhNSsNt9XZ7h4esatdCMHKPBvlcryUeMv0iCIMKiMVCt/fH/6ch7RAYUABeT8fjx6P5jvf9V3f9WE571lrzayxGWMMIiIip+jU2hMQEZG2R+EgIiIWCgcREbFQOIiIiIXCQURELBQOIiJioXA4Tz3zzDMMHjz4rIydk5ODzWZj79699T5uaUuWLMFut5+Vsf2xZ88exo4dS48ePbDZbK0yB5vNxttvv31O1zl58mQSExPP6TrPZMCAATz//POtPY3zksKhHZk8eTI2mw2bzYbdbsflcjFy5EieffZZPB5Pnb6PPvoo69ata/TYgwcP5plnnmlU31GjRlFSUkJoaGhTpn9Ge/fuxWazkZOTU6f9pptu4v/+7/9adF3NMWfOHEpLSyksLKSkpKRV5lBSUsLEiRNbZd2t4fnnn2fAgAGtPY0OReHQzlx99dWUlJTwww8/8OWXX3LPPffw97//nWHDhlFcXOzr53A4CA4ObvH1V1dX06VLF/r160enTudm9+nevTt9+/Y9J+tqjG3btnHFFVcwZMgQ+vXr16hljDEcO3asxebQr18/unXr1mLjifySwqGdOfnCHBoayrBhw5g6dSr//Oc/CQwM5P777/f1++Vppb1793LDDTcQHBxM9+7diYiI4K9//SsA8fHx7Nixg2effdZ3ZPL999/7Thd99NFH/OY3v6Fbt268/vrrDZ5GKigo4IorrqBbt24MGzaMzz//3PdcQ8vY7XaWLFkCQHh4OABjxozBZrP53inWd1rp448/5vLLL6dr166EhISQlJTE0aNHfc+fPAXy+uuv079/f3r16sXvf/97Dh48eNr6VlZWct9999GnTx+6detGTEwMn332me95m83GF198wZtvvonNZmPy5Mn1jnNyzqtXryY6OpquXbvy6aefAvD5559z1VVX0b17d8LCwpgyZQrl5eW+5wICAtizZ0+d8f7xj3/QrVs3Dh065JvHqaeVvF4vDz/8MGFhYQQGBhIdHc2KFSt8z0+aNIlJkyb5HmdkZGCz2Vi0aJGv7c477+TGG288bX1+admyZURFRdGtWzcGDBjAzJkz6/wd4uPjufvuu3nuuefo168fLpeLyZMn1+lTW1vLE088QZ8+fXA4HNx8882kpaX5/uZLlizh6aefZvfu3b7989Sj3Orqah5++GFcLhd9+/bl0Ucfpaamxvf8V199xVVXXUXPnj3p2bMnw4cP9/0t5DSMtBt33nmnGTt2bL3P/fWvfzU2m82UlpYaY4z585//bAYNGuR7/vrrrzdjx441BQUFZteuXSY7O9v813/9lzHGmPLycjNgwADzyCOPmJKSElNSUmKOHz9uVq9ebQDzq1/9yrz33ntm586dZs+ePb72PXv2GGOM7/HgwYPNBx98YLZs2WKmTp1qunXrZvbu3Vunz8llTgoICDAZGRnGGGPy8/MNYP77v//blJSU+LYlIyPDBAQE+JYpKioyAQEBJjk52WzZssV8/PHHJjw83EyaNKlOrXr16mVuvvlms3HjRvP111+biy66yNxxxx2nrfHEiRNN//79zSeffGK2bNlipk+fbjp37my+/fZbY4wxJSUlZuTIkebWW281JSUl5tChQ/WOk5GRYWw2m4mJiTFffPGF2bFjhyktLTVffPGF6d69u0lPTzfFxcXmn//8p4mPjzdXX321qa2tNTU1NSYsLMzMmTOnznjjxo0zN954o+8xYN566y1jjDG1tbUmPj7exMXFmS+//NLs2LHDvPbaa6Zz584mKyvLGGPM4sWLzQUXXOBbftKkSaZPnz7m5ptv9rWFh4ebV199tcHa/HL/y8jIML179zZLly41O3bsMLm5ueayyy6r83eIi4szQUFBJjk52Xz77bdm1apVJigoyPzpT3/y9UlNTTU9evQwS5cuNcXFxSY1NdU4nU7f37yqqsrMmjXLXHjhhb79s7Ky0hhjTP/+/U3v3r3Niy++aIqLi82yZctMQECAefPNN40xxhw/ftw4nU4zY8YMU1xcbIqLi82KFSvMmjVrGtxOOUHh0I6cLhxWrVplAPM///M/xhhrOERGRpo///nPDY49aNAgy/MnX9CXLl1ab/svw2HRokW+PseOHTMXXXSRefLJJ+td5qRTw2HPnj0GMKtXr67T55fhMGnSJDNixIg6fVauXGlsNpv5/vvvjTEnahUcHGx++uknX58XX3zR9OvXr8EabNu2zQDmo48+qtMeHR1tpkyZ4nscFxdn7rrrrgbHOTlnwPIiFBcXZ2bNmlWnbffu3QYwBQUFxhhjZs2aZS6++GLf8wcOHDB2u918+OGHvrZTw2H16tWma9eulqCaMmWK+f3vf2+MMeb77783gNm8ebMxxpiwsDDz0ksvmZCQEGOMMcXFxQYwW7dubXCbfrn/9e/f3xImubm5BjAej8e3vZdddlmdPvfdd5+JjY31PQ4NDTVPPfVUnT433XRTnb/5c889Z/r372+ZU//+/c31119fp+2aa67xhZ7H46l3n5Iz02ml84T5//dPbOjTM8nJycyZM4crr7ySWbNmsWbNmkaPfcUVVzSq38iRI33/b7fbueKKK9iyZUuj19NYmzdvZvTo0XXa4uLiMMbUWd/FF19M165dfY/DwsI4cOBAg+OeXPaXY48ePZrNmzf7NdcRI0bUeZyXl0daWhoOh8P33yWXXAKcuJYBJ07vfPvtt+Tl5QHw97//HbfbzTXXXFPvOvLy8qiuriYsLKzOuG+//bZvzP79+zNw4ECys7PZunUrhw4dIikpiZ9++olNmzaRnZ1NWFgYQ4cObdR2HTx4kN27dzNz5sw667z22msB2L59u69vVFRUnWVP/TscOXKEffv2ERsbW6fPqfvSmZxufKfTyd13380111zDtddeS0pKClu3bm302B1Z2/l8oDTLpk2bsNlsRERE1Pv8lClT+Ld/+zc++eQTVq9ezbXXXssf/vCHRn0cskePHn7NyZxyw9+TF69PbaupqaG2ttavsRsKwVPbu3TpYnnO+HETYmOMXx9ZDQgIsFw0rq2tZdasWdx+++2W/icvbl988cXExMSwdOlSRowYwdKlS7n11lsb/DhvbW0tQUFBvjA51ak1SEhI4IsvviAgIIDf/OY3dO/endGjR/PFF1+wdu1aEhISGr1tJ/9u//mf/8mYMWMsz1944YX1zgFO/B1OLn+mNzWNcbrxAd544w0efvhhPvvsMz7//HOefvpp5s2bx3333ef3OjsCHTmcB44cOcKrr77K2LFjcbvdDfa74IILmDJlCkuXLmXx4sW88847HDlyBDjxD+zUi3j+OPWjs8ePHycvL4+LL74YgJCQEAD27dvn61NYWFjnxfrkP/IzzWPYsGHk5ubWacvNzcVms/nehftj2LBhAJajqi+//NL3XHPFxMSwefNmBg8ebPnP4XD4+t1xxx0sW7aMoqIi8vPzufPOO0875qFDh/jpp58sY1500UW+fgkJCeTm5pKVlcXYsWN9bV988QU5OTlNCoe+ffsSHh7O1q1b692Wxn6SKigoiNDQUL755ps67b/8GHZz989LL72UmTNnsmrVKu666y5ef/11v8fqKBQO7Ux1dTX79++npKSELVu28Oabb3LFFVfw888/8+qrrza43EMPPcTHH3/Mjh072Lx5MytWrCA8PJyePXsCMHDgQL7++mt++OEHysrK/HpHn5KSwscff8y3337LAw88wIEDB3jggQeAE9+j6N+/P8888wzfffcdX331FTNmzKjzjjE4OBiHw8Fnn33G/v37qaioqHc9jz32GPn5+cycOZPvvvuOTz75hGnTpnHbbbfVeTFsqkGDBvHHP/6RpKQkPv30U7777jsefvhhNm3axGOPPeb3uKf693//d9577z1mzJhBYWEhO3bs4JNPPuGuu+7ixx9/9PW75ZZbOHz4MJMnTyYyMpLhw4c3OGZCQgKJiYlMmDCBzMxMdu7cyYYNG3jllVd444036vSrqKjg/fff9wVBQkICq1atorS0tEnhAPDCCy+Qnp7O888/z6ZNm9i6dSsrV65s8jvyRx55hLS0NN555x22bdtGWloan332WZ19Y+DAgezfv59vvvmGsrIyqqqqGjX29u3bmTVrFl999RW7d+/mm2++4csvv2zWm4iOQuHQznz55ZdccMEFhIeHc9VVV/H6669z6623smnTptN+I9oYQ3JyMpdeeimjR4/m6NGjrFq1yvcP8Nlnn+Xw4cP86le/ok+fPvzwww9NnttLL73E008/TVRUFF9//TXvvfee7/SC3W7nH//4B6WlpURHR/Pggw/ywgsv1PmuRKdOnZg/fz7vvvsu4eHhREdH17ueyMhI3n//fXJzcxk+fDi3334748aNY+HChU2e8y8tWrSIa665hkmTJjF8+HC+/vprPvzwQ3796183e2w48THd7OxsNm7cyNVXX01kZCQzZsygZ8+edO7c2dcvODiYcePGUVhYyB133HHaMW02G++//z4TJkxg5syZ/PrXv2bcuHF89NFHDBo0yNevX79+XHLJJfTs2dNX28jISHr37s2gQYOaHKy333477777Lh999BFXXHEFI0aM4JlnniEsLKxJ4yQnJ/PQQw/x8MMPEx0dzbp163jkkUfqHH2MHz+eP/7xj4wbN44+ffrwl7/8pVFj9+jRg23btnHzzTczdOhQbrjhBkaNGsW8efOaNMeOyGb8OQkrInIWTZ06laKiIjZs2NDaU+mwdEFaRFrVvn37yMzMZMyYMQQEBPDBBx+wdOlSvbtvZTpyEJFWdeDAAW666Sb+93//13dRfdq0adxzzz2tPbUOTeEgIiIWuiAtIiIWCgcREbFo1xekT/1CVWMFBwdTVlZ2FmbT/qk2DVNt6qe6NKyt1qaxv8OiIwcREbFQOIiIiIXCQURELBQOIiJioXAQERELhYOIiFic8aOsCxYsID8/n6CgIFJTU+s89/777/P222+zaNEievXqhTGGjIwMCgoK6Nq1K0lJSb4fn8nJyfH94PmECROIj48HYOfOncyfP5/q6mqio6OZMmVKs374Q0REmu+MRw7x8fE88cQTlvaysjI2btxIcHCwr62goID9+/eTnp7Ovffey6JFiwDwer0sX76cOXPmMGfOHJYvX47X6wVO/ErTfffdR3p6Ovv376ewsLCltk1ERPx0xnC45JJL6vxC1Ul/+9vfuO222+q8y1+/fj2jR4/GZrMxdOhQjh49SkVFBYWFhURGRvp+ZzYyMpLCwkIqKir48ccfGTp0KDabjdGjR9f7U4ciInJu+fUN6fXr1+NyuRgwYECddo/HU+dIwu124/F48Hg8dX6+0uVy1dt+sn9DsrKyyMrKAk786tip62osu93e5OUO/GFUve19M9c2ef1tmT+16ShUm/qpLg1r77Vpcjj8/PPPrFixgqeeesryXH03eD3dD8E39YawiYmJJCYm+h7789X0lvxKe1v8anxztNWv+7cFqk39VJeGtdXanLXbZxw4cIDS0lIee+wxHnzwQcrLy5k1axaHDh3C7XbXKUZ5eTlOpxOXy0V5ebmv3ePx4HQ6cbvdddrLy8txuVxNnZKIiLSwJofDRRddxKJFi5g/fz7z58/H7XbzH//xH/Tu3ZuYmBjWrFmDMYbi4mICAwNxOp1ERUVRVFSE1+vF6/VSVFREVFQUTqeT7t27U1xcjDGGNWvWEBMTcza2U0REmuCMp5XS0tLYsmULlZWV3H///dx4440kJCTU2zc6Opr8/HymT59Oly5dSEpKAsDhcHDDDTfw+OOPAzBx4kTfRe67776bBQsWUF1dTVRUVIM/Ki8iIudOu/4luHN1y+6ae35Xb3vAG+83ef1tWVs9R9oWqDb1U10a1lZro1t2i4iI3xQOIiJioXAQERELhYOIiFgoHERExELhICIiFgoHERGxUDiIiIiFwkFERCwUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsFA4iImKhcBAREQuFg4iIWNjP1GHBggXk5+cTFBREamoqAG+99RYbNmzAbrfTt29fkpKS6NGjBwCZmZlkZ2fTqVMnpkyZQlRUFACFhYVkZGRQW1vL2LFjGT9+PAClpaWkpaXh9XoZOHAg06ZNw24/47REROQsOuORQ3x8PE888USdtsjISFJTU3nppZe44IILyMzMBGDv3r2sXbuWl19+mSeffJLFixdTW1tLbW0tixcv5oknnmDu3Ll8/fXX7N27F4C3336bcePGkZ6eTo8ePcjOzj4LmykiIk1xxnC45JJLcDgcddqGDx9OQEAAAEOHDsXj8QCQl5fHqFGj6Ny5MyEhIfTr14/t27ezfft2+vXrR9++fbHb7YwaNYq8vDyMMWzevJnY2FjgRBDl5eW19DaKiEgTNfv8TXZ2NqNGjQLA4/EwZMgQ33Mul8sXHG6329fudrvZtm0blZWVBAYG+oLm1P71ycrKIisrC4CUlBSCg4ObPF+73d7k5Q400O7P+tsyf2rTUag29VNdGtbea9OscFixYgUBAQFcffXVABhj6u1XX7vNZmvy+hITE0lMTPQ9Lisra/IYwcHBfi1Xn5Yap61oydqcb1Sb+qkuDWurtQkNDW1UP78/rZSTk8OGDRuYPn2674Xe7XZTXl7u6+PxeHC5XJb28vJynE4nPXv2pKqqipqamjr9RUSkdfkVDoWFhbz33nvMmjWLrl27+tpjYmJYu3Ytx44do7S0lJKSEgYPHsygQYMoKSmhtLSU48ePs3btWmJiYrDZbAwbNox169YBJwInJiamZbZMRET8dsbTSmlpaWzZsoXKykruv/9+brzxRjIzMzl+/DjPPfccAEOGDOHee+8lPDyckSNHMnPmTDp16sRdd91Fp04n8mfq1Km88MIL1NbWMmbMGMLDwwG47bbbSEtLY9myZQwcOJCEhISzuLkiItIYNtPQhYJ2YN++fU1exp/zgDX3/K7e9oA33m/y+tuytnqOtC1QbeqnujSsrdbmrF9zEBGR85fCQURELBQOIiJioXAQERELhYOIiFgoHERExELhICIiFgoHERGxUDiIiIiFwkFERCwUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsFA4iImKhcBAREQuFg4iIWNjP1GHBggXk5+cTFBREamoqAF6vl7lz53Lw4EH69OnDjBkzcDgcGGPIyMigoKCArl27kpSUREREBAA5OTmsWLECgAkTJhAfHw/Azp07mT9/PtXV1URHRzNlyhRsNttZ2lwREWmMMx45xMfH88QTT9RpW7lyJZdddhnp6elcdtllrFy5EoCCggL2799Peno69957L4sWLQJOhMny5cuZM2cOc+bMYfny5Xi9XgDeeOMN7rvvPtLT09m/fz+FhYUtvY0iItJEZwyHSy65BIfDUactLy+PuLg4AOLi4sjLywNg/fr1jB49GpvNxtChQzl69CgVFRUUFhYSGRmJw+HA4XAQGRlJYWEhFRUV/PjjjwwdOhSbzcbo0aN9Y4mISOs542ml+hw+fBin0wmA0+nkyJEjAHg8HoKDg3393G43Ho8Hj8eD2+32tbtcrnrbT/ZvSFZWFllZWQCkpKTUWVdj2e32Ji93oIF2f9bflvlTm45Ctamf6tKw9l4bv8KhIcYYS1tD1w9sNlu9/U8nMTGRxMRE3+OysrKmTZATL+j+LFeflhqnrWjJ2pxvVJv6qS4Na6u1CQ0NbVQ/vz6tFBQUREVFBQAVFRX06tULOPHO/9RilJeX43Q6cblclJeX+9o9Hg9OpxO3212nvby8HJfL5c+URESkBfkVDjExMeTm5gKQm5vLiBEjfO1r1qzBGENxcTGBgYE4nU6ioqIoKirC6/Xi9XopKioiKioKp9NJ9+7dKS4uxhjDmjVriImJabmtExERv5zxtFJaWhpbtmyhsrKS+++/nxtvvJHx48czd+5csrOzCQ4OZubMmQBER0eTn5/P9OnT6dKlC0lJSQA4HA5uuOEGHn/8cQAmTpzou8h99913s2DBAqqrq4mKiiI6OvpsbauIiDSSzTT1xH8bsm/fviYv4895wJp7fldve8Ab7zd5/W1ZWz1H2haoNvVTXRrWVmtzVq85iIjI+U3hICIiFgoHERGxUDiIiIiFwkFERCwUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsFA4iImKhcBAREQuFg4iIWCgcRETEQuEgIiIWCgcREbFQOIiIiIW9OQt/+OGHZGdnY7PZCA8PJykpiUOHDpGWlobX62XgwIFMmzYNu93OsWPHmDdvHjt37qRnz54kJycTEhICQGZmJtnZ2XTq1IkpU6YQFRXVIhsnIiL+8fvIwePxsGrVKlJSUkhNTaW2tpa1a9fy9ttvM27cONLT0+nRowfZ2dkAZGdn06NHD1555RXGjRvHO++8A8DevXtZu3YtL7/8Mk8++SSLFy+mtra2ZbZORET80qzTSrW1tVRXV1NTU0N1dTW9e/dm8+bNxMbGAhAfH09eXh4A69evJz4+HoDY2Fg2bdqEMYa8vDxGjRpF586dCQkJoV+/fmzfvr15WyUiIs3i92kll8vF9ddfzwMPPECXLl0YPnw4ERERBAYGEhAQ4Ovj8XiAE0cabrcbgICAAAIDA6msrMTj8TBkyJA6455c5peysrLIysoCICUlheDg4CbP2263N3m5Aw20+7P+tsyf2nQUqk39VJeGtffa+B0OXq+XvLw85s+fT2BgIC+//DKFhYUN9jfGWNpsNlu97Q1JTEwkMTHR97isrKxpk+bEC7o/y9WnpcZpK1qyNucb1aZ+qkvD2mptQkNDG9XP79NKGzduJCQkhF69emG327nyyivZunUrVVVV1NTUACeOFlwuFwBut5vy8nIAampqqKqqwuFw1Gn/5TIiItI6/D5yCA4OZtu2bfz888906dKFjRs3MmjQIIYNG8a6deu46qqryMnJISYmBoDLL7+cnJwchg4dyrp16xg2bBg2m42YmBjS09O57rrrqKiooKSkhMGDB7fYBjZFzT2/a5X1ioi0NX6Hw5AhQ4iNjWXWrFkEBAQwYMAAEhMT+Zd/+RfS0tJYtmwZAwcOJCEhAYCEhATmzZvHtGnTcDgcJCcnAxAeHs7IkSOZOXMmnTp14q677qJTp/bx9YuGwiTgjffP8UxERFqWzTTlpH8bs2/fviYvc7rzgC115NBew6GtniNtC1Sb+qkuDWurtTnr1xxEROT8pXAQERELhYOIiFgoHERExELhICIiFgoHERGxUDiIiIiFwkFERCwUDiIiYqFwEBERC4WDiIhYKBxERMRC4SAiIhYKBxERsVA4iIiIhcJBREQsFA4iImKhcBAREQuFg4iIWCgcRETEQuEgIiIW9uYsfPToURYuXMiePXuw2Ww88MADhIaGMnfuXA4ePEifPn2YMWMGDocDYwwZGRkUFBTQtWtXkpKSiIiIACAnJ4cVK1YAMGHCBOLj45u9YSIi4r9mhUNGRgZRUVE88sgjHD9+nJ9//pnMzEwuu+wyxo8fz8qVK1m5ciWTJk2ioKCA/fv3k56ezrZt21i0aBFz5szB6/WyfPlyUlJSAJg9ezYxMTE4HI4W2UAREWk6v08rVVVV8e2335KQkACA3W6nR48e5OXlERcXB0BcXBx5eXkArF+/ntGjR2Oz2Rg6dChHjx6loqKCwsJCIiMjcTgcOBwOIiMjKSwsbIFNExERf/l95FBaWkqvXr1YsGABu3fvJiIigsmTJ3P48GGcTicATqeTI0eOAODxeAgODvYt73a78Xg8eDwe3G63r93lcuHxeOpdZ1ZWFllZWQCkpKTUGa+x7HZ7g8sdaPJo9fNnXm3B6WrT0ak29VNdGtbea+N3ONTU1LBr1y6mTp3KkCFDyMjIYOXKlQ32N8ZY2mw2W719G2pPTEwkMTHR97isrKyJsz7xwu3Pck1xtsc/W85Fbdor1aZ+qkvD2mptQkNDG9XP79NKbrcbt9vNkCFDAIiNjWXXrl0EBQVRUVEBQEVFBb169fL1P7VQ5eXlOJ1OXC4X5eXlvnaPx+M78hARkdbhdzj07t0bt9vNvn37ANi4cSMXXnghMTEx5ObmApCbm8uIESMAiImJYc2aNRhjKC4uJjAwEKfTSVRUFEVFRXi9XrxeL0VFRURFRbXApomIiL+a9WmlqVOnkp6ezvHjxwkJCSEpKQljDHPnziU7O5vg4GBmzpwJQHR0NPn5+UyfPp0uXbqQlJQEgMPh4IYbbuDxxx8HYOLEifqkkohIK7OZ+i4GtBMnj1qa4nTnAWvu+V1zpwRAwBvvt8g451pbPUfaFqg29VNdGtZWa3PWrzmIiMj5S+EgIiIWCgcREbFQOIiIiIXCQURELBQOIiJioXAQERELhYOIiFgoHERExELhICIiFgoHERGxUDiIiIiFwkFERCyadctuqV9Dd3dtr3drFZGOR0cOIiJioXAQERELhYOIiFgoHERExELhICIiFgoHERGxaPZHWWtra5k9ezYul4vZs2dTWlpKWloaXq+XgQMHMm3aNOx2O8eOHWPevHns3LmTnj17kpycTEhICACZmZlkZ2fTqVMnpkyZQlRUVLM3TERE/NfsI4ePP/6YsLAw3+O3336bcePGkZ6eTo8ePcjOzgYgOzubHj168MorrzBu3DjeeecdAPbu3cvatWt5+eWXefLJJ1m8eDG1tbXNnZaIiDRDs8KhvLyc/Px8xo4dC4Axhs2bNxMbGwtAfHw8eXl5AKxfv574+HgAYmNj2bRpE8YY8vLyGDVqFJ07dyYkJIR+/fqxffuQKtWmAAAM2ElEQVT25kxLRESaqVnhsGTJEiZNmoTNZgOgsrKSwMBAAgICAHC5XHg8HgA8Hg9utxuAgIAAAgMDqaysrNP+y2VERKR1+H3NYcOGDQQFBREREcHmzZvP2N8YY2mz2Wz1tjckKyuLrKwsAFJSUggODm78hP8/u93e4HIHmjxa0/gz33PpdLXp6FSb+qkuDWvvtfE7HLZu3cr69espKCigurqaH3/8kSVLllBVVUVNTQ0BAQF4PB5cLhcAbreb8vJy3G43NTU1VFVV4XA4fO0nnbrMLyUmJpKYmOh7XFZW1uR5BwcH+7VcS2it9TZWa9amrVNt6qe6NKyt1iY0NLRR/fw+rXTrrbeycOFC5s+fT3JyMpdeeinTp09n2LBhrFu3DoCcnBxiYmIAuPzyy8nJyQFg3bp1DBs2DJvNRkxMDGvXruXYsWOUlpZSUlLC4MGD/Z2WiIi0gBa/K+ttt91GWloay5YtY+DAgSQkJACQkJDAvHnzmDZtGg6Hg+TkZADCw8MZOXIkM2fOpFOnTtx111106qSvX4iItCabacpJ/zZm3759TV7mdId6Dd1qu6W09Vt2t9XD4LZAtamf6tKwtlqbs35aSUREzl8KBxERsVA4iIiIhcJBREQs9BvS55B+W1pE2gsdOYiIiIXCQURELBQOIiJioXAQERELhYOIiFgoHERExELhICIiFgoHERGxUDiIiIiFwkFERCwUDiIiYqFwEBERC4WDiIhYKBxERMSiQ96y+2z/VnRTnW4+up23iLQGHTmIiIiF30cOZWVlzJ8/n0OHDmGz2UhMTOS3v/0tXq+XuXPncvDgQfr06cOMGTNwOBwYY8jIyKCgoICuXbuSlJREREQEADk5OaxYsQKACRMmEB8f3yIbJyIi/vE7HAICArj99tuJiIjgxx9/ZPbs2URGRpKTk8Nll13G+PHjWblyJStXrmTSpEkUFBSwf/9+0tPT2bZtG4sWLWLOnDl4vV6WL19OSkoKALNnzyYmJgaHw9FiGykiIk3j92klp9Ppe+ffvXt3wsLC8Hg85OXlERcXB0BcXBx5eXkArF+/ntGjR2Oz2Rg6dChHjx6loqKCwsJCIiMjcTgcOBwOIiMjKSwsbIFNExERf7XIBenS0lJ27drF4MGDOXz4ME6nEzgRIEeOHAHA4/EQHBzsW8btduPxePB4PLjdbl+7y+XC4/HUu56srCyysrIASElJqTNeY9nt7esavD/b6C+73X5O19eeqDb1U10a1t5r0+xXyp9++onU1FQmT55MYGBgg/2MMZY2m81Wb9+G2hMTE0lMTPQ9Lisra+Jsz+2LbUvwZxv9FRwcfE7X156oNvVTXRrWVmsTGhraqH7N+rTS8ePHSU1N5eqrr+bKK68EICgoiIqKCgAqKiro1asXcOJI4dRClZeX43Q6cblclJeX+9o9Ho/vyENERFqH3+FgjGHhwoWEhYVx3XXX+dpjYmLIzc0FIDc3lxEjRvja16xZgzGG4uJiAgMDcTqdREVFUVRUhNfrxev1UlRURFRUVDM3S0REmsPv00pbt25lzZo1XHTRRTz22GMA3HLLLYwfP565c+eSnZ1NcHAwM2fOBCA6Opr8/HymT59Oly5dSEpKAsDhcHDDDTfw+OOPAzBx4kR9UukUDX1BTl+OE5GzyWbquxjQTuzbt6/JywQHB3PgD6POwmzOrbMRDm31HGlboNrUT3VpWFutzTm55iAiIucnhYOIiFgoHERExELhICIiFgoHERGxaF/3khAffcRVRM4mHTmIiIiFwkFERCwUDiIiYqFrDucZXYsQkZagIwcREbFQOIiIiIXCQURELHTNoYPQtQgRaQodOYiIiIWOHDq4U48oDpzSriMKkY5NRw4iImKhIwepl65RiHRsCgdpEoWGSMegcJAWodAQOb8oHOSsUmiItE9tJhwKCwvJyMigtraWsWPHMn78+NaekpxFDYVGQxQmIudWmwiH2tpaFi9ezFNPPYXb7ebxxx8nJiaGCy+8sLWnJm1EU8MEFCgizdEmwmH79u3069ePvn37AjBq1Cjy8vIUDtIs/gRKfRQy0hG1iXDweDy43W7fY7fbzbZt2yz9srKyyMrKAiAlJYXQ0FC/1hf+0Xr/Jiodmr/72/lOdWlYe65Nm/gSnDHG0maz2SxtiYmJpKSkkJKS4ve6Zs+e7fey5zvVpmGqTf1Ul4a199q0iXBwu92Ul5f7HpeXl+N0OltxRiIiHVubCIdBgwZRUlJCaWkpx48fZ+3atcTExLT2tEREOqw2cc0hICCAqVOn8sILL1BbW8uYMWMIDw8/K+tKTEw8K+OeD1Sbhqk29VNdGtbea2Mz9Z3wFxGRDq1NnFYSEZG2ReEgIiIWbeKaw7nS0W7RUVZWxvz58zl06BA2m43ExER++9vf4vV6mTt3LgcPHqRPnz7MmDEDh8OBMYaMjAwKCgro2rUrSUlJREREAJCTk8OKFSsAmDBhAvHx8a24ZS2ntraW2bNn43K5mD17NqWlpaSlpeH1ehk4cCDTpk3Dbrdz7Ngx5s2bx86dO+nZsyfJycmEhIQAkJmZSXZ2Np06dWLKlClERUW18lY139GjR1m4cCF79uzBZrPxwAMPEBoa2uH3mw8//JDs7GxsNhvh4eEkJSVx6NCh83OfMR1ETU2Neeihh8z+/fvNsWPHzKOPPmr27NnT2tM6qzwej9mxY4cxxpiqqiozffp0s2fPHvPWW2+ZzMxMY4wxmZmZ5q233jLGGLNhwwbzwgsvmNraWrN161bz+OOPG2OMqaysNA8++KCprKys8//ngw8++MCkpaWZF1980RhjTGpqqvnqq6+MMca89tpr5tNPPzXGGPPJJ5+Y1157zRhjzFdffWVefvllY4wxe/bsMY8++qiprq42Bw4cMA899JCpqalphS1pWa+88orJysoyxhhz7Ngx4/V6O/x+U15ebpKSkszPP/9sjDmxr6xevfq83Wc6zGmlU2/RYbfbfbfoOJ85nU7fO7ju3bsTFhaGx+MhLy+PuLg4AOLi4nx1WL9+PaNHj8ZmszF06FCOHj1KRUUFhYWFREZG4nA4cDgcREZGUlhY2Grb1VLKy8vJz89n7NixwIkvY27evJnY2FgA4uPj69Tm5Lve2NhYNm3ahDGGvLw8Ro0aRefOnQkJCaFfv35s3769VbanpVRVVfHtt9+SkJAAgN1up0ePHtpvOHGkWV1dTU1NDdXV1fTu3fu83Wc6zGmlxt6i43xVWlrKrl27GDx4MIcPH/Z9ydDpdHLkyBHgRI2Cg4N9y7jdbjwej6V2LpcLj8dzbjfgLFiyZAmTJk3ixx9/BKCyspLAwEACAgKAutt5ag0CAgIIDAyksrISj8fDkCFDfGOeD7UpLS2lV69eLFiwgN27dxMREcHkyZM7/H7jcrm4/vrreeCBB+jSpQvDhw8nIiLivN1nOsyRg2nkLTrORz/99BOpqalMnjyZwMDABvs1pUbtvXYbNmwgKCjId2R1Jg3Vpr729q6mpoZdu3bxr//6r/zlL3+ha9eurFy5ssH+HWW/8Xq95OXlMX/+fF577TV++umn0x4Jtfd9psOEQ0e9Rcfx48dJTU3l6quv5sorrwQgKCiIiooKACoqKujVqxdwokZlZWW+ZU/WyOVy1amdx+Np97XbunUr69ev58EHHyQtLY1NmzaxZMkSqqqqqKmpAU5sp8vlAuruPzU1NVRVVeFwOCz71anLtFdutxu32+17dxsbG8uuXbs6/H6zceNGQkJC6NWrF3a7nSuvvJKtW7eet/tMhwmHjniLDmMMCxcuJCwsjOuuu87XHhMTQ25uLgC5ubmMGDHC175mzRqMMRQXFxMYGIjT6SQqKoqioiK8Xi9er5eioqK2+emKJrj11ltZuHAh8+fPJzk5mUsvvZTp06czbNgw1q1bB5z4pM3JfeTyyy8nJycHgHXr1jFs2DBsNhsxMTGsXbuWY8eOUVpaSklJCYMHD26tzWoRvXv3xu12s2/fPuDEi+KFF17Y4feb4OBgtm3bxs8//4wxxleX83Wf6VDfkM7Pz+dvf/ub7xYdEyZMaO0pnVXfffcdf/rTn7jooot8h/O33HILQ4YMYe7cuZSVlREcHMzMmTN9H0lcvHgxRUVFdOnShaSkJAYNGgRAdnY2mZmZwImPJI4ZM6bVtqulbd68mQ8++IDZs2dz4MABy8cSO3fuTHV1NfPmzWPXrl04HA6Sk5N9vz+yYsUKVq9eTadOnZg8eTLR0dGtvEXN9/3337Nw4UKOHz9OSEgISUlJGGM6/H7z7rvvsnbtWgICAhgwYAD3338/Ho/nvNxnOlQ4iIhI43SY00oiItJ4CgcREbFQOIiIiIXCQURELBQOIiJioXAQERELhYOIiFj8P5BeT6+xNSCwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the distributions of len\n",
    "\n",
    "plt.hist(reviews.length,bins = 50)\n",
    "plt.title(\"Distribution of review lengths\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    50000.000000\n",
       "mean       748.574380\n",
       "std        586.307924\n",
       "min         17.000000\n",
       "25%        385.000000\n",
       "50%        548.000000\n",
       "75%        914.250000\n",
       "max       8625.000000\n",
       "Name: length, dtype: float64"
      ]
     },
     "execution_count": 193,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# some information about the lengths\n",
    "\n",
    "reviews.length.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Observation** - looks like we have a very long review i.e having length = 8677. Let's see that review"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'match 1 tag team table match bubba ray spike dudley vs eddie guerrero chris benoit bubba ray spike dudley started things tag team table match eddie guerrero chris benoit according rules match opponents tables order win benoit guerrero heated early taking turns hammering spike bubba ray german suplex benoit bubba took wind dudley brother spike tried help brother referee restrained benoit guerrero ganged corner benoit stomping away bubba guerrero set table outside spike dashed ring somersaulted rope guerrero outside recovering taking care spike guerrero slipped table ring helped wolverine set tandem set double superplex middle rope bubba table spike knocked table right brother came crashing guerrero benoit propped table corner tried irish whip spike bubba dashed blocked brother bubba caught lifted opponents body drops bubba slammed guerrero spike stomped wolverine rope bubba held benoit bay spike soar wassup headbutt shortly benoit latched spike crossface match continued spike tapped bubba came brothers rescue managed sprawl benoit table bubba leapt middle rope benoit moved sent bubba crashing wood opponents didnt force table bubba allowed stay match man eliminated shortly spike eddie table dudley dawg ring apron outside benoit spike table moments later score seconds bubba nailed bubba bomb benoit table gave dudleys win winner bubba ray spike dudleymatch 2 cruiserweight championship jamie noble vs billy kidman billy kidman challenged jamie noble brought nidia ring cruiserweight championship noble kidman locked tumbled ring raced inside grappled kidman thwarted nobles moves noble fled outside ring nidia gave encouragement fight spread outside ring noble threw girlfriend challenger kidman tossed nidia aside taken modified arm bar noble continued attack kidmans injured arm ring kidmans injured harm hampered offense continued battle hard noble tried kidman away powerbomb challenger countered facebuster kidman went finish things shooting star press noble broke attempt kidman went shooting star press time noble rolled harms way noble flipped kidman power bomb soon got pin retain wwe cruiserweight championship winner jamie noblematch 3 european championship william regal vs jeff hardy william regal took jeff hardy attempt win european championship jeff catapulted regal rope took hurracanrana ring apron ring jeff hit whisper wind knock regal loop jeff went swanton bomb regal got knees hit jeff devastating shot jeff managed surprise regal quick rollup got pin european championship regal started bawling seeing hardy celebrate way ramp winner jeff hardymatch 4 chris jericho vs john cena chris jericho promised end john cenas career match vengeance came jericho tried teach cena lesson match began suplexing mat jericho continued knock cena ring cockiness got better rope jericho began showboat allowed cena grab superplex cena followed tiltawhirl slam taken nasty dropkick gut rookie recovered hit belly belly suplex y2j away jericho launched lionsault cena dodged jericho nailed bulldog connected lionsault cover goaded cena feet walls jericho cena ideas reversing pin attempt getting 123 jericho went berserk match winner john cenamatch 5 intercontinental championship rvd vs brock lesnar disqualification big thing mr payperview tangled intercontinental championship line brock grabbed title ref draped shoulder momentarily glaring rvd van dam s quickness gave brock fits early big man rolled ring kicked steel steps frustration brock pulled began charge paul heyman beaming ringside brock slammed rvd hard floor outside ring brock began overpower rvd throwing ease rope rvd landed painfully suffer having spine cracked steel ring steps fight returned ring brock squeezing rvd ribs rvd broke away soon leveled brock kick temple rvd followed rolling thunder brock managed kick twocount fight looked like soon rvd went fivestar frog splash brock hoisted van dam shoulder went f5 rvd whirled brock ddt followed frog splash went pin heyman pulled ref ring ref immediately called disqualification soon traded blows heyman rvd leapt brock rope threatened hit van terminator heyman grabbed rvds leg brock picked champ time connected f5 steel chair winner rvdmatch 6 booker t vs big booker t faced big oneonone withstood booker ts kicks punches slapped booker corner thrown ring booker picked chair ringside big punched bookers face booker tried game choking camera cable ringside booker smashed tv monitor spanish announcers position shows skull delivered scissors kick men table booker crawled ring big staggered moments later grabbed bookers throat met low blow kick face booker climbed rope nailed somersaulting leg drop pin winner booker tannouncement triple h entered ring thunderous ovation fans hoped learn game end competing speak eric bishoff stopped game apologize getting involved personal business triple h signed raw bischoff promised personal life come play bischoff said hes spent past years networking hollywood said looking breakout wwe superstar talking triple h bischoff guaranteed triple h signed raw hed getting opportunities coming way stephanie mcmahon stepped issue pitch said personal history triple h know said unstoppable bischoff cut begged stop stephanie cited triple h told bischoff said triple h talent charisma bischoff said young time didnt know lot experience stephanie continued bicker forth triple h stepped microphone game said easy screw triple h went shake bischoffs hand pulled away said devil knows doesnt know shawn michaels came shake things hbk said thing wanted cause trouble didnt want involved remembered pledging bring triple h nwo hbk said theres world triple h better friends hbk told friend imagine making bischoffs life living hell triple h said tempting offer turned hugged hbk making official switch raw triple h hbk left bischoff gloated victory bischoff said difference hes got testicles doesnt stephanie whacked bischoff head leftmatch 7 tag team championship match christian lance storm vs hollywood hogan edge match started loud usa chants hogan shoving christian ropes ring canadians took edge scored kick christians head planted facebuster storm tag hogan hogan began hulk soon caught christian big boot leg drop storm broke count christian tossed hogan ring storm superkicked icon edge tagged soon dropped opponents speared corner turnbuckles missed spear strom hit ref hard instead edge nailed ddt ref count test raced took hogan leveled edge boot storm tried pin edge kicked riksihi sprinted fend test allowing edge recover spear storm christian distracted ref y2j dashed clocked edge tag team championship storm rolled got pinfall win title winners new tag team champions christian lance stormmatch 8 wwe undisputed championship triple threat match rock vs kurt angle undertaker wwes successful superstars lined triple threat match undisputed championship hanging balance taker rock got face face kurt angle begging attention got attention form beat form men soon taker spilled ring rock brawled angle angle gave series suplexes took rock great countered ddt managed twocount fight continued outside ring taker coming life clotheslining angle repeatedly smacking rock taker rock got ring taker dropped rock sidewalk slam twocount rock rebounded grabbed taker throat chokeslammed angle broke pin attempt likely given rock title rock retaliated latching ankle lock kurt angle angle reversed rock bottomed peoples champion soon rock disposed angle hit peoples elbow undertaker angle tried advantage disabling great outside ring covering taker kicked count outside ring rock took big swig nearby water bottle spewed liquid takers face blind champion taker didnt stay disabled long managed overpower rock turn attention angle taker landed guillotine leg drop angle laying ring apron rock picked time break pin attempt kurt angle taker nailed rock ddt set chokeslam angle tried sneaking steel chair taker caught tomfoolery smacked hands referee got caught ensuing didnt angle knock taker silly steel chair angle went cover taker rock lay prone dead man got shoulder angle tried pin rock kicked rock got landed angle sharpshooter angle looked like tap taker kicked rock submission hold taker picked rock crashed ride dead man covered win angle raced picked taker ankle lock taker went delirious pain managed counter picked angle ride angle triangle choke looked like taker pass rock broke angles hold caught ankle lock rock got hold watched taker chokeslam angle rocky hit rock taker refused kicked angle whirled taker angle slam rock bottomed great pinned winner new wwe champion rockfinally decent ppv lately ppv werent good winner ppv'"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews[reviews['length'] == 8625]['text'].iloc[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modeling\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [],
   "source": [
    "# now let's convert the text into numerical representation.\n",
    "\n",
    "from sklearn.feature_extraction.text import CountVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [],
   "source": [
    "bow_transformer = CountVectorizer().fit(reviews.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "221238"
      ]
     },
     "execution_count": 199,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(bow_transformer.vocabulary_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'typical mel brooks film slapstick movies actually plot followable leslie ann warren movie fantastic underrated actress moments fleshed bit scenes probably cut room worth price rent acting good overall brooks good job characteristic speaking directly audience warren best actor movie fume sailor played parts'"
      ]
     },
     "execution_count": 200,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_message = reviews.text.iloc[4]\n",
    "sample_message"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_transformed = bow_transformer.transform([sample_message])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 5822)\t1\n",
      "  (0, 6217)\t1\n",
      "  (0, 6425)\t1\n",
      "  (0, 6531)\t1\n",
      "  (0, 12048)\t1\n",
      "  (0, 16465)\t1\n",
      "  (0, 22506)\t1\n",
      "  (0, 23844)\t1\n",
      "  (0, 28692)\t2\n",
      "  (0, 35264)\t1\n",
      "  (0, 47685)\t1\n",
      "  (0, 54211)\t1\n",
      "  (0, 69666)\t1\n",
      "  (0, 72027)\t1\n",
      "  (0, 74301)\t1\n",
      "  (0, 75097)\t1\n",
      "  (0, 78030)\t1\n",
      "  (0, 82868)\t2\n",
      "  (0, 104470)\t1\n",
      "  (0, 112468)\t1\n",
      "  (0, 122865)\t1\n",
      "  (0, 127029)\t1\n",
      "  (0, 129017)\t2\n",
      "  (0, 129575)\t1\n",
      "  (0, 142232)\t1\n",
      "  (0, 144714)\t1\n",
      "  (0, 149395)\t1\n",
      "  (0, 149753)\t1\n",
      "  (0, 153469)\t1\n",
      "  (0, 153836)\t1\n",
      "  (0, 161589)\t1\n",
      "  (0, 165519)\t1\n",
      "  (0, 167374)\t1\n",
      "  (0, 169332)\t1\n",
      "  (0, 178569)\t1\n",
      "  (0, 182238)\t1\n",
      "  (0, 203368)\t1\n",
      "  (0, 204653)\t1\n",
      "  (0, 211461)\t2\n",
      "  (0, 217748)\t1\n",
      "shape :  (1, 221238)\n"
     ]
    }
   ],
   "source": [
    "# a sparse matrix\n",
    "\n",
    "print(sample_transformed)\n",
    "\n",
    "print(\"shape : \",sample_transformed.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of entire transformed data :  (50000, 221238)\n",
      "Non-Zero elements :  4365356\n"
     ]
    }
   ],
   "source": [
    "# now let's transform the entire reviews\n",
    "\n",
    "reviews_transformed_bow = bow_transformer.transform(reviews.text)\n",
    "\n",
    "print(\"Shape of entire transformed data : \",reviews_transformed_bow.shape)\n",
    "\n",
    "# since it will be a sparse matrix we can count no of non-zero elements\n",
    "\n",
    "print(\"Non-Zero elements : \",reviews_transformed_bow.nnz)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now we have transformed the text into numerical form.\n",
    "# But we can also assign a weights to each word by using Tf-Idf \n",
    "\n",
    "from sklearn.feature_extraction.text import TfidfTransformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf_idf_transformer = TfidfTransformer().fit(reviews_transformed_bow)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 217748)\t0.09540094558207422\n",
      "  (0, 211461)\t0.37656471727729884\n",
      "  (0, 204653)\t0.15556757911139552\n",
      "  (0, 203368)\t0.12590501523469033\n",
      "  (0, 182238)\t0.14676835873213936\n",
      "  (0, 178569)\t0.16099404381002486\n",
      "  (0, 169332)\t0.07874105947387403\n",
      "  (0, 167374)\t0.19930806413934105\n",
      "  (0, 165519)\t0.12468049502771246\n",
      "  (0, 161589)\t0.12792560668677466\n",
      "  (0, 153836)\t0.0917520041958287\n",
      "  (0, 153469)\t0.1553923035722736\n",
      "  (0, 149753)\t0.07274147234514643\n",
      "  (0, 149395)\t0.09571654478972869\n",
      "  (0, 144714)\t0.11453226369558171\n",
      "  (0, 142232)\t0.11472993830944923\n",
      "  (0, 129575)\t0.06918975589696753\n",
      "  (0, 129017)\t0.0829982523776984\n",
      "  (0, 127029)\t0.10674294639955594\n",
      "  (0, 122865)\t0.17741901690859563\n",
      "  (0, 112468)\t0.1748164292574346\n",
      "  (0, 104470)\t0.09919484093384666\n",
      "  (0, 82868)\t0.10945238499662319\n",
      "  (0, 78030)\t0.27084597030567475\n",
      "  (0, 75097)\t0.2750744049283481\n",
      "  (0, 74301)\t0.20402200751680974\n",
      "  (0, 72027)\t0.04429582437975536\n",
      "  (0, 69666)\t0.12758537871527953\n",
      "  (0, 54211)\t0.16208703814168568\n",
      "  (0, 47685)\t0.12212317390926004\n",
      "  (0, 35264)\t0.20507043558596\n",
      "  (0, 28692)\t0.3508281357964893\n",
      "  (0, 23844)\t0.09089006432773569\n",
      "  (0, 22506)\t0.07307257169777742\n",
      "  (0, 16465)\t0.10154478124798691\n",
      "  (0, 12048)\t0.17055423397998706\n",
      "  (0, 6531)\t0.08221426825943193\n",
      "  (0, 6425)\t0.11684631121743641\n",
      "  (0, 6217)\t0.09964136788154215\n",
      "  (0, 5822)\t0.07052143391947997\n"
     ]
    }
   ],
   "source": [
    "# let's see the tf-idf transformed version of the sample we have used earlier\n",
    "sample_transformed_tfidf = tf_idf_transformer.transform(sample_transformed)\n",
    "print(sample_transformed_tfidf) # each word has now a probablity assigned to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {},
   "outputs": [],
   "source": [
    "# transform all the reviews\n",
    "\n",
    "reviews_transformed_tfidf = tf_idf_transformer.transform(reviews_transformed_bow)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50000, 221238)"
      ]
     },
     "execution_count": 215,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews_transformed_tfidf.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's apply `Naive bayes classifier`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import MultinomialNB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**In case of Naive bayes**\n",
    "\n",
    "There is only one `hyper parameter` to tune i.e $\\alpha $ Which represents `Additive (Laplace/Lidstone) smoothing parameter.`\n",
    "\n",
    "* $\\alpha=0$ i.e no smoothing will lead to high variance i.e under-fitting\n",
    "* $\\alpha=large-value$ will lead to high bias i.e under fitting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_classifier = MultinomialNB().fit(reviews_transformed_tfidf,reviews.label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>label</th>\n",
       "      <th>length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>typical mel brooks film slapstick movies actua...</td>\n",
       "      <td>pos</td>\n",
       "      <td>307</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                text label  length\n",
       "4  typical mel brooks film slapstick movies actua...   pos     307"
      ]
     },
     "execution_count": 219,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's check the classifier\n",
    "\n",
    "# data \n",
    "\n",
    "reviews[reviews.text == sample_message]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {},
   "outputs": [],
   "source": [
    "# prediction from model\n",
    "\n",
    "predicted_sample = review_classifier.predict(sample_transformed_tfidf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['pos'], dtype='<U3')"
      ]
     },
     "execution_count": 221,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predicted_sample # same as true label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have a working model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's predict all data points\n",
    "\n",
    "predicted_all = review_classifier.predict(reviews_transformed_tfidf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's look at some classification metrics\n",
    "\n",
    "from sklearn.metrics import classification_report,confusion_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 233,
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_classification_report(true_labels,predicted_labels):\n",
    "    \"\"\"\n",
    "    to print classification report and confusion matrix\n",
    "    \"\"\"\n",
    "    # print classification report\n",
    "    print(\"Classification report \")\n",
    "    print(classification_report(true_labels,predicted_labels))\n",
    "\n",
    "    print(\"*\"*100)\n",
    "\n",
    "    print(\"Confusion metrics \")\n",
    "    print(pd.DataFrame(data = confusion_matrix(true_labels,predicted_labels)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification report \n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         neg       0.91      0.92      0.92     25000\n",
      "         pos       0.92      0.91      0.92     25000\n",
      "\n",
      "    accuracy                           0.92     50000\n",
      "   macro avg       0.92      0.92      0.92     50000\n",
      "weighted avg       0.92      0.92      0.92     50000\n",
      "\n",
      "****************************************************************************************************\n",
      "Confusion metrics \n",
      "       0      1\n",
      "0  23114   1886\n",
      "1   2223  22777\n"
     ]
    }
   ],
   "source": [
    "print_classification_report(reviews.label,predicted_all)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Looks like we have a nice classifier but wait we did a `blunder`. We applied classification report on training data as we did not have test data**\n",
    "\n",
    "**Hence we have no idea about the true predictive power of our classifier**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Let's apply the model in right way** \n",
    "\n",
    "We will also use pipelines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 235,
   "metadata": {},
   "outputs": [],
   "source": [
    "# perform test and train split\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_train,review_test,label_train,label_test = train_test_split(reviews.text.values,reviews.label.values,\n",
    "                                                                    test_size = 0.2, stratify = reviews.label.values,\n",
    "                                                                  random_state = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(40000,) (10000,)\n"
     ]
    }
   ],
   "source": [
    "print(review_train.shape,review_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 274,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pipeline to perform the steps in one go\n",
    "review_pipeline = Pipeline(steps = [\n",
    "    ('bow_transform',CountVectorizer()),\n",
    "    ('tfidf_transform',TfidfTransformer()),\n",
    "    ('classifier',MultinomialNB())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 275,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "         steps=[('bow_transform',\n",
       "                 CountVectorizer(analyzer='word', binary=False,\n",
       "                                 decode_error='strict',\n",
       "                                 dtype=<class 'numpy.int64'>, encoding='utf-8',\n",
       "                                 input='content', lowercase=True, max_df=1.0,\n",
       "                                 max_features=None, min_df=1,\n",
       "                                 ngram_range=(1, 1), preprocessor=None,\n",
       "                                 stop_words=None, strip_accents=None,\n",
       "                                 token_pattern='(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
       "                                 tokenizer=None, vocabulary=None)),\n",
       "                ('tfidf_transform',\n",
       "                 TfidfTransformer(norm='l2', smooth_idf=True,\n",
       "                                  sublinear_tf=False, use_idf=True)),\n",
       "                ('classifier',\n",
       "                 MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True))],\n",
       "         verbose=False)"
      ]
     },
     "execution_count": 275,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review_pipeline.fit(review_train,label_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 276,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions_test = review_pipeline.predict(review_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 277,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification report \n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         neg       0.86      0.87      0.86      5000\n",
      "         pos       0.87      0.85      0.86      5000\n",
      "\n",
      "    accuracy                           0.86     10000\n",
      "   macro avg       0.86      0.86      0.86     10000\n",
      "weighted avg       0.86      0.86      0.86     10000\n",
      "\n",
      "****************************************************************************************************\n",
      "Confusion metrics \n",
      "      0     1\n",
      "0  4358   642\n",
      "1   730  4270\n"
     ]
    }
   ],
   "source": [
    "# print the classification report\n",
    "\n",
    "print_classification_report(label_test,predictions_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 263,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's also see what is the accuracy \n",
    "\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 264,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8628"
      ]
     },
     "execution_count": 264,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy_score(label_test,predictions_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Looks like we have a pretty good model**\n",
    "\n",
    "**But What about cross validation and hyper-parameter search ?**\n",
    "\n",
    "\n",
    "- we can use `sklearn.model_selection.GridSearchCV` or `sklearn.model_selection.RandomizedSearchCV` to find best hyper-parameter. But how does `sklearn` do that -- By using `Cross validation`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 265,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV,RandomizedSearchCV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In case of `GridSearchCV` we have to provide a list of possible hyper-parameters as a dictionary in case we have only one hyper-parameter to tune or a list of dictionaries. On the other hand in `RandomizedSearchCV` we have to provide a distribution from which the value of hyper-parameters will be sampled randomly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 292,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import f1_score,make_scorer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 302,
   "metadata": {},
   "outputs": [],
   "source": [
    "# modeling using GridSearchCV\n",
    "\n",
    "f1_scorer = make_scorer(f1_score, pos_label=\"pos\")\n",
    "\n",
    "parameter_to_tune = {\"classifier__alpha\":[0.5,0.6,0.7,0.8]} # pipeline name of the our algo or estimator\n",
    "\n",
    "model_grid_search = GridSearchCV(review_pipeline,\n",
    "                                 param_grid = parameter_to_tune,\n",
    "                                 scoring = f1_scorer,cv = 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 303,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=3, error_score='raise-deprecating',\n",
       "             estimator=Pipeline(memory=None,\n",
       "                                steps=[('bow_transform',\n",
       "                                        CountVectorizer(analyzer='word',\n",
       "                                                        binary=False,\n",
       "                                                        decode_error='strict',\n",
       "                                                        dtype=<class 'numpy.int64'>,\n",
       "                                                        encoding='utf-8',\n",
       "                                                        input='content',\n",
       "                                                        lowercase=True,\n",
       "                                                        max_df=1.0,\n",
       "                                                        max_features=None,\n",
       "                                                        min_df=1,\n",
       "                                                        ngram_range=(1, 1),\n",
       "                                                        preprocessor=None,\n",
       "                                                        stop_words=None,\n",
       "                                                        strip_accents...\n",
       "                                        TfidfTransformer(norm='l2',\n",
       "                                                         smooth_idf=True,\n",
       "                                                         sublinear_tf=False,\n",
       "                                                         use_idf=True)),\n",
       "                                       ('classifier',\n",
       "                                        MultinomialNB(alpha=1.0,\n",
       "                                                      class_prior=None,\n",
       "                                                      fit_prior=True))],\n",
       "                                verbose=False),\n",
       "             iid='warn', n_jobs=None,\n",
       "             param_grid={'classifier__alpha': [0.5, 0.6, 0.7, 0.8]},\n",
       "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
       "             scoring=make_scorer(f1_score, pos_label=pos), verbose=0)"
      ]
     },
     "execution_count": 303,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_grid_search.fit(review_train,label_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 305,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "         steps=[('bow_transform',\n",
       "                 CountVectorizer(analyzer='word', binary=False,\n",
       "                                 decode_error='strict',\n",
       "                                 dtype=<class 'numpy.int64'>, encoding='utf-8',\n",
       "                                 input='content', lowercase=True, max_df=1.0,\n",
       "                                 max_features=None, min_df=1,\n",
       "                                 ngram_range=(1, 1), preprocessor=None,\n",
       "                                 stop_words=None, strip_accents=None,\n",
       "                                 token_pattern='(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
       "                                 tokenizer=None, vocabulary=None)),\n",
       "                ('tfidf_transform',\n",
       "                 TfidfTransformer(norm='l2', smooth_idf=True,\n",
       "                                  sublinear_tf=False, use_idf=True)),\n",
       "                ('classifier',\n",
       "                 MultinomialNB(alpha=0.7, class_prior=None, fit_prior=True))],\n",
       "         verbose=False)"
      ]
     },
     "execution_count": 305,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_grid_search.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 306,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8614950065570464"
      ]
     },
     "execution_count": 306,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's check model score on test data\n",
    "\n",
    "model_grid_search.score(review_test,label_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 307,
   "metadata": {},
   "outputs": [],
   "source": [
    "predicted_test_gridsearch = model_grid_search.predict(review_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 308,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification report \n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         neg       0.86      0.87      0.86      5000\n",
      "         pos       0.87      0.85      0.86      5000\n",
      "\n",
      "    accuracy                           0.86     10000\n",
      "   macro avg       0.86      0.86      0.86     10000\n",
      "weighted avg       0.86      0.86      0.86     10000\n",
      "\n",
      "****************************************************************************************************\n",
      "Confusion metrics \n",
      "      0     1\n",
      "0  4357   643\n",
      "1   730  4270\n"
     ]
    }
   ],
   "source": [
    "print_classification_report(label_test,predicted_test_gridsearch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 323,
   "metadata": {},
   "outputs": [],
   "source": [
    "# modeling using randomizedSearchCV\n",
    "\n",
    "from scipy.stats import uniform\n",
    "\n",
    "\n",
    "distributions = dict(classifier__alpha=uniform(loc=0, scale=4))\n",
    "\n",
    "\n",
    "model_randSearch = RandomizedSearchCV(review_pipeline,distributions,\n",
    "                                 scoring = f1_scorer,cv = 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 324,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomizedSearchCV(cv=3, error_score='raise-deprecating',\n",
       "                   estimator=Pipeline(memory=None,\n",
       "                                      steps=[('bow_transform',\n",
       "                                              CountVectorizer(analyzer='word',\n",
       "                                                              binary=False,\n",
       "                                                              decode_error='strict',\n",
       "                                                              dtype=<class 'numpy.int64'>,\n",
       "                                                              encoding='utf-8',\n",
       "                                                              input='content',\n",
       "                                                              lowercase=True,\n",
       "                                                              max_df=1.0,\n",
       "                                                              max_features=None,\n",
       "                                                              min_df=1,\n",
       "                                                              ngram_range=(1,\n",
       "                                                                           1),\n",
       "                                                              preprocessor=None,\n",
       "                                                              stop_words=None,\n",
       "                                                              strip_a...\n",
       "                                              MultinomialNB(alpha=1.0,\n",
       "                                                            class_prior=None,\n",
       "                                                            fit_prior=True))],\n",
       "                                      verbose=False),\n",
       "                   iid='warn', n_iter=10, n_jobs=None,\n",
       "                   param_distributions={'classifier__alpha': <scipy.stats._distn_infrastructure.rv_frozen object at 0x000001DA349740C8>},\n",
       "                   pre_dispatch='2*n_jobs', random_state=None, refit=True,\n",
       "                   return_train_score=False,\n",
       "                   scoring=make_scorer(f1_score, pos_label=pos), verbose=0)"
      ]
     },
     "execution_count": 324,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_randSearch.fit(review_train,label_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 325,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "         steps=[('bow_transform',\n",
       "                 CountVectorizer(analyzer='word', binary=False,\n",
       "                                 decode_error='strict',\n",
       "                                 dtype=<class 'numpy.int64'>, encoding='utf-8',\n",
       "                                 input='content', lowercase=True, max_df=1.0,\n",
       "                                 max_features=None, min_df=1,\n",
       "                                 ngram_range=(1, 1), preprocessor=None,\n",
       "                                 stop_words=None, strip_accents=None,\n",
       "                                 token_pattern='(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
       "                                 tokenizer=None, vocabulary=None)),\n",
       "                ('tfidf_transform',\n",
       "                 TfidfTransformer(norm='l2', smooth_idf=True,\n",
       "                                  sublinear_tf=False, use_idf=True)),\n",
       "                ('classifier',\n",
       "                 MultinomialNB(alpha=0.5448322875490428, class_prior=None,\n",
       "                               fit_prior=True))],\n",
       "         verbose=False)"
      ]
     },
     "execution_count": 325,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_randSearch.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 326,
   "metadata": {},
   "outputs": [],
   "source": [
    "predicted_test_Randsearch = model_randSearch.predict(review_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 328,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification report \n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         neg       0.86      0.87      0.86      5000\n",
      "         pos       0.87      0.86      0.86      5000\n",
      "\n",
      "    accuracy                           0.86     10000\n",
      "   macro avg       0.86      0.86      0.86     10000\n",
      "weighted avg       0.86      0.86      0.86     10000\n",
      "\n",
      "****************************************************************************************************\n",
      "Confusion metrics \n",
      "      0     1\n",
      "0  4354   646\n",
      "1   724  4276\n"
     ]
    }
   ],
   "source": [
    "print_classification_report(label_test,predicted_test_Randsearch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**As we can see we are not observing any improvement in our model at all**\n",
    "\n",
    "Yup no use of hyper-parameter search in this case. Or we may have failed to find best hyper-param space.\n",
    "\n",
    "**Let's try one more simple model i.e `logistic regression`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 329,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 330,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create pipeline for logistic regression\n",
    "\n",
    "\n",
    "logistic_review_pipeline = Pipeline(steps = [\n",
    "    ('bow_transform',CountVectorizer()),\n",
    "    ('tfidf_transform',TfidfTransformer()),\n",
    "    ('classifier',LogisticRegression())\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 331,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "         steps=[('bow_transform',\n",
       "                 CountVectorizer(analyzer='word', binary=False,\n",
       "                                 decode_error='strict',\n",
       "                                 dtype=<class 'numpy.int64'>, encoding='utf-8',\n",
       "                                 input='content', lowercase=True, max_df=1.0,\n",
       "                                 max_features=None, min_df=1,\n",
       "                                 ngram_range=(1, 1), preprocessor=None,\n",
       "                                 stop_words=None, strip_accents=None,\n",
       "                                 token_pattern='(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
       "                                 tokenizer=None, vocab...\n",
       "                ('tfidf_transform',\n",
       "                 TfidfTransformer(norm='l2', smooth_idf=True,\n",
       "                                  sublinear_tf=False, use_idf=True)),\n",
       "                ('classifier',\n",
       "                 LogisticRegression(C=1.0, class_weight=None, dual=False,\n",
       "                                    fit_intercept=True, intercept_scaling=1,\n",
       "                                    l1_ratio=None, max_iter=100,\n",
       "                                    multi_class='warn', n_jobs=None,\n",
       "                                    penalty='l2', random_state=None,\n",
       "                                    solver='warn', tol=0.0001, verbose=0,\n",
       "                                    warm_start=False))],\n",
       "         verbose=False)"
      ]
     },
     "execution_count": 331,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logistic_review_pipeline.fit(review_train,label_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 332,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification report \n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         neg       0.91      0.88      0.89      5000\n",
      "         pos       0.89      0.91      0.90      5000\n",
      "\n",
      "    accuracy                           0.90     10000\n",
      "   macro avg       0.90      0.90      0.90     10000\n",
      "weighted avg       0.90      0.90      0.90     10000\n",
      "\n",
      "****************************************************************************************************\n",
      "Confusion metrics \n",
      "      0     1\n",
      "0  4413   587\n",
      "1   462  4538\n"
     ]
    }
   ],
   "source": [
    "predicted_test_log = logistic_review_pipeline.predict(review_test)\n",
    "\n",
    "print_classification_report(label_test,predicted_test_log)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Wow looks like logistic regression is performing well**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 339,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's do some hyper-param search for logistic reg\n",
    "\n",
    "f1_scorer = make_scorer(f1_score, pos_label=\"pos\")\n",
    "\n",
    "parameter_to_tune = {\"classifier__C\":[1,2,3,4,5,6,7,8,9,10,10,11,12]} # pipeline name of the our algo or estimator\n",
    "\n",
    "model_grid_search = GridSearchCV(logistic_review_pipeline,\n",
    "                                 param_grid = parameter_to_tune,\n",
    "                                 scoring = f1_scorer,cv = 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 340,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise-deprecating',\n",
       "             estimator=Pipeline(memory=None,\n",
       "                                steps=[('bow_transform',\n",
       "                                        CountVectorizer(analyzer='word',\n",
       "                                                        binary=False,\n",
       "                                                        decode_error='strict',\n",
       "                                                        dtype=<class 'numpy.int64'>,\n",
       "                                                        encoding='utf-8',\n",
       "                                                        input='content',\n",
       "                                                        lowercase=True,\n",
       "                                                        max_df=1.0,\n",
       "                                                        max_features=None,\n",
       "                                                        min_df=1,\n",
       "                                                        ngram_range=(1, 1),\n",
       "                                                        preprocessor=None,\n",
       "                                                        stop_words=None,\n",
       "                                                        strip_accents...\n",
       "                                                           l1_ratio=None,\n",
       "                                                           max_iter=100,\n",
       "                                                           multi_class='warn',\n",
       "                                                           n_jobs=None,\n",
       "                                                           penalty='l2',\n",
       "                                                           random_state=None,\n",
       "                                                           solver='warn',\n",
       "                                                           tol=0.0001,\n",
       "                                                           verbose=0,\n",
       "                                                           warm_start=False))],\n",
       "                                verbose=False),\n",
       "             iid='warn', n_jobs=None,\n",
       "             param_grid={'classifier__C': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 10,\n",
       "                                           11, 12]},\n",
       "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
       "             scoring=make_scorer(f1_score, pos_label=pos), verbose=0)"
      ]
     },
     "execution_count": 340,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_grid_search.fit(review_train,label_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 341,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "         steps=[('bow_transform',\n",
       "                 CountVectorizer(analyzer='word', binary=False,\n",
       "                                 decode_error='strict',\n",
       "                                 dtype=<class 'numpy.int64'>, encoding='utf-8',\n",
       "                                 input='content', lowercase=True, max_df=1.0,\n",
       "                                 max_features=None, min_df=1,\n",
       "                                 ngram_range=(1, 1), preprocessor=None,\n",
       "                                 stop_words=None, strip_accents=None,\n",
       "                                 token_pattern='(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
       "                                 tokenizer=None, vocab...\n",
       "                ('tfidf_transform',\n",
       "                 TfidfTransformer(norm='l2', smooth_idf=True,\n",
       "                                  sublinear_tf=False, use_idf=True)),\n",
       "                ('classifier',\n",
       "                 LogisticRegression(C=5, class_weight=None, dual=False,\n",
       "                                    fit_intercept=True, intercept_scaling=1,\n",
       "                                    l1_ratio=None, max_iter=100,\n",
       "                                    multi_class='warn', n_jobs=None,\n",
       "                                    penalty='l2', random_state=None,\n",
       "                                    solver='warn', tol=0.0001, verbose=0,\n",
       "                                    warm_start=False))],\n",
       "         verbose=False)"
      ]
     },
     "execution_count": 341,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_grid_search.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 342,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification report \n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         neg       0.91      0.89      0.90      5000\n",
      "         pos       0.89      0.91      0.90      5000\n",
      "\n",
      "    accuracy                           0.90     10000\n",
      "   macro avg       0.90      0.90      0.90     10000\n",
      "weighted avg       0.90      0.90      0.90     10000\n",
      "\n",
      "****************************************************************************************************\n",
      "Confusion metrics \n",
      "      0     1\n",
      "0  4430   570\n",
      "1   457  4543\n"
     ]
    }
   ],
   "source": [
    "predicted_test_log = model_grid_search.predict(review_test)\n",
    "\n",
    "print_classification_report(label_test,predicted_test_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.7.5 64-bit ('pytorch_env': conda)",
   "language": "python",
   "name": "python37564bitpytorchenvconda1a599ba2df0f415ab16aa82c434e76bb"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}