02_Lab_2_Data_analysis_visualisation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introductory applied machine learning (INFR10069)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 2: Data analysis and visualisation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this lab we work with a spam filtering dataset. We will perform exploratory data analysis, visualisation and, finally, we learn how to perform classification tasks using Naive Bayes. For this, we will use the the packages introduced in Lab 1, and `scikit-learn` package (`sklearn`): a machine learning library for Python which works with numpy array, and pandas DataFrame objects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Please Note**: Throughout this lab we make reference to [`methods`](https://en.wikipedia.org/wiki/Method_%28computer_programming%29) for specific objects e.g. \"make use of the predict method of the MultinomialNB classifier\". If you get confused, refer to the documentation and just ctrl+f for the object concerned:\n",
    "* [Scikit-learn API documentation](http://scikit-learn.org/stable/modules/classes.html) \n",
    "* [Seaborn API documentation](https://stanford.edu/~mwaskom/software/seaborn/api.html)\n",
    "* [Matplotlib Pyplot documentation](http://matplotlib.org/1.5.3/api/pyplot_summary.html)\n",
    "* [Pandas API documentation](http://pandas.pydata.org/pandas-docs/stable/api.html)\n",
    "* [Numpy documentation](http://docs.scipy.org/doc/numpy/reference/)\n",
    "\n",
    "There are also tonnes of great examples online; googling key words with the word \"example\" will serve you well."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we need to import the packages (run all the code cells as you read along):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Import packages\n",
    "from __future__ import division, print_function # Imports from __future__ since we're running Python 2\n",
    "import os\n",
    "import numpy as np \n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Clarification*:\n",
    "\n",
    "* The `%matplotlib inline` command is a special ipython [built in magic command](http://ipython.readthedocs.io/en/stable/interactive/magics.html) which forces the matplotlib plots to be rendered within the notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Spambase dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [Spambase](http://archive.ics.uci.edu/ml/datasets/Spambase) dataset consists of tagged emails from a single email account. You should read through the description available for this data to get a feel for what you're dealing with. We have downloaded the dataset for you.\n",
    "\n",
    "You will find the dataset located at `./datasets/spambase.csv` (the `datasets` directory is adjacent to this file). Execute the cell below to load the csv into in a pandas DataFrame object. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Load the dataset\n",
    "data_path = os.path.join(os.getcwd(), 'datasets', 'spambase.csv')\n",
    "spambase = pd.read_csv(data_path, delimiter = ',')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have now loaded the data. Let's get a feeling of what the data looks like by using the `head()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>word_freq_make</th>\n",
       "      <th>word_freq_address</th>\n",
       "      <th>word_freq_all</th>\n",
       "      <th>word_freq_3d</th>\n",
       "      <th>word_freq_our</th>\n",
       "      <th>word_freq_over</th>\n",
       "      <th>word_freq_remove</th>\n",
       "      <th>word_freq_internet</th>\n",
       "      <th>word_freq_order</th>\n",
       "      <th>word_freq_mail</th>\n",
       "      <th>...</th>\n",
       "      <th>char_freq_;</th>\n",
       "      <th>char_freq_(</th>\n",
       "      <th>char_freq_[</th>\n",
       "      <th>char_freq_!</th>\n",
       "      <th>char_freq_$</th>\n",
       "      <th>char_freq_#</th>\n",
       "      <th>capital_run_length_average</th>\n",
       "      <th>capital_run_length_longest</th>\n",
       "      <th>capital_run_length_total</th>\n",
       "      <th>is_spam</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.00</td>\n",
       "      <td>0.64</td>\n",
       "      <td>0.64</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.32</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>...</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.778</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>3.756</td>\n",
       "      <td>61.0</td>\n",
       "      <td>278.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.21</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.50</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.21</td>\n",
       "      <td>0.07</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.94</td>\n",
       "      <td>...</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.132</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.372</td>\n",
       "      <td>0.180</td>\n",
       "      <td>0.048</td>\n",
       "      <td>5.114</td>\n",
       "      <td>101.0</td>\n",
       "      <td>1028.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.06</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.71</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.23</td>\n",
       "      <td>0.19</td>\n",
       "      <td>0.19</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.64</td>\n",
       "      <td>0.25</td>\n",
       "      <td>...</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.143</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.276</td>\n",
       "      <td>0.184</td>\n",
       "      <td>0.010</td>\n",
       "      <td>9.821</td>\n",
       "      <td>485.0</td>\n",
       "      <td>2259.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.63</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.63</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.63</td>\n",
       "      <td>...</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.137</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.137</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>3.537</td>\n",
       "      <td>40.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.63</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.63</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.63</td>\n",
       "      <td>...</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>3.537</td>\n",
       "      <td>40.0</td>\n",
       "      <td>191.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 58 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   word_freq_make  word_freq_address  word_freq_all  word_freq_3d  \\\n",
       "0            0.00               0.64           0.64           0.0   \n",
       "1            0.21               0.28           0.50           0.0   \n",
       "2            0.06               0.00           0.71           0.0   \n",
       "3            0.00               0.00           0.00           0.0   \n",
       "4            0.00               0.00           0.00           0.0   \n",
       "\n",
       "   word_freq_our  word_freq_over  word_freq_remove  word_freq_internet  \\\n",
       "0           0.32            0.00              0.00                0.00   \n",
       "1           0.14            0.28              0.21                0.07   \n",
       "2           1.23            0.19              0.19                0.12   \n",
       "3           0.63            0.00              0.31                0.63   \n",
       "4           0.63            0.00              0.31                0.63   \n",
       "\n",
       "   word_freq_order  word_freq_mail   ...     char_freq_;  char_freq_(  \\\n",
       "0             0.00            0.00   ...            0.00        0.000   \n",
       "1             0.00            0.94   ...            0.00        0.132   \n",
       "2             0.64            0.25   ...            0.01        0.143   \n",
       "3             0.31            0.63   ...            0.00        0.137   \n",
       "4             0.31            0.63   ...            0.00        0.135   \n",
       "\n",
       "   char_freq_[  char_freq_!  char_freq_$  char_freq_#  \\\n",
       "0          0.0        0.778        0.000        0.000   \n",
       "1          0.0        0.372        0.180        0.048   \n",
       "2          0.0        0.276        0.184        0.010   \n",
       "3          0.0        0.137        0.000        0.000   \n",
       "4          0.0        0.135        0.000        0.000   \n",
       "\n",
       "   capital_run_length_average  capital_run_length_longest  \\\n",
       "0                       3.756                        61.0   \n",
       "1                       5.114                       101.0   \n",
       "2                       9.821                       485.0   \n",
       "3                       3.537                        40.0   \n",
       "4                       3.537                        40.0   \n",
       "\n",
       "   capital_run_length_total  is_spam  \n",
       "0                     278.0      1.0  \n",
       "1                    1028.0      1.0  \n",
       "2                    2259.0      1.0  \n",
       "3                     191.0      1.0  \n",
       "4                     191.0      1.0  \n",
       "\n",
       "[5 rows x 58 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spambase.head(5) # Display the 5 first rows of the dataframe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 1 =========="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**a)** Display the number of attributes in the dataset (i.e. number of columns)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "58"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "len(spambase.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**b)** Display the number of observations (i.e. number of rows)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4601"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "spambase.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**c)** Display the mean and standard deviation of each attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "word_freq_make                  0.104553\n",
      "word_freq_address               0.213015\n",
      "word_freq_all                   0.280656\n",
      "word_freq_3d                    0.065425\n",
      "word_freq_our                   0.312223\n",
      "word_freq_over                  0.095901\n",
      "word_freq_remove                0.114208\n",
      "word_freq_internet              0.105295\n",
      "word_freq_order                 0.090067\n",
      "word_freq_mail                  0.239413\n",
      "word_freq_receive               0.059824\n",
      "word_freq_will                  0.541702\n",
      "word_freq_people                0.093930\n",
      "word_freq_report                0.058626\n",
      "word_freq_addresses             0.049205\n",
      "word_freq_free                  0.248848\n",
      "word_freq_business              0.142586\n",
      "word_freq_email                 0.184745\n",
      "word_freq_you                   1.662100\n",
      "word_freq_credit                0.085577\n",
      "word_freq_your                  0.809761\n",
      "word_freq_font                  0.121202\n",
      "word_freq_000                   0.101645\n",
      "word_freq_money                 0.094269\n",
      "word_freq_hp                    0.549504\n",
      "word_freq_hpl                   0.265384\n",
      "word_freq_george                0.767305\n",
      "word_freq_650                   0.124845\n",
      "word_freq_lab                   0.098915\n",
      "word_freq_labs                  0.102852\n",
      "word_freq_telnet                0.064753\n",
      "word_freq_857                   0.047048\n",
      "word_freq_data                  0.097229\n",
      "word_freq_415                   0.047835\n",
      "word_freq_85                    0.105412\n",
      "word_freq_technology            0.097477\n",
      "word_freq_1999                  0.136953\n",
      "word_freq_parts                 0.013201\n",
      "word_freq_pm                    0.078629\n",
      "word_freq_direct                0.064834\n",
      "word_freq_cs                    0.043667\n",
      "word_freq_meeting               0.132339\n",
      "word_freq_original              0.046099\n",
      "word_freq_project               0.079196\n",
      "word_freq_re                    0.301224\n",
      "word_freq_edu                   0.179824\n",
      "word_freq_table                 0.005444\n",
      "word_freq_conference            0.031869\n",
      "char_freq_;                     0.038575\n",
      "char_freq_(                     0.139030\n",
      "char_freq_[                     0.016976\n",
      "char_freq_!                     0.269071\n",
      "char_freq_$                     0.075811\n",
      "char_freq_#                     0.044238\n",
      "capital_run_length_average      5.191515\n",
      "capital_run_length_longest     52.172789\n",
      "capital_run_length_total      283.289285\n",
      "is_spam                         0.394045\n",
      "dtype: float64 word_freq_make                  0.305358\n",
      "word_freq_address               1.290575\n",
      "word_freq_all                   0.504143\n",
      "word_freq_3d                    1.395151\n",
      "word_freq_our                   0.672513\n",
      "word_freq_over                  0.273824\n",
      "word_freq_remove                0.391441\n",
      "word_freq_internet              0.401071\n",
      "word_freq_order                 0.278616\n",
      "word_freq_mail                  0.644755\n",
      "word_freq_receive               0.201545\n",
      "word_freq_will                  0.861698\n",
      "word_freq_people                0.301036\n",
      "word_freq_report                0.335184\n",
      "word_freq_addresses             0.258843\n",
      "word_freq_free                  0.825792\n",
      "word_freq_business              0.444055\n",
      "word_freq_email                 0.531122\n",
      "word_freq_you                   1.775481\n",
      "word_freq_credit                0.509767\n",
      "word_freq_your                  1.200810\n",
      "word_freq_font                  1.025756\n",
      "word_freq_000                   0.350286\n",
      "word_freq_money                 0.442636\n",
      "word_freq_hp                    1.671349\n",
      "word_freq_hpl                   0.886955\n",
      "word_freq_george                3.367292\n",
      "word_freq_650                   0.538576\n",
      "word_freq_lab                   0.593327\n",
      "word_freq_labs                  0.456682\n",
      "word_freq_telnet                0.403393\n",
      "word_freq_857                   0.328559\n",
      "word_freq_data                  0.555907\n",
      "word_freq_415                   0.329445\n",
      "word_freq_85                    0.532260\n",
      "word_freq_technology            0.402623\n",
      "word_freq_1999                  0.423451\n",
      "word_freq_parts                 0.220651\n",
      "word_freq_pm                    0.434672\n",
      "word_freq_direct                0.349916\n",
      "word_freq_cs                    0.361205\n",
      "word_freq_meeting               0.766819\n",
      "word_freq_original              0.223812\n",
      "word_freq_project               0.621976\n",
      "word_freq_re                    1.011687\n",
      "word_freq_edu                   0.911119\n",
      "word_freq_table                 0.076274\n",
      "word_freq_conference            0.285735\n",
      "char_freq_;                     0.243471\n",
      "char_freq_(                     0.270355\n",
      "char_freq_[                     0.109394\n",
      "char_freq_!                     0.815672\n",
      "char_freq_$                     0.245882\n",
      "char_freq_#                     0.429342\n",
      "capital_run_length_average     31.729449\n",
      "capital_run_length_longest    194.891310\n",
      "capital_run_length_total      606.347851\n",
      "is_spam                         0.488698\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# Your code goes here\n",
    "print(spambase.mean(),spambase.std())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now want to *remove* some of the attributes from our data. There are various reasons for wanting to do so, for instance we might think that these are not relevant to the task we want to perform (i.e. e-mail classification) or they might have been contaminated with noise during the data collection process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 2 =========="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**a)** Delete the `capital_run_length_average`, `capital_run_length_longest` and  `capital_run_length_total` attributes. *Hint*: You should make use of the [`drop`](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.drop.html) method. *Tip*: some pandas methods have the argument `inplace` which you can use to determine whether they alter the object they are called upon and return nothing, or return a new object. This is particularly useful if you are dealing with huge datasets where you would typically want to operate `inplace`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code goes here\n",
    "df2 = spambase.drop(['capital_run_length_average','capital_run_length_longest','capital_run_length_total'],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**b)** Display the new number of attributes. Does it look like what you expected?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "55"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "df2.shape[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The remaining attributes represent relative frequencies of various important words and characters in emails. This is true for all attributes except `is_spam` which represents whether the e-mail was annotated as spam or not. So each e-mail is represented by a 55 dimensional vector representing whether or not a particular word exists in an e-mail. This is the so called [bag of words](http://en.wikipedia.org/wiki/Bag_of_words_model) representation and is clearly a very crude approximation since it does not take into account the order of the words in the emails."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 3 =========="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's get a feeling of the distribution of ham (i.e. valid) vs. spam emails. We can do this by using a [countplot](https://stanford.edu/~mwaskom/software/seaborn/generated/seaborn.countplot.html) in seaborn."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**a)** Produce a seaborn [countplot](https://stanford.edu/~mwaskom/software/seaborn/generated/seaborn.countplot.html) object that shows the distribution of ham/spam e-mails. Assign it to a variable (e.g. `ax` to emphasise it is a [matplotlib.axes.Axes](http://matplotlib.org/api/axes_api.html#axes) object)\n",
    "  \n",
    "**b)** In the same cell, modify the labels on the x axis (`xticklabels`) to `Ham` and `Spam` (by default they should be set to `0.0` and `1.0`). *Hint: Axes objects have a [`set_xticklabels`](http://matplotlib.org/api/axes_api.html#matplotlib.axes.Axes.set_xticklabels) method!* \n",
    "  \n",
    "**c)** Finally, again in the same cell, remove the `is_spam` label from the x axis (`xlabel`) since it does not add any information to the graph\n",
    "\n",
    "You may notice `<matplotlib.text.Text at ...memory_location...>` printed by the ipython notebook. This is just because the notebook is inferring how to display the last object in the cell. To explicitly plot the Axes object, use the `matplotlib.pyplot.show()` method at the very end of the cell, i.e. `plt.show()` (we imported the `matplotlib.pyplot` module as `plt` above)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfkAAAFVCAYAAADsaJZxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGTBJREFUeJzt3X9MVff9x/HXhXvvlHtALxv1a+rinewStSVqoY7FlJgO\nF2mXuGyyb7kT+4N2YkbKSmPRSqVVWoktZU0qDQ3LfmCDYDdak5llX6bFZdJqWCxTh21KTRerHQip\n3Ou4XOF+/1h6v5J+h1flcuXj8/EX99zDue+bePI851w81xYOh8MCAADGSYj3AAAAIDaIPAAAhiLy\nAAAYisgDAGAoIg8AgKGIPAAAhrLHcuNjY2OqrKzUxx9/rISEBD333HNyOp3avHmzEhIS5PV6VVVV\nJUlqbW1VS0uLHA6HSkpKtHLlSgWDQW3atEkXLlyQZVmqqamR2+2O5cgAABgjpmfyBw8elM1mU3Nz\ns8rKyvTyyy9r586dKi8v1549ezQ2Nqb29nb19/erqalJLS0tamxsVG1trUKhkJqbm5WRkaE33nhD\na9asUX19fSzHBQDAKDGNfF5ennbs2CFJ+vTTTzVr1iydOnVK2dnZkqTc3FwdOXJE3d3dysrKkt1u\nl2VZ8ng86unpUVdXl3JzcyPrdnZ2xnJcAACMEvPP5BMSErR582ZVV1fre9/7nq68wZ7L5ZLf71cg\nEFBycnJkeVJSUmS5ZVnj1gUAANGJ6WfyX6ipqdGFCxe0du1aBYPByPJAIKCUlBRZljUu4FcuDwQC\nkWVXHgj8J5cvj8puT5z8NwEAwDQT08i//fbb+uyzz/STn/xEX/nKV5SQkKA777xTR48e1fLly3X4\n8GHl5OQoMzNTdXV1GhkZUTAYVG9vr7xer5YtW6aOjg5lZmaqo6Mjcpl/IoODl2L5lgAAuKmkpf3n\nE2BbLL+g5l//+pe2bNmi/v5+Xb58WRs2bNCCBQtUWVmpUCik9PR0VVdXy2azad++fWppaVE4HNbG\njRuVl5en4eFhVVRUqK+vT06nU7W1tfrqV7864Wv29Q3F6u0AAHDTiVvk44HIAwBuJRNFnpvhAABg\nKCIPAIChiDwAAIYi8gAAGIrIAwBgKCIPAIChiDwAAIYi8gAAGIrIAwBgKCIPAIChiDwAAIYi8gAA\nGIrIAwBgKCIPAIChiDwAAIYi8gAAGIrIAwBgKHu8B5gORkdHdeZMb7zHACaFx7NAiYmJ8R4DwBQg\n8lE4c6ZXW2pb5JqVFu9RgBsS+LxPO5/8b6Wne+M9CoApQOSj5JqVppTUufEeAwCAqPGZPAAAhiLy\nAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGI\nPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAo\ne6w2fPnyZT399NM6e/asQqGQSkpKNHfuXG3YsEEej0eSVFhYqPz8fLW2tqqlpUUOh0MlJSVauXKl\ngsGgNm3apAsXLsiyLNXU1MjtdsdqXAAAjBOzyO/fv19ut1u7du3S559/ru9///v66U9/qkceeUQP\nPfRQZL3+/n41NTWpra1Nw8PDKiws1IoVK9Tc3KyMjAyVlpbqwIEDqq+v19atW2M1LgAAxonZ5fr8\n/HyVlZVJksbGxmS323Xy5EkdOnRI69atU2VlpQKBgLq7u5WVlSW73S7LsuTxeNTT06Ouri7l5uZK\nknJzc9XZ2RmrUQEAMFLMzuRnzpwpSfL7/SorK9PPfvYzjYyMqKCgQIsXL1ZDQ4NeffVVLVq0SMnJ\nyZHfS0pKkt/vVyAQkGVZkiSXyyW/3x+rUQEAMFLMIi9J586dU2lpqdatW6f7779fQ0NDkaDn5eWp\nurpay5cvHxfwQCCglJQUWZalQCAQWXblgcBE3O4k2e2Jk/o+BgetSd0eEE+pqZbS0qLbnwBMbzGL\nfH9/v4qLi7Vt2zbl5ORIkoqLi/XMM88oMzNTnZ2duuOOO5SZmam6ujqNjIwoGAyqt7dXXq9Xy5Yt\nU0dHhzIzM9XR0aHs7OyoXndw8NKkv5eBAa4iwBwDA3719Q3FewwAk2Sig/aYRb6hoUEXL15UfX29\ndu/eLZvNpi1btuiFF16Qw+FQWlqatm/fLpfLpaKiIvl8PoXDYZWXl8vpdKqwsFAVFRXy+XxyOp2q\nra2N1agAABjJFg6Hw/EeYjLF4gzlo48+VHXjQaWkzp30bQNT6eLAOVU+eq/S073xHgXAJJnoTJ6b\n4QAAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACG\nIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCA\noYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMA\nYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGssdq\nw5cvX9bTTz+ts2fPKhQKqaSkRN/85je1efNmJSQkyOv1qqqqSpLU2tqqlpYWORwOlZSUaOXKlQoG\ng9q0aZMuXLggy7JUU1Mjt9sdq3EBADBOzCK/f/9+ud1u7dq1SxcvXtSaNWu0cOFClZeXKzs7W1VV\nVWpvb9fSpUvV1NSktrY2DQ8Pq7CwUCtWrFBzc7MyMjJUWlqqAwcOqL6+Xlu3bo3VuAAAGCdml+vz\n8/NVVlYmSRodHVViYqJOnTql7OxsSVJubq6OHDmi7u5uZWVlyW63y7IseTwe9fT0qKurS7m5uZF1\nOzs7YzUqAABGilnkZ86cqaSkJPn9fpWVlemJJ55QOByOPO9yueT3+xUIBJScnBxZ/sXvBAIBWZY1\nbl0AABC9mF2ul6Rz586ptLRU69at0/33368XX3wx8lwgEFBKSoosyxoX8CuXBwKByLIrDwQm4nYn\nyW5PnNT3MThoTer2gHhKTbWUlhbd/gRgeotZ5Pv7+1VcXKxt27YpJydHkrRo0SIdO3ZMd999tw4f\nPqycnBxlZmaqrq5OIyMjCgaD6u3tldfr1bJly9TR0aHMzEx1dHRELvNfzeDgpUl/LwMDXEWAOQYG\n/OrrG4r3GAAmyUQH7TGLfENDgy5evKj6+nrt3r1bNptNW7duVXV1tUKhkNLT07V69WrZbDYVFRXJ\n5/MpHA6rvLxcTqdThYWFqqiokM/nk9PpVG1tbaxGBQDASLbwlR+UGyAWZygfffShqhsPKiV17qRv\nG5hKFwfOqfLRe5We7o33KAAmyURn8twMBwAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQB\nADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMFbPvkweAGzU6OqozZ3rjPQYw\nKTyeBUpMTJzS1yTyAG5aZ8706pl922V9LSXeowA3xN9/UTsKtik93Tulr0vkAdzUrK+laNZ/ueM9\nBjAt8Zk8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiK\nyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoaKK/I4d\nO760rKKiYtKHAQAAk8c+0ZNbt27VP/7xD504cUIffvhhZPnly5c1NDQU8+EAAMD1mzDyGzdu1Nmz\nZ/X888+rtLQ0sjwxMVHp6ekxHw4AAFy/CSM/b948zZs3T/v375ff79fQ0JDC4bAk6dKlS5o9e/aU\nDAkAAK7dhJH/QkNDgxoaGsZF3Waz6U9/+lPMBgMAADcmqsjv27dP7e3tSk1NjfU8AABgkkT11/Vz\n587VrFmzYj0LAACYRFGdyXs8Hvl8Pn3rW9+S0+mMLL/yj/H+k/fff18vvfSSmpqa9Pe//10bNmyQ\nx+ORJBUWFio/P1+tra1qaWmRw+FQSUmJVq5cqWAwqE2bNunChQuyLEs1NTVyu93X9y4BALgFRRX5\nOXPmaM6cOde88cbGRr399ttyuVySpBMnTuiRRx7RQw89FFmnv79fTU1Namtr0/DwsAoLC7VixQo1\nNzcrIyNDpaWlOnDggOrr67V169ZrngEAgFtVVJGP5oz9/zN//nzt3r1bTz31lCTp5MmTOnPmjNrb\n2+XxeLRlyxZ1d3crKytLdrtdlmXJ4/Gop6dHXV1deuyxxyRJubm5qq+vv64ZAAC4VUUV+YULF8pm\ns41bdtttt6mjo2PC31u1apXOnj0bebxkyRL96Ec/0uLFi9XQ0KBXX31VixYtUnJycmSdpKQk+f1+\nBQIBWZYlSXK5XPL7/VG9Ibc7SXZ7YlTrRmtw0JrU7QHxlJpqKS0t+eor3gTY92CSeOx7UUW+p6cn\n8nMoFFJ7e7uOHz9+zS+Wl5cXCXpeXp6qq6u1fPnycQEPBAJKSUmRZVkKBAKRZVceCExkcPDSNc91\nNQMD0R1gANPBwIBffX3T446V7HswSaz2vYkOHK75C2ocDofy8/P17rvvXvMgxcXF+tvf/iZJ6uzs\n1B133KHMzEx1dXVpZGREQ0ND6u3tldfr1bJlyyJXCjo6OpSdnX3NrwcAwK0sqjP5t956K/JzOBzW\nhx9+KIfDcc0v9uyzz2rHjh1yOBxKS0vT9u3b5XK5VFRUJJ/Pp3A4rPLycjmdThUWFqqiokI+n09O\np1O1tbXX/HoAANzKoor8e++9N+6x2+1WXV1dVC9w++23a+/evZKkxYsXq7m5+UvrFBQUqKCgYNyy\nGTNm6JVXXonqNQAAwJdFFfmdO3cqFArp448/1ujoqLxer+z2qH4VAADESVSlPnHihB5//HHNnj1b\nY2Nj6u/v1+7du7VkyZJYzwcAAK5TVJGvrq5WXV1dJOrHjx/Xjh079Oabb8Z0OAAAcP2i+uv6S5cu\njTtrX7p0qYLBYMyGAgAANy6qyM+aNUvt7e2Rx+3t7XyXPAAAN7moLtfv2LFDGzZsGHfv+C/+Yh4A\nANycojqTP3z4sGbOnKlDhw7p17/+tVJTU3X06NFYzwYAAG5AVJFvbW1Vc3OzkpKStHDhQv3ud7/T\nnj17Yj0bAAC4AVFFPhQKjbvD3fXc7Q4AAEytqD6Tz8vL04MPPqj8/HxJ0h//+Ed95zvfielgAADg\nxkQV+U2bNukPf/iDjh07JrvdrvXr1ysvLy/WswEAgBsQ9b1pV69erdWrV8dyFgAAMImu+atmAQDA\n9EDkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQRB4A\nAEMReQAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEH\nAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQMY/8+++/r6KiIknS\nJ598Ip/Pp3Xr1um5556LrNPa2qof/vCHeuCBB/TOO+9IkoLBoB5//HH9+Mc/1oYNGzQ4OBjrUQEA\nMEpMI9/Y2KjKykqFQiFJ0s6dO1VeXq49e/ZobGxM7e3t6u/vV1NTk1paWtTY2Kja2lqFQiE1Nzcr\nIyNDb7zxhtasWaP6+vpYjgoAgHFiGvn58+dr9+7dkccnT55Udna2JCk3N1dHjhxRd3e3srKyZLfb\nZVmWPB6Penp61NXVpdzc3Mi6nZ2dsRwVAADjxDTyq1atUmJiYuRxOByO/OxyueT3+xUIBJScnBxZ\nnpSUFFluWda4dQEAQPTsU/liCQn/d0wRCASUkpIiy7LGBfzK5YFAILLsygOBibjdSbLbE6++4jUY\nHLQmdXtAPKWmWkpLi25/ijf2PZgkHvvelEZ+8eLFOnbsmO6++24dPnxYOTk5yszMVF1dnUZGRhQM\nBtXb2yuv16tly5apo6NDmZmZ6ujoiFzmv5rBwUuTPvfAAFcRYI6BAb/6+obiPUZU2PdgkljtexMd\nOExp5CsqKvTMM88oFAopPT1dq1evls1mU1FRkXw+n8LhsMrLy+V0OlVYWKiKigr5fD45nU7V1tZO\n5agAAEx7MY/87bffrr1790qSPB6PmpqavrROQUGBCgoKxi2bMWOGXnnllViPBwCAsbgZDgAAhiLy\nAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGI\nPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAo\nIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAY\nisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoezxe9Ac/+IEs\ny5IkzZs3TyUlJdq8ebMSEhLk9XpVVVUlSWptbVVLS4scDodKSkq0cuXKeIwLAMC0NOWRHxkZkST9\n5je/iSzbuHGjysvLlZ2draqqKrW3t2vp0qVqampSW1ubhoeHVVhYqBUrVsjhcEz1yAAATEtTHvme\nnh5dunRJxcXFGh0d1RNPPKFTp04pOztbkpSbm6u//OUvSkhIUFZWlux2uyzLksfj0enTp3XnnXdO\n9cgAAExLUx75GTNmqLi4WAUFBTpz5owee+wxhcPhyPMul0t+v1+BQEDJycmR5UlJSRoaGprqcQEA\nmLamPPIej0fz58+P/Dx79mydOnUq8nwgEFBKSoosy5Lf7//S8qtxu5NktydO6syDg9akbg+Ip9RU\nS2lpyVdf8SbAvgeTxGPfm/LI//a3v9UHH3ygqqoqffbZZ/L7/VqxYoWOHj2q5cuX6/Dhw8rJyVFm\nZqbq6uo0MjKiYDCo3t5eeb3eq25/cPDSpM88MOC/+krANDEw4Fdf3/S4Ksa+B5PEat+b6MBhyiO/\ndu1abdmyRT6fTwkJCaqpqdHs2bNVWVmpUCik9PR0rV69WjabTUVFRfL5fAqHwyovL5fT6ZzqcQEA\nmLamPPIOh0MvvfTSl5Y3NTV9aVlBQYEKCgqmYiwAAIzDzXAAADAUkQcAwFBEHgAAQxF5AAAMReQB\nADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5\nAAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBE\nHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAU\nkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQ9njPcBEwuGwnn32WZ0+fVpOp1PPP/+8vv71\nr8d7LAAApoWb+ky+vb1dIyMj2rt3r5588knt3Lkz3iMBADBt3NSR7+rq0j333CNJWrJkiU6cOBHn\niQAAmD5u6sv1fr9fycnJkcd2u11jY2NKSJj6Y5PA531T/prAZJuO/479/RfjPQJww+L17/imjrxl\nWQoEApHH0QQ+LS15wuevR1raXTq0765J3y6AiaWl3aX/yXkr3mMA09ZNfbn+rrvuUkdHhyTp+PHj\nysjIiPNEAABMH7ZwOByO9xD/yZV/XS9JO3fu1De+8Y04TwUAwPRwU0ceAABcv5v6cj0AALh+RB4A\nAEMReQAADEXkAQAwFJFHzB09elTl5eXjltXW1uqtt/j/z0CsvP7663r44YdVVFSkBx98UCdPnoz3\nSIiDm/pmODCHzWaL9wjALeOjjz7SwYMHtXfvXklST0+PNm/ezIH1LYjIY0r8f/9Tc3R0VJWVlTp/\n/rz6+vp07733qqysTFu2bJHdbtenn36qkZER3XfffTp06JDOnTun+vp6vokQuArLsnT+/Hm9+eab\nuueee7Rw4UK1traqqKhICxYsUG9vryTp5z//udxut7Zt28Z+aCgu12NKvPvuu1q/fr3Wr1+voqIi\n/f73v1diYqKWLl2qxsZG7du3T83NzZH1582bp1/84hdasGCBzp49q9dff13f/e53dejQoTi+C2B6\nmDNnjl577TX99a9/1QMPPKD77rtP77zzjqR/30m0qalJ+fn5eu2113T+/Hn2Q4NxJo8p8e1vf1u1\ntbWRxy+//LL8fr8++OADvffee3K5XAqFQpHnFy9eLElKSUlRenp65OdgMDi1gwPT0CeffCKXy6UX\nXnhBknTy5Ek9+uijuu2225STkyPp37E/ePCgUlJS1N3dzX5oKM7kERfhcFjhcFizZs3Siy++qIcf\nfljDw8OR5/kMH7h+p0+f1vbt2yPBnj9/vlJSUpSYmBj5A7yuri55vV61tbWxHxqMM3nEhc1mU2Ji\nov785z/r+PHjcjgc8ng8+uc///ml9QBcm1WrVqm3t1dr165VUlKSwuGwnnrqKf3qV79SW1ubfvnL\nXyopKUm7du1SX1+fnnzySfZDQ3HvegC4RRQVFWn79u180dcthMv1AHCL4Iz81sOZPAAAhuJMHgAA\nQxF5AAAMReQBADAUkQcAwFBEHgAAQ/0vUanfKVpMjIAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x3d42978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "sns.set(style='darkgrid')\n",
    "ax = sns.countplot(x=\"is_spam\",data=df2)\n",
    "ax.set_xticklabels([\"Ham\",\"Spam\"])\n",
    "ax.set_xlabel(\"\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we want to simplify the problem by transforming our dataset. We will replace all numerical values which represent word frequencies with a binary value representing whether each word was present in a document or not."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 4 ==========\n",
    "\n",
    "**a)** Crate a new dataframe called `spambase_binary` from `spambase`. *Hint*: Look into the [`copy`](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.copy.html) method in pandas. *Tip*: Be careful, in python, unless you explictly say not to, assigment is typically just reference e.g.\n",
    "```python\n",
    "i = [1, 3]\n",
    "j = i\n",
    "i[1] = 5\n",
    "print(j)\n",
    "```\n",
    "outputs:\n",
    "```\n",
    "[1, 5]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Your code goes here\n",
    "spambase_binary = spambase.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**b)** Convert all attributes in `spambase_binary` to Boolean values: 1 if the word or character is present in the email, or 0 otherwise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code goes here\n",
    "spambase_binary[spambase_binary>0.00]=1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**c)** Display the 5 last observations of the transformed dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>word_freq_make</th>\n",
       "      <th>word_freq_address</th>\n",
       "      <th>word_freq_all</th>\n",
       "      <th>word_freq_3d</th>\n",
       "      <th>word_freq_our</th>\n",
       "      <th>word_freq_over</th>\n",
       "      <th>word_freq_remove</th>\n",
       "      <th>word_freq_internet</th>\n",
       "      <th>word_freq_order</th>\n",
       "      <th>word_freq_mail</th>\n",
       "      <th>...</th>\n",
       "      <th>char_freq_;</th>\n",
       "      <th>char_freq_(</th>\n",
       "      <th>char_freq_[</th>\n",
       "      <th>char_freq_!</th>\n",
       "      <th>char_freq_$</th>\n",
       "      <th>char_freq_#</th>\n",
       "      <th>capital_run_length_average</th>\n",
       "      <th>capital_run_length_longest</th>\n",
       "      <th>capital_run_length_total</th>\n",
       "      <th>is_spam</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4596</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4597</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4598</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4599</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4600</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 58 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      word_freq_make  word_freq_address  word_freq_all  word_freq_3d  \\\n",
       "4596             1.0                0.0            1.0           0.0   \n",
       "4597             0.0                0.0            0.0           0.0   \n",
       "4598             1.0                0.0            1.0           0.0   \n",
       "4599             1.0                0.0            0.0           0.0   \n",
       "4600             0.0                0.0            1.0           0.0   \n",
       "\n",
       "      word_freq_our  word_freq_over  word_freq_remove  word_freq_internet  \\\n",
       "4596            0.0             1.0               0.0                 0.0   \n",
       "4597            0.0             0.0               0.0                 0.0   \n",
       "4598            0.0             0.0               0.0                 0.0   \n",
       "4599            1.0             0.0               0.0                 0.0   \n",
       "4600            0.0             0.0               0.0                 0.0   \n",
       "\n",
       "      word_freq_order  word_freq_mail   ...     char_freq_;  char_freq_(  \\\n",
       "4596              0.0             0.0   ...             0.0          1.0   \n",
       "4597              0.0             0.0   ...             0.0          0.0   \n",
       "4598              0.0             0.0   ...             1.0          1.0   \n",
       "4599              0.0             0.0   ...             0.0          1.0   \n",
       "4600              0.0             0.0   ...             0.0          0.0   \n",
       "\n",
       "      char_freq_[  char_freq_!  char_freq_$  char_freq_#  \\\n",
       "4596          0.0          0.0          0.0          0.0   \n",
       "4597          0.0          1.0          0.0          0.0   \n",
       "4598          0.0          0.0          0.0          0.0   \n",
       "4599          0.0          0.0          0.0          0.0   \n",
       "4600          0.0          1.0          0.0          0.0   \n",
       "\n",
       "      capital_run_length_average  capital_run_length_longest  \\\n",
       "4596                         1.0                         1.0   \n",
       "4597                         1.0                         1.0   \n",
       "4598                         1.0                         1.0   \n",
       "4599                         1.0                         1.0   \n",
       "4600                         1.0                         1.0   \n",
       "\n",
       "      capital_run_length_total  is_spam  \n",
       "4596                       1.0      0.0  \n",
       "4597                       1.0      0.0  \n",
       "4598                       1.0      0.0  \n",
       "4599                       1.0      0.0  \n",
       "4600                       1.0      0.0  \n",
       "\n",
       "[5 rows x 58 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "spambase_binary.tail(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualisation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we want to get a feeling for how the presence or absence of some specific words could affect the outcome (whether an email is classifed as *ham* or *spam*). We will be focusing on three specific words, namely `make`, `internet` and `edu`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 5 ==========\n",
    "\n",
    "**a)** Using seaborn, produce one figure with three [countplots](https://stanford.edu/~mwaskom/software/seaborn/generated/seaborn.countplot.html), one for each of the frequency variables for the words `make`, `internet` and `edu`. For each variable, the count plot should have two bars: the number of emails containing the word (i.e. the variable = 1), and the number not containing that word (i.e. the variable = 1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfkAAAFkCAYAAAAjTkJ5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X9wVPW9//HXJpsVNyeJQWNL1SE1TUQxhZgUMzftGmhw\nwtWObTWV3ZLolavEGvU2XgxpwFTAknpNmVpJSwsz1sDERAuFaanDpELoaK4wmStUaaxjoMVIlQQM\n2Y1sfp3vHx33mxQNUfYHfHw+/krOnj3nvcrO85yTnbMO27ZtAQAA48TFegAAABAZRB4AAEMReQAA\nDEXkAQAwFJEHAMBQRB4AAENFPPK9vb0qLCzUoUOH9Pe//10+n0+LFi3So48+GlqnpaVFt956qxYu\nXKjdu3dLkoLBoB544AF997vf1ZIlS3TixIlIjwoAgFEiGvnh4WHV1tZqypQpkqQ1a9aosrJSmzZt\n0ujoqFpbW9XT06PGxkY1Nzdrw4YNqq+v19DQkJqampSVlaXNmzfrlltuUUNDQyRHBQDAOBGN/I9/\n/GN5vV5deumlsm1bBw8eVF5eniTJ4/Ho5Zdf1oEDB5Sbmyun0ynLspSenq7Ozk51dHTI4/GE1m1v\nb4/kqAAAGCdikd+yZYsuvvhiFRQU6MOb6o2OjoYeT0xMlN/vVyAQUFJSUmi52+0OLbcsa9y6AABg\n8pyR2vCWLVvkcDj00ksv6Y033lBVVdW4v6sHAgElJyfLsqxxAR+7PBAIhJaNPRCYyPDwiJzO+PC+\nGAAAzkMRi/ymTZtCP5eVlenRRx/V448/rn379ukrX/mK9uzZo/z8fGVnZ2vt2rUaHBxUMBhUV1eX\nMjMzlZOTo7a2NmVnZ6utrS10mf9MTpwYiNRLAgDgnJOW9vEnwRGL/EepqqrSihUrNDQ0pIyMDBUX\nF8vhcKi0tFQ+n0+2bauyslIul0ter1dVVVXy+XxyuVyqr6+P5qgAAJz3HKZ9C92xY/2xHgEAgKiZ\n6Eyem+EAAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGI\nPAAAhiLyAAAYKqrfQne+GhkZ0eHDXbEeAwiL9PQrFR8fH+sxAEQBkZ+Ew4e7VF3frMSUtFiPApyV\nQN8xrXnodmVkZMZ6FABRQOQnKTElTclTp8V6DAAAJo2/yQMAYCgiDwCAoYg8AACGIvIAABiKyAMA\nYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvKT\nMDIyGusRgLDh3zPw2eGM5MZHR0e1fPlyHTp0SHFxcXr00Uc1NDSkJUuWKD09XZLk9Xq1YMECtbS0\nqLm5WQkJCSovL1dhYaGCwaCWLl2q3t5eWZaluro6paamRnLkj9TdfSTq+wQipbv7iLKyror1GACi\nIKKRf/HFF+VwONTU1KS9e/fqJz/5iebOnau77rpLd955Z2i9np4eNTY2auvWrTp16pS8Xq8KCgrU\n1NSkrKwsVVRUaMeOHWpoaFBNTU0kRwYAwBgRjXxRUZHmzZsnSeru7lZKSopef/11HTp0SK2trUpP\nT1d1dbUOHDig3NxcOZ1OWZal9PR0dXZ2qqOjQ3fffbckyePxqKGhIZLjAgBglIhGXpLi4uK0bNky\ntba26sknn9S7776r73znO7rmmmu0fv16PfXUU7r66quVlJQUeo7b7Zbf71cgEJBlWZKkxMRE+f3+\nM+4vNdUtpzM+rK8hJcUd1u0BsZSS4lZaWtKZVwRw3ot45CWprq5Ovb29Kikp0bPPPqtLL71U0j/P\n9FevXq05c+aMC3ggEFBycrIsy1IgEAgtG3sg8HFOnBgI+/x9feHfJhArfX0DOnasP9ZjAAiTiQ7a\nI/rp+m3btumXv/ylJOmCCy6Qw+HQ/fffrwMHDkiS2tvbNXPmTGVnZ6ujo0ODg4Pq7+9XV1eXMjMz\nlZOTo7a2NklSW1ub8vLyIjkuAABGieiZ/I033qjq6motWrRIw8PDqqmp0bRp07Ry5UolJCQoLS1N\nK1euVGJiokpLS+Xz+WTbtiorK+VyueT1elVVVSWfzyeXy6X6+vpIjgsAgFEctm3bsR4inCJxGXLX\nrlY9s/MtJU+dFvZtA9F08vhRld2Yoblzi2I9CoAwidnlegAAEDtEHgAAQxF5AAAMReQBADAUkQcA\nwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQB\nADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5\nAAAMReQBADAUkQcAwFBEHgAAQzkjufHR0VEtX75chw4dUlxcnB599FG5XC4tW7ZMcXFxyszMVG1t\nrSSppaVFzc3NSkhIUHl5uQoLCxUMBrV06VL19vbKsizV1dUpNTU1kiMDAGCMiJ7Jv/jii3I4HGpq\natKDDz6on/zkJ1qzZo0qKyu1adMmjY6OqrW1VT09PWpsbFRzc7M2bNig+vp6DQ0NqampSVlZWdq8\nebNuueUWNTQ0RHJcAACMEtHIFxUVadWqVZKkd955RykpKTp48KDy8vIkSR6PRy+//LIOHDig3Nxc\nOZ1OWZal9PR0dXZ2qqOjQx6PJ7Rue3t7JMcFAMAoEf+bfFxcnJYtW6bVq1fr5ptvlm3boccSExPl\n9/sVCASUlJQUWu52u0PLLcsaty4AAJiciP5N/kN1dXXq7e3VbbfdpmAwGFoeCASUnJwsy7LGBXzs\n8kAgEFo29kDg46SmuuV0xod1/pQUd1i3B8RSSopbaWlnfi8BOP9FNPLbtm3Tu+++q3vuuUcXXHCB\n4uLidO2112rv3r2aM2eO9uzZo/z8fGVnZ2vt2rUaHBxUMBhUV1eXMjMzlZOTo7a2NmVnZ6utrS10\nmX8iJ04MhP119PWFf5tArPT1DejYsf5YjwEgTCY6aI9o5G+88UZVV1dr0aJFGh4e1vLly3XllVdq\n+fLlGhoaUkZGhoqLi+VwOFRaWiqfzyfbtlVZWSmXyyWv16uqqir5fD65XC7V19dHclwAAIzisMf+\nkdwAkThD2bWrVc/sfEvJU6eFfdtANJ08flRlN2Zo7tyiWI8CIEwmOpPnZjgAABiKyAMAYCgiDwCA\noYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMA\nYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIA\nABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYChnpDY8PDysH/zgB+ru7tbQ0JDKy8s1bdo0LVmy\nROnp6ZIkr9erBQsWqKWlRc3NzUpISFB5ebkKCwsVDAa1dOlS9fb2yrIs1dXVKTU1NVLjAgBgnIhF\nfvv27UpNTdXjjz+uvr4+ffOb39R9992nu+66S3feeWdovZ6eHjU2Nmrr1q06deqUvF6vCgoK1NTU\npKysLFVUVGjHjh1qaGhQTU1NpMYFAMA4Ebtcv2DBAj344IOSpNHRUTmdTr3++uvatWuXFi1apOXL\nlysQCOjAgQPKzc2V0+mUZVlKT09XZ2enOjo65PF4JEkej0ft7e2RGhUAACNF7Ez+wgsvlCT5/X49\n+OCD+q//+i8NDg6qpKRE11xzjdavX6+nnnpKV199tZKSkkLPc7vd8vv9CgQCsixLkpSYmCi/3x+p\nUQEAMFLEIi9JR48eVUVFhRYtWqSbbrpJ/f39oaAXFRVp9erVmjNnzriABwIBJScny7IsBQKB0LKx\nBwITSU11y+mMD+vrSElxh3V7QCylpLiVlja59xOA81vEIt/T06PFixfrkUceUX5+viRp8eLFWrFi\nhbKzs9Xe3q6ZM2cqOztba9eu1eDgoILBoLq6upSZmamcnBy1tbUpOztbbW1tysvLm9R+T5wYCPtr\n6esL/zaBWOnrG9CxY/2xHgNAmEx00B6xyK9fv14nT55UQ0OD1q1bJ4fDoerqav3oRz9SQkKC0tLS\ntHLlSiUmJqq0tFQ+n0+2bauyslIul0ter1dVVVXy+XxyuVyqr6+P1KgAABjJYdu2HeshwikSZyi7\ndrXqmZ1vKXnqtLBvG4imk8ePquzGDM2dWxTrUQCEyURn8twMBwAAQxF5AAAMReQBADAUkQcAwFBE\nHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAU\nkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFCTivyqVatOW1ZV\nVRX2YQAAQPg4J3qwpqZGR44c0WuvvaY333wztHx4eFj9/f0RHw4AAHx6E0b+3nvvVXd3tx577DFV\nVFSElsfHxysjIyPiwwEAgE9vwshffvnluvzyy7V9+3b5/X719/fLtm1J0sDAgC666KKoDAkAAD65\nCSP/ofXr12v9+vXjou5wOPTHP/4xYoMBAICzM6nIP/fcc2ptbdXUqVMjPQ8AAAiTSX26ftq0aUpJ\nSYn0LAAAIIwmdSafnp4un8+n66+/Xi6XK7R87Ifx/tXw8LB+8IMfqLu7W0NDQyovL9eXvvQlLVu2\nTHFxccrMzFRtba0kqaWlRc3NzUpISFB5ebkKCwsVDAa1dOlS9fb2yrIs1dXVKTU19SxfLgAAnx2T\nivznPvc5fe5zn/tEG96+fbtSU1P1+OOP6+TJk7rllls0Y8YMVVZWKi8vT7W1tWptbdXs2bPV2Nio\nrVu36tSpU/J6vSooKFBTU5OysrJUUVGhHTt2qKGhQTU1NZ/qRQIA8Fk0qchPdMb+cRYsWKDi4mJJ\n0sjIiOLj43Xw4EHl5eVJkjwej1566SXFxcUpNzdXTqdTlmUpPT1dnZ2d6ujo0N133x1at6Gh4RPP\nAADAZ9mkIj9jxgw5HI5xyy699FK1tbV97HMuvPBCSZLf79eDDz6o73//+/rxj38cejwxMVF+v1+B\nQEBJSUmh5W63O7Tcsqxx605GaqpbTmf8pNadrJQUd1i3B8RSSopbaWlJZ14RwHlvUpHv7OwM/Tw0\nNKTW1la9+uqrZ3ze0aNHVVFRoUWLFummm27S//zP/4QeCwQCSk5OlmVZ4wI+dnkgEAgtG3sgMJET\nJwYmtd4n0dcX/m0CsdLXN6Bjx7hjJWCKiQ7aP/EX1CQkJGjBggX63//93wnX6+np0eLFi7V06VJ9\n61vfkiRdffXV2rdvnyRpz549ys3NVXZ2tjo6OjQ4OKj+/n51dXUpMzNTOTk5oSsFbW1tocv8AABg\nciZ1Jv/b3/429LNt23rzzTeVkJAw4XPWr1+vkydPqqGhQevWrZPD4VBNTY1Wr16toaEhZWRkqLi4\nWA6HQ6WlpfL5fLJtW5WVlXK5XPJ6vaqqqpLP55PL5VJ9ff3ZvVIAAD5jHPaH96mdQHV19bjfU1NT\n5fV6dcUVV0RssE8rEpchd+1q1TM731Ly1Glh3zYQTSePH1XZjRmaO7co1qMACJOJLtdP6kx+zZo1\nGhoa0qFDhzQyMqLMzEw5nZN6KgAAiJFJlfq1117TAw88oIsuukijo6Pq6enRunXrNGvWrEjPBwAA\nPqVJRX716tVau3ZtKOqvvvqqVq1apeeffz6iwwEAgE9vUp+uHxgYGHfWPnv2bAWDwYgNBQAAzt6k\nIp+SkqLW1tbQ762trXyXPAAA57hJXa5ftWqVlixZMu7e8c8++2zEhgIAAGdvUmfye/bs0YUXXqhd\nu3bp17/+taZOnaq9e/dGejYAAHAWJhX5lpYWNTU1ye12a8aMGdqyZYs2bdoU6dkAAMBZmFTkh4aG\nxt3h7kx3uwMAALE3qb/JFxUV6Y477tCCBQskSTt37tTXv/71iA4GAADOzqQiv3TpUr3wwgvat2+f\nnE6nysrKVFTEbTEBADiXTfretMXFxSouLo7kLAAAIIw+8VfNAgCA8wORBwDAUEQeAABDEXkAAAxF\n5AEAMBSRBwDAUEQeAABDEXkAAAxF5AEAMBSRBwDAUEQeAABDEXkAAAxF5AEAMBSRBwDAUEQeAABD\nEXkAAAxF5AEAMBSRBwDAUBGP/P79+1VaWipJ+stf/iKPx6OysjKVlZXpD3/4gySppaVFt956qxYu\nXKjdu3dLkoLBoB544AF997vf1ZIlS3TixIlIjwoAgFGckdz4hg0btG3bNiUmJkqSXnvtNd111126\n8847Q+v09PSosbFRW7du1alTp+T1elVQUKCmpiZlZWWpoqJCO3bsUENDg2pqaiI5LgAARonomfz0\n6dO1bt260O+vv/66du/erUWLFmn58uUKBAI6cOCAcnNz5XQ6ZVmW0tPT1dnZqY6ODnk8HkmSx+NR\ne3t7JEcFAMA4EY38/PnzFR8fH/p91qxZevjhh7Vp0yZdccUVeuqpp+T3+5WUlBRax+12y+/3KxAI\nyLIsSVJiYqL8fn8kRwUAwDgRvVz/r4qKikJBLyoq0urVqzVnzpxxAQ8EAkpOTpZlWQoEAqFlYw8E\nJpKa6pbTGX/mFT+BlBR3WLcHxFJKiltpaZN7PwE4v0U18osXL9aKFSuUnZ2t9vZ2zZw5U9nZ2Vq7\ndq0GBwcVDAbV1dWlzMxM5eTkqK2tTdnZ2Wpra1NeXt6k9nHixEDY5+7rC/82gVjp6xvQsWP9sR4D\nQJhMdNAe1cj/8Ic/1KpVq5SQkKC0tDStXLlSiYmJKi0tlc/nk23bqqyslMvlktfrVVVVlXw+n1wu\nl+rr66M5KgAA5z2Hbdt2rIcIp0icoeza1apndr6l5KnTwr5tIJpOHj+qshszNHduUaxHARAmE53J\nczMcAAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcA\nwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQB\nADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAM5Yz1AADwcUZGRnT4cFesxwDC\nIj39SsXHx0d1nxGP/P79+/XEE0+osbFRf//737Vs2TLFxcUpMzNTtbW1kqSWlhY1NzcrISFB5eXl\nKiwsVDAY1NKlS9Xb2yvLslRXV6fU1NRIjwvgHHL4cJdWPLdS1iXJsR4FOCv+npNaVfKIMjIyo7rf\niEZ+w4YN2rZtmxITEyVJa9asUWVlpfLy8lRbW6vW1lbNnj1bjY2N2rp1q06dOiWv16uCggI1NTUp\nKytLFRUV2rFjhxoaGlRTUxPJcQGcg6xLkpXyeQ7wgU8jon+Tnz59utatWxf6/fXXX1deXp4kyePx\n6OWXX9aBAweUm5srp9Mpy7KUnp6uzs5OdXR0yOPxhNZtb2+P5KgAABgnopGfP3/+uL8/2LYd+jkx\nMVF+v1+BQEBJSUmh5W63O7Tcsqxx6wIAgMmL6gfv4uL+/zFFIBBQcnKyLMsaF/CxywOBQGjZ2AOB\niaSmuuV0hveDDSkp7rBuD4illBS30tIm936KtRMnrFiPAITN1KlW1N97UY38Nddco3379ukrX/mK\n9uzZo/z8fGVnZ2vt2rUaHBxUMBhUV1eXMjMzlZOTo7a2NmVnZ6utrS10mf9MTpwYCPvcfX3h3yYQ\nK319Azp2rD/WY0zK8eNcwYM5jh/3R+S9N9GBQ1QjX1VVpRUrVmhoaEgZGRkqLi6Ww+FQaWmpfD6f\nbNtWZWWlXC6XvF6vqqqq5PP55HK5VF9fH81RAQA470U88pdddpmeffZZSVJ6eroaGxtPW6ekpEQl\nJSXjlk2ZMkU//elPIz0eAADG4o53AAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGI\nPAAAhiLyAAAYKqq3tQWAT2JkZET+npOxHgM4a/6ekxoZGYn6fjmTB3DO6u5+O9YjAGETi3/PnMkD\nOKdZlyQr5fOpsR4DOC9xJg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAo\nIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAY\nyhmLnX7729+WZVmSpMsvv1zl5eVatmyZ4uLilJmZqdraWklSS0uLmpublZCQoPLychUWFsZiXAAA\nzktRj/zg4KAk6Zlnngktu/fee1VZWam8vDzV1taqtbVVs2fPVmNjo7Zu3apTp07J6/WqoKBACQkJ\n0R4ZAIDzUtQj39nZqYGBAS1evFgjIyP6/ve/r4MHDyovL0+S5PF49NJLLykuLk65ublyOp2yLEvp\n6el64403dO2110Z7ZAAAzktRj/yUKVO0ePFilZSU6PDhw7r77rtl23bo8cTERPn9fgUCASUlJYWW\nu91u9ff3n3H7qaluOZ3xYZ05JcUd1u0BsZSS4lZaWtKZVzwH8N6DSWLx3ot65NPT0zV9+vTQzxdd\ndJEOHjwYejwQCCg5OVmWZcnv95+2/ExOnBgI+8x9feHfJhArfX0DOnbszAfM5wLeezBJpN57Ex04\nRP3T9b/5zW9UV1cnSXr33Xfl9/tVUFCgvXv3SpL27Nmj3NxcZWdnq6OjQ4ODg+rv71dXV5cyMzOj\nPS4AAOetqJ/J33bbbaqurpbP51NcXJzq6up00UUXafny5RoaGlJGRoaKi4vlcDhUWloqn88n27ZV\nWVkpl8sV7XEBADhvRT3yCQkJeuKJJ05b3tjYeNqykpISlZSURGMsAACMw81wAAAwFJEHAMBQRB4A\nAEMReQAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEH\nAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXk\nAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQzlgPMBHbtvXDH/5Qb7zxhlwulx577DFdccUV\nsR4LAIDzwjl9Jt/a2qrBwUE9++yzeuihh7RmzZpYjwQAwHnjnI58R0eHvva1r0mSZs2apddeey3G\nEwEAcP44py/X+/1+JSUlhX53Op0aHR1VXFz0j00Cfceivk8g3P757zgj1mN8Iv6ek7EeAThr/p6T\nMXnrOWzbtqO/28mpq6vT7NmzVVxcLEkqLCzU7t27YzsUAADniXP6cv11112ntrY2SdKrr76qrKys\nGE8EAMD545w+kx/76XpJWrNmjb74xS/GeCoAAM4P53TkAQDAp3dOX64HAACfHpEHAMBQRB4AAEMR\neQAADEXkEXW2bau2tlYLFy5UWVmZjhw5Mu7xF198UbfddpsWLlyo5557LkZTAubav3+/SktLT1vO\ne8885/Qd72Cmsd9JsH//fq1Zs0YNDQ2SpOHhYdXV1WnLli264IIL5PV69fWvf11Tp06N8dSAGTZs\n2KBt27YpMTFx3HLee2biTB5RN9F3Erz11luaPn26LMtSQkKCcnNztW/fvliNChhn+vTpWrdu3WnL\nee+Zicgj6j7uOwk+6rHExET19/dHfUbAVPPnz1d8fPxpy3nvmYnII+osy1IgEAj9PvZLhyzLkt/v\nDz0WCASUnJwc9RmBzxree2Yi8oi6ib6TICMjQ3/729908uRJDQ4Oat++fZo9e3asRgWM9a83O+W9\nZyY+eIeomz9/vl566SUtXLhQ0j+/k+B3v/udPvjgA5WUlKi6ulp33XWXbNtWSUmJLr300hhPDJjH\n4XBIEu89w3HvegAADMXlegAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEHPgNKS0vPeB/y\nJ598UvPnz9fTTz8dsTmWLFmiY8eOfezjfr9f9913X8T2/6Hq6modPXo04vsBYo2b4QCQJG3fvl0b\nN27U9OnTI7aP9evXT/j4+++/r87Ozojt/0OvvPLKaXd8A0xE5IFzzDe+8Q399Kc/1ZVXXqmHHnpI\nycnJqq2t1f79+7Vu3Trl5uZq+/btio+PV0FBgR5++GG98847+s///E+lpqZqypQp+sUvfqGamhod\nPHhQX/jCF/T+++9PuM/a2lr94x//0H333acnnnhC//Ef/6GZM2eqt7dXzz//vDZu3KgXXnhBo6Oj\n+upXv6r//u//liT96le/0vPPP6/U1FRlZGRo2rRpqqio+Nj9zJs3T5s2bdIrr7yiP/3pT+rr69OR\nI0f01a9+VY888ogee+wxvffee7r//vv1s5/9TL/97W/1zDPPyLZtzZw5U4888ohcLpfy8/N17bXX\nqre3V0uXLtXGjRs1ZcoUvfXWW7rqqqtUX18vp9P5kc9/+umn9d577+mee+7R5s2blZKSEtb/f8A5\nxQZwTqmvr7c3bdpk27Zt33zzzfY3vvEN27Zt+8knn7Q3b95s33777XYwGLRHRkbse++91968ebP9\n9ttv2zNmzLDfeecd27Zte+PGjfbDDz9s27ZtHz582P7yl79s7927d8L9zp07N/T8q666yt63b59t\n27a9Z88e+4EHHrBHR0ft0dFR+6GHHrK3b99u79+/3y4uLrYHBgbsQCBg33zzzfbPfvazCfcxb948\nu7u7296yZYs9d+5ce2BgwP7ggw/sG264wf7rX/9qv/322/a8efNs27btN9980/b5fHYwGAz9d/n5\nz39+2nyvvPKKnZOTY7/77rv26Oiofdttt9m7du2a8PljXytgMs7kgXOMx+PR008/rfz8fGVmZurQ\noUM6fvy49uzZo6ysLN10001yuVySpFtvvVXbtm3TDTfcoIsvvljTpk2TJO3duzf03QDTp0/Xdddd\nN6l922MuYX/5y1+WJL388sv685//rG9/+9uybVvBYFCXXXaZ3nvvPRUWFurCCy+UJN18880aGhqa\n9PZzcnK9nlj0AAACm0lEQVRCz73iiivU19cnt9sdevyVV17R3/72N91+++2ybVvDw8OaOXPmafNJ\nUlZWVug+6xkZGXr//ffV3d094fNtLtfjM4DIA+eY6667TlVVVWpvb9f111+vSy65RC+88IKGh4dP\n++rPD+MlSRdccMG4x0ZHR0M/f/hVvpPlcDhCBxKjo6MqKyvTnXfeKemfH46Li4vT1q1bx4XS6XSe\nMfIffimKpND2x76WsUZGRrRgwQLV1NRIkj744AONjIycNt+/buvDfUz0fOCzgk/XA+eYuLg4zZo1\nS42NjZozZ46uv/56/eIXv5DH49H111+v3//+9woGgxoeHtaWLVuUn58vaXwk/+3f/k2/+93vZNu2\nuru79X//93+faIax28rPz9f27ds1MDCg4eFh3Xvvvdq5c6fy8/O1e/du+f1+DQ4OaufOnZ9oux/F\n6XSGQjxnzhy1trbq+PHjsm1btbW1oU/+T+YsfKLnj90PYDLO5IFz0A033KB9+/bpi1/8oi655BId\nP35c8+bN06xZs/SXv/xFt956q0ZGRvS1r31NixYt0tGjR8edJft8Pr355pv693//d33hC19QVlbW\nGfc59vljf547d67eeOMNfec739Ho6Kg8Ho+++c1vSpLKy8vl9XrldrtlWdYn2sdHLb/44ov1+c9/\nXnfccYd+/etf63vf+57uuOMO2batq6++Wvfcc8+E2xlrxowZuu+++z7y+YWFhbr77ru1ceNGXXbZ\nZWfcFnC+4qtmAYTFU089JUkTfroeQHRxJg98Rhw5ckT333//uLNg27blcDi0evXqcR9KOxtlZWXq\n7+8/bR8LFy7U7bffHpZ9AJgczuQBADAUH7wDAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQ/w/175Gt\naNqmtgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xab3a518>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "ax_make = sns.countplot(x=\"word_freq_make\",data=spambase_binary)\n",
    "ax_internet = sns.countplot(x=\"word_freq_internet\",data=spambase_binary)\n",
    "# haven't got it right"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**b)** Repeat the above but split the bars showing the proportion of emails that are spam/ham. *Hint*: This only requires you to use the `hue` input argument to use different colours for the `is_spam` variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfkAAAFkCAYAAAAjTkJ5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X1YVHX+//HXwEDKDCgk5W2gLGS65A1uupqkpt9Lbft1\nY+4Kid3YXuLmTVIumhSltrom2Y2Ylrq12ppaUq5Z21IJlWx6sSmrhutqpLm2C0LKDMnt/P7o2llZ\nDbHmBj4+H3/BmcM577Fres45DOdYXC6XSwAAwDgB/h4AAAB4B5EHAMBQRB4AAEMReQAADEXkAQAw\nFJEHAMBQVm9tuK6uTg8//LCOHz+u2tpapaamqlOnTpoyZYqio6MlSUlJSRozZow2bdqkjRs3Kigo\nSKmpqRo2bJiqq6s1e/ZsnTx5Una7XYsXL1Z4eLi3xgUAwDgWb/2d/JYtW3Tw4EHNnTtXp06d0q23\n3qr7779fDodDd999t3u9srIy3XPPPcrJydGZM2eUlJSkLVu26JVXXpHD4dC0adO0fft2ffrpp5o3\nb543RgUAwEheO10/ZswYzZw5U5LU0NAgq9Wq/fv364MPPtDEiROVkZEhp9OpoqIiJSQkyGq1ym63\nKzo6WsXFxSosLFRiYqIkKTExUQUFBd4aFQAAI3ntdH3btm0lSQ6HQzNnztQDDzygmpoajR8/Xr16\n9dKqVau0fPlyXXPNNQoNDXX/XEhIiBwOh5xOp+x2uyTJZrPJ4XB4a1QAAIzk1Q/enThxQnfddZdu\nu+023XTTTRo5cqR69eolSRo5cqSKi4sVGhraKOBOp1NhYWGy2+1yOp3uZWe/EWhKXV29558IAACt\nkNeO5MvKyjR58mQ9+uijGjRokCRp8uTJeuSRRxQfH6+CggL17t1b8fHxWrZsmWpqalRdXa0jR44o\nNjZW/fr1U15enuLj45WXl6cBAwY0a78VFVXeekoAALQ4kZHffRDstQ/ePfHEE3r77bfVo0cPuVwu\nWSwWzZo1S0uWLFFQUJAiIyM1f/582Ww2bd68WRs3bpTL5dLUqVM1cuRInTlzRunp6SotLVVwcLCy\nsrJ0+eWXX3C/paWV3ng6AAC0SH6JvL8QeQDApaSpyHMxHAAADEXkAQAwFJEHAMBQRB4AAEMReQAA\nDEXkAQAwFJEHAFzyPvmkQLm5f/L3GB7H38kDANCKNfV38l67rC0AAK3F229vU1lZmXbu/FABAQHq\n0qWrHn4487zr7tnzV61cuVwWi0V9+/bXlCn3a/r0KerW7Sp9/vlhdenSTRkZj+urr04oK2uxamtr\nVVVVpfnzF+vEieN65ZWXJUmnTn2tn/3sVn344Q5VVFRo2bJshYWFefR5cboeAABJ9fV1Gjx4qLKz\nX9R11w3SmTNnzrveRx/la9y4n+v559eoS5eu7uWJicP1/PNrFRQUpI8//lDHjn2hX/5yqp5+eoUS\nE4epoOAjSVJVVZWWLn1W//d/Y/XZZ/u1dOmz6tu3n4qKPvX4c+JIvhnq6+tVUnLE32MYLzq6hwID\nA/09BoBLVGCgVTU11Zo581e66qooDR067LzrpaTcrZdfXqNt27aqV6/eamhokCT17dtfknT11dfo\n+PEv1b//AK1f/5KCg4NVWvpvDR48VJLUo0eMJMlut6tbt6skSTabXdXVNR5/TkS+GUpKjmhu1kbZ\n2kX6exRjOU+VatGDv1BMTKy/RwFwibJYpISEn2jy5Cl6+uml+utfd+unP73+nPX+/Od3dMst4xQV\nFa05c9JUUvK5JOnvfy/Wtdf21Wef7deIEaO0Zs0q3XPPfYqL66nFixfovx+Bs/jsORH5ZrK1i1RY\nRCd/jwEA8JK6ujqtWpWt4ODLZLPZ1KdP//OuFxfXUwsXZiokxKYrr7xS3bv3kCRt3vyqnn/+OcXF\nXa2BA3+qr7+u0GOPzVN4eITCwtrp5MkyxcbG+fIp8en65jh8+JAWrn6fyHvR6fITyrhvBEfyAFql\n6dOnaP78RQoPj/D5vvl0PQAAF6mg4CP94Q/rZLF8e3rd5XLJYrFoyZKn1aZNm0br/medlobIAwBw\nHj/96fXn/Z38+Tz77EovT/P98Cd0AAAYisgDAGAoTtcDAC4p3rj2SUu9zgeRBwBcUjx97ZOWfJ0P\nIg8AuOT4+tonLpdLWVmL9Y9/HFJwcLDS0zMaXRL3o4/y9fLLq2W1WjV27P/TzTff6pH98jt5AAC8\nLD9/h2pqarRy5VpNmTJNy5cvcz9WV1en5cuX6emnV+i5517Q1q1bVFFR4ZH9EnkAALysqGiPBg4c\nLEnq3fvHKi7+zP3YF1+UqGvXbrLZ7LJarbr22r7au/evHtkvkQcAwMuqqpyy2+3u7wMDA903tnE6\nHbLZ/vtYSIhNDofDI/sl8gAAeFlIiE1VVU739w0NDQoI+DbBNpu90WNVVU6Fhn73pWovBh+8AwBc\ncpynSn26rWuv7aOPP/5Qw4eP1L59f1NMzI/cj0VFRevLL4+psrJSbdq00Z49nyopaZJHZiPyAIBL\nSnR0Dy168Bce32ZTEhOHa/fuTzR16r2SpLlzM/XnP7+jM2fO6Oabb9X06WlKS7tfLpd08823qEOH\nDh6Zi8gDAC4pgYGBPv+bdovFoocemtto2VVXRbm/Hjz4eg0e3Lzr5F8MficPAIChiDwAAIYi8gAA\nGIrIAwBgKD54BwC4pHAXOgAADFVSckSPbJ4ve4cwj2zPUXZaC8Y/yl3oAABoCewdwtSuY7jP97t/\n/z6tXPmcnntuVaPl3roLHZEHAMAH/vCH3+tPf9qutm1DGi3/z13o1qxZp8sua6OpU+/V9dffoPDw\nH/4mhA/eAQDgA126dNNvfrP0nOXchQ4AgFbuhhuGn/fDedyFDgAAQ3EXOgAAPMhRdtpv23K5XI2+\n5y50AAB4SHR0Dy0Y/6jHt9lcFotFknxyFzqL63/fUrRypaWVHt/m4cOHtHD1+wqL6OTxbeNbp8tP\nKOO+ES3y70wBoCWLjPzuU/v8Th4AAEMReQAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEH\nAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQXrsLXV1dnR5++GEdP35ctbW1Sk1N1Y9+9CPNmTNHAQEB\nio2NVWZmpiRp06ZN2rhxo4KCgpSamqphw4apurpas2fP1smTJ2W327V48WKFh4d7a1wAAIzjtchv\n3bpV4eHhWrJkiU6fPq1bbrlFPXv2VFpamgYMGKDMzEzl5uaqb9++WrdunXJycnTmzBklJSVpyJAh\n2rBhg+Li4jRt2jRt375dK1as0Lx587w1LgAAxvHa6foxY8Zo5syZkqT6+noFBgbqwIEDGjBggCQp\nMTFRO3fuVFFRkRISEmS1WmW32xUdHa3i4mIVFhYqMTHRvW5BQYG3RgUAwEheO5Jv27atJMnhcGjm\nzJmaNWuWfvvb37oft9lscjgccjqdCg39771wQ0JC3MvtdnujdZsjPDxEVmugB5+JVFFh9+j2cH4R\nEfYm74sMALg4Xou8JJ04cULTpk3TxIkTddNNN+nJJ590P+Z0OhUWFia73d4o4Gcvdzqd7mVnvxFo\nSkVFlWefhKTy8ua9wcAPU17uUGlppb/HAIBWpamDI6+dri8rK9PkyZM1e/Zs3XbbbZKka665Rrt3\n75Yk5efnKyEhQfHx8SosLFRNTY0qKyt15MgRxcbGql+/fsrLy5Mk5eXluU/zAwCA5vHakfyqVat0\n+vRprVixQtnZ2bJYLJo3b54WLlyo2tpaxcTEaPTo0bJYLEpJSVFycrJcLpfS0tIUHByspKQkpaen\nKzk5WcHBwcrKyvLWqAAAGMnicrlc/h7Ck7xxuvfw4UNauPp9hUV08vi28a3T5SeUcd8IxcTE+nsU\nAGhV/HK6HgAA+BeRBwDAUEQeAABDEXkAAAxF5AEAMBSRBwDAUEQeAABDEXkAAAxF5AEAMBSRBwDA\nUEQeAABDEXkAAAxF5AEAMBSRBwDAUEQeAABDEXkAAAxF5AEAMBSRBwDAUEQeAABDEXkAAAxF5AEA\nMBSRBwDAUEQeAABDEXkAAAxF5AEAMBSRBwDAUEQeAABDEXkAAAxF5AEAMBSRBwDAUEQeAABDEXkA\nAAxF5AEAMBSRBwDAUEQeAABDEXkAAAxF5AEAMBSRBwDAUEQeAABDEXkAAAxF5AEAMBSRBwDAUEQe\nAABDEXkAAAxF5AEAMBSRBwDAUEQeAABDEXkAAAxF5AEAMBSRBwDAUEQeAABDEXkAAAxF5AEAMJTX\nI793716lpKRIkj777DMlJiZq0qRJmjRpkt5++21J0qZNmzRu3DhNmDBBO3bskCRVV1drxowZuvPO\nOzVlyhRVVFR4e1QAAIxi9ebGV69erTfffFM2m02StG/fPt177726++673euUlZVp3bp1ysnJ0Zkz\nZ5SUlKQhQ4Zow4YNiouL07Rp07R9+3atWLFC8+bN8+a4AAAYxatH8lFRUcrOznZ/v3//fu3YsUMT\nJ05URkaGnE6nioqKlJCQIKvVKrvdrujoaBUXF6uwsFCJiYmSpMTERBUUFHhzVAAAjOPVyI8aNUqB\ngYHu7/v06aNf//rXWr9+vbp166bly5fL4XAoNDTUvU5ISIgcDoecTqfsdrskyWazyeFweHNUAACM\n49XT9f9r5MiR7qCPHDlSCxcu1HXXXdco4E6nU2FhYbLb7XI6ne5lZ78RaEp4eIis1sALr3gRKirs\nHt0ezi8iwq7IyOb9dwYAXJhPIz958mQ98sgjio+PV0FBgXr37q34+HgtW7ZMNTU1qq6u1pEjRxQb\nG6t+/fopLy9P8fHxysvL04ABA5q1j4qKKo/PXV7OWQRfKC93qLS00t9jAECr0tTBkU8j/9hjj2nB\nggUKCgpSZGSk5s+fL5vNppSUFCUnJ8vlciktLU3BwcFKSkpSenq6kpOTFRwcrKysLF+OCgBAq2dx\nuVwufw/hSd44Ejx8+JAWrn5fYRGdPL5tfOt0+Qll3DdCMTGx/h4FAFqVpo7kuRgOAACGIvIAABiK\nyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACGIvIAABiKyAMAYCgiDwCAoYg8AACG\nIvIAABiKyAMAYCgiDwCAoZoV+QULFpyzLD093ePDAAAAz7E29eC8efN07Ngx7du3T4cOHXIvr6ur\nU2VlpdeHAwAA31+TkZ86daqOHz+uJ554QtOmTXMvDwwMVExMjNeHAwAA31+Tke/atau6du2qrVu3\nyuFwqLKyUi6XS5JUVVWl9u3b+2RIAABw8ZqM/H+sWrVKq1atahR1i8Wi9957z2uDAQCAH6ZZkd+8\nebNyc3MVERHh7XkAAICHNOvT9Z06dVK7du28PQsAAPCgZh3JR0dHKzk5WQMHDlRwcLB7+dkfxgMA\nAC1LsyJ/5ZVX6sorr/T2LAAAwIOaFXmO2AEAaH2aFfmePXvKYrE0WnbFFVcoLy/PK0MBAIAfrlmR\nLy4udn9dW1ur3Nxc7dmzx2tDAQCAH+6ib1ATFBSkMWPG6C9/+Ys35gEAAB7SrCP5N954w/21y+XS\noUOHFBQU5LWhAADAD9esyH/yySeNvg8PD9eyZcu8MhAAAPCMZkV+0aJFqq2t1eeff676+nrFxsbK\nam3WjwIAAD9pVqn37dunGTNmqH379mpoaFBZWZmys7PVp08fb88HAAC+p2ZFfuHChVq2bJk76nv2\n7NGCBQv02muveXU4AADw/TXr0/VVVVWNjtr79u2r6upqrw0FAAB+uGYdybdr1065ubkaOXKkJCk3\nN5d7ycOjXA0NOnr0C3+PcUmIju6hwMBAf48BwAcsLpfLdaGVSkpKNGXKFH399dfuZa+++qq6d+/u\n1eG+j9LSSo9v8/DhQ1q4+n2FRXTy+LbxrX9+XqSgqw7K3iHM36MYzVF2WgvGP6qYmFh/jwLAQyIj\nQ7/zsWYdyefn56tt27bKycnR0aNHNWvWLO3atatFRh6tl71DmNp1DPf3GABgjGb9Tn7Tpk3asGGD\nQkJC1LNnT23ZskXr16/39mwAAOAHaFbka2trG13hjqvdAQDQ8jXrdP3IkSN11113acyYMZKkd999\nVzfeeKNXBwMAAD9MsyI/e/ZsvfPOO9q9e7esVqsmTZrk/qQ9AABomZp9bdrRo0dr9OjR3pwFAAB4\n0EXfahYAALQORB4AAEMReQAADEXkAQAwFJEHAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQRB4AAEN5\nPfJ79+5VSkqKJOno0aNKTk7WxIkT9fjjj7vX2bRpk8aNG6cJEyZox44dkqTq6mrNmDFDd955p6ZM\nmaKKigpvjwoAgFG8GvnVq1crIyNDtbW1kqRFixYpLS1N69evV0NDg3Jzc1VWVqZ169Zp48aNWr16\ntbKyslRbW6sNGzYoLi5Or7zyim655RatWLHCm6MCAGAcr0Y+KipK2dnZ7u/379+vAQMGSJISExO1\nc+dOFRUVKSEhQVarVXa7XdHR0SouLlZhYaESExPd6xYUFHhzVAAAjOPVyI8aNUqBgYHu710ul/tr\nm80mh8Mhp9Op0NBQ9/KQkBD3crvd3mhdAADQfM2+1awnBAT89z2F0+lUWFiY7HZ7o4CfvdzpdLqX\nnf1GoCnh4SGyWgMvvOJFqKiwe3R7gD9FRNgVGdm81xOA1s2nke/Vq5d2796tn/zkJ8rPz9egQYMU\nHx+vZcuWqaamRtXV1Tpy5IhiY2PVr18/5eXlKT4+Xnl5ee7T/BdSUVHl8bnLyzmLAHOUlztUWlrp\n7zEAeEhTb9p9Gvn09HQ98sgjqq2tVUxMjEaPHi2LxaKUlBQlJyfL5XIpLS1NwcHBSkpKUnp6upKT\nkxUcHKysrCxfjgoAQKtncZ39i3IDeOMI5fDhQ1q4+n2FRXTy+LbxrX9+XqTw/ifUrmO4v0cx2qmv\nKpQ+/AHFxMT6exQAHtLUkTwXwwEAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQB\nADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5\nAAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBE\nHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMZfX3AAAA\n/6qvr1dJyRF/j2G86OgeCgwM9Ok+iTwAXOJKSo7okc3zZe8Q5u9RjOUoO60F4x9VTEysT/dL5AEA\nsncIU7uO4f4eAx7G7+QBADAUkQcAwFBEHgAAQxF5AAAMReQBADAUkQcAwFBEHgAAQxF5AAAMReQB\nADAUkQcAwFBEHgAAQxF5AAAM5Zcb1Nx+++2y2+2SpK5duyo1NVVz5sxRQECAYmNjlZmZKUnatGmT\nNm7cqKCgIKWmpmrYsGH+GBcAgFbJ55GvqamRJP3+9793L5s6darS0tI0YMAAZWZmKjc3V3379tW6\ndeuUk5OjM2fOKCkpSUOGDFFQUJCvRwYAoFXyeeSLi4tVVVWlyZMnq76+XrNmzdKBAwc0YMAASVJi\nYqI+/vhjBQQEKCEhQVarVXa7XdHR0Tp48KB+/OMf+3pkAABaJZ9Hvk2bNpo8ebLGjx+vkpIS/fKX\nv5TL5XI/brPZ5HA45HQ6FRoa6l4eEhKiysrKC24/PDxEVmugR2euqLB7dHuAP0VE2BUZGXrhFXHJ\n4P9xvuGP157PIx8dHa2oqCj31+3bt9eBAwfcjzudToWFhclut8vhcJyz/EIqKqo8PnN5uePCKwGt\nRHm5Q6WlF37DjEsH/4/zDW+99pp64+DzT9e//vrrWrx4sSTpX//6lxwOh4YMGaJdu3ZJkvLz85WQ\nkKD4+HgVFhaqpqZGlZWVOnLkiGJjY309LgAArZbPj+TvuOMOzZ07V8nJyQoICNDixYvVvn17ZWRk\nqLa2VjExMRo9erQsFotSUlKUnJwsl8ultLQ0BQcH+3pcAABaLZ9HPigoSEuXLj1n+bp1685ZNn78\neI0fP94XYwEAYBwuhgMAgKGIPAAAhiLyAAAYisgDAGAoIg8AgKH8coMaAGiO+vp6lZQc8fcYxjt6\n9At/jwAvIfIAWqySkiOam7VRtnaR/h7FaKVfHlTnG/w9BbyByANo0WztIhUW0cnfYxjNcapU0gl/\njwEv4HfyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAo\nIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAY\nisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAA\nhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8AgKGIPAAAhiLyAAAYisgDAGAoIg8A\ngKGIPAAAhiLyAAAYyurvAZricrn02GOP6eDBgwoODtYTTzyhbt26+XssAABahRZ9JJ+bm6uamhq9\n+uqrevDBB7Vo0SJ/jwQAQKvRoiNfWFiooUOHSpL69Omjffv2+XkiAABajxZ9ut7hcCg0NNT9vdVq\nVUNDgwICfP/exHmq1Of7vJR8U1muoLLT/h7DeI5W+G/Ma8/7eP15n79eey068na7XU6n0/19cwIf\nGRna5OPfR2Rkf32wub/Htwugabz2gB+mRZ+u79+/v/Ly8iRJe/bsUVxcnJ8nAgCg9bC4XC6Xv4f4\nLmd/ul6SFi1apO7du/t5KgAAWocWHXkAAPD9tejT9QAA4Psj8gAAGIrIAwBgKCIPAIChiDx8zuVy\nKTMzUxMmTNCkSZN07NixRo+///77uuOOOzRhwgRt3rzZT1MC5tq7d69SUlLOWc5rzzwt+mI4MNPZ\n9yTYu3evFi1apBUrVkiS6urqtHjxYm3ZskWXXXaZkpKSdOONNyoiIsLPUwNmWL16td58803ZbLZG\ny3ntmYkjefhcU/ckOHz4sKKiomS32xUUFKSEhATt3r3bX6MCxomKilJ2dvY5y3ntmYnIw+e+654E\n53vMZrOpsrLS5zMCpho1apQCAwPPWc5rz0xEHj7X1D0J7Ha7HA6H+zGn06mwsDCfzwhcanjtmYnI\nw+eauidBTEyMvvjiC50+fVo1NTXavXu3+vbt669RAWP978VOee2ZiQ/ewedGjRqljz/+WBMmTJD0\n7T0Jtm3bpm+++Ubjx4/X3Llzde+998rlcmn8+PG64oor/DwxYB6LxSJJvPYMx7XrAQAwFKfrAQAw\nFJEHAMBQRB4AAEMReQAADEXkAQAwFJEHAMBQRB64RKWkpFzw2uTPPvusRo0apZdeesk3Q12k5cuX\na/ny5f4eA2ixuBgOgO+0detWrVmzRlFRUf4eBcD3QOSBVuDmm2/WM888ox49eujBBx9UWFiYMjMz\ntXfvXmVnZyshIUFbt25VYGCghgwZol//+tf65z//qfvuu0/h4eFq06aNVq5cqXnz5unAgQPq3Lmz\nvv766yb3mZmZqa+++kr333+/li5dqnvuuUe9e/fWyZMn9dprr2nNmjV655131NDQoOuvv14PPfSQ\nJOnFF1/Ua6+9pvDwcMXExKhTp06aNm3ad+5nxIgRGjt2rHbs2CGr1apZs2Zp7dq1Onr0qNLT0zV6\n9GgdOnRICxYs0DfffKOTJ0/q3nvv1cSJE93baGho0AMPPKCrrrpKDz30kPLz8/Xcc8+pvr5eXbt2\n1YIFC9SuXTvP/McAWhFO1wOtwPDhw1VQUCBJ+vvf/67CwkJJUn5+voYNG6YPPvhAOTk5euONN/TF\nF19ow4YNkqSSkhJlZWVp7dq1Wr9+vQICAvTWW28pIyNDR48ebXKfjz/+uK644gq9+OKL6tmzpyoq\nKpSamqqcnBzt3LlT+/fv1+uvv66cnBx99dVX+uMf/6iioiJt2bJFb7zxhtauXauioqJmPb+OHTtq\n27Ztuuaaa/Tiiy/qd7/7nZYsWaIXXnhBkrR582b96le/0ubNm/Xyyy/rqaeecv+sy+VSRkaGOnfu\nrIceekjl5eV66qmntHbtWm3ZskVDhgzRk08+edH/5oAJOJIHWoHExES99NJLGjRokGJjY/X555+r\nvLxc+fn5iouL00033aTg4GBJ0rhx4/Tmm2/qhhtu0OWXX65OnTpJknbt2uW+X0BUVJT69+/frH2f\nfeXra6+9VpK0c+dO/e1vf9Ptt98ul8ul6upqdenSRf/+9781bNgwtW3bVpL0s5/9TLW1tRfcx9Ch\nQyVJXbp0UceOHRUQEKDOnTvr1KlTkqQ5c+boww8/1AsvvKCDBw/qm2++cf/sq6++KofDoffee0+S\nVFRUpBMnTmjSpElyuVxqaGhQ+/btm/VcAdMQeaAV6N+/v9LT01VQUKCBAweqQ4cOeuedd1RXV3fO\n7UBdLpfq6uokSZdddlmjxxoaGtxf/+f2vs1lsVjcbyQaGho0adIk3X333ZK+vRd5QECAcnJyGr0p\nsFqtzYp8UFCQ++vz3et85syZat++vYYPH66xY8dq+/bt7sf69++vXr16acGCBXrmmWdUX1+vhIQE\nrVixQpJUU1PT6NbGwKWE0/VAKxAQEKA+ffpo3bp1uu666zRw4ECtXLlSiYmJGjhwoN566y1VV1er\nrq5OW7Zs0aBBgyQ1PgofPHiwtm3bJpfLpePHj+vTTz+9qBnO3tagQYO0detWVVVVqa6uTlOnTtW7\n776rQYMGaceOHXI4HKqpqdG7777rkee/c+dOzZgxQyNGjNCuXbsazXP11Vfrvvvu0z/+8Q/l5eWp\nT58+2rNnj0pKSiRJ2dnZWrJkiUfmAFobjuSBVuKGG27Q7t271b17d3Xo0EHl5eUaMWKE+vTpo88+\n+0zjxo1TfX29hg4dqokTJ+rEiRPu24lKUnJysg4dOqSxY8eqc+fOiouLu+A+z/75s78ePny4Dh48\nqJ///OeGyOtcAAAAxUlEQVRqaGhQYmKibr31VklSamqqkpKSFBISIrvdflH7+C7Tp09XUlKSwsLC\n1L17d3Xt2lVffvml+/GgoCBlZmZqzpw5euutt/Sb3/xGDzzwgBoaGtSxY0d+J49LFreaBeA1//kb\n9qY+XQ/AeziSBy5hx44d0/Tp0xsdTbtcLlksFi1cuFC9e/f2yH4mTZqkysrKc/YxYcIE/eIXv/DI\nPgCciyN5AAAMxQfvAAAwFJEHAMBQRB4AAEMReQAADEXkAQAw1P8H/tWxAlnyL80AAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xab5c8d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "ax=sns.countplot(x=\"word_freq_make\",data=spambase_binary,hue=\"is_spam\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multinomial Naive Bayes classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given the transformed dataset, we now wish to train a Naïve Bayes classifier to distinguish spam from regular email by fitting a distribution of the number of occurrences of each word for all the spam and non-spam e-mails. Read about the [Naive Bayes classifier](https://en.wikipedia.org/wiki/Naive_Bayes_classifier) and the underlying assumption if you are not already familiar with it. In this lab we focus on the [Multinomial Naive Bayes classifier](https://en.wikipedia.org/wiki/Naive_Bayes_classifier#Multinomial_naive_Bayes). \n",
    "\n",
    "We will make use of the `MultinomialNB` class in `sklearn`. **Check out the user guide [description](http://scikit-learn.org/stable/modules/naive_bayes.html#multinomial-naive-bayes) and [documentation](http://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html#sklearn.naive_bayes.MultinomialNB) to familiarise yourself with this class.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All classifiers in `sklearn` implement a `fit()` and `predict()` [method](https://en.wikipedia.org/wiki/Method_%28computer_programming%29). The first learns the parameters of the model and the latter classifies inputs. For a Naive Bayes classifier, the [`fit()`](http://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.GaussianNB.html#sklearn.naive_bayes.GaussianNB.fit) method takes at least two input arguments `X` and `y`, where `X` are the input features and `y` are the labels associated with each example in the training dataset (i.e. targets). \n",
    "\n",
    "As a first step we extract the input features and targets from the DataFrame. To do so, we will use the [`values`](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.values.html) property. For the input features we want to select all columns except `is_spam` and for this we may use the [`drop`](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.drop.html) method which discards the specified columns along the given axis. In fact, we can combine these two operations in one step."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 6 =========="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**a)** Create a Pandas DataFrame object `X` containing only the features (i.e. exclude the label `is_spam`). We need to do this as it is the input Scikit-learn objects expect for fitting. *Hint*: make use of the `drop` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code goes here\n",
    "x = spambase_binary.drop('is_spam',axis=1) #axis 1 denotes that we refer to column not row\n",
    "x = x.drop(['capital_run_length_average','capital_run_length_longest','capital_run_length_total'],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**b)** Create a Pandas Series object `y` that contains only the label from `spambase_binary`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1    1.0\n",
      "2    1.0\n",
      "3    1.0\n",
      "4    1.0\n",
      "Name: is_spam, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# Your code goes here\n",
    "y = pd.Series(spambase_binary['is_spam'])\n",
    "print (y[1:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**c)** Display the dimensionality (i.e. `shape`) of each of the two arrays. *Hint:* The shape of `X` and `y` should be `(4601, 54)` and `(4601,)` respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(4601, 54)\n",
      "(4601L,)\n"
     ]
    }
   ],
   "source": [
    "# Your code goes here\n",
    "print (x.shape)\n",
    "print (y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 7 ==========\n",
    "\n",
    "Now we want to train a Multinomial Naive Bayes classifier. Initialise a `MultinomialNB` object and [`fit`](http://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html#sklearn.naive_bayes.MultinomialNB.fit) the classifier using the `X` and `y` arrays extracted in the cell above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "clf = MultinomialNB()\n",
    "clf.fit(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can evaluate the classifier by looking at the classification accuracy, and the [confusion matrix](https://en.wikipedia.org/wiki/Confusion_matrix). \n",
    "\n",
    "Scikit-learn model objects have built in scoring methods. The default [`score` method for `MultinomialNB`](http://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html#sklearn.naive_bayes.MultinomialNB.score) estimates the classification accuracy score. Alternatively, you can compute the prediction for the training data and make use of the [`accuracy_score`](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html) function (that is in fact what the classifier's `score()` method does under the hood).\n",
    "\n",
    "Scikit-learn also has a [`confusion_matrix`](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html#sklearn.metrics.confusion_matrix) implementation which returns a numpy array (square matrix) of dimensionality `K`, where `K` is the number of classes (2 in our case)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 8 ========== "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**a)** Display the log-prior probabilities for each class. *Hint:* use tab-completion to figure out which attribute of the `MultinomialNB` structure you are interested in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.50094918, -0.93129074])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Your code goes here \n",
    "clf.class_log_prior_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**b)** Predict the output of the classifier by using the training data as input. *Hint*: make use of the `predict` method of the `MultinomialNB` classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.  1.  1. ...,  0.  0.  0.]\n"
     ]
    }
   ],
   "source": [
    "# Your code goes here \n",
    "print(clf.predict(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**c)** Compute the classification accuracy on the training data by either using the `accuracy_score` metric or the `score` method of the `MultinomialNB`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.892414692458\n"
     ]
    }
   ],
   "source": [
    "# Your code goes here \n",
    "print (clf.score(x,y,sample_weight=None))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**d)** Compute the resulting confusion_matrix by using the builtin scikit-learn class and display the result. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[2412  376]\n",
      " [ 119 1694]]\n"
     ]
    }
   ],
   "source": [
    "# Your code goes here \n",
    "cm = confusion_matrix(y, clf.predict(x))\n",
    "print (cm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**e)** Normalise the produced confusion matrix by the true class and display the result. In other words, the matrix should show you what proportion of `Ham` emails were predicted as `Ham`/`Spam` and vice versa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.8651363   0.20739106]\n",
      " [ 0.04268293  0.93436293]]\n"
     ]
    }
   ],
   "source": [
    "# Your code goes here \n",
    "cm_norm = cm /cm.sum(axis=1) #axis 1 means the actual, 0 is the predicted\n",
    "print (cm_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**f)** By making use of the `plot_confusion_matrix` provided below, visualise the normalised confusion matrix. Plot the appropriate labels on both axes by making use of the `classes` input argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_confusion_matrix(cm, classes=None, title='Confusion matrix'):\n",
    "    \"\"\"Plots a confusion matrix.\"\"\"\n",
    "    if classes is not None:\n",
    "        sns.heatmap(cm, xticklabels=classes, yticklabels=classes, vmin=0., vmax=1., annot=True)\n",
    "    else:\n",
    "        sns.heatmap(cm, vmin=0., vmax=1.)\n",
    "    plt.title(title)\n",
    "    plt.ylabel('True label')\n",
    "    plt.xlabel('Predicted label')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAc0AAAFtCAYAAABlWTHzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucjeX+//H3mhlrGDMTwxBhMpgGhejXlzI70ewdIeOQ\nIUPanegks1PEMEqDsFOziZRyajqOaqKDPeSwC181MkJyPsSMU8zBHKz794evtU0a607utaw1r+d+\nrMdj3Yd1XZ/bVp8+133d120zDMMQAABwyc/TAQAA4C1ImgAAmETSBADAJJImAAAmkTQBADCJpAkA\ngEkkTXgdh8OhuXPnqlevXoqLi1PXrl01ZcoUFRcX/6k2hwwZojvvvFMLFy78w7/Pzs7Wk08+ecn9\nX255eXkaNGhQucfj4uKUl5fnxogA32DjOU14mzFjxujUqVN64YUXFBwcrNOnTysxMVHBwcGaNGnS\nJbV58OBBde7cWVlZWbLZbJc5Yvfbv3+/unfvru+++87ToQA+hUoTXmX//v3KyMjQiy++qODgYElS\n5cqVNX78eMXGxko6W2U9/fTT6tatm7p3766XXnpJDodDktSiRQulpqaqX79+uuOOOzRv3jzl5+fr\nwQcfVGlpqXr27Kl9+/YpOjpaJ06ccPZ7brugoEBPPvmk4uLi1LNnTyUlJUmS1q1bp27duv3h/t9+\n++3fvc4WLVpo2rRp6tatmzp27KilS5fqySefVOfOnXXffffp9OnTkqQPPvhA99xzj3r27KmOHTsq\nLS1NkjRq1CidPn1acXFxcjgcuv766zVs2DB17txZ2dnZzutJTU1VfHy8DMNQbm6uYmJitG7dusv9\nfxvgOwzAi3zxxRdGnz59LnrOM888Y0yYMMEwDMMoLi427r//fmP27NmGYRjGddddZyxcuNAwDMPI\nzs42brjhBqOoqMjYv3+/ceONNzrbiI6ONo4fP37B9uLFi40HHnjAMAzDOHPmjDFmzBhj7969xtq1\na42uXbsahmEYI0aM+MP9/9Z1111nLFiwwDAMw5g9e7bRpk0bIycnx3A4HEZcXJyRkZFh5OfnG337\n9jVOnDhhGIZhZGVlOa/ht9dz3XXXGZ988skF13PmzBljwIABxqxZs4z77rvPmDVr1kX/bIGKjkoT\nXsXPz89ZtZVn5cqVGjBggCSpUqVK6tevn1auXOk83qlTJ0lS8+bNVVJSosLCwgvaMH5z1+Lcdps2\nbfTzzz8rISFBs2fP1qBBg1S/fv0y565atepP9y/JWTk3aNBAUVFRCg8Pl81mU7169XTixAkFBQXp\ntdde0/LlyzV9+nTNnDmz3LbOxf7b6/Hz89NLL72kOXPmyM/PTw899FC5vwfA8Cy8zA033KAdO3ao\noKCgzP7Dhw/r4YcfVlFR0QVJ1eFwqLS01LkdGBjo/G4YxgUJ8vxjklRSUuLcV69ePX355Zd65JFH\nlJ+fr0GDBunLL7+8oD+z/Z/fz2/Z7Xbn94CAgAuOHz58WD169NAvv/yim266ScOGDfvdds4JCgr6\n3f0HDhxQYGCg9uzZo1OnTl20DaCiI2nCq9SuXVvdunXTqFGjnLM/8/LylJycrLCwMAUGBiomJsY5\nA7a4uFjvvvuubr31Vpdtn5+8atSooezsbEnSl19+6Zwc9M477+jZZ5/VrbfeqsTERMXExOinn34q\n00779u1N919ewjRj06ZNCgsL05AhQ3Trrbdq+fLlzjYDAgJcVuSSdPLkSY0YMUKTJ0/WXXfdpVGj\nRl1yPEBFQNKE1xk3bpwaNWqkfv36KS4uTn379lWTJk30/PPPS5Kee+45HTlyxDkRJzIyUo888ogk\nXTAz9vzt878/99xzSk5OVs+ePbV161aFh4dLknr06CGHw6EuXbqoV69ezmrzfKNHj76k/s3sP/9Y\nTEyMateurb/97W/q2bOnDh06pLCwMO3Zs0fh4eFq2rSpunTpohMnTpTb75gxY3T77berXbt2euyx\nx7Rv3z6988475fYNVHQ8cgIAgElUmgAAn7Zx40YlJCRcsD8zM1O9e/dWfHy83n//fVNtXTi7AAAA\nHzFnzhx9/PHHqlq1apn9paWlmjhxoj766CMFBgaqX79+6tSpk8LCwi7aHpUmAMBnRURE6F//+tcF\n+3fs2KGIiAgFBwerUqVKatOmjdavX++yPZImAMBnxcbGyt/f/4L9eXl5CgkJcW5XrVrV1CNXV+zw\nbO82gz0dAvCnvTL5Xk+HAFwWdTvdYVnbLSJuu+Tf/rDn60v6XXBwcJmXFuTn5ys0NNTl76g0AQA+\n77cPijRq1Eh79uzRyZMnVVxcrPXr16tVq1Yu27liK00AQMXgjjcLnesjIyNDhYWF6tOnj0aOHKn7\n779fhmGoT58+qlWrlst2SJoAAI+y2awd9LzmmmucbwDq2rWrc3+HDh3UoUOHP9QWw7MAAJhEpQkA\n8Cg/ec+L30maAACPcsc9zcuF4VkAAEyi0gQAeJSfxROBLieSJgDAoxieBQDAB1FpAgA8ysbsWQAA\nzPGme5reEykAAB5GpQkA8ChvmghE0gQAeJSfFyVNhmcBADCJShMA4FE2L6rfSJoAAI/ypnua3pPe\nAQDwMCpNAIBHedNEIJImAMCjvGlFIIZnAQAwiUoTAOBR3rSMHkkTAOBRzJ4FAMAHUWkCADyK2bMA\nAJjE7FkAAHwQlSYAwKOYPQsAgEnMngUAwAdRaQIAPIrZswAAmMTsWQAAfBCVJgDAo7xpIhBJEwDg\nUd50T5PhWQAATKLSBAB4lDdNBCJpAgA8yptWBPKeSAEA8DAqTQCARzF7FgAAk5g9CwCAD6LSBAB4\nFLNnAQAwieFZAAB8EJUmAMCjmD0LAIBJDM8CAOCDqDQBAB7F7FkAAExieBYAAB9EpQkA8ChmzwIA\nYBLDswAA+CAqTQCARzF7FgAAk6wanjUMQ+PGjdO2bdtkt9s1YcIE1a9f33n8k08+0VtvvSV/f3/1\n7NlT/fr1c9kmSRMA4JOWLVum4uJipaWlaePGjUpJSdGMGTOcxydPnqylS5eqcuXKuuuuu9S1a1eF\nhIRctE2SJgDAo6yaPbthwwbFxMRIklq2bKns7Owyx6Ojo/Xrr786+zcTB0kTAOBRVg3P5uXllakc\nAwIC5HA45Od3dg5skyZN1KtXLwUFBSk2NlbBwcGuY7UkUgAAPCw4OFj5+fnO7fMT5rZt27RixQpl\nZmYqMzNTR48e1RdffOGyTZImAMCjbDbbJX8upnXr1vr6668lSVlZWYqKinIeCwkJUZUqVWS322Wz\n2RQWFqaTJ0+6jJXhWQCAR1n1yElsbKzWrFmj+Ph4SVJKSooyMjJUWFioPn366J577lH//v1lt9vV\noEEDxcXFuWyTpAkA8Ek2m03Jycll9jVs2ND5PT4+3plQzSJpAgA8ys971jYgaQIAPMubFmxnIhAA\nACZZXmnu27dPy5cvV1FRkXPfgw8+aHW3AAAvwVtOzjN06FD9+uuvstvtzg8AAOdY9ciJFSyvNOvU\nqaPHH3/c6m4AALCc5Unz9ttv15QpU9S4cWPnvh49eljdLQDAS/jxarD/WrJkiSIjI7Vjxw5J3jVL\nylu1bt9C/R/tpYBKAdqzfb9mjn9TpwuLypxz8+2tdc9Dd8vhMJR/Ml8zn5+rnINHlDhpqGrXqyVJ\nskmqdU24Nm/YqsmJr3rgSlDRfbMpW3M++USlpaWKvOYajRhwr6pUrlzmnK/WrtO7/14mP5ufAu2V\n9FifPrquQQPn8Zxjx/XolCl647lRCq1a1d2XABO8KS9YnjTtdvsFD5fCOiHVgjU06X6NGjxBOQdy\nde/jvTXgiT6aM2mB85xK9gA9Pv5BJfYdo5yDR3RXv1j9fcS9Shk2XVOf+e9rcxo1vVbDJw3V6ynz\nPXEpqOB+zcvTSwsWKPXpf6huzZqavXixZi3+WMPi+zrP2Xf4sGYvXqzZo0aqekiI1m7erLGzZyvt\nhRckSV98u1ZvffaZjv36q6cuAz7G8olAdevW1axZs7Rq1SqtXr1aq1evtrrLCq1l2+v18+ZdyjmQ\nK0n64v3liuncrsw55xYsrhoSJEmqHFRZxUUlZc7xD/DXY8kPaO6URTp+5IQbIgfKWr9li6IjIlS3\nZk1JUveYGC1bv77MOZUCAvSPAfeq+v+9ySKqQQMdO3lKZ86c0dFff9V/Nv2gSY8OdXvs+GP8bLZL\n/rib5ZVmaWmpdu/erd27dzv3tW/f3upuK6yatcN05PAx5/axnGOqHFRZlasEOodoi04X6/WUeZow\nd7ROnTglP38/PXf/i2Xa6dTjLzqWe1z/uzLLrfED5+QeP67w6tWd2+HVq6vw9GkVnj7tHKK9ukYN\nXV2jhvOcGR98qFtbtJC/v79qXHWVkv/v8TbDvaHjD/Ki0Vnrk2ZKSkqZ7ZycHKu7rNBs5axH5XA4\nnN/rN7pGfR68W0/2GqncX46qc99Oevqlx/R0/7HOc+7qF6vXnp9rebxAeRyO309150ZKzne6uFgT\n356nIydOaNJjj1odGiowy4dnp0+frrZt26pNmzZq3ry5Bg8ebHWXFdqRQ0cVFl7NuR1WK0z5p/LL\nDL+2ane9tmT9pNxfjkqSPn8vUw0aXaPg0LOTJK6Nqi8/fz9tydru3uCB89QKq66j592LzD1+XMFB\nQQr8zbPeh48d02NTpiggwF//fGqYqlap4u5Q8Sd50/Cs5UkzMzNTK1euVLdu3bRkyRLVrl3b6i4r\ntI3fbFaT6yNVu164JOmvvTpo/Yrvy5yzc+seNW99nUKrn70PdPPtrXX4QK7yTp59WWuzNtHKXr/F\nvYEDv/H/mjbVlt27dSD37P35T1ev1q0tWpQ551RBgYb982X9pdWNGj14sCoFsJy2N7L9if+5m+V/\nw8LDw2W325Wfn6+IiAiVlJS4/hEu2ckTp/SvcW/oH5MfU0CAvw7tz9GrSa8rMjpCj4wZrBH3jtPm\n/92qj+d/ruTZz6i0uFR5J/M1afgrzjbq1K+t3INHPHgVgFQtJETPJAzQ2NdfV+mZM7qmZrhGDhqo\nbXv3aurCRZo98ll9vHKlco8f1+qNG7Uq6+z9d5vNpqlPPqGQoCBnW150ywxXOJthGJbeIx89erRa\ntWqlH374QVdddZVWrVqlxYsXu/xd7zYM48L7vTL5Xk+HAFwWdTvdYVnbo/428pJ/++IXKa5Puows\nqzTPJcYbb7xR/v7+ioqKkmEYuueee6zqEgDghbxpwXbLkua5FYAk6bPPPlPXrl1lGIZXrfwAALCe\nN6UFy5JmYmKi83tWVpaGDx9uVVcAALiFW6aaUV0CAMrjTcOzlj9yAgCAr7Cs0hw+fLhsNpsMw9DP\nP/9cZrh26tSpVnULAPAynnje8lJZljTj4+N/9zsAAOfzpuFZy5LmzTffbFXTAAAf4kU5k3uaAACY\nxUKNAACP8qYnLKg0AQAwiUoTAOBRTAQCAMAkL8qZJE0AgGd5U6XJPU0AAEyi0gQAeJQ3rQhEpQkA\ngElUmgAAj/Km5zRJmgAAj/LznpxJ0gQAeJY3VZrc0wQAwCQqTQCAR3lTpUnSBAB4lDfd02R4FgAA\nk6g0AQAexfAsAAAmeVHOZHgWAACzqDQBAB7lTW85IWkCADyKBdsBAPBBVJoAAI/yotFZkiYAwLO8\n6Z4mw7MAAJhEpQkA8CgWNwAAwCQvypkMzwIAYBaVJgDAoxieBQDAJKteDWYYhsaNG6dt27bJbrdr\nwoQJql+/vvP4Dz/8oEmTJkmSatasqZdeekl2u/3isVoTKgAAnrVs2TIVFxcrLS1NiYmJSklJKXM8\nKSlJEydO1MKFCxUTE6ODBw+6bJNKEwDgUVYNz27YsEExMTGSpJYtWyo7O9t5bNeuXapWrZrmzp2r\n7du3q0OHDrr22mtdtkmlCQDwKJvt0j8Xk5eXp5CQEOd2QECAHA6HJOn48ePKyspSQkKC5s6dq//8\n5z9au3aty1hJmgAAnxQcHKz8/HzntsPhkJ/f2bRXrVo1NWjQQA0bNlRAQIBiYmLKVKLlIWkCADzK\nz2a75M/FtG7dWl9//bUkKSsrS1FRUc5j9evXV0FBgfbt2yfp7FBu48aNXcbKPU0AgEdZdU8zNjZW\na9asUXx8vCQpJSVFGRkZKiwsVJ8+fTRhwgQNHz5cknTjjTfqtttuc9kmSRMA4JNsNpuSk5PL7GvY\nsKHz+//8z//o/fff/0NtkjQBAB7lRWsbkDQBAJ7lTSsCMREIAACTqDQBAB7lRYVm+UkzNTX1oj98\n7LHHLnswAICKx9WjI1cShmcBADCp3Erz/EqyoKBAe/fuVVRUlE6fPq2goCC3BAcA8H1eVGi6rjS/\n+eYb3X333Ro6dKiOHDmijh07avXq1e6IDQBQAdhstkv+uJvLpDlt2jQtWrRIoaGhqlWrlhYsWKDJ\nkye7IzYAAK4oLmfPOhwOhYeHO7fNrM0HAIBZ3jQ86zJpXn311Vq+fLlsNptOnjyphQsXqm7duu6I\nDQBQAfjU4gbjx4/Xp59+ql9++UV33HGHtmzZovHjx7sjNgAAriguK80aNWpo2rRpysvLU0BAgCpX\nruyOuAAAFYQXFZquk+a2bdv07LPP6uDBg5KkyMhITZo0SQ0aNLA8OACA7/OpxQ3Gjh2rYcOGae3a\ntVq7dq3uv/9+jRo1yh2xAQBwRXGZNIuKisq8mDM2NlZ5eXmWBgUAqDhstkv/uFu5w7PnhmOjo6M1\ne/Zs9e7dW/7+/vr000910003uS1AAIBv86bZs+UmzQEDBshms8kwDK1du1ZpaWnOYzabTaNHj3ZL\ngAAAXCnKTZqZmZnujAMAUEF5UaHpevbszp07tWjRIhUUFMgwDDkcDu3fv18LFy50R3wAAB/nTcOz\nLicCPfXUUwoNDdWWLVvUtGlTHT16VE2aNHFHbAAAXFFMrT37xBNPqLS0VM2aNVN8fLzi4+PdERsA\noALwokLTdaVZpUoVFRcX69prr9XmzZtlt9tVVFTkjtgAABWAT70arHv37nrkkUfUoUMHLViwQA88\n8IBq167tjtgAALiiuByeHTBggHr06KHg4GDNnz9fmzZtUvv27d0RGwCgAvCm4dlyk2Zqamq5P9q2\nbZsee+wxSwICAFQsPjV7FgAAnFVupUklCQBwBy8qNF3f0/SUeUuSPR0C8Kd16fCEp0MALosVW+6w\nrG1vejXYFZs0AQAVgxflTHP3NAsKCrR161YZhqGCggKrYwIA4IrkMml+8803uvvuuzV06FDl5uaq\nY8eOWr16tTtiAwBUAD61uMG0adO0aNEihYaGqlatWlqwYIEmT57sjtgAALiimFp7Njw83LnduHFj\nSwMCAFQs3nRP02XSvPrqq7V8+XLZbDadPHlSCxcuVN26dd0RGwCgArD5eU/WdDk8O378eH366af6\n5ZdfdMcdd2jLli0aP368O2IDAFQANtulf9zNZaVZo0YNTZs2zR2xAABwRXOZNDt27Pi7M5T+/e9/\nWxIQAKBi8aa1Z10mzfnz5zu/l5aW6quvvlJxcbGlQQEAcCVyeU/zmmuucX4iIiL0wAMPaNmyZe6I\nDQBQAfjUPc3169c7vxuGoe3bt6uoqMjSoAAAFYdPDc++8sorzu82m03Vq1fXxIkTLQ0KAFBxeFHO\ndJ00O3furP79+7sjFgAArmgu72kuWrTIHXEAACoqL7qpaWpFoIEDB6ply5YKDAx07ucl1QCAisZl\n0mzVqpU74gAAVFA+MREoPT1dcXFxVJQAAEt5Uc4s/57mvHnz3BkHAKCCsvnZLvnjbi4nAgEAgLPK\nHZ7dvn27OnXqdMF+wzBks9lYexYAcFl40/BsuUkzIiJCs2fPdmcsAABc0cpNmpUqVdI111zjzlgA\nABWQT8yebd26tTvjAABUUF6UM8tPmklJSe6MAwBQQXlTpcnsWQCATzIMQ2PHjlV8fLwGDhyoffv2\n/e55SUlJmjZtmqk2SZoAAI+yaunZZcuWqbi4WGlpaUpMTFRKSsoF56Slpemnn34yHStJEwDgkzZs\n2KCYmBhJUsuWLZWdnV3m+Pfff69NmzYpPj7edJskTQCAR9lstkv+XExeXp5CQkKc2wEBAXI4HJKk\n3NxcpaamKikpSYZhmI7V5YLtAABYyqLyLTg4WPn5+c5th8MhP7+znX3++ec6ceKEHnzwQeXm5qqo\nqEiRkZHq0aPHRdskaQIAPMqq2bOtW7fW8uXLdeeddyorK0tRUVHOYwkJCUpISJB09gUlu3btcpkw\nJZImAMBHxcbGas2aNc57likpKcrIyFBhYaH69OlzSW2SNAEAHmXVY5o2m03Jycll9jVs2PCC8+Li\n4ky3yUQgAABMotIEAHiUN60IRNIEAHiUF+VMkiYAwMO8KGtyTxMAAJOoNAEAHmXzo9IEAMDnUGkC\nADzKi25pkjQBAJ7FIycAAJjkRTmTe5oAAJhFpQkA8CwvKjWpNAEAMIlKEwDgUd70nCZJEwDgUV40\nOkvSBAB4mBdlTe5pAgBgEpUmAMCjvKjQpNIEAMAsKk0AgEcxexYAAJNYexYAALO8J2dyTxMAALOo\nNAEAHuVNw7NUmgAAmESlCQDwKG+qNEmaAADP8qIxT5ImAMCjvKnS9KL8DgCAZ1FpAgA8ikoTAAAf\nZHmluW/fPi1fvlxFRUXOfQ8++KDV3QIAvIX3FJrWV5pDhw7Vr7/+Krvd7vwAAHCOzc92yR93s7zS\nrFOnjh5//HGruwEAeCsvuqdpedK8/fbbNWXKFDVu3Ni5r0ePHlZ3CwDAZWd50lyyZIkiIyO1Y8cO\nSd41SwoAYD1vSguWJ0273a7k5GSruwEAwHKWJ826detq1qxZatasmbPKbN++vdXdVjirvlmrV2e/\nqdKSEjVpFKmxzyQqKKjKHz4n8blxqlUrXM88+agkaf13WXr5tddVWlqqyoGVNeKJoWre9Dq3XRcq\ntra3tdEDwwaoUqUA7fhpj14anarCgtNlzom79y7F9e+s06eLtGfHfk1/YbbyTuYrqGoVjXjhMTWI\nrCebTfri4xVKeyPdQ1eCi/GmEUjLZ8+WlpZq9+7dWrJkiT777DN99tlnVndZ4Rw/8avGTZyiaRPG\n6aMFb6punas1/bXX//A5by16V1nZm53bJaWlGjn+RY19JlHvvjlLDwzsr9ETJrnlmoCrqoVoxAuP\na8zjEzWo6+M6tP+wHkocWOacVjdfr/j7e2jYfWP0UK9ErVv1nf6RPFSSdP8T/ZVz6Ijuv/tJPXLP\n07o7/k41bdHEE5cCV/xsl/5xM8srzZSUlDLbOTk5VndZ4Xy7foOubxqtenXrSJL69Oimvvc/rJHD\nnzB9zvrvsvTN+g3qfXdXnTyVJ0mqFBCgLz58R/7+/jIMQ/sOHFS1q0LdfHWoqG669UZt3bRdv+w/\nLEn6OO1zvZH+sqY/P9t5TlSzRtrwzUYdyz0uSVr51Td6+vlH5e/vp9SUN5wVTM1aYQqoFKD8UwXu\nvxC4RKV5nunTp6tt27Zq06aNmjdvrsGDB1vdZYVzKCdHtWuFO7drh9dUQUGhCgoKTZ2Tc+SIpqTO\n1ItjRsrPVvavhL+/v44dP647e/fXK6/N0aB+91h/QYCkWnVqKOfQEed27qGjqlK1sqoEVXbu27Jp\nu1q3baHwq2tKkrr0vEP+Af4KrRYiSTIMQ6MmDdMbi6cra1229u464N6LgM+xPGlmZmZq5cqV6tat\nm5YsWaLatWtb3WWF4zCM393v5+/n8hxDhkYmv6inHx+qGmHVf/ecsOrV9cWH7+itGdM1NmWK9u7n\nXzywns32+/96OnPG4fy+acOPeutfaXohdaRmvjtZZ86c0alf81RSUuo858VnXtbdtyToqmohGjS0\nr+Vx4xLY/sTHzSwfng0PD5fdbld+fr4iIiJUUlJidZcVTp1atZT941bn9uHcIwoNCVblwECX5+zc\nvUcHDx3W1NTXZMjQ0aPH5TAcKi4u1vBHH9a6Dd/r9phbJUnRUY0V1ThSP+/cpQb1rnHfBaJCyvkl\nV81aRDm3w6+uobyT+SouKnbuqxJUWT/874/6PD1TklQt7Crd/0R/5Z3M1023tNLO7Xt0LPe4ik4X\n69+frVJMbFu3Xwd8i+WV5tVXX60PPvhAVapU0dSpU3Xy5Emru6xw2t7cRtk/btW+AwclSR9+kqHb\nbr3F1Dk3NGuqpe8v1DtvzFTaG6+p991d9deOHTTm6afkZ7Np3MSp2pj9oyRpx67d2r13n65vFu3W\n60PFtH5Nlpq2bKK69a+WJHW7529anbmuzDk1aoXp5bdfcA7ZDhxyj5ZlrJQk3d75Vg0aevZ2QqVK\nAerQ+VZ9v3aTG68AZtlstkv+uD1Wwyhn3O4ycTgcOnTokEJDQ5Wenq527dqVWR2oPAWH91oZls9Z\ns3a9Xpk1R6WlZ1Svbh298Nwz2nfwoJ6f/E+988bMcs8JCQku086sufN14uRJ5yMn323cpGn/mqUz\nZ86okr2Snnj477qpVUu3X5+36tLhCdcnoVw3t79RDw0fKP9K/jq495BSnp2uuvWv1j+eH6qHeiVK\nku7u11lx/bvIZpM2fbdF05+frZKSUgVVraLE5CFq2CRCDodDq5et1Vv/SvPwFXmvFVsWW9b2vowl\nl/zb+l27XMZIXLM8aR49elQzZ87U7t271aRJEz3yyCO66qqrXP6OpAlfQNKEr7A0aX629JJ/W/+u\nzpcxEtcsH54dNmyYIiMj9Y9//EP16tXTiBEjrO4SAOBFvGl41vKJQJLUv39/SVJ0dLQ+//xzd3QJ\nAMBlZ3mlGRkZqY8//liHDx9WZmamqlWrpl27dmnXrl1Wdw0AwGVleaW5c+dO7dy5U2+++ab8/f1V\ntWpVJSUlyWazad68eVZ3DwC40nnPgkDWJc3Nmzfrueee03vvvacVK1Zo7NixCg0N1aOPPqpOnTpZ\n1S0AwMvYPLCG7KWybHh28uTJmjhxoux2u15++WXNmTNHH374oV5//XXXPwYAVBw226V/3MyyStPh\ncCg6OlqHDx9WYWGhmjdvLsm7FuYFAFjPm/KCZUkzIOBs06tWrVK7du0kSSUlJSoo4C0DAADrGYah\ncePGaduAOwwaAAAL/0lEQVS2bbLb7ZowYYLq16/vPJ6RkaF58+YpICBAUVFRGjdunMs2LRuebdeu\nneLj45WamqqEhATt3btXQ4YMUZcu7l29AQBQMS1btkzFxcVKS0tTYmJimVdVFhUV6ZVXXtGCBQu0\naNEinTp1SsuXL3fZpmWV5kMPPaROnTopODhYtWvX1t69e9W3b1/FxsZa1SUAwBtZNBFow4YNiomJ\nkSS1bNlS2dnZzmN2u11paWmy2+2SpNLSUgWe95KL8lj6yEmjRo2c3xs0aKAGDRpY2R0AwAtZdU8z\nLy9PISEhzu2AgAA5HA75+fnJZrMpLCxMkjR//nwVFhbqlltuKa+p/7ZhSaQAAJhl0Tyg4OBg5efn\nO7fPJcxzDMPQ5MmTtWfPHqWmpppq0/IVgQAAuBir1p5t3bq1vv76a0lSVlaWoqKiyhwfM2aMSkpK\nNGPGDOcwrStUmgAAnxQbG6s1a9YoPj5ekpSSkqKMjAznY5AfffSR2rRpo4SEBNlsNg0cOFB33HHH\nRdskaQIAfJLNZlNycnKZfQ0bNnR+//HHH/9wmyRNAIBnedEyeiRNAIBHsSIQAABmkTQBADDHmypN\nHjkBAMAkkiYAACYxPAsA8CxmzwIAYI433dMkaQIAPIukCQCAOTYvGp5lIhAAACaRNAEAMInhWQCA\nZ3FPEwAAc5g9CwCAWSRNAADMYfYsAAA+iKQJAIBJDM8CADyLe5oAAJhE0gQAwBweOQEAwCxmzwIA\n4HtImgAAmMTwLADAo2w276nfSJoAAM9iIhAAAOYwexYAALOYPQsAgO8haQIAYBLDswAAj+KeJgAA\nZpE0AQAwiec0AQAwx8bsWQAAfA9JEwAAkxieBQB4FhOBAAAwh0dOAAAwi9mzAACYw+xZAAB8EEkT\nAACTGJ4FAHgWE4EAADCH2bMAAJjF7FkAAExi9iwAAL6HpAkAgEkMzwIAPIqJQAAAmMVEIAAAzKHS\nBADALC+qNL0nUgAAPIykCQCASQzPAgA8yqpXgxmGoXHjxmnbtm2y2+2aMGGC6tev7zyemZmpGTNm\nKCAgQL169VKfPn1ctknSBAB4lkUTgZYtW6bi4mKlpaVp48aNSklJ0YwZMyRJpaWlmjhxoj766CMF\nBgaqX79+6tSpk8LCwi7aJsOzAACPstn8LvlzMRs2bFBMTIwkqWXLlsrOznYe27FjhyIiIhQcHKxK\nlSqpTZs2Wr9+vctYqTQBAJ5lUaWZl5enkJAQ53ZAQIAcDof8/PwuOFa1alWdOnXKZZtXbNIMqt3A\n0yEAf9qKLYs9HQJwxbOH1rCk3eDgYOXn5zu3zyXMc8fy8vKcx/Lz8xUaGuqyTYZnAQA+qXXr1vr6\n668lSVlZWYqKinIea9Sokfbs2aOTJ0+quLhY69evV6tWrVy2aTMMw7AsYgAAPOT82bOSlJKSos2b\nN6uwsFB9+vTRihUrlJqaKsMw1Lt3b/Xr189lmyRNAABMYngWAACTSJoAAJhE0gQAwCSSpo9at26d\nhg8fXmbf1KlTtXgxj0Dgyjd79mwNHjxYCQkJGjRokDZv3uzpkABJV/BzmvjzvOkddcA5O3bsUGZm\nptLS0iRJW7du1bPPPst/8OGKQNL0Yb83MfrMmTMaPXq0Dh06pNzcXHXs2FFPPvmkRo4cqYCAAB08\neFDFxcXq0qWLli9frl9++UUzZswos8gxYKXg4GAdOnRIH3zwgWJiYhQdHa333ntPCQkJioyM1M6d\nOyVJL7/8sqpXr66kpCT+PsNtGJ71Yd9++60GDhyogQMHKiEhQZ999pn8/f3VqlUrzZkzR++//77e\neecd5/n16tXTG2+8ocjISB04cECzZ8/WX//6Vy1fvtyDV4GKpnbt2po5c6a+++47xcfHq0uXLlqx\nYoWksw+rz58/X507d9bMmTN16NAh/j7Drag0fVi7du00depU5/a0adOUl5enn376SWvXrlXVqlVV\nUlLiPN6sWTNJUmhoqBo1auT8XlRU5N7AUaHt3btXVatW1YsvvihJ2rx5sx544AHVqlVLbdu2lXQ2\neWZmZio0NFQ//PADf5/hNlSaFYhhGDIMQ1dddZVeeuklDR48WKdPn3Ye5x4orgTbtm3T+PHjnQkw\nIiJCoaGh8vf3d04I2rBhg5o0aaL09HT+PsOtqDQrEJvNJn9/f61atUpZWVmqVKmSrr32WuXk5Fxw\nHuApsbGx2rlzp3r37q2goCAZhqERI0borbfeUnp6uubOnaugoCBNnjxZubm5SkxM5O8z3IZl9AB4\nhYSEBI0fP14NGzb0dCiowBieBeAVqBhxJaDSBADAJCpNAABMImkCAGASSRMAAJNImgAAmETShE85\ncOCArr/+esXFxSkuLk5du3bV3//+dx0+fPiS20xPT9fIkSMlSQ8//LByc3PLPffVV1/Vhg0b/lD7\n0dHRF+xLTU1VamrqRX/XsWNHHTx40HQ/ZtoEcHEkTfic2rVrKz09Xenp6crIyFDz5s31/PPPX5a2\nZ82apfDw8HKPr1u3Tg6H4w+1eamPUvAIBuB+rAgEn3fTTTc5F+nu2LGjWrZsqa1bt2rhwoVauXKl\n5s2bJ8Mw1Lx5cyUlJclut2vx4sV67bXXFBISojp16qhq1arO3y9YsEA1a9ZUcnKyNmzYoEqVKmnI\nkCEqLi5Wdna2Ro8erdTUVAUGBmrcuHE6ceKEqlSpotGjR6tp06Y6cOCAnn76aRUWFqpFixYu41+w\nYIE++eQTFRYWys/PT//85z8VGRkpwzD06quvauvWrQoMDFRycrKuu+46HT161PnmDz8/Pw0fPlzt\n2rWz9M8YqCioNOHTSkpKtHTpUrVu3dq577bbbtPSpUt17Ngxvf/++0pLS1N6errCwsL05ptvKicn\nR1OmTNGiRYv07rvvKj8/3/nbc9Xd/PnzVVhYqM8//1xz587VzJkz1bVrV11//fWaMGGCmjRpomee\neUYjRozQRx99pPHjx+upp56SJD3//PPq1auX0tPTy8T1e/Ly8pSZmakFCxbo008/VadOnbRo0SLn\n8YYNGyo9PV1DhgzRs88+K0maMGGCevfurQ8//FAzZsxQUlKSCgoKLtufKVCRUWnC5xw+fFhxcXEy\nDEMlJSVq0aKFEhMTncfPVXdr167Vnj171LdvXxmGodLSUjVr1kzff/+9WrdurbCwMElS9+7d9e23\n30r67ztK169fr759+0qSatasqU8//dTZvmEYKigo0KZNmzRy5Ejnb06fPq0TJ05o7dq1mjZtmrPt\n0aNHl3stwcHBmjJlijIyMrR7926tWrVKTZs2dR7v3bu3pLP/ITBixAjl5eXpP//5j3bt2qXp06dL\nOvsO1b179/6JP1EA55A04XPO3dMsT+XKlSWdTSadO3fWc889J0kqLCxUaWmpvvnmmzL3JQMC/vuP\nyblK8/x90tnXWdWpU8e57XA4VLly5TJxHD58WNWqVZOfn5+zfZvNJj+/8gd8Dh06pISEBA0YMEB/\n+ctfVLNmTW3ZssV53N/fv8z5AQEBcjgcevvttxUaGipJysnJUc2aNbVs2bJy+wFgDsOz8DlmV4a8\n+eabtWzZMh07dkyGYWjs2LF6++231aZNG23cuFE5OTlyOBxasmTJBW3fdNNNWrp0qSTp6NGjSkhI\nUElJiQICAlRaWqrg4GBFRETok08+kSStWbNGAwYMkCTdcsst+vjjjyVJX3zxhYqLi8uNcdOmTYqI\niNCgQYPUokULrVy5skxCP1fhfvXVV4qMjFTlypXVtm1bLVy4UJL0888/q3v37mVemQXg0lFpwudc\nbFbp+ceio6P16KOPatCgQTIMQ02bNtVDDz0ku92uMWPG6L777lNQUJAaN258we/79++vF154Qd27\nd5fNZtOYMWMUFBSkmJgYjRs3TpMmTdKUKVOUlJSkOXPmyG636+WXX5YkjR49WiNGjNB7772nG264\nQcHBweXG2759e73zzju66667FBgYqBYtWmj79u3OWHbv3q0ePXooODhYkyZNcraflJSk7t27S5Km\nTJmioKCgS/zTBHA+FmwHAMAkhmcBADCJpAkAgEkkTQAATCJpAgBgEkkTAACTSJoAAJhE0gQAwCSS\nJgAAJv1/gJWTy003pakAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xda15d30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "labels=['Ham','Spam']\n",
    "plot_confusion_matrix(cm_norm,classes=labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 9 ==========\n",
    "\n",
    "Study the output produced, most importantly the percentages of correctly and incorrectly classified instances. You probably will notice that your classifer does rather well despite making a very strong assumption on the form of the data. If we didn't make this assumption, what would be the main practical problems? *Hint*: If you've forgotten the assumption of the Naive Bayes model, check wikipedia and/or sklearn documentation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Your answer goes here:*\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The assumption is the occurrence of one sample word or character in a 'Spam' email (or 'Ham') does not affect the probability of the other. However in the practical word, some word might be more likely appeared in one class of email (e.g Spam),and thus it will reduce the probability of it appearance in another class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ========== Question 10 =========="
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "The empirical log probability of input features given a class $P\\left(x_i  |  y\\right)$ is given by the attribute `feature_log_prob` of the classifier. For each feature there are two such conditional probabilities, one for each class. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**a)** What dimensionality do you expect the `feature_log_prob_` array to have? Why?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Your answer goes here:*\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2 x 54, because in this occasion we have 54 features, that are being used to determined whether an email is a spam or not"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**b)** Inspect the log probabilities of the features. Verify that it has the expected dimensionality (i.e. `shape`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2L, 54L)"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "clf.feature_log_prob_.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**c)** Create a list of the names of the features that have higher log probability when the email is `Ham` than `Spam` i.e. what features imply an email is more likely to be `Ham`? *Hint:* There are a many ways to do this. Try it on your own then, if you get stuck, you can do it using index numbers (look up [`np.argwhere`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.argwhere.html)), or using a boolean mask (look up [pandas indexing](http://pandas.pydata.org/pandas-docs/stable/indexing.html)). The column names of a Pandas DataFrame are contained in the `columns` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[11, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50]\n",
      "word_freq_will\n",
      "word_freq_you\n",
      "word_freq_hp\n",
      "word_freq_hpl\n",
      "word_freq_george\n",
      "word_freq_650\n",
      "word_freq_lab\n",
      "word_freq_labs\n",
      "word_freq_telnet\n",
      "word_freq_857\n",
      "word_freq_data\n",
      "word_freq_415\n",
      "word_freq_85\n",
      "word_freq_technology\n",
      "word_freq_1999\n",
      "word_freq_parts\n",
      "word_freq_pm\n",
      "word_freq_direct\n",
      "word_freq_cs\n",
      "word_freq_meeting\n",
      "word_freq_original\n",
      "word_freq_project\n",
      "word_freq_re\n",
      "word_freq_edu\n",
      "word_freq_table\n",
      "word_freq_conference\n",
      "char_freq_;\n",
      "char_freq_(\n",
      "char_freq_[\n"
     ]
    }
   ],
   "source": [
    "# Your code goes here\n",
    "# Probability is smaller when they have smaller log probabilites\n",
    "lp = clf.feature_log_prob_ #moving all of the log prob to an array\n",
    "idx_list_feature = []\n",
    "for j in range(len(lp[0])): #for every features we have,\n",
    "    if(lp[0,j]>lp[1,j]): #if the log probabilities for class 0 (ham) > class 1 (spam)\n",
    "        #print (j)\n",
    "        idx_list_feature.append(j)#add the index to be shown later as names\n",
    "print (idx_list_feature)\n",
    "column_names = list(spambase.columns.values)\n",
    "for i in range(len(idx_list_feature)):\n",
    "    print (column_names[idx_list_feature[i]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 11 =========="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the final part of this section we will now pretend we are spammers wishing to fool a spam checking system based on Naïve Bayes into classifying a spam e-mail as ham (i.e. a valid e-mail). For this we will use a test set consisting of just one data point (i.e. e-mail). This tiny dataset is called `spambase_test` and has already been pre-processed for you which means that the redundant attributes have been removed and word frequencies have been replaced by word presence/absence."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**a)** Load `./datasets/spambase_test.csv` dataset into a new pandas structure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Your code goes here\n",
    "data_path_new = os.path.join(os.getcwd(), 'datasets', 'spambase_test.csv')\n",
    "spambase_test = pd.read_csv(data_path, delimiter = ',')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**b)** Use `spambase_test` to create a pandas DataFrame object X_test, contatining the test features, and pandas Series object y_test, containing the test outcome"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code goes here\n",
    "X_test=spambase_test.drop(['capital_run_length_average','capital_run_length_longest','capital_run_length_total','is_spam'],axis=1)\n",
    "y_test=pd.Series(spambase_test['is_spam'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**c)** Feed the input features into the classifier and compare the outcome to the true label. Make sure you don't feed the target into the classifier as you will receive an error (why?). Does the classifer classify the spam e-mail correctly?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.  1.  1. ...,  0.  0.  0.]\n",
      "0.896544229515\n",
      "[[ 0.87876614  0.18643133]\n",
      " [ 0.04949785  0.92388307]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAc0AAAFtCAYAAABlWTHzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VFX6x/HvTMIkhEnESOgQCYIUBQRWQciKYFxBQEKR\noATUBQUbCIqCEJoYRMAWQRBFqrFGJYIFE2kqsCglCKj0XkVIIYW5vz/4MUsWQ67InWEmn/e+5vWa\nW+ac57Lo43PuuefaDMMwBAAAimX3dgAAAPgKkiYAACaRNAEAMImkCQCASSRNAABMImkCAGASSRM+\nx+VyaebMmerSpYtiY2PVvn17TZw4UXl5eX+rzf79++uOO+7QvHnz/vLvMzIyNGDAgIvu/1LLzMxU\n7969izweGxurzMxMD0YE+Acbz2nC14wYMUInT57Uc889J6fTqVOnTmnw4MFyOp164YUXLqrNffv2\nqW3btlq7dq1sNtsljtjz9uzZo44dO+rHH3/0diiAX6HShE/Zs2ePUlNT9fzzz8vpdEqSgoODNWbM\nGMXExEg6U2U99dRT6tChgzp27KgXX3xRLpdLktSgQQMlJSWpR48euu222zR79mxlZWWpb9++Kigo\nUOfOnbV7927VqVNHx48fd/d7djs7O1sDBgxQbGysOnfurISEBEnSqlWr1KFDh7/c/6xZs/70Ohs0\naKDJkyerQ4cOat26tRYtWqQBAwaobdu2uu+++3Tq1ClJ0ocffqi7775bnTt3VuvWrZWcnCxJGjZs\nmE6dOqXY2Fi5XC5dd911GjhwoNq2bauMjAz39SQlJSkuLk6GYejw4cOKjo7WqlWrLvX/bYD/MAAf\n8uWXXxrdunW74DlPP/20MW7cOMMwDCMvL8944IEHjOnTpxuGYRjXXnutMW/ePMMwDCMjI8O4/vrr\njdzcXGPPnj3GDTfc4G6jTp06xu+//37e9ieffGL06dPHMAzDOH36tDFixAhj165dxsqVK4327dsb\nhmEYQ4YM+cv9/69rr73WmDt3rmEYhjF9+nSjSZMmxqFDhwyXy2XExsYaqampRlZWltG9e3fj+PHj\nhmEYxtq1a93X8L/Xc+211xqfffbZeddz+vRpo2fPnsa0adOM++67z5g2bdoF/2yBko5KEz7Fbre7\nq7aiLF26VD179pQklSpVSj169NDSpUvdx9u0aSNJql+/vvLz85WTk3NeG8b/3LU4u92kSRP99ttv\nio+P1/Tp09W7d29Vq1at0LnLli372/1LclfO1atXV+3atRURESGbzaaqVavq+PHjCgkJ0RtvvKH0\n9HS98sormjp1apFtnY39f6/HbrfrxRdf1IwZM2S32/Xggw8W+XsADM/Cx1x//fXaunWrsrOzC+0/\nePCgHnroIeXm5p6XVF0ulwoKCtzbQUFB7u+GYZyXIM89Jkn5+fnufVWrVtVXX32lfv36KSsrS717\n99ZXX311Xn9m+z+3n//lcDjc3wMDA887fvDgQXXq1En79+9X06ZNNXDgwD9t56yQkJA/3b93714F\nBQVp586dOnny5AXbAEo6kiZ8SoUKFdShQwcNGzbMPfszMzNTo0ePVnh4uIKCghQdHe2eAZuXl6f3\n3ntPLVq0KLbtc5PXVVddpYyMDEnSV1995Z4c9O677+qZZ55RixYtNHjwYEVHR+uXX34p1E7Lli1N\n919UwjRjw4YNCg8PV//+/dWiRQulp6e72wwMDCy2IpekEydOaMiQIZowYYLuvPNODRs27KLjAUoC\nkiZ8zqhRo1SzZk316NFDsbGx6t69u2rVqqWxY8dKkp599lkdOXLEPREnKipK/fr1k6TzZsaeu33u\n92effVajR49W586dtXnzZkVEREiSOnXqJJfLpXbt2qlLly7uavNcw4cPv6j+zew/91h0dLQqVKig\nf/3rX+rcubMOHDig8PBw7dy5UxEREapbt67atWun48ePF9nviBEjdOutt6p58+Z69NFHtXv3br37\n7rtF9g2UdDxyAgCASVSaAAC/tm7dOsXHx5+3Py0tTV27dlVcXJw++OADU22dP7sAAAA/MWPGDH36\n6acqU6ZMof0FBQUaP368Pv74YwUFBalHjx5q06aNwsPDL9gelSYAwG9FRkbq9ddfP2//1q1bFRkZ\nKafTqVKlSqlJkyZavXp1se2RNAEAfismJkYBAQHn7c/MzFRoaKh7u0yZMqYeubpsh2djb+jl7RCA\nv+2NVx/wdgjAJVEhupVlbTeIvOWif7t+55KL+p3T6Sz00oKsrCyFhYUV+zsqTQCA3/vfB0Vq1qyp\nnTt36sSJE8rLy9Pq1avVqFGjYtu5bCtNAEDJ4Ik3C53tIzU1VTk5OerWrZuGDh2qBx54QIZhqFu3\nbipfvnyx7ZA0AQBeZbNZO+hZpUoV9xuA2rdv797fqlUrtWrV6i+1xfAsAAAmUWkCALzKLt958TtJ\nEwDgVZ64p3mpMDwLAIBJVJoAAK+yWzwR6FIiaQIAvIrhWQAA/BCVJgDAq2zMngUAwBxfuqfpO5EC\nAOBlVJoAAK/ypYlAJE0AgFfZfShpMjwLAIBJVJoAAK+y+VD9RtIEAHiVL93T9J30DgCAl1FpAgC8\nypcmApE0AQBe5UsrAjE8CwCASVSaAACv8qVl9EiaAACvYvYsAAB+iEoTAOBVzJ4FAMAkZs8CAOCH\nqDQBAF7F7FkAAExi9iwAAH6IShMA4FXMngUAwCRmzwIA4IeoNAEAXuVLE4FImgAAr/Kle5oMzwIA\nYBKVJgDAq3xpIhBJEwDgVb60IpDvRAoAgJdRaQIAvIrZswAAmMTsWQAA/BCVJgDAq5g9CwCASQzP\nAgDgh6g0AQBexexZAABMYngWAAA/RKUJAPAqZs8CAGASw7MAAPghKk0AgFcxexYAAJMYngUAwA9R\naQIAvIrZswAAmGTV8KxhGBo1apS2bNkih8OhcePGqVq1au7jn332md555x0FBASoc+fO6tGjR7Ft\nkjQBAH5p8eLFysvLU3JystatW6fExERNmTLFfXzChAlatGiRgoODdeedd6p9+/YKDQ29YJskTQCA\nV1k1e3bNmjWKjo6WJDVs2FAZGRmFjtepU0d//PGHu38zcZA0AQBeZdXwbGZmZqHKMTAwUC6XS3b7\nmTmwtWrVUpcuXRQSEqKYmBg5nc7iY7UkUgAAvMzpdCorK8u9fW7C3LJli7799lulpaUpLS1NR48e\n1ZdffllsmyRNAIBX2Wy2i/5cSOPGjbVkyRJJ0tq1a1W7dm33sdDQUJUuXVoOh0M2m03h4eE6ceJE\nsbEyPAsA8CqrHjmJiYnRihUrFBcXJ0lKTExUamqqcnJy1K1bN919992655575HA4VL16dcXGxhbb\nJkkTAOCXbDabRo8eXWhfjRo13N/j4uLcCdUskiYAwKvsvrO2AUkTAOBdvrRgOxOBAAAwyfJKc/fu\n3UpPT1dubq57X9++fa3uFgDgI3jLyTkefvhh/fHHH3I4HO4PAABnWfXIiRUsrzQrVaqkxx57zOpu\nAACwnOVJ89Zbb9XEiRN1zTXXuPd16tTJ6m4BAD7CzqvB/mvhwoWKiorS1q1bJfnWLClf1aRlQ937\nWDeVCgzUjl936/XRM3QqJ7fQOTfd2kTd+8XKddqlrBNZmjL2bR3ce1g2m019n+ml+k2ulQxpzfJ1\nmv3Ke166EpR036/foOkff6L8ggLVrFpFT9/XSyHBwX96buLb7yiqahV1vz1GknQyK0uT5s7Xb7t3\nq3RQkO5ocbO6tL7Vk+HDJF/KC5YnTYfDcd7DpbBOaFmnHh3VR8/0HqODew8r/vG71WtAd00fP9t9\nTilHKQ147iEN7DZMh/YdUft7/qU+Q+I1bsBktWrfQpUjK2pA12Gy220aPytBzdo01Q/f/MeLV4WS\n6PjJTI2fOUtThz2tyhEReuPDj/XGRx9r0L33FDpv5/4DemnefG3avkNRVau497/23gcKCQ7W3OfG\nqKDgtIa9PkWVy5VT8wbXe/pS4EcsnwhUuXJlTZs2TcuWLdPy5cu1fPlyq7ss0Ro1u16/ZmzTwb2H\nJUlffPCN/tmueaFz7P//JHGZ0DKSpOCQIOXl5v3/MbuCSwfJEVRKjiCHAksFKj8334NXAJyx+uef\nVbfG1aocESFJ6nTrLVr8w6rzzktJT1e7li10a9MmhfZv2blTtze/SZIUGBig5g2u15I1P1oeN/46\nu8120R9Ps7zSLCgo0I4dO7Rjxw73vpYtW1rdbYlVrmK4jhw85t4+evCYgkNKK7h0kHuINvdUnqY9\n/47GzxqhE8czZbfbNez+sZKktM+W6eaYf2jGl6/Ibrdr3Q8ZWrN8nVeuBSXboWPHVD483L0dceWV\nyj51StmnThUaoh14Tw9J0pqfNxX6ff2oKH31/UpdV7Om8vLztWTNjyoVyHoulyMfGp21PmkmJiYW\n2j506JDVXZZotiLWo3K5XO7v1WtW0d0PdtKjsc/o8P4jahcXo6cnPa5BcSMU1y9Wfxw7qftaPyJH\ncJCGvTRQHe79lxbMK/6VOcClZBjGn+4PsJsbIHvk7q6a8sGH+veY51SubFn9o349ZWzddilDRAlk\n+fDsK6+8ombNmqlJkyaqX7++7r//fqu7LNGO7D+q8Iiy7u2rKoQr60SW8s4ZYm108/Xa9NMvOrz/\niCRp0XuLVS2qipxhZXTTrU30zadL5HIZOpV9Sumpy3XdP+p6/DqA8uHhOnL8uHv78O+/K7RMiIJM\nPuudmZOjfl27aNbokZr0xADZZFOV/x/qxeXFl4ZnLU+aaWlpWrp0qTp06KCFCxeqQoUKVndZoq39\nIUO1rqupilXLS5Ju73KrVi0pfB9n26adqt+kjq648swbzW9q3USH9h1R5oksbdu8Uy1iztwHCggM\n0D9uuUG/rN/q2YsAJN1Yv55+3r5dew+duT//2ZKlatGokenff/btUr31yaeSpGN/nNCCZcsVc9ON\nlsSKv8f2N/7naZYPz0ZERMjhcCgrK0uRkZHKz2dSiZVO/H5SSaPe1JCJjykgMEAH9hzSK8OnK6ru\n1Xp4xAN68p4EZfxnkz6ZvVBjZwxTfl6BMv/IVOITL0uS3p44T32ejtdrH43X6dOntX7Vz0p5J9XL\nV4WSqGxoqIbe31sjpr6hgtOnVTkiQs/++35t2bFTE2bP0VsJwwud/7+PLdzb7g6Ne2umeo88M3v/\n33d10LVXR3osfvgnm1HUjYNLZPjw4WrUqJHWr1+vK664QsuWLdMnn3xS7O9ib+hlZViAR7zx6gPe\nDgG4JCpEt7Ks7WH/GnrRv33+y8TiT7qELKs0zybGG264QQEBAapdu7YMw9Ddd99tVZcAAB/kSwu2\nW5Y0z64AJEmff/652rdvL8MwfGrlBwCA9XwpLViWNAcPHuz+vnbtWg0aNMiqrgAA8AiPPOlLdQkA\nKIovDc9a/sgJAAD+wrJKc9CgQbLZbDIMQ7/99luh4dpJkyZZ1S0AwMd443nLi2VZ0oyLi/vT7wAA\nnMuXhmctS5o33sjKGwCA4vlQzuSeJgAAZvGeHACAV/nSExZUmgAAmESlCQDwKiYCAQBgkg/lTJIm\nAMC7fKnS5J4mAAAmUWkCALzKl1YEotIEAMAkKk0AgFf50nOaJE0AgFfZfSdnkjQBAN7lS5Um9zQB\nADCJShMA4FW+VGmSNAEAXuVL9zQZngUAwCQqTQCAVzE8CwCAST6UMxmeBQDALCpNAIBX+dJbTkia\nAACvYsF2AAD8EJUmAMCrfGh0lqQJAPAuX7qnyfAsAAAmUWkCALyKxQ0AADDJh3Imw7MAAJhFpQkA\n8CqGZwEAMMmqV4MZhqFRo0Zpy5YtcjgcGjdunKpVq+Y+vn79er3wwguSpHLlyunFF1+Uw+G4cKzW\nhAoAgHctXrxYeXl5Sk5O1uDBg5WYmFjoeEJCgsaPH6958+YpOjpa+/btK7ZNKk0AgFdZNTy7Zs0a\nRUdHS5IaNmyojIwM97Ht27erbNmymjlzpn799Ve1atVKV199dbFtUmkCALzKZrv4z4VkZmYqNDTU\nvR0YGCiXyyVJ+v3337V27VrFx8dr5syZ+u6777Ry5cpiYyVpAgD8ktPpVFZWlnvb5XLJbj+T9sqW\nLavq1aurRo0aCgwMVHR0dKFKtCgkTQCAV9lttov+XEjjxo21ZMkSSdLatWtVu3Zt97Fq1aopOztb\nu3fvlnRmKPeaa64pNlbuaQIAvMqqe5oxMTFasWKF4uLiJEmJiYlKTU1VTk6OunXrpnHjxmnQoEGS\npBtuuEG33HJLsW2SNAEAfslms2n06NGF9tWoUcP9/aabbtIHH3zwl9okaQIAvMqH1jYgaQIAvMuX\nVgRiIhAAACZRaQIAvMqHCs2ik2ZSUtIFf/joo49e8mAAACVPcY+OXE4YngUAwKQiK81zK8ns7Gzt\n2rVLtWvX1qlTpxQSEuKR4AAA/s+HCs3iK83vv/9ed911lx5++GEdOXJErVu31vLlyz0RGwCgBLDZ\nbBf98bRik+bkyZM1f/58hYWFqXz58po7d64mTJjgidgAALisFDt71uVyKSIiwr1tZm0+AADM8qXh\n2WKTZsWKFZWeni6bzaYTJ05o3rx5qly5sidiAwCUAH61uMGYMWO0YMEC7d+/X7fddps2bdqkMWPG\neCI2AAAuK8VWmldddZUmT56szMxMBQYGKjg42BNxAQBKCB8qNItPmlu2bNEzzzyjffv2SZKioqL0\nwgsvqHr16pYHBwDwf361uMHIkSM1cOBArVy5UitXrtQDDzygYcOGeSI2AAAuK8Umzdzc3EIv5oyJ\niVFmZqalQQEASg6b7eI/nlbk8OzZ4dg6depo+vTp6tq1qwICArRgwQI1bdrUYwECAPybL82eLTJp\n9uzZUzabTYZhaOXKlUpOTnYfs9lsGj58uEcCBADgclFk0kxLS/NkHACAEsqHCs3iZ89u27ZN8+fP\nV3Z2tgzDkMvl0p49ezRv3jxPxAcA8HO+NDxb7ESgJ554QmFhYdq0aZPq1q2ro0ePqlatWp6IDQCA\ny4qptWcff/xxFRQUqF69eoqLi1NcXJwnYgMAlAA+VGgWX2mWLl1aeXl5uvrqq7Vx40Y5HA7l5uZ6\nIjYAQAngV68G69ixo/r166dWrVpp7ty56tOnjypUqOCJ2AAAuKwUOzzbs2dPderUSU6nU3PmzNGG\nDRvUsmVLT8QGACgBfGl4tsikmZSUVOSPtmzZokcffdSSgAAAJYtfzZ4FAABnFFlpUkkCADzBhwrN\n4u9pesu8z0d7OwTgb+tw25PeDgG4JL75uZVlbfvSq8Eu26QJACgZfChnmrunmZ2drc2bN8swDGVn\nZ1sdEwAAl6Vik+b333+vu+66Sw8//LAOHz6s1q1ba/ny5Z6IDQBQAvjV4gaTJ0/W/PnzFRYWpvLl\ny2vu3LmaMGGCJ2IDAOCyYmrt2YiICPf2NddcY2lAAICSxZfuaRabNCtWrKj09HTZbDadOHFC8+bN\nU+XKlT0RGwCgBLDZfSdrFjs8O2bMGC1YsED79+/Xbbfdpk2bNmnMmDGeiA0AUALYbBf/8bRiK82r\nrrpKkydP9kQsAABc1opNmq1bt/7TGUrffPONJQEBAEoWX1p7ttikOWfOHPf3goICff3118rLy7M0\nKAAALkfF3tOsUqWK+xMZGak+ffpo8eLFnogNAFAC+NU9zdWrV7u/G4ahX3/9Vbm5uZYGBQAoOfxq\nePbVV191f7fZbLryyis1fvx4S4MCAJQcPpQzi0+abdu21T333OOJWAAAuKwVe09z/vz5nogDAFBS\n+dBNTVMrAvXq1UsNGzZUUFCQez8vqQYAlDTFJs1GjRp5Ig4AQAnlFxOBUlJSFBsbS0UJALCUD+XM\nou9pzp4925NxAABKKJvddtEfTyt2IhAAADijyOHZX3/9VW3atDlvv2EYstlsrD0LALgkfGl4tsik\nGRkZqenTp3syFgAALmtFJs1SpUqpSpUqnowFAFAC+cXs2caNG3syDgBACeVDObPopJmQkODJOAAA\nJZQvVZrMngUA+CXDMDRy5EjFxcWpV69e2r1795+el5CQoMmTJ5tqk6QJAPAqq5aeXbx4sfLy8pSc\nnKzBgwcrMTHxvHOSk5P1yy+/mI6VpAkA8Etr1qxRdHS0JKlhw4bKyMgodPynn37Shg0bFBcXZ7pN\nkiYAwKtsNttFfy4kMzNToaGh7u3AwEC5XC5J0uHDh5WUlKSEhAQZhmE61mIXbAcAwFIWlW9Op1NZ\nWVnubZfLJbv9TGdffPGFjh8/rr59++rw4cPKzc1VVFSUOnXqdME2SZoAAK+yavZs48aNlZ6erjvu\nuENr165V7dq13cfi4+MVHx8v6cwLSrZv315swpRImgAAPxUTE6MVK1a471kmJiYqNTVVOTk56tat\n20W1SdIEAHiVVY9p2mw2jR49utC+GjVqnHdebGys6TaZCAQAgElUmgAAr/KlFYFImgAAr/KhnEnS\nBAB4mQ9lTe5pAgBgEpUmAMCrbHYqTQAA/A6VJgDAq3zoliZJEwDgXTxyAgCAST6UM7mnCQCAWVSa\nAADv8qFSk0oTAACTqDQBAF7lS89pkjQBAF7lQ6OzJE0AgJf5UNbkniYAACZRaQIAvMqHCk0qTQAA\nzKLSBAB4FbNnAQAwibVnAQAwy3dyJvc0AQAwi0oTAOBVvjQ8S6UJAIBJVJoAAK/ypUqTpAkA8C4f\nGvMkaQIAvMqXKk0fyu8AAHgXlSYAwKuoNAEA8EOWV5q7d+9Wenq6cnNz3fv69u1rdbcAAF/hO4Wm\n9ZXmww8/rD/++EMOh8P9AQDgLJvddtEfT7O80qxUqZIee+wxq7sBAPgqH7qnaXnSvPXWWzVx4kRd\nc8017n2dOnWyulsAAC45y5PmwoULFRUVpa1bt0ryrVlSAADr+VJasDxpOhwOjR492upuAACwnOVJ\ns3Llypo2bZrq1avnrjJbtmxpdbclzrLvV+q1Ge+ooKBAtaJqaOSQJxRSuvRfPmdwwhiVL1dOTz/+\nsCRp9U/r9NLUN3Xa5VLZsDANfuRB1a4Z5bHrQsl20z8b698D71VgqUBt+2WnJo6YolPZpwqd0+ne\ntrqrR1vlnsrVzm179NpzM5R5IkulHKU0YERfXXvdNZJN2rz+V7069k3l5xd46WpQFF8agbR89mxB\nQYF27NihhQsX6vPPP9fnn39udZclzu9//KFRL76kyWMT9PGsN1W5UkW9Mu2tv3zOO+9+oLUZP7u3\nM7Oy9OTIsRr0cF+9N2OKhg58RE+Pfl75BfxLB9YLKxuqp557RCMfn6AHOgzQgT2H9OCgnoXOaXTj\ndbr7gbs0+L4E9ev6lFYv+0mDRveTJN37UBfZ7Xb1jR2kvp0GKSg4SD0e7OyNS0Fx7LaL/3g6VKs7\nSExMLPR54oknrO6yxPlh9Y+6rs61qlq5kiSpW8c7tXBx+l86Z/VP6/T9f9aoa4c73ft27dmnUKdT\nTRs1lCRdXb2aypQJ0fqNm6y+JEBNWzTU5g2/af+eg5Kkz977Um3a/7PQObXq1dCP36/XsSPHJUnL\nvv5BzVs1lT3ArvX/+Vlzp33oPve3zdtVoXKE5y4Aptlstov+eJrlSfOVV15Rs2bN1KRJE9WvX1/3\n33+/1V2WOAcOH1aFiP/+y6BCRDll5+QoOyfH1DmHjhzVxNen6flnn5b9nP9yi6xWRdk5OfphzY+S\npI2bt2jrjp06cvSYB64KJV35iuV06MAR9/aRA0dVukywgkOC3fs2b/hNN9x0vSIqXiVJuqNzGwUE\nBuiKsqH68fv12rfrwJm2KkeoS/ydWvLFd569CPgdy5NmWlqali5dqg4dOmjhwoWqUKGC1V2WOC6X\n60/32+32Ys8xDENDx47XU48+pKvCryx0rExIiF56bqTempusuL6P6POv03Rj40YqVYoli2G9oh5c\nd53+79/lDWs2afaU9zXmtaf1evJ4nT59Wif/yCx037JWvSi9PHusUuYu1KplP1keNy6C7W98PMzy\nf/tFRETI4XAoKytLkZGRys/Pt7rLEqdS+fLK2LTFvX3w8BGFOZ0KDgoq9pxtO3Zp38GDmjRlugxD\nOnrsmFyGoby8fI14coBKBwfrzZcmuH/X+b4HVa1KZc9cGEq0Q/uPqG6D2u7tchWvUuaJLOXl5rn3\nBYcEa/1/ftaXKWduNZQND9P9j8Up80SWJOnWti302PA+enXsm/qWKhOXgOWVZsWKFfXhhx+qdOnS\nmjRpkk6cOGF1lyVOs380Vsamzdq9d58k6aMFC3VLi2amzrm+Xh0tSp6td6e/ruQ3X1fXjnfq9lb/\n1IgnB0iSHnsmQT9v+VWS9PW3S1UqMFC1omp48OpQUv1nxTrVaVBLlaqdGZ3qcHeMvktbXeicchFX\navI7Y1T6/4dse/bvprTPl0uS/nl7Mz0y9AE93WcMCfMy50v3NG2GYRhWduByuXTgwAGFhYUpJSVF\nzZs3L7Q6UFGy9223Miy/s2LVf/Tq9LdVUFCgqlUq6bmhT2n33v0aO+llvTv99SLPCXU6C7UzbdZc\nHf/jhPuRkx/XZ+jFpKkqKDitcleFa8Tgx1W5YkWPX5+v6nDbk94Owaf9o2Uj9RnUU4GBgdq364DG\nD31VlatV1OAx/dWv61OSpI5x/1Kne9tKsinjx0167bkZys8v0KyFr6mMM0RHDh07M4xnSBk/bVbS\nuLcu2Cf+3Dc/f2RZ27tTF170b6u1b3cJIyme5Unz6NGjmjp1qnbs2KFatWqpX79+uuKKK4r9HUkT\n/oCkCX9hadL8fNFF/7banW0vYSTFs3x4duDAgYqKitKTTz6pqlWrasiQIVZ3CQDwIb40POuRaZD3\n3HOPJKlOnTr64osvPNElAACXnOWVZlRUlD799FMdPHhQaWlpKlu2rLZv367t2xl+BQD4FssrzW3b\ntmnbtm16++23FRAQoDJlyighIUE2m02zZ8+2unsAwOXOd5aetS5pbty4Uc8++6zef/99ffvttxo5\ncqTCwsL0yCOPqE2bNlZ1CwDwMUUtZHE5smx4dsKECRo/frwcDodefvllzZgxQx999JHefPNNq7oE\nAPgim+3IHQZEAAAMR0lEQVTiPx5mWaXpcrlUp04dHTx4UDk5Oapfv74k33oFDADAer6UFyxLmoGB\nZ5petmyZmjdvLknKz89Xdna2VV0CAOBmGIZGjRqlLVu2yOFwaNy4capWrZr7eGpqqmbPnq3AwEDV\nrl1bo0aNKrZNy4Znmzdvrri4OCUlJSk+Pl67du1S//791a6dZ1dvAACUTIsXL1ZeXp6Sk5M1ePBg\nJSYmuo/l5ubq1Vdf1dy5czV//nydPHlS6enpF2jtDMsqzQcffFBt2rSR0+lUhQoVtGvXLnXv3l0x\nMTFWdQkA8EUWTQRas2aNoqOjJUkNGzZURkaG+5jD4VBycrIcDockqaCgQEHnvOSiKJY+clKzZk33\n9+rVq6t69epWdgcA8EFW3dPMzMxUaGioezswMFAul0t2u102m03h4eGSpDlz5ignJ0c333xzsW3y\nYkQAgHdZNA/I6XQqKyvLvX02YZ5lGIYmTJignTt3KikpyVSblq8IBADAhVi19mzjxo21ZMkSSdLa\ntWtVu3btQsdHjBih/Px8TZkyxT1MWxwqTQCAX4qJidGKFSsUFxcnSUpMTFRqaqr7MciPP/5YTZo0\nUXx8vGw2m3r16qXbbrvtgm2SNAEAfslms2n06NGF9tWoUcP9/eeff/7LbZI0AQDe5UPL6JE0AQBe\nxYpAAACYRdIEAMAcX6o0eeQEAACTSJoAAJjE8CwAwLuYPQsAgDm+dE+TpAkA8C6SJgAA5th8aHiW\niUAAAJhE0gQAwCSGZwEA3sU9TQAAzGH2LAAAZpE0AQAwh9mzAAD4IZImAAAmMTwLAPAu7mkCAGAS\nSRMAAHN45AQAALOYPQsAgP8haQIAYBLDswAAr7LZfKd+I2kCALyLiUAAAJjD7FkAAMxi9iwAAP6H\npAkAgEkMzwIAvIp7mgAAmEXSBADAJJ7TBADAHBuzZwEA8D8kTQAATGJ4FgDgXUwEAgDAHB45AQDA\nLGbPAgBgDrNnAQDwQyRNAABMYngWAOBdTAQCAMAcZs8CAGAWs2cBADCJ2bMAAPgfkiYAACYxPAsA\n8ComAgEAYBYTgQAAMIdKEwAAs3yo0vSdSAEA8DKSJgAAJjE8CwDwKqteDWYYhkaNGqUtW7bI4XBo\n3Lhxqlatmvt4WlqapkyZosDAQHXp0kXdunUrtk2SJgDAuyyaCLR48WLl5eUpOTlZ69atU2JioqZM\nmSJJKigo0Pjx4/Xxxx8rKChIPXr0UJs2bRQeHn7BNhmeBQB4lc1mv+jPhaxZs0bR0dGSpIYNGyoj\nI8N9bOvWrYqMjJTT6VSpUqXUpEkTrV69uthYqTQBAN5lUaWZmZmp0NBQ93ZgYKBcLpfsdvt5x8qU\nKaOTJ08W2+ZlmzRDKtfwdgjA3/bNzx95OwTgsucIu8qSdp1Op7KystzbZxPm2WOZmZnuY1lZWQoL\nCyu2TYZnAQB+qXHjxlqyZIkkae3atapdu7b7WM2aNbVz506dOHFCeXl5Wr16tRo1alRsmzbDMAzL\nIgYAwEvOnT0rSYmJidq4caNycnLUrVs3ffvtt0pKSpJhGOratat69OhRbJskTQAATGJ4FgAAk0ia\nAACYRNIEAMAkkqafWrVqlQYNGlRo36RJk/TJJ594KSLAvOnTp+v+++9XfHy8evfurY0bN3o7JEDS\nZfycJv4+X3pHHXDW1q1blZaWpuTkZEnS5s2b9cwzz/AffLgskDT92J9NjD59+rSGDx+uAwcO6PDh\nw2rdurUGDBigoUOHKjAwUPv27VNeXp7atWun9PR07d+/X1OmTCm0yDFgJafTqQMHDujDDz9UdHS0\n6tSpo/fff1/x8fGKiorStm3bJEkvv/yyrrzySiUkJPD3GR7D8Kwf++GHH9SrVy/16tVL8fHx+vzz\nzxUQEKBGjRppxowZ+uCDD/Tuu++6z69atareeustRUVFae/evZo+fbpuv/12paene/EqUNJUqFBB\nU6dO1Y8//qi4uDi1a9dO3377raQzD6vPmTNHbdu21dSpU3XgwAH+PsOjqDT9WPPmzTVp0iT39uTJ\nk5WZmalffvlFK1euVJkyZZSfn+8+Xq9ePUlSWFiYatas6f6em5vr2cBRou3atUtlypTR888/L0na\nuHGj+vTpo/Lly6tZs2aSziTPtLQ0hYWFaf369fx9hsdQaZYghmHIMAxdccUVevHFF3X//ffr1KlT\n7uPcA8XlYMuWLRozZow7AUZGRiosLEwBAQHuCUFr1qxRrVq1lJKSwt9neBSVZglis9kUEBCgZcuW\nae3atSpVqpSuvvpqHTp06LzzAG+JiYnRtm3b1LVrV4WEhMgwDA0ZMkTvvPOOUlJSNHPmTIWEhGjC\nhAk6fPiwBg8ezN9neAzL6AHwCfHx8RozZoxq1OANSPAehmcB+AQqRlwOqDQBADCJShMAAJNImgAA\nmETSBADAJJImAAAmkTThV/bu3avrrrtOsbGxio2NVfv27fXvf/9bBw8evOg2U1JSNHToUEnSQw89\npMOHDxd57muvvaY1a9b8pfbr1Klz3r6kpCQlJSVd8HetW7fWvn37TPdjpk0AF0bShN+pUKGCUlJS\nlJKSotTUVNWvX19jx469JG1PmzZNERERRR5ftWqVXC7XX2rzYh+l4BEMwPNYEQh+r2nTpu5Fulu3\nbq2GDRtq8+bNmjdvnpYuXarZs2fLMAzVr19fCQkJcjgc+uSTT/TGG28oNDRUlSpVUpkyZdy/nzt3\nrsqVK6fRo0drzZo1KlWqlPr376+8vDxlZGRo+PDhSkpKUlBQkEaNGqXjx4+rdOnSGj58uOrWrau9\ne/fqqaeeUk5Ojho0aFBs/HPnztVnn32mnJwc2e12vfTSS4qKipJhGHrttde0efNmBQUFafTo0br2\n2mt19OhR95s/7Ha7Bg0apObNm1v6ZwyUFFSa8Gv5+flatGiRGjdu7N53yy23aNGiRTp27Jg++OAD\nJScnKyUlReHh4Xr77bd16NAhTZw4UfPnz9d7772nrKws92/PVndz5sxRTk6OvvjiC82cOVNTp05V\n+/btdd1112ncuHGqVauWnn76aQ0ZMkQff/yxxowZoyeeeEKSNHbsWHXp0kUpKSmF4vozmZmZSktL\n09y5c7VgwQK1adNG8+fPdx+vUaOGUlJS1L9/fz3zzDOSpHHjxqlr16766KOPNGXKFCUkJCg7O/uS\n/ZkCJRmVJvzOwYMHFRsbK8MwlJ+frwYNGmjw4MHu42eru5UrV2rnzp3q3r27DMNQQUGB6tWrp59+\n+kmNGzdWeHi4JKljx4764YcfJP33HaWrV69W9+7dJUnlypXTggUL3O0bhqHs7Gxt2LBBQ4cOdf/m\n1KlTOn78uFauXKnJkye72x4+fHiR1+J0OjVx4kSlpqZqx44dWrZsmerWres+3rVrV0ln/kNgyJAh\nyszM1Hfffaft27frlVdekXTmHaq7du36G3+iAM4iacLvnL2nWZTg4GBJZ5JJ27Zt9eyzz0qScnJy\nVFBQoO+//77QfcnAwP/+Y3K20jx3n3TmdVaVKlVyb7tcLgUHBxeK4+DBgypbtqzsdru7fZvNJru9\n6AGfAwcOKD4+Xj179tQ///lPlStXTps2bXIfDwgIKHR+YGCgXC6XZs2apbCwMEnSoUOHVK5cOS1e\nvLjIfgCYw/As/I7ZlSFvvPFGLV68WMeOHZNhGBo5cqRmzZqlJk2aaN26dTp06JBcLpcWLlx4XttN\nmzbVokWLJElHjx5VfHy88vPzFRgYqIKCAjmdTkVGRuqzzz6TJK1YsUI9e/aUJN1888369NNPJUlf\nfvml8vLyioxxw4YNioyMVO/evdWgQQMtXbq0UEI/W+F+/fXXioqKUnBwsJo1a6Z58+ZJkn777Td1\n7Nix0CuzAFw8Kk34nQvNKj33WJ06dfTII4+od+/eMgxDdevW1YMPPiiHw6ERI0bovvvuU0hIiK65\n5przfn/PPffoueeeU8eOHWWz2TRixAiFhIQoOjpao0aN0gsvvKCJEycqISFBM2bMkMPh0MsvvyxJ\nGj58uIYMGaL3339f119/vZxOZ5HxtmzZUu+++67uvPNOBQUFqUGDBvr111/dsezYsUOdOnWS0+nU\nCy+84G4/ISFBHTt2lCRNnDhRISEhF/mnCeBcLNgOAIBJDM8CAGASSRMAAJNImgAAmETSBADAJJIm\nAAAmkTQBADCJpAkAgEkkTQAATPo/5VStOlul3oUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xda0cfd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "print(clf.predict(X_test))\n",
    "print (clf.score(X_test,y_test,sample_weight=None))\n",
    "cm_test = confusion_matrix(y_test, clf.predict(X_test))\n",
    "cm_test_norm = cm_test /cm_test.sum(axis=1) #axis 1 means the actual data\n",
    "print (cm_test_norm)\n",
    "labels=['Ham','Spam']\n",
    "plot_confusion_matrix(cm_test_norm,classes=labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**d)** Pick one (perhaps random) attribute that has higher probability for the ham class (using your feature names in Question 10c) and set the corresponding value in `X_test` to 1. Now predict the new outcome. Has it changed? If not, keep modifying more attributes until you have achieved the desired outcome (i.e. model classifies the e-mail as ham)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      "1.0\n"
     ]
    }
   ],
   "source": [
    "# Your code goes here\n",
    "#selecting attribute that was a spam (output 1), attribute no 14\n",
    "#y_test[ytest==1]\n",
    "\n",
    "#selecting feature that have high log prob to 0 (ham),word_freq_will , idx 11\n",
    "#from answer 10C\n",
    "\n",
    "X_test_2 = X_test.copy() #copy the data to save initial dataframe\n",
    "X_test_2.set_value(14, 'word_freq_will', 1.00) #set the value of attribute no 14 column 11 to 1\n",
    "\n",
    "output =  clf.predict(X_test_2) #insert prediction to array\n",
    "print(output[14]) #still output 1\n",
    "\n",
    "#word_freq_direct\n",
    "X_test_2.set_value(14, 'word_freq_original', 1.00)\n",
    "X_test_2.set_value(14, 'word_freq_project', 1.00)\n",
    "X_test_2.set_value(14, 'word_freq_re', 1.00)\n",
    "X_test_2.set_value(14, 'word_freq_edu', 1.00)\n",
    "X_test_2.set_value(14, 'word_freq_table', 1.00)\n",
    "\n",
    "output =  clf.predict(X_test_2)\n",
    "print(output[14]) #still output 1\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========== Question 12 =========="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**This is an extension for people keen to learn more advanced plotting.** We'll be happy to discuss your conclusions in the lab."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**a)** Create a plot of the spam/ham log probabilities for all of the features. This will help you find the spammiest/hammiest words to use in your emails! *Hint*: you can do this however you like, but try 'adapting' [this matplotlib demo](http://matplotlib.org/examples/api/barchart_demo.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfMAAAFeCAYAAACRjfZ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVPX+P/DXbIDKoiiaO4osCYJb7prQ4so3d7yK3rRF\nrdRSr2WZS2rcMq1Mb9rtVpb5y3LLupVFmKWSJve6ZC5ZiIKIyibDNjDz+f3BPccZmGEAGWYOvJ6P\nhw+Z+cz5nPf5zJnzPp/P2VRCCAEiIiJSLLWzAyAiIqI7w2RORESkcEzmRERECsdkTkREpHBM5kRE\nRArHZE5ERKRwTObUoKWlpaFHjx4Oqz8qKgojRozA2LFjMXbsWERHRyMuLq7a9WzcuBGrV6+u1jTH\njh1DdHS01bINGzbg888/BwCEhIQgJycHCQkJWLNmDQDg4MGD2LBhQ7XjtMZkMmHOnDkYPnw4Pv74\n4wrl7777LsaMGYMxY8YgOjoar7zyCkpKSmpl3lUxefJk6PV6TJs2Dd9++61FmaPXD6LaonV2AETO\nplKpHFr/unXr0LVrVwBAaWkpYmNj8fHHH2Pq1KkOnW9l5s2bJ/8tLX9UVBSioqIAAKdPn8atW7dq\nZV7Xrl3DkSNHcOLEiQpt/c033yA+Ph6ffvop3NzcYDAYMG/ePGzcuBHPPPNMrcy/MhkZGWjSpAk8\nPT1tfsbR6wdRbWAyJ7JBr9dj5cqVOHfuHFQqFQYPHoyFCxdCrVbj4MGDeO2116DVahESEoIjR47g\n//2//4c2bdpUqMf8vkxarRa9evVCcnIy0tLSMHXqVAQEBCAtLQ3btm3DiRMnsGnTJphMJnh6euLZ\nZ59FeHg4AODixYuIjY1Fbm4uunbtiuXLl6Nx48Y4cOAAtmzZgtLSUmRlZeGhhx7C/PnzAQD5+fmY\nN28eLl++DG9vb6xatQodO3bEkiVLEBQUhBkzZsjx7dmzB/v378cTTzyBTz75RI7h1KlTGDFiBCZO\nnAgA2Lx5M3JycvDcc89ZLOfx48exdu1aFBUVQafTYf78+ejZsycee+wxlJaWYty4cdiwYQPat28v\nT3Pjxg2YTCYUFhbCzc0Nbm5uWLZsGTIzMwEAS5YsAQD8+eefyM7OxoABA/Diiy9Co9Fg586d+PTT\nT1FaWoqcnBw8/vjjmDx5srwcxcXFSEtLQ+vWrTF16lRs27YNKSkpePjhhzFjxgwAQHx8vLwDY435\nd1dYWIgVK1YgJSUFOTk5aNKkCdatWwd/f39MmzYNYWFh+Pnnn5GVlYVp06YhMzMTx44dQ1FREd54\n4w0EBgZWcc0jqgFB1IClpqaKHj16WC179tlnxZo1a4QQQhgMBjFz5kzxzjvviOzsbNGnTx9x/vx5\nIYQQe/bsESEhISItLa1CHZGRkeLXX3+VX1+7dk2MGDFCfPfddyI1NVUEBweLpKQkIYQQf/zxhxg4\ncKBITU0VQgiRmJgoBg4cKPR6vXjrrbdEZGSkyM7OFkIIsWjRIvHaa68JIYSYPn26SElJEUIIkZGR\nIbp27Sqys7PF0aNHRdeuXcWJEyeEEELs2LFDTJw4UQghxHPPPSfee+89IYQQwcHBIjs7W+zevVvM\nmjVLCCHEW2+9JVatWiWEEOK7774TEyZMEEIIYTKZRFRUlLh06ZLFcmZnZ4sBAwaIU6dOCSGE+P33\n30Xfvn1FampqpW2cl5cnZs6cKUJDQ0VMTIz4+9//Ln755Re5/LnnnhNjx44VhYWFwmAwiNjYWLFt\n2zaRn58vYmJiRE5OjhBCiBMnTsjz2L17t7jnnnvEtWvXhBBCjBo1SsyfP18IIcTZs2dFeHi4XP/M\nmTNFenq6EEKI2NhYcd9994kxY8bI/0aOHCnX+80334jVq1fL0y5btkxuo9jYWDF37lwhhBAnT54U\nwcHB4ocffhBCCPHyyy+LF1980eryE9UW9syJbPjxxx/xySefAAB0Oh3+8pe/YOvWrejUqRMCAwMR\nFBQEABgzZkylx7MXLVoEDw8PGI1G6HQ6TJo0Cffffz/S0tKg1WrRvXt3AMDPP/+M/v37o23btgCA\nfv36oUWLFjhz5gwA4MEHH0TTpk0BAOPGjcPatWuxcOFCvP322/jhhx+wb98+/PnnnwDKepEAEBwc\njIiICADA2LFjsXLlSuj1+mq1Q1RUFF5++WWcP38eGRkZaN++PTp27GjxmZMnT6Jjx47o1q0bAKBL\nly7o2bMnjh07hj59+tis29PTE//617+QmpqKo0eP4tixY5g1axamTJmChQsXysvq4eEBAHjooYfw\n/fffY+rUqdi8eTMOHDiAlJQUnD17Vl5mAOjWrRtatWoFAGjXrh0GDhwIAOjQoQMMBgMKCwthNBqR\nn5+Pu+66S55u8eLFePDBB+XXaWlp8nkHw4YNQ/v27eUe/rFjxyyOp0vTtW/fHiqVCoMGDZLneezY\nseo0OVG1MZkT2SDKPbbAZDKhtLQUWq0WJpPJokyttn0uqfkx8/Lc3NzkacvPDwCMRiNKS0utzkOn\n06GwsBBjxozBgw8+iN69e2PChAmIj4+X6zKfRggBlUoFnU5nM1Zr1Go1Jk+ejJ07d+L69euYPHly\nhc9Yi11qr8q8++676NWrF3r06IF27dph/PjxSEpKwmOPPSYnc41GYzEfjUaDjIwMxMTEICYmBr17\n98awYcNw8OBBi7YxZ22ZDx48iCFDhlS+8Ga2b9+Ozz77DLGxsYiOjoaPjw/S0tLkcjc3N4vPm8dN\n5Gg8m50aPGuJCAAGDRokn31tMBiwY8cODBw4ED169EBKSgouXLgAANi/fz/y8vJsnihlq/7yZf36\n9cPhw4eRmpoKAEhMTERGRoZ8zDwhIQF5eXkwGo3YsWMHBg8ejJSUFBQUFODpp5/G0KFDcfToUZSU\nlMBoNAIAzp07h3PnzgEAduzYgZ49e8Ld3d1um2g0GoszyqWdhN9++w0PPPBAhc9HREQgOTkZp0+f\nBgD8/vvvSEpKQt++fSttg6KiIqxfvx65ubnye3/++SdCQ0Pl11999RUMBgOKi4uxZ88eREZG4vTp\n0/D19cWcOXMwcOBAHDhwoNL5mJM+Ex8fj/vuu8/u5yWHDx/GuHHjMH78ePj7++PAgQMVdurKz4Oo\nrrBnTg1eUVERevbsCeB273XHjh144YUXsGrVKkRHR6OkpARDhgzB7NmzodVq8dprr2Hx4sVQq9UI\nCwuDRqORh4LN2TsT2rw8ICAAy5cvx1NPPQWj0YhGjRph8+bN8pnWXbp0weOPP468vDz07NkTjz/+\nOHQ6HYYOHYrhw4fD29sbHTt2RJcuXXD58mXodDoEBARg06ZNuHz5Mlq0aIFXXnmlSjH2798fc+fO\nhU6nw9KlS+Hr64uwsDAEBARY7XE2a9YMb775JlatWoXCwkJoNBrExcWhQ4cOSEtLs9kOTz75pNzz\nV6vVMBqN6NatG9544w35M40aNcLUqVNx69YtDB8+HOPHj0dRURF2796NYcOGoUmTJujWrRt8fX2R\nkpJSaXtLy2symZCcnIzg4OBK28HczJkzsWzZMuzevRtqtRqhoaHyDl35aXkGPNU1leAuJFG16PV6\nvP3225g3bx7c3d3x22+/YdasWfjpp5+cHZrDZGVlYdKkSfj444/lY9F1wfyseyKyzWV65kIIrFix\nAufPn4ebmxvWrFljcQkLkavw9PSETqfD+PHjodVqodPp8Oabbzo7LIf57LPP8Prrr2P27Nl1msiJ\nqOpcpmf+3XffISEhAXFxcTh58iS2bNmCf/zjH84Oi4iIyOW5zAlwSUlJGDx4MICyk2l+/fVXJ0dE\nRESkDC4zzK7X6+Hl5SW/li7/sXXJz36VCp0A/AQg8xWgdWsgPR0YM+Y8ACA5OBiplZSZTwuUlQUF\nBeHChQt1Pq21sjtZHraF8paHbcH1vCEsjyu1hXSfiPrCZZK5p6cn8vPz5deVJXIA6AQgCEAyALfW\ngHR4PStLL5ejkjLzaaWyGzfykJWlr/Npa3t52BbKWx62BdfzhrA8rtQWN27kQYn8/Lysvu8yw+w9\ne/aUb/pw4sSJerfXRERE5Cgu0zN/4IEHcPjwYfnuUjV5TCQREVFD5DLJXKVSYeXKlc4Og4iISHFc\nZpidiIiIaobJnIiISOGYzImIiBTOZY6ZExER2WM0ll0r7u5e9lAd6Xl7qQAy08v+lsouX05Bbrky\n82kl/v6dFf/IWiZzIiJSjKtXgYhngU4YZ/H+YAB4VnpVVub7v1edLMpulwNl16dfSkxCQECgA6Kt\nO0zmRESkKNKNYmpLlp3y//43CXv37sLKlS/L723evBEdO/pjxIjRtRhJzfGYORERkR2u/ox69syJ\niIjssPaAUZPJhFdeWY3r168jM/MmBg0agkcfnY2XX14JjUaLjIx0GAwG3H//gzh8+Cdcv56BuLh1\naNOmba3Hx545ERGRHf/5z3HMmzcb8+bNxty5sxAfvx9qtRqhod2wbt0GvPPOB9i7d6f8+TZt2mD9\n+o3w9++E9PR0rF37JoYMicThwz85JD72zImIiOzo1eserFixRn69ZcsmFBTk488//8B//3scjRo1\nQUlJiVweFBQCAPD09ELHjv4AAC8vbxgMxQ6Jj8mciIgUJbmW6/KpwXRCCAgh4OXljb/97Xmkpl7B\nF1/skcvr+hg7kzkRESlGmzZA0itArv9uAIBPzLgKzzP399+NDh064vLlFPjEjKvwPHOpHChL5P7+\nnasdh0qlgkajxdGjR/Drr6eg0+nQvn1H3Lx5s8Ln6gKTORERKYZGU5aUpWTsi4rPM+/QoaN83bh0\nrbn588zNy6uiR49e6NGjl8V7s2Y9CQAYO3ZChc8///zyCp8DgEmT/lLleVYXT4AjIiJSOCZzIiIi\nhWMyJyIiUjgmcyIiIoXjCXBERKQYfGqadUzmRESkGFevlv1fXFz25LPrHwLXAbgB+N/J7CguHoff\nf4dcbl7WurVleXo6APCpaURERHWqtdllZnVl27YPcPz4MZSWlkKj0eCJJ+YjODikboOoBJM5ERFR\nJS5dSsbhwz/i7bffAwBcvPg71qxZjvff3+7kyG5jMiciIqqEp6cnMjIy8OWXn6NfvwHo0iUQ77yz\nFXPnzkLHjv5ISbkEAHjppTj4+DTF2rUv1/mT1Hg2OxERUSVatPDDK6+sx+nTJzFr1gzExk7EkSNl\nTz/r1i0Cb721BVFRD2Dr1vdw/fp1pzxJjT1zIiKiSqSlpaJx4yZYsmQZAOD8+XNYuHAuWrTwQ69e\n9wAAwsLCcejQj/D29sLZs2fq/ElqTOZERKQo6em1W1egnRPZL178Hfv27cErr6yHVqtFu3bt4Onp\nBbVajXPnzmLQID+cPn0SnTp1xldffeGUJ6kxmRMRkWK0aSNdK147T00LDLT/1LR7743E5cuX8Oij\n09GoUSMAAk89NR87dmzH119/gU8+2YbGjRtj6dKXkJl5EytXLq3zJ6kxmRMRkWI446lpADBt2gxM\nmzbD4r0dO7Zj1qyn5FgAwNvbGx98UPEsd0c/SY0nwBEREdVAXT2rvCrYMyciIqqBDRs2OzsEGXvm\nRERECsdkTkREpHBM5kRERArHZE5ERKRwTOZEREQKx2RORESkcEzmRERECsdkTkREpHBM5kRERArH\nZE5ERKRwTOZEREQKx2RORESkcEzmRERECsdkTkREpHBM5kRERArHZE5ERKRwTOZEREQKx2RORESk\ncEzmRERECsdkTkREpHBM5kRERArHZE5ERKRwTOZEREQKx2RORESkcFpnBwAAer0eixYtQn5+PkpK\nSvDcc8+he/fuzg6LiIhIEVwimb///vsYMGAApk+fjuTkZCxcuBC7d+92dlhERESK4BLJfMaMGXBz\ncwMAlJaWwt3d3ckRERERKUedJ/OdO3di69atFu/FxcUhLCwMN27cwOLFi/HCCy/UdVhERESKVefJ\nfMKECZgwYUKF98+fP49Fixbh2WefRe/evWtcv6+vZ43L/Py8kJ1t/TOOnLYyzpi2vrVFfVueO5mW\nbWFZVp/aor4tz51MW5Xl8fPzqnTeSuMSw+wXL17E008/jTfeeAPBwcF3VFdWlh4A4FuDshs38pCV\npa/zaW2VOWva+tYW9W157mRatoVlWX1qi/q2PHcybVWW58aNPBtzdW22dkJcIpmvX78eBoMBa9as\ngRAC3t7e2LRpk7PDIiIiUgSXSOb/+Mc/nB0CERGRYvGmMURERArHZE5ERKRwTOZEREQKx2RORESk\ncEzmRERECsdkTkREpHBM5kRERArHZE5ERKRwTOZEREQKx2RORESkcEzmRERECsdkTkREpHBM5kRE\nRArHZE5ERKRwLvEIVKLakvy//1MBuDkzECKiOsRkTvWGv39nXEpMQhYAr8spuHRpHAAgMNC5cRER\nd7Qdjcmc6g2NRoOAgLLM7e/fGd27n0dWlh7+/p2dHBlRw1Z+R7u4eJyzQ6p3mMypXtJoNAgKCsKN\nG3nODoWowTPf0QaA33+vWT3p6bf/54ibJSZzIiukjQY3GESuoQMARHwDHx8/BAb+r7d/6U9nh+Uy\nmMyJyjHfaHCInsg1aAB06tQJzZq1dnYoLonJnKgcbjSISGl4nTkREZHCMZkTEREpHJM5ERGRwvGY\neQPHGzkQESkfk3kDxhs5EJEz8NLP2sdkfgeUvkKWv5HDjz+W/a/U5SEi1+fv3xm+vrfvzshrxWtH\ng0/mNU3I9e1a5PI/MKo7PNRBDYkj786o9A7WnWjQyfxOEnJ9uxaZtz91Dh7qIKod9a2DVV0NOpnX\nt4RMylNb96wmauga+vacl6aR06Wn3x4ec6ZkABdwe9ibiEgpGnTPnJyv/NCYs06GMR/u9gEQEBCA\nrKwCp8RCRFRdTObkVK4yNFZ+uFuj0TgxGiKi6uEwOxERkcIxmRMRESkckzkRUT2TjLL7FlDDwWPm\n9Yx0Vnh6esO8cUJ9wxvKUHVJJ3Pm8r4FDUqDTObJZv93cmYgtcz8zPDAQPBWiQpXlRvKSD0wJnqS\nmJ/MyfsWNBwNLpm7yiVIjrjtoKucGU61w94NZdgDqzqOcCib9P3Vp85XbWtwydwVLkFylWurSdnY\nA6sa3jJX2aTvD+D9HyrT4JK5K2APmqju8Ja5ymb+/fH+D7bxbHZyaenpwJUr1m/3WlkZEVFDwp65\ng/Cs8jtX2Ql91sqUgOsFETkCk7kD8Kzy2lHZ4QglHqpQ6g4IEbm+epHMK+vt1HZPqCqXtSkx0ZDj\ncb0gIkdRfDKvy6HYurqsjUOxyuaIyw6JiCqj+GRel0OxdXFZG4folY2XHRKRMyg+mdc3jhyKZY/f\n8TiUTkTOwGSuMDVNyDz5ioio/mIyV5A7GYJnj5GIHIm3zHUuJnMFcWZC5hA9EdmihFvm1vdtGJN5\nPeDop8DxpDxSAj5Bznlc/Za5DeEwI5O5wtXF5XIcoidXxyfIWVffe6NV1RC2YUzmCucKT4Gj+kdp\nxz/5BLmK/P07w9f3PLKy9BYjakr7bqlq7D5o5bHHHsPXX3+NkpIShwfzxx9/oHfv3jAYDA6fF1F9\nlwzgAso22tXh798ZPolJyEpMgteO3Q6IrGZqujwNlUajQVBQEAICAhEQEAiNRuOy3y3dObs988cf\nfxx79uzB2rVrce+992Ls2LEIDw+v9UD0ej1effVVuLu713rdRA3NnZyQ5IrHP5VwgpUSuOJ3S7XD\nbjK/5557cM8996CoqAjffPMN5s2bB09PT0yYMAFTpkyBm1vtDNQsW7YMCxYswBNPPFGlzzv6pC8i\nJatvG+36tjxEta1Kx8yPHj2Kzz//HIcPH8aQIUMwcuRIHD58GHPmzMG//vWvas1w586d2Lp1q8V7\nbdq0wahRoxAcHAwhRJXqaX/mN0CrQSeUnfRlfqw4O9vT4rN+fl6V1mVeXn5aia+vp9V6KpvWXr1S\neWVl9uq1pi6mtddOlZVXNt+6bIvyMdd02srma295zJl/tnxM5esxL6+szFa9VW2L6k5rS2VlVZnW\nWW1hLyZntIW9mMqrzvp2J21hrd6qxJWd7WnzOL61aau6XbVWbm97oWR2k3lkZCTatWuH8ePHY9my\nZfDw8AAA9OnTBxMmTKj2DCdMmFBhumHDhmHnzp347LPPcPPmTTzyyCP46KOPKq3Hz6+d/LdGo8GN\nG3ny66wsPXzNPmteVrEer0qnNX+/fD32prVXr1ReWVltL09tTWuvnSorB+ASbVE+5ppOC9R8eazV\nZS0m87Ly5ZWV2aq3qm1R3WmBim1hr6wq0zqrLezF5Iy2sBeTufLrsbW6zKe9k7awVm9V4vL2bll2\nHB8VD6GUn7Yqv8uabi+UwtYOid1kvmXLFgQFBVm8d+LECXTv3h179uypleD2798v/x0VFYX33nuv\nVuolIiLXxkMotcNmMk9KSoLJZMLSpUuxZs0aefi7tLQUK1assEjAtUmlUlV5qJ2opnj9LW+yQlSf\n2EzmR44cwbFjx3D9+nW8+eabtyfQahETE+OwgL7//nuH1U31g3R8raYnPjaEu0HZ46ybrPAaZyLH\nsJnM586dCwDYu3cvxowZU2cBkWu508RZ26QkBNT8bncN4W5Q9jjjJiu8vIzIcWwm87feegtz587F\n0aNHcfTo0QrlcXFxDg2MnK82EmdtM09CvNudsij12KijRhM4SkG1yWYyDw0NBVB21jpVnav1ZO8E\nEyc1dI4aTeAoBdU2m8k8JCQEV69eRd++fesyHkVzxZ4sEdWco0YTlDpKQa7LZjKPjY21eWa5SqXi\niWpWsCdLRFS3eDfQMjaTeUJCQl3GQUQKwsvaHI9tbF9dPAJaKeyeALdkyRKr5a5+Alx9OnatdLym\nu37hs8Mdz14b8+S5MnwE9G318gQ4Hrt2Hdau6b506c9Kp+GwmWvjs8Mdr7I25slzZI3NZB4VFQUA\nGDt2LDIzM3Hy5ElotVqEh4ejadOmdRZgTfDYteuo7jXdHDYjqpwSTp7jaFzds3tv9q+//hpr1qxB\nz549YTKZsGzZMrz00ksYMmRIXcRH9ZSt3jeHzUhpeGzbkr9/Z/j6nkdWlr7B3mHRGewm87fffhu7\nd+9Gy5YtAQBpaWmYM2cOkznVGHvfVF/w/IGKNBoNgoKC6sUTypTEbjLXarXw8/OTX7dt2xZabZUe\ng05kFXvfVF/w/AFyFTaz8t69ewEA7dq1w+zZszFmzBhotVp8+eWXCA4OrrMAiYiIaqqhXNlkM5lL\n92Nv0qQJmjRpgh9//BEA0Lhx47qJjCy4+gopnfDCk12IyFU0pCubbCbzyq4jLyoqckgwZJ2rr5Dm\nl5/xZBcichUN6comuwe/9+/fj02bNqGgoABCCJhMJhQVFSExMbEu4mswKru22tVXSD5SlMzdyQ1N\neGY4Uc3YTeZr167F6tWr8f7772P27Nk4dOgQsrOz6yK2BoNnd1N9cSc3NOGZ4UQ1ZzeZe3t7o1+/\nfvjPf/6DvLw8zJ07F+PG8YdWm3h2N9UXd3JDE6WfGc4bpZAz2U3mHh4eSE5ORkBAAI4dO4Z+/foh\nL4/XDxIRSazdKMXebYuViPeEd11qex94+umn8cYbbyAyMhKJiYkYOHAg7r///rqIjYhIEaQbpQQE\nBCIgILBejq75+3eGT2ISshKT4LVjt7PDoXLs9sz79OkjP2xl165dyM3NhY+Pj8MDIyIi16GEe8I3\nZHZ75teuXcNTTz2FPn36YODAgVi1ahWysrLqIjYiIiKqArvJ/Pnnn8eAAQOQkJCA/fv3IywszOYz\nzomIiKju2U3mWVlZmDJlCjw9PeHp6YmHH34Y165dq4vYiIiIqArsJvPw8HD8+9//ll8fOHAAYWFh\nDg2KiIiIqs7mCXAhISFQqVQQQuDTTz/FCy+8ALVajYKCAvj4+GDNmjV1GScRkeLwUi6qKzaT+blz\n5+oyDnIgV39IC1F9dCd3wyOqLruXphUWFmLjxo1ITEyE0WhEv379MH/+fD49TSFc/SEtRPUVL+Wi\numT3mPlLL72EwsJCvPzyy3jllVdQUlKC5cuX10VsVAukDUp9vZEFERFVoWd+5swZ7Nu3T369bNky\njBw50qFBERERUdXZ7ZkLIXDr1i359a1bt9jDIwBlx+Iv4PYxeSKiO5WeDly5cvvBNVQ1dnvmDz/8\nMCZOnIjIyEgAQEJCAh5//HGHB0aujY9tJaLaZu2BNVQ1dpN5ZGQkunXrhl9++QUmkwlvvfUWgoOD\n6yI2cmF8bCsR1TbpgTU3bvDJnNVlN5lPnToVX3/9NYKCguoiHiIiIqomu8k8JCQEe/fuRXh4ODw8\nPOT327Rp49DAiIiIqGrsJvOTJ0/i5MmTFu+pVCp8//33DguKiKimpBOn0tOBwMDKP0tUX9hN5gkJ\nCXURBxHRHeMJVNRQ2U3mV69exerVq/Hzzz9Dq9ViyJAheP755+Hr61sX8RERVRlPoKKGyu515osW\nLcKAAQPw448/Ij4+HmFhYXj22WfrIjYiolqXns5rmKn+sdsz1+v1iI2NlV8//PDD2L17t0ODIiJy\nBPNheA7BU31it2ceGhqKzz//XH79ww8/oGvXrg4NisiV8Q5VyiUNw/NZBVTf2O2Z//DDD9izZw+W\nL18OlUqFwsJCAMDevXuhUqlw9uxZhwdJ5Co6AEDEN/Dx8ZNPsLp06U9nh0VEDZzdZJ6YmFgXcRAp\nggZAp06d0KxZa2eHQkQkszvMTkREzsPDOlQVdnvmRETkHLxunqqKyZyoFiSb/d/JmYFQvcLr5qmq\n7CbzjRs3WrxWqVTw8PBAQEAAhg4d6qi4iBSDj4MlImezm8wvX76MlJQUjBo1CgDw7bffwtPTE0lJ\nSTh27BgWL17s8CCJXBkfB0tEzmY3mScnJ+Pjjz+Gm5sbAGDy5MmYNm0aduzYgf/7v/9jMiciInIy\nu2ez37p1C6WlpfLrkpISFBSUDSEKIRwXGREREVWJ3Z751KlTMX78eAwdOhQmkwk//vgjYmNj8cEH\nHyAoKKguYiQiIgeQLnfjo2KVz24ynz59Ovr27YvExERoNBps2LABgYGBuHTpEqZMmVIXMRIRUS3j\nferrF7tlsKdXAAAgAElEQVTJXAiBpKQkJCUlwWg0wmQyISAgAP7+/nUQHhEROQIve6tf7CbzV199\nFSkpKRg/fjyEENi9ezdSU1Pxwgsv1FoQJpMJcXFxOHPmDAwGA+bOnYt777231uonItcjDfGmpytn\nmJfD0uSq7Cbzw4cPY+/evVCry86VGzp0KKKjo2s1iM8//xxGoxHbt29HRkYG9u/fX6v1E5FrUeKd\nzTgsTa7MbjI3Go0oLS2VL00zGo21fh3toUOHEBgYiFmzZgEAli5dWqv1E5FrUeIQrxJjpobDbjKP\njo7G9OnT5ZvG/Pvf/5b/romdO3di69atFu/5+vrC3d0dW7ZswS+//IIlS5Zg27ZtNZ4HkSvhrV6J\nyNHsJvPZs2fj7rvvxs8//wwhBGbPnn1Ht3GdMGECJkyYYPHeggULEBkZCQC45557cOnSJbv1NGvW\nGFrt7RECPz8vm5+trKx8eXa2p9XP+Pp6Wq2nqvO1Vm9NY67O8jhr2vLL66j51lVMNZ3W1zcCf5w/\nD6AskQcEBNgc2TKfR/n1rfz8zcsrK7uTeh05rcTae9Wp11bd1f1ukwGkAnCzU6+9um2V3Uk71rTe\nqsRbvryq27/anm9tld3ptEpWpQet3HvvvRYnpK1YsQIrVqyotSB69eqFgwcP4oEHHsC5c+fQpk0b\nu9NkZ9++97Wfn5fNoa/KyqyVZ2Xp4Wvlc1lZ+gr1VGe+1uqtSczVXR5nTVt+eR0x37qM6U6mNX/2\nuUajsTltVpbe4u/y8y//Wam8srI7qdeR0wK2v7/q1GutvLrrhbd3S/gkJiH3cgqKi8fVKGZ7ZXfS\njjWt19601sqruv2r7fnWRtmdTqsUtnZIavQ883379t1RMOVNnDgRJpMJMTExWL58OVauXFmr9ZNz\nJOP2EDNVTXo6n1td16R763fo0NHZoRDVWI0egVrbt3F1c3PDyy+/XKt1knNJTxIDXOspYtLOhSse\nu+bZ0kRUUzVK5iqVqrbjoHrG/ElirvIUMVfdwZDwbGmqCV77TkAlyXzatGlWk7YQAsXFxQ4NisgR\nXHEHg2pGiTeccQRHjOZYOxmQXJ/NZD537ty6jKNB4CVKRHdOiTeccZTaHs2RRq/KnwxIrs9mMu/T\np09dxlHvST+SLAA+cM1hXiIl4OEIxzEfvfr9dycHQ9VSo2PmVH3mPxLpNRERUW2o0aVpRERE5DqY\nzCuRng5cucLrfomIyLVxmN2GDgAQ8Q18fPwa/Ek2RETk2pjMbdAA6NSpk8WtOImIiFwRh9mJiIgU\njsmciIhI4ZjMiYiIFI7JnIjof3gFCykVT4AjIgJvE0vKxmRO1ADwyVr28TaxpGRM5kT1HJ+TTlT/\nMZkT1XPscRLVfzwBjkgheHIWEdnCnjmRAvDkLCKqDJM5kQJwqJyIKsNhdiIiIoVjMieqouT//SMi\ncjUcZieqAn//zriUmAQACAgIQFZWgZMjIiK6jcmcqAo0Gg0CAgLlv4mIXAmH2YmIiBSOyZyIiEjh\nmMyJiJyMNwSiO8Vj5kRETuSqNwSSdizS0/mAHiVgMiciciJXvCGQq+5gkG1M5kRUKT4+teFxxR0M\nqhyTORHZxMenEikDkzkR2cQeGpEy8Gx2IiKqtvR0nn3vStgzJyKiauHhF9fDZE5ERNXCwy+uh8Ps\nRERECsdkTkREpHBM5kRERArHZE5ERKRwTOZEREQKx2RO1MDxiV1EysdL04gaMD5Qg6h+YDInasB4\nvTBR/cBhdiIiIoVjMiciIlI4JnMicjk8KY+oenjMnIhcCk/KI6o+JnMicik8KY+o+jjMTkREpHBM\n5kRERArHZG5F8v/+ERERKQGPmZfj798ZlxKTAAABAQHIyipwckRERESVYzIvR6PRICAgUP6bSAmk\nS7jS04HAQOfGQkR1zyWSuV6vxzPPPIOCggK4u7tj7dq1aN68ubPDIlIEXspFRC5xzHz37t0IDg7G\nxx9/jBEjRuDdd991dkhEiiFdyhUQEIiAgECOKBE1QC6RzIOCgqDX6wGU9dJ1Op2TIyIiIlKOOh9m\n37lzJ7Zu3Wrx3rJly3D48GGMGjUKubm52L59e12HRUREpFgqIYRwdhBz587F4MGDMWnSJJw/fx5/\n+9vfsG/fvkqnKS01QqvlcCKRs1y4cAFHjwajffuy+6j37XseQUFBFcqBimUNDduCHM0lToDz8fGB\np6cnAMDX1xf5+fl2p8nOvn3JmJ+fl81bP1ZWpsRpXTEmZ03rijE5a1pnxJSVpa/w2vxz5uXlyxwV\nk6tOy7aonWkdGZNS+Pl5WX3fJZL5vHnzsHTpUmzfvh2lpaVYvXq1s0MiIiJSDJdI5i1btsQ777zj\n7DCIiIgUySXOZiciIqKaYzInIiJSOCZzIiIihWMyJyIiUjgmcyIiIoVjMiciIlI4JnMiIiKFYzIn\nIiJSOCZzIiIihWMyJyIiUjgmcyIiIoVjMiciIlI4JnMiIiKFYzInIiJSOCZzIiIihWMyJ6IaS08H\nrlwp+5+InEfr7ACISJn8/TvD1/c8srL0CAwse01EzsFkTkQ1otFoEBQUhBs38pwdClGDx2F2IiIi\nhWMyJyIiUjgmcyIiIoVjMiciIlI4JnMiIiKFYzInIiJSOCZzInKY9HTeUIaoLvA6cyJyCPObyvCG\nMkSOxWRORA7Bm8oQ1R0OsxMRESkckzkREZHCMZkTEREpHJM5ERGRwjGZExERKRyTORERkcIxmRMR\nESkckzkREZHCMZkTEREpHJM5ERGRwjGZExHVAT50hhyJ92YnInIwPnSGHI3JnIjIwfjQGXI0DrMT\nEREpHJM5ERGRwjGZExERKRyTORERkcIxmRMRESkckzkREZHCMZkTEREpHJM5ERGRwjGZExERKRyT\nORERkcIxmRMRESkckzkREZHCMZkTEREpHJM5ERGRwjktmX/33XdYuHCh/PrkyZOYNGkSpkyZgo0b\nNzorLCIiIsVxSjJfs2YNXn/9dYv3li9fjvXr12P79u04deoUzp0754zQiIiIFMcpybxnz55YsWKF\n/Fqv16OkpATt2rUDAAwaNAhHjhxxRmhERESKoxJCCEdVvnPnTmzdutXivbi4OISFheHYsWPYsWMH\n1q1bh4yMDMybNw87duwAAOzatQupqamYP3++o0IjIiKqN7SOrHzChAmYMGGC3c81adIEer1efp2f\nnw9vb29HhkZERFRvuMTZ7J6ennBzc8OVK1cghMChQ4fQq1cvZ4dFRESkCA7tmVfHypUrsWjRIphM\nJgwcOBDh4eHODomIiEgRHHrMnIiIiBzPJYbZiYiIqOaYzImIiBSOyZyIiEjhnJrMY2JiMHbsWFy9\netVqeXx8PAYNGoT+/ftj27Zt1arXWp3S+/Hx8Rg2bJhcp7X3pdj69+9v8X5UVBQOHDiAXr16oX//\n/jZjM5/m2LFjePjhh9GjRw907doV69atsxr3Pffcg/DwcNx7772Vtkl4eHilbTJ8+HCMHj3aZht8\n+umnCA0NRZ8+fSrUIcX90EMPYeDAgRXKP/30UxiNRos2XrhwISZNmoT58+fj0KFDcj0DBw7EsmXL\nLOqdM2cOoqOj8f7772PdunVIS0tDTEyMXP+gQYMqfD8jRoxAv379kJycjJiYGAwZMgQGg0GeZt26\ndQgNDcX8+fPx+eefY+DAgYiKirLZhu+88w4iIyNx3333VdqG3bp1w6ZNm6yWx8fHo0+fPujZsyc2\nbtxosQzl29s8jgULFuDee+9FaGgonnjiCcTHx2PAgAH48MMP5c/885//RGhoKEJDQ/HYY4/J7w8Y\nMAAPPPAA/vKXv+DUqVMAgOzsbDzyyCOIjY3FlClTLGKOioqCwWCwiCE+Ph5du3ZFZGSkxXvm7W3+\nXp8+fTBx4kQcP34cM2bMwLRp0/DII48gNzcXMTExmDlzJkaOHImIiAiEhobCYDDIMYWHhyM4OFi+\n2+OgQYMAAMePH8eFCxcQHx+Pu+++G/3797f4vjZu3Cjfc+LIkSMYP348hg8fjq5du8rL9sQTT2DK\nlCmYPn067r//fuzduxcRERHo1q0bHnzwQQwdOhQPPfQQAGDt2rWYOnWqxXKmpKQgIiICV69exciR\nIzFq1CiEhISgb9++mD59Oj766CNMnDgR4eHhGDRoEEJCQvDWW29h3rx58joeExODJUuWYPz48Rg9\nejTCwsIQFRWFxMRExMbGYtSoUejZsyc++OADAGV3vgwJCUFkZCR27twJADAYDPjss88AlK3Ha9as\nQWRkJGbNmmV1fSq/Xtnahg4YMABRUVFYunQpBg8ejP79+2PcuHHIzMzEl19+ifj4ePTr1w+vvvoq\n9uzZg/Xr12PevHkV5rNnzx4888wzcozlPfnkk3j00Uct3jt+/DimTZuGgQMHYunSpYiKirIonzRp\nEgYPHozExESMGDECS5YsAXB7fZ00aRIGDRqE5OTkCvNbsGABJk6caLUMKGvPxYsXW51uxIgR6Nu3\nL1577TWr00rzt+bUqVPIzMy0WmYymeTv68knn0RhYaHVzzmaS/fMExISEBQUhLi4OMTGxtZqvUuW\nLKlQZ1XeV6lUAMpWms6dO9uMrXxdv/32G/z9/eHj44OTJ09avV2twWBAmzZt4OvrW2nsvXv3rrRN\nrl+/DpPJZLOOI0eOwMPDA88995zNZRVCYPTo0RXKN2/eDKPRaPHeunXr5HYxr2fNmjV46aWXLOp9\n++230bhxYwCQp7E2rbXvQVL+899++y1mz56NN998E19++SX8/f3h5+dnc/kff/xxFBUV4cknn7Q5\nj9zcXLRq1Qpjx461Wp6QkICwsDCsX78eY8eOrRBTZfLz89G6dWssXboUCQkJAIDJkyfL5f/85z8R\nHh6OTZs2QaVS4dy5c/jtt9+Ql5eH8ePHY/369XK7btq0CdHR0di2bRuKi4vh6ekpx2wtpoSEBDRt\n2hQtWrSweK98e0vveXl5AQBee+01PPPMM/joo48QExODS5cuAQBSU1PRvXt3vPnmm7jrrrssYmrS\npInVGHbt2oWMjAwkJCRAo9EgLi7O5ve1du1arF27Fj179oRWq0X37t0BACkpKdi+fTs+/PBDjBgx\nAkDZ7+fFF1/EV199heLiYkyZMgUHDx7Ed999h+zsbItlWrBgAUpLSwEAV69exfjx49GtWzd4e3vj\nxRdfxLRp05CcnIwNGzZgyJAhaNeuHX744Qf8+uuvcmy5ublITU3Frl27EBISApVKhRYtWuDdd9/F\nlClTEBERgXHjxqGoqAjZ2dnYuXMnOnTogG3btuGLL77A1atXcf36dTmxA2XbieXLl2PLli1W26Oq\nsrOzMXv2bJhMJnTp0gVxcXHYvXs3Ll68iISEBCQkJODvf/+7ReLbsGGD1brat2+PiRMnVnneu3bt\nwqlTpxAWFgZvb+8K68DZs2fRpk0bZGRkYOjQoYiLiwNwe329ePEixo0bh06dOlWoOzExEZ999lmF\nMinJv/766xg5cqTV6VauXIlBgwZh0aJFVuO2tq4aDAY8+uijeO655zBz5ky8/fbbFT5z/vx5dOnS\nBQBQWFiIRo0aWa3f0ap9Nvu4ceNgNBrx7rvvYtiwYRBCYPv27XjhhRdw8+ZNqFQq3Lp1C15eXnjo\noYfg7u6Ozz//HNnZ2Wjfvj3Cw8Px/fffo7i4GKWlpfD19cVHH32Ep556Cu+++y68vb3Rt29fzJs3\nDxs3bsStW7fg7e2N/Px8GI1GNGrUCNHR0fKeohR+586dcfnyZfkHKuncuTOuXr2KkpKSCklIo9FU\neK8q1Gq11WRp6/3KSCuQta9Bo9HAZDJZLasKrVYLIUSNlvFOmLeDTqdDSUmJzc+qVKoqL1/59lWr\ny/ZFq9vm1ZkHADRv3hw5OTk1bkcvLy8UFhZWWDebNm2KnJwcAGXfVWlpabXaQ4pXCGExjVSX+fJU\nti5UZ72tbnwSe+uBef0qlcpmPFWZf01+h1WZj7S9qOw3W5vzq+n00t/m27c7aROpPpVKBbVabbEO\nlY+7uq+Bst9BUVERioqK5PfUajW0Wi0MBgNUKhVat26NW7duyTcX0+l0CA0NRVpaGjIzM2EymeTv\n5dFHH8UDDzyAl156CZcuXUJJSQlKSkrg4eEhL4NWq0VcXByKi4vxwQcfQKPR4Pz58zhy5Ahyc3Ox\ncOFCqFQqeHh44MyZM1i0aBG6d++ODz74wOJW5KtXr8YPP/wALy8vqFQq5OTk4JFHHsHUqVNr1NZ3\noto98/vvvx8dO3bEtm3b0Lx5c3h6euKLL76Qv5BWrVohKSkJ3bt3x/Hjx5GcnIxmzZph3LhxWLVq\nFU6dOgV/f38cOHAAjRo1QkFBgVzvoUOHkJSUhPbt26O0tBR9+vRBo0aN4OHhIa+Ibdu2xZkzZ6DR\naODh4YHFixfDw8MDly5dgru7u7yBlzYKqamp6NSpE7TaskvqlyxZAh8fHwCQex0A0KhRI/lLbt68\nuVwHUPYj1mq1cHNzA1B2xzprpHkAZQlAigWA3Bs1FxQUBJVKBV9fX+h0urIvxGya4cOHw9/fX57e\n3d1dLtNoNPLfERER8t8zZ860+IzJZIKnpyeaNm1aYf4tW7aU/zbfKzWv2zzu8nuuGo0GarVa/ryb\nmxu0Wi1MJhO8vLygVqvl5QKA0NBQ+W+pZyiEgFqttqhHIvW4pHlLsajVavj4+MBkMsnt5e7uXmF6\n6bNhYWHya1ujHlL7SN+hRqOR68vMzJT/lpZH2iiYa9mypRyjRqORv6+CggKUlpZCrVbL6xYAlJSU\nQK1Wo1GjRggJCQEAeHh4yOXmd0Esv65aS3pt2rQBcHvnRq1Wy+uPRqORy8vPR2oTjUYDlUolr+eV\nufvuu+X1wfx3J3Fzc7N4T0rkXbp0kdtWKm/durXFMkht3axZswrtXH59lD4rrU8eHh4W64G0bCqV\nCj4+PvIG2nx+5rRarcWyRUdHy2X9+/eX5/Hggw/K9Q4bNkz+jPlvqkOHDnL7mJPiM3+/skRuvh1q\n3ry5/P1otVq5TFqmoUOHyu3p6ekpb+vMb8Ll4eEBrVYLnU4HT09Pi2WX/O1vf5P/9vT0tNguAmXb\nQGn+arUa/fr1Q/v27SGEQLt27aBSqdCxY0fodDqoVCp5WaXlbNSokTz9rVu35OWQns8hhMCQIUPk\n91u1agW9Xi8nXa1WiytXrliM6EjtvXv3bkybNg2enp4YPXo0WrVqBaCsl92uXTt0794dzZo1w8qV\nK7Fs2TJoNBp8/PHHKC0tRWJiIv7xj38gOjoaW7duRe/evZGfn49Lly5Bp9NZJHIAWLp0KXr16oV3\n3nkH06dPx9NPP+2URA4AmhXlo7OjadOm+PHHH3HixAmEhISgpKQEFy5cQKdOneDr64uQkBAMGTIE\nGRkZuHnzJvR6PTw9PdGjRw8UFhYiNTUVwcHBiIqKwqFDh+Dl5YUHH3wQ7du3x65du5CdnY3x48fj\nyy+/xM2bN+Hm5gaj0Qij0YjGjRvDw8MDqampMBgMKC0txS+//AIhBEwmk7znZTKZ5I09UPaD9vDw\nkDcoaWlpKCkpgUqlkntD5j0HrVaL4uJiqNVquT4pQRUVFcnzk1ZkaQU132MtLCyssEcKQN7DValU\nyM3NhdFoRLNmzdCiRQvk5uZaTGMwGNCxY0ekpKSgtLTUomfn4eFh8To/Px8A8Ouvv8rLLf2v0Whg\nMBhgNBrljai0DNLyS4lfilFirzfVtGlTlJaWytNIdbi7u8NgMFj0EnNycuRy8+NKUs+y/AatRYsW\nSEtLs9igG41GuYcpfS9S+7q5uclJU6pLCIEbN25YfC/W6HQ6GAwGuY7y8bi7u6OkpMRifuXj1Wq1\nKCgokKct34ORNnbSDqyPjw+MRiPc3d3h6emJmzdvyj0QqS3NE7PUjuV7+B4eHhBCwM/PD9nZ2fDx\n8ZGP/fn6+iI3NxclJSXIy8uTp2ncuDFKSkoghIDBYLD47ivrwUnrj7SDIvUAzeOW2sZaPQUFBfJO\njNR+5r+V0tJSebmlnpo0XG80GuVRDmk+0jykZZB2CqU2MhgMct3FxcUALH+nbm5uKCwslH8DQgiU\nlJTI04WEhODixYvy9yZ9v9JoHwBcunRJnoe0Hknfn1SP+bpibXshLU955X+bhYWFFr9rrVYr95bV\najUyMzORl5cn1y2NSOp0OuTm5kKtVsvfu9ForPD7lmLLzs7GzZs35RikbaH0nUjtK40c3rx5Ezk5\nORBCwNfXV17XioqKYDKZoNPpLH6b0giHNHpg/puRYkpPT5e306tXr8a+ffsghMDmzZvlnrYQAu7u\n7igoKICfnx8yMzPlNrnrrrvw66+/4t///jd27NiB/Px85Ofn49q1a2jUqBG6dOmCtLQ0ZGVl4aef\nfsLVq1cRERGB06dPY+zYsWjRogU6dOiAQ4cOIScnB9u2bYMQwqLztHr1auzfvx//+c9/8NVXXyEl\nJQXFxcVOuelZtXvmgYGByMnJQUZGBnJycuDu7o4bN27gwoULuHbtGk6ePAmj0YhffvkFWVlZFnvX\nXbp0wfXr13Hq1CkUFBTg999/R0pKilzvlStXcOrUKdx7773Iz8/H1atX0bRpU2RnZ6O0tBRubm7I\nysqCSqWCn58fNBoN7r77bowbNw5CCBQVFckbXGlY3mAwIC8vD7du3UJhYSGuXr0qJ+SioiL5B2Q+\nHCXtOUtJV/pb2jOV9oKlFcl8NED6X+ptSqRkav65IUOGQKvV4tq1a0hPT68wpH7z5k35GJ1Wq7XY\nizb/4S9YsED++5FHHpGXo0mTJvLySKMJ0g8HuJ2ohRAWPWjzPXBpT9larxco2zhLiVWal7QzVH66\nkpISuVdh/r5arYabm1uFEY/09HSLRN+sWTN52aVjzFJvws3NTW6T8kkkICBAnp/UU5HmK00r9Yyl\nQznSdy991wUFBRYxW+tJ5ebmWvT63Nzc5J6JtMEz32ErKiqCwWCARqPBlStXAEDuSQOw+L7N20bq\n9UvrmbRuSclUGooUQiA7O1seIZFGTqTlkdpJWi91Oh1atGghjwJYG62REq15j9ZoNFbYOTPveUv1\nqNVqebRBapfyy6nT6eQNvbQehoSEyOtn+aFYidQDl5ahfNxNmjSRvxvzUQ7zmKXRt+LiYnk680Qs\nzU+tVssjBFqtFs8++6xcT+PGjeUdfamtmzZtajGKUH4UwzwGa+9J7aTVatGuXTt5OaQd5pKSEuh0\nOphMJvj7+6N3794Aynq/fn5+0Ol0uO+++yxil35zUq9VWj8ks2fPlv9u2rSpvF01j106NqxWqxEQ\nEICuXbsCuL1NyMvLk2OXRiyk77FJkyZo2bJlhRGvjh07yssWGhoqj5hK72s0Gpw+fRre3t5o0aIF\nmjdvLndSpG1Cz5490apVKzRp0gSZmZkYNWoUsrOzoVKp0KxZMzRv3hxPPPEEhg0bhrZt2yIkJAQf\nffQRVq1ahV69eiE4OBjHjx8HAJw4cQKNGzfGQw89hLfffhu7d+/G5cuX5TZ48sknMXjwYHz44Yfo\n0qULPvnkE+X0zIGyk09u3LiBDh064O6770Z6ejo6deqE++67D0eOHMGGDRuQmZmJyMhItGrVCtev\nX0enTp0wdOhQZGdn4/Dhw/jnP/+JkpIS+Pn5ITo6Gl5eXkhJSYHRaMSwYcNw+fJl/Pnnn2jZsqW8\nw2AwGBAYGAh/f3/k5uYiLy8P165dw++//w5vb2+UlJRY7DFKPW5p41BcXIysrCyYTCZ5SFH6XKtW\nreTehrTBkIaApTqk45vS9NJevPQjlIaYpWltDZ1J76empsojBuZnUUo/gNLSUnlEorCw0OIzzZs3\nlzfaBw4ckOuUVkIhBPr27Yvr168jPz/fYiMobSBatmwJvV4PtVqNESNG4Pz58wAse6/SHratZZE2\nKADkpK5SqeQNjL+/P27cuGGxRy61obSs5nvk5gkkNzfXYl5Go1H+fqWzuaXYrPUyJFlZWRV6Z+bL\nZDQakZWVJfeSzZfffANs65i5eczmMUi9pZCQEGRmZsqjSdI0gYGByMzMREFBgdyGo0ePxn//+1+Y\nTCaLdcv8b/N2NE86Uo/f3d0dWq1WnqZDhw7IycmxGFkov8Mj9fwLCgpQXFwsj3RI85F+C9LfmZmZ\n8npkMplQXFxscczWfKfRnPQ589EF8+9E6oWa98zNe8HmI2jm9RcXF8ttZt57ldYv87Y3b0vz71pa\nv4QQGDZsGC5evIjz58/Ln42IiMCVK1eg0WjkjoDRaMRPP/0kz8t8R0qKX+o8mH9/1WE+CpSbm2sx\n6iCR2iwjIwNXr16Vvz9pVEan0yEtLQ0ajUb+DoxGoxyvtH5Lh0u/+eYbuW5pR1ZaDvP2kpbnxo0b\nuH79uty+5UfOpO2M1GExGAzQ6/XyzrM0oiGNoplMJuj1enmeFy5cQHZ2NoqLi7Fp0yaUlpaibdu2\n6N69O06dOoXS0lL897//RXFxMbRaLVatWoWvv/4a165dk3v/TZs2xa1bt1BcXIwzZ85g1qxZyM3N\nxcGDB/H9999j//79mDZtGvr374/3338fn332GRITE3H16lX0798f7dq1Q3x8PIYPHy53CpKSkqDR\naBAWFoZvv/0Wo0ePrtZ3W6tEPTVp0iQxZswYkZaWVuv1WquzOu/bi81WXTWJqzrzrep8bNUxadIk\nMXjwYDFx4sQaz+Po0aOiT58+ol+/fhb1VrW+6rad+edtTbty5Urx888/V3kekyZNEosXL7aYpnx5\nbaybNVm35s2bJ1599VUhhBCLFi0SzzzzjNi1a5e4//77xYoVK6o9v8jISPHbb78JIYR4/fXXxaZN\nm2rl+6pKG1e1Dav627A3v8rKIyMjhcFgqPD58tNt2bJFGAwGERkZKRYsWCD27t0rJk2aJP744w8x\nYcIEm/OrjWWQYty3b5+4fPmyEEKITz/9VDz//PNW64mMjBSjR48W77zzjpgyZYrYtGlTlWMxV1xc\nLIhzlkcAAAQVSURBVBYsWCBiY2PFlClTxHvvvSfWrVsnl0VFRQkhhHj66adFdHS06Nmzp5g1a5bY\nvHmzOHr0qFiwYIEQQog//vhDTJs2Tbz88ssV5jFw4EDxxx9/iH79+omnnnpKfn/fvn0iJiZGREdH\ni969e4t58+aJ3NxcMWfOHPHXv/5VzJ07V0ybNs1m7GfPnhX9+vUTo0ePFjExMWLLli1VXu665vR7\ns586dQrLly/HlStXLPZAW7RogYcfftjikp3qiI6Olus03/OXhoJbt26Nli1byr2LiIgInDhxwuLY\ndn5+PrKysqDX6+XYCgoK0KpVK3lYWBpyKSgokHu6zZs3R6tWrSCEwG+//YYpU6bIdev1epw7d04e\n1pP2bqUhvhYtWiA/Px+enp7yIQiJTqfD/fffb/U69cmTJ+OFF17Aq6++Ks8nOTkZxcXF0Ol00Ol0\nKCgokIc6peNb5sdkmzVrhi5dumDkyJE2233y5MkoKCiAVquV65dIe93SyWpSz0Sn0+Guu+5Cq1at\n5LpPnTqFtWvXVjiXoGvXrti7dy+EEPIJgmfOnEHLli2RkpICnU4n9xKAsiHAgIAAqFQqjBw5Env3\n7pXbIT8/X/5+pBMNy8cwY8YM+Pv7IyUlBQUFBfK5CNJx7GvXrsmHU6ShQIPBgL/+9a8V1hcphtde\new3du3eXe7h6vR6XL1+Wh45v3Lgh917KH//WarUYPny4zXsRmLfbmTNn0LFjR1y+fFle586cOWNx\nTFQi/tez9vDwQHFxMdzd3eXh5PDwcJw8eRJeXl4oLi62WH+lZeratatFW0ltmp+fb3EM18PDA4WF\nhfKIWpMmTSzWE2ldbN68OWbMmGHz+7L1e5KOe968eVOer/k6LvUIzX/f5X+D5t+X9Ls/ffo01Gq1\nxTpbPs6YmBisWbPGog7pXB29Xo+IiAjo9XpcuXJFHkU8e/asvE65ubnJo4elpaVo0qQJQkNDcfr0\nafk8H+mSqzNnzsDf3x96vV7+Xjt06IArV66gRYsWaNmyJfLy8nDlyhV4eXkhJydH/n1L3410qMPN\nzQ2+vr5IT09HREQEwsLCcOzYMTRq1AgajQZr1qyRD6eZb0tiY2MtRvQ0Go08MtKyZUsUFhbKo57m\n2+1//etf2Lt3L37++ecK7R0XF4e2bdti06ZNVsvnzJmDpUuXAigbhTCvu23btvDw8ECvXr1w+vTp\nCtO2adMG8fHx8rC71A6dOnWCEAJ6vR7Xrl2Dj4+PfLJcQUEBUlJSsHfvXrRt27bC7+3PP//EihUr\n4OPjg+vXr1ucyGe+PK7A6cmciIjIHls7AHeSUKVk3a9fPyQmJtZq3XWNyZyIiEjhXPoOcERERGQf\nkzkREZHCMZkTEREpHJM5ERGRwjGZExERKdz/By1xROizv79vAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xde45780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Your code goes here\n",
    "N = 54\n",
    "hamMeans = lp[0,:] #height, in this case log probability \n",
    "\n",
    "ind = np.arange(N)  # the x locations for the groups\n",
    "width = 0.35       # the width of the bars\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "rects1 = ax.bar(ind, hamMeans, width, color='r')\n",
    "\n",
    "spamMeans = lp[1,:]\n",
    "rects2 = ax.bar(ind + width, spamMeans, width, color='y')\n",
    "\n",
    "# add some text for labels, title and axes ticks\n",
    "ax.set_ylabel('Log probability')\n",
    "ax.set_title('Log Probability of Spam/Ham')\n",
    "ax.set_xticks(ind + width)\n",
    "ax.set_xticklabels(column_names) #column names are generated on code up above\n",
    "\n",
    "ax.legend((rects1[0], rects2[0]), ('Ham', 'Spam'))\n",
    "\n",
    "def autolabel(rects):\n",
    "    # attach some text labels\n",
    "    for rect in rects:\n",
    "        height = rect.get_height()\n",
    "        ax.text(rect.get_x() + rect.get_width()/2., 1.05*height,\n",
    "                '%d' % int(height),\n",
    "                ha='center', va='bottom')\n",
    "\n",
    "#autolabel(rects1)\n",
    "#autolabel(rects2)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**b)** The features are in the order they appear in the dataset. Can you order them by probability of being `Ham`?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-3.98858041 -4.3988999  -3.36045613 -7.81480343 -3.59040574 -4.2468369\n",
      " -6.22783837 -4.68415184 -4.62295628 -3.84661015 -5.04918338 -2.93621914\n",
      " -4.20388552 -5.16784092 -6.100005   -4.47863852 -4.42477935 -4.15124178\n",
      " -2.62184658 -6.140827   -3.14718023 -6.87653379 -5.65531918 -6.00469482\n",
      " -3.06409094 -3.34634429 -3.36174896 -3.93898347 -4.12315005 -3.89834583\n",
      " -4.33870474 -4.69390801 -4.16848359 -4.68415184 -3.92525328 -3.8217126\n",
      " -3.42035427 -6.06078429 -4.23437568 -4.48259892 -5.01481573 -4.23747646\n",
      " -4.33870474 -4.37367334 -3.29664462 -3.90278042 -6.20536552 -4.77558604\n",
      " -3.7581992  -2.67313987 -4.02056346 -3.39462503 -4.3352742  -4.5739487 ]\n"
     ]
    }
   ],
   "source": [
    "# Your code goes here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**c)** What about ordering by the absolute difference between `Ham` and `Spam` log probability?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code goes here\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [iaml_env]",
   "language": "python",
   "name": "Python [iaml_env]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}