loan project.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width=\"400\" align=\"center\"></a>\n\n<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "In this notebook we try to practice all the classification algorithms that we learned in this course.\n\nWe load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n\nLets first load required libraries:"
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [],
            "source": "import itertools\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.ticker import NullFormatter\nimport pandas as pd\nimport numpy as np\nimport matplotlib.ticker as ticker\nfrom sklearn import preprocessing\n%matplotlib inline"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### About dataset"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n\n| Field          | Description                                                                           |\n|----------------|---------------------------------------------------------------------------------------|\n| Loan_status    | Whether a loan is paid off on in collection                                           |\n| Principal      | Basic principal loan amount at the                                                    |\n| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n| Effective_date | When the loan got originated and took effects                                         |\n| Due_date       | Since it\u2019s one-time payoff schedule, each loan has one single due date                |\n| Age            | Age of applicant                                                                      |\n| Education      | Education of applicant                                                                |\n| Gender         | The gender of applicant                                                               |"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets download the dataset"
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "--2020-07-03 12:10:40--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 23101 (23K) [text/csv]\nSaving to: \u2018loan_train.csv\u2019\n\n100%[======================================>] 23,101      --.-K/s   in 0.002s  \n\n2020-07-03 12:10:40 (13.1 MB/s) - \u2018loan_train.csv\u2019 saved [23101/23101]\n\n"
                }
            ],
            "source": "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Load Data From CSV File  "
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>9/8/2016</td>\n      <td>9/22/2016</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30       9/8/2016   \n1           2             2     PAIDOFF       1000     30       9/8/2016   \n2           3             3     PAIDOFF       1000     15       9/8/2016   \n3           4             4     PAIDOFF       1000     30       9/9/2016   \n4           6             6     PAIDOFF       1000     30       9/9/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   45  High School or Below    male  \n1  10/7/2016   33              Bechalor  female  \n2  9/22/2016   27               college    male  \n3  10/8/2016   28               college  female  \n4  10/8/2016   29               college    male  "
                    },
                    "execution_count": 3,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df = pd.read_csv('loan_train.csv')\ndf.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "(346, 10)"
                    },
                    "execution_count": 4,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.shape"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Convert to date time object "
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  \n0 2016-10-07   45  High School or Below    male  \n1 2016-10-07   33              Bechalor  female  \n2 2016-09-22   27               college    male  \n3 2016-10-08   28               college  female  \n4 2016-10-08   29               college    male  "
                    },
                    "execution_count": 5,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['due_date'] = pd.to_datetime(df['due_date'])\ndf['effective_date'] = pd.to_datetime(df['effective_date'])\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Data visualization and pre-processing\n\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Let\u2019s see how many of each class is in our data set "
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "PAIDOFF       260\nCOLLECTION     86\nName: loan_status, dtype: int64"
                    },
                    "execution_count": 6,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['loan_status'].value_counts()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "260 people have paid off the loan on time while 86 have gone into collection \n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Lets plot some columns to underestand data better:"
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Solving environment: done\n\n## Package Plan ##\n\n  environment location: /opt/conda/envs/Python36\n\n  added / updated specs: \n    - seaborn\n\n\nThe following packages will be downloaded:\n\n    package                    |            build\n    ---------------------------|-----------------\n    seaborn-0.10.1             |             py_0         160 KB  anaconda\n    ca-certificates-2020.6.24  |                0         133 KB  anaconda\n    certifi-2020.6.20          |           py36_0         160 KB  anaconda\n    openssl-1.1.1g             |       h7b6447c_0         3.8 MB  anaconda\n    ------------------------------------------------------------\n                                           Total:         4.2 MB\n\nThe following packages will be UPDATED:\n\n    ca-certificates: 2020.1.1-0         --> 2020.6.24-0       anaconda\n    certifi:         2020.6.20-py36_0   --> 2020.6.20-py36_0  anaconda\n    openssl:         1.1.1g-h7b6447c_0  --> 1.1.1g-h7b6447c_0 anaconda\n    seaborn:         0.9.0-pyh91ea838_1 --> 0.10.1-py_0       anaconda\n\n\nDownloading and Extracting Packages\nseaborn-0.10.1       | 160 KB    | ##################################### | 100% \nca-certificates-2020 | 133 KB    | ##################################### | 100% \ncertifi-2020.6.20    | 160 KB    | ##################################### | 100% \nopenssl-1.1.1g       | 3.8 MB    | ##################################### | 100% \nPreparing transaction: done\nVerifying transaction: done\nExecuting transaction: done\n"
                }
            ],
            "source": "# notice: installing seaborn might takes a few minutes\n!conda install -c anaconda seaborn -y"
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "import seaborn as sns\n\nbins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "bins = np.linspace(df.age.min(), df.age.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'age', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Pre-processing:  Feature selection/extraction"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Lets look at the day of the week people get the loan "
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "df['dayofweek'] = df['effective_date'].dt.dayofweek\nbins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\ng.axes[-1].legend()\nplt.show()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below    male          3        0  \n1 2016-10-07   33              Bechalor  female          3        0  \n2 2016-09-22   27               college    male          3        0  \n3 2016-10-08   28               college  female          4        1  \n4 2016-10-08   29               college    male          4        1  "
                    },
                    "execution_count": 12,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## Convert Categorical features to numerical values"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets look at gender:"
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "Gender  loan_status\nfemale  PAIDOFF        0.865385\n        COLLECTION     0.134615\nmale    PAIDOFF        0.731293\n        COLLECTION     0.268707\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 14,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "86 % of female pay there loans while only 73 % of males pay there loan\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets convert male to 0 and female to 1:\n"
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>1</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below       0          3        0  \n1 2016-10-07   33              Bechalor       1          3        0  \n2 2016-09-22   27               college       0          3        0  \n3 2016-10-08   28               college       1          4        1  \n4 2016-10-08   29               college       0          4        1  "
                    },
                    "execution_count": 15,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## One Hot Encoding  \n#### How about education?"
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "education             loan_status\nBechalor              PAIDOFF        0.750000\n                      COLLECTION     0.250000\nHigh School or Below  PAIDOFF        0.741722\n                      COLLECTION     0.258278\nMaster or Above       COLLECTION     0.500000\n                      PAIDOFF        0.500000\ncollege               PAIDOFF        0.765101\n                      COLLECTION     0.234899\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 16,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "#### Feature befor One Hot Encoding"
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>education</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>High School or Below</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>Bechalor</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender             education\n0       1000     30   45       0  High School or Below\n1       1000     30   33       1              Bechalor\n2       1000     15   27       0               college\n3       1000     30   28       1               college\n4       1000     30   29       0               college"
                    },
                    "execution_count": 17,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df[['Principal','terms','age','Gender','education']].head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
        },
        {
            "cell_type": "code",
            "execution_count": 18,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    },
                    "execution_count": 18,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "Feature = df[['Principal','terms','age','Gender','weekend']]\nFeature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\nFeature.drop(['Master or Above'], axis = 1,inplace=True)\nFeature.head()\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Feature selection"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets defind feature sets, X:"
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    },
                    "execution_count": 19,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X = Feature\nX[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "What are our lables?"
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n      dtype=object)"
                    },
                    "execution_count": 20,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "y = df['loan_status'].values\ny[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## Normalize Data "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  return self.partial_fit(X, y)\n/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  if __name__ == '__main__':\n"
                },
                {
                    "data": {
                        "text/plain": "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n        -0.38170062,  1.13639374, -0.86968108],\n       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n         2.61985426, -0.87997669, -0.86968108],\n       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679]])"
                    },
                    "execution_count": 21,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X= preprocessing.StandardScaler().fit(X).transform(X)\nX[0:5]"
        },
        {
            "cell_type": "code",
            "execution_count": 22,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.model_selection import train_test_split\nX_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=4)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Classification "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\nYou should use the following algorithm:\n- K Nearest Neighbor(KNN)\n- Decision Tree\n- Support Vector Machine\n- Logistic Regression\n\n\n\n__ Notice:__ \n- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n- You should include the code of the algorithm in the following cells."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# K Nearest Neighbor(KNN)\nNotice: You should find the best k to build the model with the best accuracy.  \n**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
        },
        {
            "cell_type": "code",
            "execution_count": 23,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.neighbors import KNeighborsClassifier\nfrom sklearn import metrics"
        },
        {
            "cell_type": "code",
            "execution_count": 24,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "array([0.67142857, 0.65714286, 0.71428571, 0.68571429, 0.75714286,\n       0.71428571, 0.78571429, 0.75714286, 0.75714286, 0.67142857,\n       0.7       , 0.72857143, 0.7       , 0.7       , 0.68571429,\n       0.72857143, 0.72857143, 0.72857143, 0.7       , 0.68571429,\n       0.71428571, 0.68571429, 0.7       , 0.7       ])"
                    },
                    "execution_count": 24,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "Ks = 25\nmean_acc = np.zeros((Ks-1))\nstd_acc = np.zeros((Ks-1))\nConfustionMx = [];\nfor n in range(1,Ks):\n    \n    #Train Model and Predict  \n    neigh = KNeighborsClassifier(n_neighbors = n).fit(X_train,y_train)\n    yhat=neigh.predict(X_test)\n    mean_acc[n-1] = metrics.accuracy_score(y_test, yhat)\n\n    \n    std_acc[n-1]=np.std(yhat==y_test)/np.sqrt(yhat.shape[0])\n\nmean_acc"
        },
        {
            "cell_type": "code",
            "execution_count": 25,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXd4VFXawH9nWoYECC0U6QoIoQWIWEAEK64FK2Bby7qu+9l1VVRWsaPr2lbXvmsHARWxiwoCrgQSREoAKdIkSIe0ybTz/TGZGMKUOzP3ztyZnN/zzAOZueXNzZ3z3rcLKSUKhUKhUJgNS6oFUCgUCoUiFEpBKRQKhcKUKAWlUCgUClOiFJRCoVAoTIlSUAqFQqEwJUpBKRQKhcKUKAWlUCgUClOiFJRCoVAoTIlSUAqFQqEwJbZUC6AXbdq0kd26dUu1GAqFQqGIQElJyS4pZZ6WbTNGQXXr1o3i4uJUi6FQKBSKCAghNmndVrn4FAqFQmFKlIJSKBQKhSlRCkqhUCgUpiRjYlAKhUKRCB6Ph61bt+JyuVItSkbgdDrp1KkTdrs97mMoBaVQKBTA1q1badasGd26dUMIkWpx0hopJbt372br1q1079497uMoF59CoVAALpeL1q1bK+WkA0IIWrdunbA1qhSUQqFQ1KKUk37ocS2VglIoFAqFKVEKSmEoUkr8fplqMRSKtOHDDz9ECMHq1atTLUrKUQpKYShun5/yGm+qxVAo0oYpU6YwfPhwpk6dauh5fD6focfXA5XFpzAUt9ePy+Ojid2Kw6aehxTpwc1f3MzS7Ut1PWZB+wKeHv10xG0qKir4/vvvmTNnDmeffTaTJk2q++zxxx/nrbfewmKxcPrppzN58mTWrVvHtddey86dO7FarUyfPp0tW7bwxBNP8MknnwBw/fXXU1hYyBVXXEG3bt246qqr+Oqrr7j++uspLy/n5Zdfxu1206NHD9566y2ys7P57bffuPbaa9mwYQMAL7zwAp9//jlt2rThpptuAuCee+6hXbt23Hjjjbpep/ooBaUwFI8v4N4rd3lo3TQrxdIoFOZm5syZjB49ml69etGqVSuWLFnC4MGD+fzzz5k5cyZFRUVkZ2ezZ88eAC655BImTJjAueeei8vlwu/3s2XLlojncDqdLFiwAIDdu3fz5z//GYCJEyfy2muvccMNN3DjjTdywgkn8OGHH+Lz+aioqOCwww7jvPPO46abbsLv9zN16lQWLVpk6PVQCkphGFJKvD4/AF6/pMrtJduhbjmF+Ylm6RjFlClTuPnmmwEYP348U6ZMYfDgwXz99ddceeWVZGdnA9CqVSvKy8v59ddfOffcc4GA4tHCuHHj6v6/YsUKJk6cyL59+6ioqOC0004D4Ntvv+XNN98EwGq1kpubS25uLq1bt+bHH3/kt99+Y9CgQbRu3Vq33z0UarVQGIbHJ6mfHlFR48Vps2KxqFRehaIhu3fv5ttvv2XFihUIIfD5fAghePzxx5FSHpK2LWXo5CObzYbf76/7uWEtUk5OTt3/r7jiCmbOnMnAgQN5/fXXmTt3bkQZr776al5//XW2b9/OVVddFeNvGDsqKKAwDLfPf9DPUqISJhSKMMyYMYM//vGPbNq0iY0bN7Jlyxa6d+/OggULOPXUU/nPf/5DVVUVAHv27KF58+Z06tSJmTNnAlBTU0NVVRVdu3altLSUmpoa9u/fzzfffBP2nOXl5XTo0AGPx8M777xT9/5JJ53ECy+8AASSKQ4cOADAueeeyxdffMHixYvrrC0jUQpKYRger/+Q91weH+4Q7ysUjZ0pU6bUueuCnH/++bz77ruMHj2as88+m8LCQgoKCnjiiScAeOutt3j22WcZMGAAxx13HNu3b6dz586MHTuWAQMGcMkllzBo0KCw53zwwQc5+uijOeWUU+jdu3fd+8888wxz5syhf//+DBkyhJUrVwLgcDgYNWoUY8eOxWq1GnAVDkaEMxPTjcLCQqkGFpqLHQdchLq7bBahEiYUpmPVqlX06dMn1WKYGr/fz+DBg5k+fTo9e/aMun2oayqEKJFSFmo5n7KgFIbg8flDKicIJExUKlefQpFWlJaW0qNHD0466SRNykkPVJKEwhCiufEqa7w47VasKmFCoUgL8vPz6+qikoWyoBSG4PFFVlASqHApK0qhUIRHKSiFITTM4AuFy+ujxmv+disKhSI1KAWl0B2vz4/W3JtylzdsPYdCoWjcKAWl0B0t1lMQn19S5VZWlEKhOBSloBS64/HGZhFV1njxqZEcCpPx2wGXri+9+OGHH+r654XjxRdfpH///hQUFDB8+HBKS0tjPs/GjRt59913w34+cuRIjC7tUQpKoTuxWFAQSJgod3mMEUahSEPmzp3LFVdcEfKzL774gtGjR0fc/+KLL2b58uUsXbqUO+64g1tvvTVmGaIpqGSgFJRCV3x+iT+OmFKN168SJhQKDXzzzTecfPLJEbdp3rx53f8rKyvr+vg9+eSTdT30li9fTr9+/aiqquK7776joKCAgoICBg0aRHl5ORMmTGD+/PkUFBTw1FNPUV1dzfjx4xkwYADjxo2jurrauF+yFlUHpdCVaOnlkSh3eXHkWA5piqlQKALs2rULu91Obm5u1G2ff/55nnzySdxuN99++y0AN998MyNHjuTDDz/k4Ycf5qWXXiI7O5snnniC559/nmHDhlFRUYHT6WTy5MkHzZV68sknyc7OZtmyZSxbtozBgwcb+ruCsqAUOlOTQJ89n19SqRImFI2Yo48+moKCAq6++mpmzZpVZ9V8+eWXAHz11Veceuqpmo513XXXsX79eh577DEeeughACwWC6+//jqXXXYZJ5xwAsOGDQNg2LBh3HrrrTz77LPs27cPm+1Q22XevHlceumlAAwYMIABAwbo8StHRCkoha4kYkEBVKmECUUjpqioiKVLl/Lqq69y9tlns3TpUpYuXVrXOfzzzz+viz9deeWVFBQU8Ic//CHiMcePH1/X8Rxg7dq1NG3alG3bttW9N2HCBF599VWqq6s55phjWL16dchjJdu7oRSUQjf8fpmwclEJEwpFaKSULFu2jIKCAgD++9//snTpUj777LNDtl27dm3d/z/99NO63nn79+/npptuYt68eezevZsZM2YAsH79evr378+dd95JYWEhq1evplmzZpSXl9cdZ8SIEXUjOVasWMGyZcsM+12DqBiUQjdizd4LR43Xj8vjw2k3vp2/QhGOds21TahNFiUlJQwaNEiTFfPcc8/x9ddfY7fbadmyJW+88QYAt9xyC//3f/9Hr169eO211xg1ahQjRozg6aefZs6cOVitVvLz8zn99NOxWCzYbDYGDhzIFVdcwV//+leuvPJKBgwYQEFBAUOHDjX6V1bjNhT6ccDloVqnGJJFCNo0daiECUXSMPu4jYceeogePXowfvz4VIuimUTHbSgLqpHh9vpx2Izx7IYaUBgvfinZVeEmHv1kt1rIbWLXTRaFwgxMnDgx1SIkHaWgGhFSSspdHkOGBUop8eqc3OCXkrBDpSLu5wOUglIo0h2VJNGIcPv8eP3SkJHresWf9EBKVCagIi4yJeRhBvS4lkpBNSI8vsANU+3Rv9bICKWXCImmuysaH06nk927dyslpQNSSnbv3o3TmViiiXLxNSKCMaIajw/ptOmagBBUfmZBb3ejIvPp1KkTW7duZefOnakWJSNwOp106tQpoWMYqqCEEKOBZwAr8KqUcnKDz7sAbwAtareZIKX8rPazu4A/AT7gRinll0bK2hgIWhWSgBWV7dDnzy+lxGsyi8Vs8ijMj91up3v37qkWQ1EPw1x8Qggr8DxwOpAPXCSEyG+w2URgmpRyEDAe+Hftvvm1P/cFRgP/rj2eIk7cXv9B+QZ6pYNDwHoym71iNotOoVDEjpExqKHAOinlBimlG5gKjGmwjQSCbXdzgWDvjTHAVClljZTyF2Bd7fEUcdIwJuP1S93iNGZKkAjilxK/cvMpFGmNkQqqI7Cl3s9ba9+rzyTgUiHEVuAz4IYY9kUIcY0QolgIUaz8xpEJpYz0mmSrZ/2TnphRcSoUCu0YqaBCReAbPtJeBLwupewE/AF4Swhh0bgvUsqXpZSFUsrCvLy8hAXOZEIt1jUeny4ZS2bNmFOJEgpFemOkgtoKdK73cyd+d+EF+RMwDUBK+QPgBNpo3FehEa/PTyg9JAGXJzHl4vH5TRd/CqISJRSK9MZIBbUY6CmE6C6EcBBIepjVYJvNwEkAQog+BBTUztrtxgshsoQQ3YGewCIDZc1oIiUMVLm9CR7bvEpAJUooFOmNYWnmUkqvEOJ64EsCKeT/kVKuFEI8ABRLKWcBtwGvCCFuIfBAf4UM+JxWCiGmAaWAF7hOSqkm2cVJpCLaYLKE3Rrfs4rZCnTrE0yUsFhUw1mFIh1R3cwbATvLawJ97cLQxGGluTO+3nU7yl0h3YdmoUW2nSybqlBQKMxCLN3MVaujDMfnlxGVE4DLHV+yRLjYlpnwKjefQpG2KAWV4WiJEcWbLJEOMR6loBSK9EUpqAxHay1QPA1kzRx/CqJqoRSK9EUpqAxHaxGtx+ePOSMvHRZ/1VFCoUhflILKYPz+2IYIxmJFaYltmQWP3/yKVKFQHIpSUBlMrBaOK4bOElqtrQp3OZWeypjk0BsVh1Io0hOloDKYWF12UmpPlqjR6Dq87KPzGfv+H/DL1FkxSkEpFOmJUlAZTDxZdlrdfFqUX7W3msXbfqCkbBEz10yLWRa9UC4+hSI9UQoqQ4l3iKDH54+6n98v8WmIbS37bQlev5dsew4PL7iXam91zPLogc8v1RhvhSINUQoqQ0lkiGBVFCtKa2yruKwIgKdOeZFfy7fw6pLn4pQocdKhZkuhUByMoSPfFakjkRRwl8dHsywbQoTuYac1tlVSVkS33MMZc+QFvL96Ks8s/gfj+11OXnbbuGWLF4/Pj8NmvuexKreXJnZr2GudCfj8kr1V7qSe0yIELbPtGX1dGwPm+8YqdCGRIYJSRk6C0FKgK6WkeNsiCjscDcDfj3+Yak8V/1z4SNxyJYIZEyW8Pj8VLq9ugyPNSrXHh6/WLZysl8fnpzLDr2tjQCmoDCXRMRjhFk0ptdVWbT6wiR1V2xlyWEBB9Wx1JH8ccDVvLXuVn3evTki2eDBjokS5y4sEKmu8mmJ66Up1ihRFVYZf18aAUlAZiB5DBMMlS2h1HZaULQSos6AAbjvmHrLtOTw4/54EpYsdsyVKuDy+umspgQpXYnO5zIrL40tZQXcmX9fGglJQGYheQwRDpZxrTTYoLisi255Dnzb96t5rk53HjUNvZ/Yvn7Fg81xdZIwFsyRKSCkpb7Bwury+tOhtGCuuOHo86np+r48ar3L1pStKQWUgHq8+C3F1iM4SWhfR4m2LGNS+EJvl4DycPw+6nk7NuzBp3gR8/uQuHF6TuPkq3aGtinKXJwXSGIfPLzUXdBtJw4cBRfqgFFQGUuPTZ+FvmCyhtbaqylNF6a5lB7n3gjhtTu4Z9gArdv7EjFVTdJFTK2awoHx+SVVN6AXT65dUuTNnMY2nQ74R+PySyjDXXGFulILKMPQeIlg/wK21tuqn2gLdISEUFMA5R45lUPtCJv/vPqo8VTpJGh29XJ+JUO7yRLyGFTXejOm+nqrkiFBUZtB1bUwoBZVh6G0luOslS2hd4ItrEySGdBga8nMhBJNGTKasYhsvljyjj6AaSHWiRI3XF9XlJSVUZIAVVeNNXXJEKCRQrqyotEMpqAzDiBlNQVeN9vhTEYe36EHrJm3CbnN0x2Gc0WMMzxX/k98qynSRUwupcvOFSowIR7XbZwprLxFcbvPJ7/JkZiJKJqMUVIZhxMIWTJbQND5eSorLisK69+oz8fiH8fjcPP7Dg3qIqYlUJUpUuX0x1eSkc2Df75emzZzLtESUTEcpqAxCaxPXWJEyEBvRcuRN+39hd/XOkAkSDene4giuHPgXpqx8g1W7ViQuqAZSYUHFE6T3+PymiuHEQrXHl3AdnlF4/TJtr2tjRCmoFKL3U6aRI9i1tuMJNog96rBjNG1/yzF30czRnAfm3R23bLEQT4f3RKlwaVPuDSmv8ZiquFgrZsneC0d5jcdUCROxDAptbCgFlUIqa3y6FjIaqaC0UlxWRFNHM45sna9p+5bOVtxy9ATmbJrNnI2zDZYu+YkSbq8fV5wPIkHLNZ1we/2mby9kpkQUv19yoNqT8f0Y40UpqBThr21oWe7y6rZgJtIgVi9KyooY1K4Qq8WqeZ8rB15L19zu3D/vLsOLdyVo6iWoF4nGPKrcvpRYffFiduspiFkSUcprfu/HaCarziwoBZUigtaOX0pdnpK1NnE1kkpPJaU7l1N4WPT4U32ybFlMHP4Qq3evZOrKNw2S7neStTBVub26/E3SJWHC75fUpImCgtRfV7fXX+dBUWnwoVEKKkXUr4ep1uEp2QzuvaXbS/BJn6YMvoac2fNchh52LI/97wEq3RUGSPc7yUiU8Pv1efCAwN821T3ttODymjc5IhSeFF/Xhta1SoM/FKWgUkT9BAlJ4rEGM7TxiVagG4lg8e6Oqu08V/yk3qIdRDJcZuU1Xl07eujpCjaKdMyOS9V1rXb7QlrXKg3+YJSCSgFu76HtiGq8iT3NmeHJq3hbET1a9qKls1Vc+w/uMJQxvS7gxZKn2Va+VWfpfsfoRAkjnsz9Upp6AJ/H50+5izke9HKxx3ROv6S8JrQiUmnwB6MUVAoI546rqInvaU5rE1cjkVJSsn1RXO69+tw9/EF80sfk/92vk2SHYnSihFGxDTMP4EvnLDQ9XOyxUOGObF2bLQ0+lSgFlQLCBZJ9fhnXF11rE1cj+WXfevZU79Jc/xSOrrnduLrgOqaXvsPyHUt1ku5QjBoBb2R2mMScLiAp0ys5oiGB65ocK8qroQDbTGnwqcYWfROFnvj8kbPtKmu8OO1WrBah+ZhaF8SdVTv4Yet8zu51vuZjayVYoJuoBQVw09A7mLryTW6bfR2jjzgz5v3tFhuX9L+KVk1ah93G4/fTBO2p8FqQMrzrRi9qvH5qvD6ybPrKngguT+ITnFNNMBHFaTf2usbSjzHbbsVmbdw2hFJQSSZarCg4pjo32675mFoV1Eslz/Jc8T85omVP+uYN0Hx8LRSXFdHM0ZwjW/dJ+Fi5zhZMOuExbpv9V5btWBLXMWxWO38dcnPYz42woCp0TowIR7nLiyPHghDaH2KMJF1qn6JR7vKSZTPuuro8vpiybctdXlrmOAyRJV1QCirJaGlv5PL6aOK14rBpe3rSetMHs+ymlb7N/Sc8rmkfrZSUFTG4/VFYhD5PfGPzL+H83uPj2veo145k5Y5lEbfRO+bg9fmTFocJuoJzslL/9fX6/KYoeNWDYCJKUwOuayzd7IMky6ozM6m/wxsRUkrN2XblLg+tm2ZF3c6jcUChx+fhp98C1sj7q99j4vCHsVu1W2mRqHCXs2rXCm4eOkGX4wWJpRtFffLzBrBiZ2QFJQlcO7tOLpRkF31W1sRfBGy3CrId+nz1qzLEegpSVePFYbVofjjUSqU7vvlYRlh1sjZzMZ7bR897RwuGOjiFEKOFEGuEEOuEEIesXkKIp4QQS2tfPwsh9tX7zFfvs1lGypks3D7tvnqt47+1Pr2u3LmMam815/Uex66qHczZpF/fux+3F+OX/pg7SBhFv7wBrNu7BpfXFXE7vdx8sbpu9EDWnjeeV7nLy/7qxBvRSinTooA4FiSwr8qt6+/l80uq4kxl17u8wOeX7Kl0U+WO796p8ST3PjdMQQkhrMDzwOlAPnCREOKgDqJSyluklAVSygLgX8AH9T6uDn4mpTzbKDmTSbRpqg3RMv7b49W2yASTGO489j5aN8ljWunbMcmi5diD2x+l2zETIT+vP16/l593r4q4nUeH2VDxuG7MgMvjY19VYunMNSHq+TIBCeyv9uhWH1Xu8iSURKJXeYHH52dPpTut6tWMtKCGAuuklBuklG5gKjAmwvYXAVMMlCflxFpMqyXdVOuTe0lZER2aHkbXFt05r/dYvtrwKXtde2KSJxzFZUX0bNWbFs6WuhwvUfrlDQSI6ubTw4KK13VjBtw+P3uq3HHH4zK9oLSyxsv+qsQszRqvL+YH04boUV7g8vjYW+lOu3vVSAXVEdhS7+ette8dghCiK9Ad+Lbe204hRLEQYqEQ4hzjxEwOXl98Ywgi1dX4/FLzDbe4bGFdCvjY/Etx+9zMXDM9ZnkaIqVkSdmihOuf9KRbi8PJtudQGlVBJbZwJOK6MQs+v2RPlTvmhyevz2+K/o9G4/L62BunpamndR0sL4iHyppal64ukiQXIxVUqKheuGs0Hpghpaz/F+gipSwELgaeFkIcccgJhLimVokV79y5M3GJDSSRL3O4m1xr/Om3ijK2HthcN+W2X95A+rTpp4ubb/3etex17dE0QTdZWISF/Db9NCVKJKKk4h1EaDakjD3ukimp5Vrw+Pzsrozd0qxy+3Tt/BFP30A9XZWpwEgFtRXoXO/nTsC2MNuOp4F7T0q5rfbfDcBcYFDDnaSUL0spC6WUhXl5eXrIbBiJBBfD9XbT6joIxoiCSkQIwdj8S/hxezFr96yJW676x9ajQFdP+uYNpHTX8qhf6Hj98TVeX9yDCM1IrHGXxqSgIJCssKfKrdmK8fsllTorhlg6zfj9kr2V+iZ7pAIjFdRioKcQorsQwkFACR2SjSeEOBJoCfxQ772WQois2v+3AYYBpQbKaihSyoRrRUI9PWk9ZnFZEQ6rg/5tf9fx5/cej1VYE7aiirctJDerBT1bHZnQcfSmb15/DtTsZ8uBzRG3i/fvko6JEVrQEncJjChPolAmQUrYX+XRFHsLDiLUGy2DDevcthnggjVMQUkpvcD1wJfAKmCalHKlEOIBIUT9rLyLgKny4G9EH6BYCPETMAeYLKVMWwVV4028FUzDrst+v9TsPigpK6J/2wKybL/XVbXNac+obqcyY9W7CU2xLda5QFcv+tV2yli586eI28UzpqTKbd6mrXoQLe6S6ckRkZDAAZcnYtKCO8HJBNHOH2mwodvrZ3dlTcbcn4auKlLKz6SUvaSUR0gpH659714p5ax620ySUk5osN//pJT9pZQDa/99zUg5jSbRLJ4g9bsua306cvvc/PTbkpAxorH5l1BWsY35W+bEJc+Bmv2s2V1qOvceQO82/bAICyt3Lo+4XaxxBT0HEZqZcHEXn19mxJN5olS5feyrcoe0NI1u6BtusGGgdMCdUdatuR57M5R4s28aUr/rslbX1IqdP1HjqwmpRE49/Axys1rE7eb7cXsxEmmaAt36ZNuzObxFj6gWVKyJEnoPIjQzoeIujS32FIkab6CuqL61Em4Qod40VIIVaZypFwmloAxGaysirQT7c2l1TZVsOzhBoj5Om5NzjhzL5+tmUV5zIGZZisuKEAjTFOg2pG/egKgWFGhPlEj1iPBUEMjw+z3u0pjde6Hw1nZm8Pj8EQcRGnHearcPKSX7qzy6J2SYBaWgDEYv9159yl3emBIkOjbrxGHNOoX8fFz+JVR7q/l47QchP4927CNb59M8KzfmfZNB37wBbD6wkf2ufRG303otMzUxQgsHXJ60LPRMBn4ZyJgLtI9K3nnLazzsrfJkVDZpQ5SCMhgjRrHHskgU1yvQDcWg9kfRo2WvmN18fulnSdkiU9U/NSQ4UqR014qI22npKBGwWht37EXFnsIjSf71kTL+LNR0IaqCEkJcL4QwRw+bNMPvTzy9PBHKKn7l1/KtEZVIoCbqUhb++j0b923QfOx1e35mf80+hpgw/hSkb15/INAoNxLRevKla789hSLd0WJBtQcWCyGm1XYnN8eUtDTACPdeLBRHiD/V54I+FyEQTCt9R/uxa2dLmdmCapfTgdZN8qIqKCmJmJYbGE2gXFsKRbKJqqCklBOBnsBrwBXAWiHEI6FaDykOxgj3XiwUlxWRZc2iX9uCiNsd1qwTI7qcyPRV7+CX2mNbLbJackTLnhG3S+XTjBCCfhoTJcJZul6fXyUGKBQpQlMMqraIdnvty0ug88MMIYS+Y1kzjBpfahe2krIiBrQdhMMafWz02PxL2HJgEwt//V7TsYu3FTG4w9CoBbpWi0jpRND8vP6s2b0Sjy9ydlU4BVVhUEcAhUIRHS0xqBuFECXA48D3QH8p5V+BIcD5BsuXttR4U9sOpsZbw7IdP2qOEZ3eYwxNHc00JUvsd+3j5z2rNLn37DYLTbNsKbOk+uUNoMZXw7q9P0fcLlSihB6jEhQKRfxosaDaAOdJKU+TUk6XUnoApJR+4ExDpUtjUu3eW7FzKW6fm6M6aBuDkW3P5qye5/Hxzx9Q6amMuO2S7YsBbfEnu8USsKIcqbGi8oOZfDEmSqjECIUi9WhRUJ8BdZPthBDNhBBHA0gpI48sbcSk+sm7YQdzLYzLv5RKTwWfrf0o6rEtwsKg9oVRj2m3Bmynpo7UWFE9WvYiy5oVffRGg0QJvUclKBSK2NGioF4AKur9XFn7niIMvhgauRpF8bYiOjXvQrumHTTvc3THYXTN7R7VzVdSVkTv1n1pltU84nYCsFkDt5jFImiSAivKbrVzZOu+lMaQKGHEqASFQhE7WhSUqN9pvNa1ZzNOpPRHr957iVBSVhRzCrgQggv7XMKCLXP5tXxLyG380k+JxgLdoHIKkpMiK6pvXn9W7FymeTaUUaMSFApFbGhRNBuEEDfyu9X0f4D2is5GSCLDCfXg1/ItbKv4Na4apQvzL+GJhQ8xY9UUbhp6xyGfr9m9inL3AU0dzIPuvSAWiyA7y5Z066Rv3gCmrHyDHZXbI1qUHq8ftzV5/fYq3RXcNecWrhp4LQXthyTlnKngxZJn+OjnGUk9Z5Y1i78f/7ApO+0rtKNFQV0LPAtMJNDR4xvgGiOFSmf0GE6YKMEC3Xi+nF1zu3FMx+FMK32bG4+6nYZ12SXB2JaG7EC79VADPcdhpcqd3I7gwZZHK3Yui6yg/H7DRyXU598lTzOt9G1Kdy7ny4u/x2pJXTq+UVR6KvnHDw/RNqdHHd5vAAAgAElEQVQd3Vokr3Ry+Y6l3PzVtcy5bDE2i3L4pCtR/3JSyh0EpuFmNH5/IGureRPbIYtyLOgxnDBRSsqKcFqddQtzrIzrexm3fPWXgJvwsIOzAIvLimjlbM3hLXpEPU4oBSWEIMdhS+pMpfotj07qflrY7aQEb5I05/aKbfy7+Cm6NO/Gip0/MX3Vu4zve1lSzp1MPl83i0pPBU+e8iHHdhqetPN+tu4jrvp4PO8s/y+XD/xz0s6r0BctdVBOIcR1Qoh/CyH+E3wlQ7hkE22SqBbM0FCzuKyIge0GayrQDcVZPc+liS07ZOujkrJAgW40JS5EoEg3FNkOK8lsmNU8K5cuzbtFbXmUTCb/73580su08z9lUPtCJn9/X9T0/nRkWunbdGnejaM7HpfU855+xNkc03EYj//wYFyjZBTmQEuSxFsE+vGdBnwHdALKjRQqlXh8fvZUHTpJVCupjj+5vC6W71iakO+9qaMZZ/QYw0c/z8DlddW9v9e1h7V71miKbTlCWE9BhBA0zUqu26VvXn/TKKgVO37ivZVvcVXBX+nW4nDuH/EY2yvLeLHkmVSLpiu/lm9h/uY5XJh/SdSOI3ojhGDSiMfYXb2Tfy1+IqnnVuiHlrumh5Ty70CllPIN4Aygv7FipRafPzBJNNZiW4/Pn/Kmost3/IjH7+Gow7QV6IZjbP6l7K/Zx5frP6l7b0lZbYGuhmM3zOBrSBO7FUsSzai+eQNZv3dtyq0UKSX3z7uLFs6W3Dz0TgCGdjyOM3qcw/PFT/JbRVlK5dOTGaumIJGM7XNJSs5f0H4I5/cez8tL/sXWA5tTIoMiMbQoqGDUeJ8Qoh+QC3QzTCKTEJgk6o4poyvV3SMgvgLdUAzrfAIdm3Vi2qrf3XzFZQsDBbrtomec2cK494Ik24rqm9cfiWTNrpVJO2covtn4JfO3zOG2Y+6mhfP3KTZ/P/5hPD43j//wYAql0w8pJdNK3+aYjsPp2qJ7yuSYMGwSEsnk7yelTAZF/GhRUC/XzoOaCMwCSoHHDJXKJEhgf7VHc0A/1d0jIJDB16V5N/Jy2iV0HKvFygV9LmbOxq/qnuqLtxWR36Y/OY6mUfeP5OIL0sRhDRun0pu+eQMBonaUMBKv38v98+7i8BY9+OOAgwP33VoczlUF1zJl5RusijJgMR1Ysn0R6/euZWz+pSmVo3Pzrlwz+AZmrJ7C0u0lKZVFETsRVxEhhAU4IKXcK6WcJ6U8XErZVkr5UpLkMwWVNV72V3kiFnqmejghBJ5aA5l3+tR+XNjnEvzSz/urp+Lz+1iyfbGm2JZFCCwaFU+yrKjOzbvQPCtX0+gNo3hn+X9Zu2c1E49/KGQCy81HT6C5I5f7592VAun05b3Sd2hia8JZPc9NtSjceNTttG6Sx6R5E6IWayvMRUQFVds14vokyWJqomX4mSF7b2v5ZrZXlulWnNijVS+GdBjKtNJ3WLO7lEpPRcIJEg1x2pNjRQkhyG/Tn5U7fzL8XKEorznAPxY+xDEdh3H6EWeH3KalsxW3HDOBuZu+5tuNXyVZQv1weV18tGY6f+gxJmo7rGTQLKs5tx87kYW/LuCL9R+nWhxFDGhZSWYLIf4mhOgshGgVfBkumQmJlOGX6uw9qFdEq2P1/Ng+l7J690pe/+nlwLE1WGc2a2wKJ1lWVL+8AZTuWqF5KKOePFf8T3ZV7WDSiMcipuhfOfBauuUezv3z7sLrT89+gF+u/4T9NftS7t6rz6X9r6Jnq948OP8e3D53qsVRaESLgroKuA6YB5TUvoqNFMrMhMvwS/VwQgjEiJrYmpDfRr8kyzFHXkCWNYu3lr9GqyZt6JZ7eNR9QhXoRsJpt0ZNqtCD/LwBVHkq2bgvuZ26fi3fwkslz3J+7/FRWxo5rA4mHv8Qa3aXMmXFG0mSUF+mrXqHw5p2ZHjnkakWpQ6bxca9xz/Chn3reHPZK6kWR6ERLSPfu4d4RV+lMpiGGX5urz+lwwmDBAp0h2C32nU7ZgtnS0474kwkksIOR2vqstGwB58WmjqNt6L61bU8Sq6b79EF9yGR3DXsfk3bn9HjHIYediyP//AgFe70KjncUbmduRtnc0Gfi03Xuunk7qMZ3nkk/1z4CPtd+1ItjkIDWjpJ/DHUKxnCmZn6GX7xdC/f69rDqLeOYv7mObrIU+2tZsXOnxKufwpF0FWjxXVotYi4WkVl2awxW16x0qt1H2wWm6bRG3qxdHsJM1ZP4ZrBN9CpeRdN+wSKTCezs+o3ni9+0mAJ9eX91VPxSZ+p3HtBgtd1n2svTy8yRyLyftc+xrx3Ep+unZlqUUyJlhXhqHqv44FJQOgobyOkssZLtTt2BfXtL1+yatcK7p5ziy6xhmW/LcHr9+oafwoyquspPHDC41zc74qo2yaiZHKyjH3idtqc9Gh5ZNJSzaWU3D//Llo3yePGo26Pad/BHYZyzpEX8mLJM2wr32qQhPoipeS9lW8zuP1R9GjVK9XihKRf24GMzb+U15b+m037N6ZaHJ5d/A+Ktv2Pu+fcQqW7IvoOjQwtLr4b6r3+DAwC4mvylqHE492bs+lrbBYba/es4e3libc2DBboDu4wNOFjNcRqsXLN4Btok50Xddt43HtBsmzWmDIA46Ff3oCo49/14ssNn/DD1vncfuzEuLLZ7h72IH7p59E0KTJdvmMpq3evNKX1VJ8Jw+7DKmw8vGBiSuXYvH8Tr/z4HIUdjua3yu38u+TplMpjRuJZDaqAnnoL0pjwSz9zN33N2b3O59hOx/OPHx5KuKFlSVkR3XIPJy+7rU5SxkeibrocgzP68vP6s63iV3ZX7zL0PB6fhwfn30PPVr25tP9VcR2jS25Xrh50HdNXvcOy337UWUL9mVb6Ng6rg3OOvDDVokSkQ9OO/HXIzcz6+X2Kty1MmRyPfP93rMLKy2e8zZheF/Dv4qfYXrEtZfKYES0xqI+FELNqX58Aa4CPjBctc1m5cxm7qnYwsuspTBoxmd3VO3l28T/iPp6UkuJt2qbcGokgeoujaDhsFrJsxllR/Wo7Shgdh3pj2Sus37uWe49/JKF5RDcNvYNWztamLzJ1+9x8sGYapx1+5kEtnMzKdYW30Da7PffNuzMl13VJ2SJmrpnOtUNu4rBmnbh7+IP4pJfJ/9OWSNNY0LISPAH8s/b1KDBCSjnBUKkynLkbZwMwsutJDGw3mAv6XMzLS/7FlgOb4jrelgOb2VG1nSE6dZCIF5vVktAsrSBGWlH59WZDGcV+1z6eXPgIwzuP5OTuoxM6VvOsXG479h7+t3Ueszd8ppOE+vPNL1+wp3qX6d17QXIcTbnzuHspKVvEx2s/SOq5pZTc992d5GW347rCW4HAoNA/Ffwf7618ixU7UlNMbka0KKjNQJGU8jsp5ffAbiFEN0OlynC+3TSbfnkDaZvTHoC7hk1CIHj0+/viOl5x2Q+AvgW68RBrgW447FYLTpsxCRNtsvNon9PB0ESJZxY9zl7XHiaNmKyLwv5j/6vp0bIXD8y/G48veRN/Y2Fa6dvkZbdjVLdTUi2KZsb3/SN92vTjofkTqfHWJO28n6z9kMVlC7nzuHtp6mhW9/7NQ++kpbMV98+7y9TWcjLRoqCmA/WrUn217ynioMJdzuJtPxz0Re7YrDN/GXIjH6x+jyXbF8d8zOKyIrLtOfRp009PUWPGbtHPNWdkRl/ftgMNc/Ft2r+RV5c+z9j8S+nXdqAux7Rb7fz9+IdZt/dn3lr+mi7H1JPd1bv4+pcvOK/3uLQar261WLlvxKNsPrCR//z0QlLO6fa5eWjBRHq37stFfS8/6LNcZwtuPeZu5m+Zwzcbv0yKPGZHy4pik1LW9Qap/b/K4ouTBVvm4vV7Gdn14CfNG476G22y2/JAHE9PxdsWUdBuiKbFwcgZTIlk8DXEZrWQ28ROc2fsryaOyMqtb5v+/LxnlSFPzY8sCAS+JwyLzxoOx6mHn8FxnUbwxMKHOVCzX9djJ8qHq6fh8XsYlybuvfqM7Hoyo7qdytNFj7Gnerfh5/vvTy+yaf8v3Dfi0ZCFzJcP+DOHt+iR1q2u9ESLgtophKirexJCjAGMTYHKYOZs/Joce9NDCmqbOppxx7F/Z+Gv3/P5+lmaj1flqaJ01zLNBbpG9b0TRB9SGCtOu5Umjthf2fYoCipvAF6/l5/3rNJV3pKyIj76eQbXDrmZDk076npsIQT3jXiUPdW7eGbR47oeO1Gmlb5D/7YFdfG9dOPe4x+h3H2AJxc+Yuh59rr28NTCybVKMbQrNGgtr92zmneW/9dQedIBLSvKtcDdQojNQojNwJ3AX7QcXAgxWgixRgixTghxSGKFEOIpIcTS2tfPQoh99T67XAixtvZ1ecN90xEpJXM2fsXwziNDjlu4uN8V9GrVJ6aGlj/VFuhq6WButQicdgtGGFFGd4GIBZvVQqRfMeh603P0hpSSSd9NoG12e66vDXzrTTCh5pUfn2Pz/vgSavRm1a6VLNuxJGVTc/WgT5u+XNz3Cl5f9jLr96417DxPLXyUA+793Dfi0YjbjT7iLI7pOJzHf3gw4fKTdEdLoe56KeUxQD7QV0p5nJRyXbT9hBBW4Hng9Np9LxJC5Dc49i1SygIpZQHwL+CD2n1bAfcBRwNDgftqhyamNb/sW8/mAxsZ2e3kkJ/bLDbuHfEIv+xbzxu13cOjUVwWqOMYoqFA124JZNllGZCAoFeChF5EGuHRLfdwmtiydR29UT/wrWWgY7zcNWwSFiw8+v29hp0jFqaVvo3NYuPc3uNSLUpC3HHc38myOg0r3v1l33r++9NLXNT38qix4mBLpt3VO/nX4icMkSdd0FIH9YgQooWUskJKWS6EaCmEeEjDsYcC66SUG2rjVlOBMRG2vwiYUvv/04DZUso9Usq9wGwgsXxdEzBnUyC9/MSup4bd5qRupzGiy4k8WfQo+1x7ox6zpKyIw1v0oHWTNlG3tdsCi7bTrr+1YyYLCiK7G60WK/l5/XSzoGq8NTy0YCJ92vRjfF9j21R2bNaZa4fcxIdrprFg81x2Ve2M+VXlqdJFFq/fy/urpnJSt9GauoyYmbY57bn+qNv4bN0sfti6QPfjPzT/HuxWB3cep+3BoqD9EC7ofREvL/kXWw9s1l2e/a59cd07et4/WtASkDhdSnl38Acp5V4hxB8IjICPREdgS72ftxKwiA5BCNEV6A58G2FffZ36KWDOxtl0b3EEXVt0D7tNMNZw8tvH8FTRZO4/IXxTSyklxWWLGNVVW2qvrTbLLstmRQiPrh3YTaegohQM920zgI9+noGUMuFU8GDge+p5Hyelg/f1R93G2yv+ywXvnx7X/k1s2fz79P9yeo/EWmp+t+kbdlRtZ1zf9EuOCMVfBt/Im8te4dbZ1/LeeZ/SJberLsct+vV7Pl33EXcce29daYkWJgybxCdrP2Ty95N47vTE26FBoIvNA/Pv5sWSZ+I+xiMnPsJdxydn6rMWBWUVQmRJKWsAhBBNgCwN+4X61odbEscDM6SUwa6rmvYVQlwDXAPQpYu2TtGposZbw/dbvuOiftGfsPvmDWB83z/yn6UvcNXAa8MqtM37N7Kraoem+ifBwVl2Trs1ria3IY8tIrvUUkE0l2PfvAG8ufxVtpZvpnPz+BeiPdW7eapoMqO6nsLIrqFdt3rT1NGMmRfOZt7mb6NvHILpq97hT59cxKOjnubygX+OW473St+ilbM1J3ePT1GajWx7Ni/94S0u++h8zpw6knfPnZlwqYBf+pk0bwLtczrwlyE3xrRvp+ZduGbwDTy7+B9cPei6qLPEolHjreHmr67hwzXTGN/3jwxoOyjmY9gsghMPH56QHDGdT8M2bwPfCCH+S0BJXAW8qWG/rUDnej93AsI1mhpPYChi/X1HNth3bsOdpJQvAy8DFBYWmrqybdG2/1HtrTokvTwcdx53LzPXTOehBRN55cx3Qm6zuDb+pGXKbcMxGE6bfgrK6Aav8WCLUpPVr21gNlTpzuUJKainih6l3H2Ae6MEvvWmR6tecXcMH9f3Mq759FLu/PZGtleWccexf4/Zitzn2suX6z/hkv5Xhkz4SVeGdjyOj8Z+w8Ufns0500/h9bOmMbzLyLiP99Ga6fy4vZinT32ZHHtOzPvfcNTfeHflG0yaN4EPL/wqbmu/vOYAV308nvlb5jBx+ENcV3hrXMdyWC20zEne31tLksTjwENAH6Av8KCUUsswlcVATyFEdyGEg4ASOiR/WghxJNAS+KHe218Cp9bGu1oCp9a+l7bM2Tgbu8XOsE4jNG3fvulhXFd4Kx+v/YDF234IuU1xWRE59qb0bt036vHsDfrbOWwW3Wqi9E4v14OAQg7/ee82/RCIhDpKbNi7jv/+9BIX972CPm2i/w3MQo49hzfOns5FfS/nqaJHue3r/4u55mbWz+9T46thbJ/McO/Vp3ebfD4eP4eOzTpz0YdnM3PNtLiO4/K6ePj7e+mXN5Cx+fFlOTbLas7tx0xk4a8L+GL9x3Ed47eKMs6dfio//Dqff532GtcfdZsuHU6SgaaVRUr5hZTyb1LK24AKIcTzGvbxAtcTUCyrgGlSypVCiAfq11URSI6YKutVp0op9wAPElByi4EHat9LW+Zu+pqjOw6LKcPrr4U30y6nfdhGoSVlRQxuf5SmuEeoLg/RClq1omeBrp5EsqJy7Dkc3rJHQqM3Hl4wkSyrkzuO+3vcx0gVNouNJ095gVuOnsC7K17nylnjYgp+Tyt9myNb5zOw3WADpUwdHZt15qOxXzOkw1Cu/exyXl7yr5iP8cqPz7H1wGYmjZiMRcT/EHdJ/yvp2ap3TOUnQdbt+Zkz3xvFhn3reGvMB1yYf3HccqQCTVdNCFEghHhMCLGRgDW1Wst+UsrPpJS9pJRHSCkfrn3vXinlrHrbTArVfFZK+R8pZY/aV1pXrG2v2EbpruUxxyhy7DncedwkSsoW8dHPMw76rNJTSenO5ZrcexBaiTh16hquZ4sjPYkWh8pv0z9uC2rh1gV8uu4jrj/qtpgC32ZCCMGdx93H5BOf4etfPufC90/XNIZk/d61FJcVMTb/krR5Eo+HFs6WTD3vE87oMYZ7v7uD++fdhV/6o+8I7KraybOL/sHJ3U9PyEUIteUnxz/Chn3reHPZK5r3W1K2iLPfO5EqTxUfXPBlWvVJDBJ2ZRFC9BJC3CuEWAU8RyAuJKSUo6SUsT9ONGLmbvoGgBO7hU8vD8e4/EvJb9Ofhxf8HZfXVff+0u0l+KRPU4GuEKHdcDarJeHxGBYhsJgsQSJINMXZr+1ANu3/JeZiyGDgu0PTw/jL4NgC32bkioHX8NpZU1ix4yfOfu/EqEXA00rfxiIsXND7oiRJmDqcNicvn/EOVwz8Cy+UPM0NX/xJkxXzz4UPU+Wp5N7j9elOcXL30RzfeRT/XPgI+137om7/1YbPOH/GaJpn5fLJ+DkJJ1ikikjf4NXAScBZUsrhtUpJn6h6I2POxq9ol9M+rmauVouVSSMms+XAJv6z9PeGliW1E3S1FuiGwxmlLVA0zJggESRaZmHfvNpEiV2x1UPNXDONpb+VMOG4+8m2Z8ctn5n4Q48xTDv/U3ZV7eTMqSPDjnzwSz8zVr3LCV1Ppl3TDkmWMjVYLVYeHfUUdw27n/dXT+WymedR4S4Pu/3aPWt4c9mrXNr/T/Rq3VsXGYQQTDphMvtce3l6UeQUgHdXvMGVs8bSs1UfPh4/h+4tjtBFhlQQaXU5H9gOzBFCvCKEOInQ6d+KCPj8PuZt/pYTup4ctztkRNcTOan7aJ5e9FidC6a4rIgeLXvR0tkq6v6RXF2JKiizdZCoT7TYWN/a3nGxuPmqvdU8vOBe+rctSDt/fjSO7jiMj8Z+g81i5Zzpp7Bg89xDtvl+y3f8Wr6VcXEG/dMVIQQ3Db2Dp059iQVb5nLe9NPYWflbyG0fnH8PTezZ/O3Ye3SVoW/eAMb1vYzXlv6bTfs3HvK5lJKniiZz6+xrGdHlRD688MuUT9hOlLAKSkr5oZRyHNCbQIr3LUA7IcQLQojYfVWNlJ9+K2Gvaw8nJuj/vff4R6hwl/PkwkdqC3SLNLn3IHIRrdUiEiqyNVuBbn2EEBEzFdvnHEarJm1iGr3x6o/P82v5Fu5LMPBtVnq3yeeT8XPp1KxLyAy290rfpnlWLqcdcVaKJEwtF/X9I2+MmcHaPWsCyQd7D+76tmDLd3y14VNuOup2Q5TDncfdi1XYDmnJ5PP7uPPbm3jsf/dzYZ9LeHPM+4a23EoWWtLMK6WU70gpzyRQj7QUUBN1NTJn09cIBCO6nJTQcY5s3YdL+13FG8te4ZuNX7KnepfmAYXRlEgirY/MmsEXJJJ8Qgj6tunPCo09+YKB71MPP4PhnU/QS0TTcVizTswcO/uQDLYKdzmfrp3JmF4X0MTWJMVSpo6Tu4/m/Qu/oNxdzpnvjeLH7cVAwP15/7wJdGzWmasHX2/IuTs07chfh9zMrJ/fp3hboA6y2lvNnz+9mDeXvcINR/2NZ097BbvVbsj5k01MK1Ntb7yXpJQnGiVQpjF342wK2g+hVZPWCR/r9mMDac03fPEnQFuBrkWIqLGYeKfX2hoU/5qRqHGotgNYvWulpjqgJ2oD338//mG9xDMtv2ewnVOXwTbr5w+o9lbFXdOTSQxufxQfj/uWpvamnDf9NL755Uuml77L8h1LuXvY/YYq8OsKb6Ftdnvum3cne117GPf+GXy+7mMeGvlP7hn+oOm/k7GQeT4KE7HPtZeS7Ys0d4+IRl5OO24cejt7XXto6mhGr1Z9ou6jxcKxWERcyQ5mLNBtSDTrsV/eQGp8NVHHLPy8ezVvLXuVywZcTc9WR+opomkJZLC9zZW1GWx3f3szh7foQWEHbbPHMp0jWvbkk/Fz6NGqF3/86HwmzbuTge0GG97ZPcfRlAnD7qOkbBEj3hjM0t9KeOmMt7h60P8Zet5UYP4VJo2Zv3kOfulnlI492q4ZfAMdm3XmmI7DtRXoalQi8SRLmN29BxqaxtYmSkQbvfHg/HvItufwt2P0DXybHavFyiOjnuLuYQ/g8rm4qN/lGfWEnihtc9rz4YVfMazzCex17WHSiMeSEpscl38ZffMG4PJWM+Xcjzm71/mGnzMVGDNeVQEExms0z8plsIZUcK00sTXh84vmYbNo8zFrzbJz2i2Uu8J38w2FmRMkggSHF4b7vXq0PBKH1cHKncs5r/f4kNss2DyX2b98xj3DH0z7sRLxIITgxqG3c1av8+iaG74Tf2OlqaMZ75wzk63lm5OW0m21WJlxwed4fR7yctol5ZypQCkog5BSMnfjbI7vPAqbRd/LHEvnAq2uOyEEDpuFGq+2SvlAd3TzKygIxKG8/tAqym61c2TrfFaGSTUPFuV2at6FPw8yJvCdLqRzPY3R2K32pF8fLSUm6U56rDBpyM97VrOt4te4ukfoRcMO5tGIxc2XDvGnINE6m/fNGxC2Fmp66bus2PkT9wx7AKfNaYR4CoUiDOmzyqQZczZ+BRB2vHsyiNXCybJZNFdim7lAtyHRZO2XN4BdVTvYUbn9oPerPFVM/t99FLQbwpgjLzRSRIVCEQKloAxi7qav6dmqNx2bdY6+sUHEmsQghCBLoxVl5hZHDYnaNLa25VFDK+rFkmcoq9jG/SckJ/CtUCgORn3rDKDKU8UPW+cn3D0iUeKJEWkt2k20yWwyieriaxPM5PtdQf1WUcZzxf/kjB5jOLrjMEPlUygUoVEKygAW/rqAGl+NbvVP8SCIT4lk2awRB/1B+O7oZiXa8MJcZws6Ne9ykIJ6/IcH8fjcTGwERbkKhVlJn1UmjZiz8SucVifHdBqeMhlsVkvc9SrRkiXMOv8pElFHwOcNqFNQq3atYMrKN7hy4F9U5ppCkUJUmrkBzN30Ncd2Oj6l/coSSWJw2qxUu8NPVkmnBIkgNqvAE2FYTN+8gXy14TOqPFU8MO9umjmac8sxdxkuV6QarUwi2SUJfinxhSktUKQPSkHpzJYDm1i7Zw2X9r8qpXIkksTgsFmwCIE/xJh5SJ/6p/po6Sjhl35eLHmGOZtmM2nE5KTUmTRvYueAy0OYS50R2K0WWuU4knpOKSW7Ktxh72FFepB+K43JmbvxawBGdU3tRJJEkxgiJUukp4KKVgs1EIAnFj5E19zuXDnwWsNlslstOO1WmmZl9nNiIt3y40UIQU5WYrPOFKkn/VYag/hs7Wcs37E04ePM3TSbjs06pbShqB5JDOHiUFq6o5uRaAq7S/OuNHM0xy/9TBz+EFm2LMNlCiqmJnZrxLlV6Ywg/m75iZLtsKXlvar4HaWgAK/fy99m38a5009l3qZv4z6Ox+dh3uY5jOp6akobauqRxGC3WkJ+udOhQWwoLJbIwwuFEBzTcTjDOp/AmT3PNVyeLJsFh81Sd+5MtaIcNguWFCqJTL2ujQWloACbxcZXl86mU7MuXDLzHD5c/V5cx1myfRHl7gMp7R4B+iUxhLKi0im9vCHRlOt/zprKlHNnJeXhouHC2cRhzcin/Xi65Ot9/nSq2VMcTPquNjrTqXknPhr7NUM6HM1fP7+CF0qejvkYczd9jVVYOb7zKAMk1I5eMaImIRaXdLWgIPrwQrvVjsNqfDDfabOGVPSZ9rQvCFiKqSYnw65rYyL1d4+JyHW2YOp5H3Nmz3O5f95d3Pfdnfiltu7eAN9u/IohHYaS62xhoJTR0UtBWS3ikGOlYw1UEDMkdwigqTP0gplpT/uBou/U/z5Ou9UUf3tF7Ki/WgOcNicv/eEtriq4lpeWPMt1n1+J2+eOut+uqp0s++3HlHaPAP2TGOpnYFktIqXxhEQxgwvNGcWVF055pSNZKcjeC0emWaeNBfVXC4HVYuXhkU/SIecwHv7+XnZV7eQ/Z02lWcXv6+kAABTDSURBVFbzsPvM2/wNEsmolPff03cRdtqslOMNHDuNrScIZPKlsjBWAE0dkb9yWTYrDqsPt0+75W5GhEh9/Kk+DpuFrBjmnSnMQXqvOAYihOCGobfzzKmv8L+t8zh3+qmHjGOoz5yNX9PK2ZoBbQclUcpD0duVYbGIuqJfuy31FkgiCJFaC7CJw6rp/JkQMzGTcgqSCde1saEUVBTG9b2Ut875gA371nHG1JGs37v2kG380s/cTV8zouuJWC2p/WIa0YYouNhEK3ZNB1JlBQoBOVGspyDBp/10JlW1T5GwWy2mlEsRnvT+FiSJE7udyvsXfEGVp4qz3juRJWWLDvq8dOdydlb9lvLuEWDMnKbgIMN0zuALkqo+gjkOW0zWWzrHTCxC1NV4mQ3VXSK9MOddZEIGtS/kk/FzaOZoxvkzTmf2hs/rPpuzaTYAI7uelCrxgNhHvGvFYhFkZ9lMkZGVKKlIlBACsh2xLYy22jZI6UgqWhtpxWa10CTGv4UidZj3TjIh3VscwSfj5tCz1ZFcMetC3l3xBgBzNs6mb94A2jXtENPx9F4sjUylzcmQL3Uq0o2bxqncm2bZSMdHArMr1hyHua5rJpUW6I1SUDGSl9OODy78kuO7jOLW2dcy+ftJLN72AyO7xt49ommWTdeb00gXXCZYT1BrZSbxfBYhQhY8a8FqETjT7MHAFqJ2zmxYLcI0VpQAcpvY0z7maBTqqsRBU0cz3hzzPhf0voinFz2Gx+9hVBz1Tw6rRde6F7MvDGYhme2amjkTc402NdnTfjTMbj0FyXHYok6OTgZOR6CriMowDI26KnHisDp4dvSrdGzemfmb53DUYcfGtL+ttug1y6JP3Uu8I94bI9GGF+qF1SISXrCD8b/KGq9OUhlLuigoi0WQ7Ujtda1fFxfMMHR5k3BjphHqkTsBLMLCXcPu57OL5sU8nqF+lpMeVlQiI94bG8lS5Hpl4mXbraZ42o9GuA74ZiXHkdrr2rAuLifLmlbWcjJQCipF1FdQdmvidS/pOIY9VSSjnsuuYxaexSI011ClEjNn74UilWNOQtXF2ayWtIs5Go2hd5QQYrQQYo0QYp0QYkKYbcYKIUqFECuFEO/We98nhFha+5plpJzJRnBovVKiXxQj6p8ylWRYUHovfNkOcw81TOVgwkRI1bDIcHVx6RZzNBrDHh+EEFbgeeAUYCuwWAgxS0pZWm+bnsBdwDAp5V4hRNt6h6iWUhYYJV8qsYdwxwXrXlxxBkdU/Ek7weGFfmlMVz6H1aJ7oWpwhHm5y5yxqFQPJoyXoBV1wOVJ2jktQoSti7PUZhhWuVUsCoy1oIYC66SUG6SUbmAqMKbBNn8GnpdS7gWQUu4wUB7TEG7xirfuRY8R740NIxW6UR3JzTzCPF2SI0KR7GGROVmRx5CYJcPQDBi5qnUEttT7eWvte/XpBfQSQnwvhFgohBhd7zOnEKK49v1zDJQz6YRTUPHWZ6R7l/FUYFTMzmkzdvaQGVsgmWUwYSIk67paa7MHI5EuMcdkYORVCLUCNPSp2ICewEigEzBfCNFPSrkP6CKl3CaEOBz4VgixXEq5/qATCHENcA1Aly5d9JbfEISIXK+U47BR7fbFNBJCJUjETuBvoL8bxeheb067lcoaL15/qoaGHIpZBhMmgtMecKt5DB5zolURZte6+YxyQ6cLRj72bAU61/u5E7AtxDYfSSk9UspfgDUEFBZSym21/24A5gKHzLGQUr4spSyUUhbm5eXp/xsYQJY18gIWrHuJBVWgGztGuHSc9tCj3PXGbEWdZhpMmAhGP1zYYqiLC8YcGztG3lmLgZ5CiO5CCAcwHmiYjTcTGAUghGhDwOW3QQjRUgiRVe/9YUApGYCW4Hms9Rkqgy929I5BCZLnJjLTCHOzDSZMhMCwSAPdszHGJs0cc0wWhv01pJRe4HrgS2AVME1KuVII8YAQ4uzazb4EdgshSoE5wO1Syt1AH6BYCPFT7fuT62f/pTNaFFQs9RmWFA/hS1eEELp++ZMdaA8m1MT70otMUU5BjEpwCdQ6xn6tjHroifveSfJSI2SG+DgLCwtlcXFx3Pv7/ZKdFTU6SnQoVougTVNtHSeklOyqcEf1QWfZLLTIdughXqNjf5VHl9YyAmjTNCttHhS8Pj97qzy6xDdaZjtMO/spXqrdPt3TzhO5TrsranSNOTqsFlpk21MWNxRClEgpC7Vsm1l3lsmJ5QbVakWZxdWTjlh1Si7JzoptGGGqsVkttMpxJFygaubBhInQxGElt4ldN0szy5ZYXZyeVl3ggTZ1yilWMu/uMjGx+re1uI1UBl/86BGHCrSsST83l9UiaJXjSMgtmW6tjWLBabfSXCcllaibLkun0gWnzUqLbEfaKCdQCiqpxBOAjXZzqwSJ+NHjSx/vMEIzYLUIWmU74lbUmRZ/aojTXrugJ3IMmz6ZnYkquSYOK7nZ9oTlSDZqdUsSwfEasRIpY8uoEe+NhUSHFyYyjNAsWGotqViVtTUNBhPqgcNmoWWOI67kAIF+7jmHLf6G0tkOK82d6aecQCmopJGIDzpcPURjWCCMJpGn20SHEZoFIQQts+0xWePprphjwW610Co79pidU+fMznisqKZZNpqlqXICpaCSRiIKKlx9hpEj3hsL8S4gsRRdpgNCCFpk2zV3JM+k310LsSaW1B9GqKcMsXSMb+a0ma6oO1aUgkoCocZrxEooV4GyoBInXiWf7l/8UAghyM22R1U+6TaYUC9iSSwxKrOzqVNbQ+ncJvaoPf/SAbXCJYFQ4zXiOUb9pyc14l0f4llo9RxGaEZym9gjNi3O5Oy9aGhJLDEys9NqERGHGgoCf79MuT8b752WRPSqFakfi1Ij3vUhnk7wZuworjfNnfawVmI6DibUk2iJJYFxGQaOcwkz1FCAJgs4nVAKKgnopaBs9Z7cVf2TPgSHF2ol0aLLdKJplu0QZZyVpoMJ9SZcYkmkYYR6EaqhtBDQItsRVzslM9M4vmkpJNp4jVgJ9mBT9U/6EYurNBNjT5HIybIdlKKcSU/niRIqsSRZdXHZ9t8bSguRmS2nQCkow4k2XiNWgkMNVfxJP7Rao0YPIzQrda1/RPoPJtSb+okl8Q4cjYfgUEOLCMTEMvW+bFyPgynAiKeaHEd69X4zOzZL9OGFehZdpiPBgnEV9wxNbhM7XoOHHTYk22GtU4yZSuP9xiUJIxSUUk76osWC0rvoMh1p7L9/NJIxrLI+QggyPRSdmXahSbDoPHNIYQzR3KVGFF0qFIroKAVlIJkYtMxEog0vTLdxGgpFpqBWUANRAeX0IVw9VLqO01AoMgG1ghqISgVPH8INLzS66FKhUIRHraAGEe94DUVqCBWHSkbRpUKhCI9SUAah4k/pRSgFlc7DCBWKTECtogahFFR6YbNaDupvlsyiS4VCERq1ihqAakWUntSvY2kMDWEVCrOjVlEDUBX36Ukw1TzTx2koFOmCUlAGoNx76UlweKGynhQKc6BWUgNQCio9sVoEDmvjGaehUJgd9ahYi14eOb3HayiSh91ioalTuWYVCrOgVtJahBA006FbtUqOSF8sFqEeLhQKE6G+jfXIdtgCc28SOIZyDykUCoU+qNW0AU67leYJKCllQSkUCoU+qNU0BE67lRbZjpjjUhYhkj4TRqFQKDIVtZqGwWGz0DJGJaXcewqFQqEfakWNgN1qoVW2A4tGLaXGaygUCoV+qBU1CjarhVY5Dk2TcVX8SaFQKPRDragasFoErbIdEUeDq/EaCoVCoS9KQWnEYhG0zHaErZNR8SeFQqHQF7WqxkBASdlDuvKUglIoFAp9MXRVFUKMFkKsEUKsE0JMCLPNWCFEqRBipRDi3XrvXy6EWFv7utxIOWNBCEGLbDtO2+/drtV4DYVCodAfw3rxCSGswPPAKcBWYLEQYpaUsrTeNj2Bu4BhUsq9Qoi2te+3Au4DCgEJlNTuu9coeWNBCEFuth2qweXxqfEaCoVCYQBGPvYPBdZJKTdIKd3AVGBMg23+DDwfVDxSyh21758GzJZS7qn9bDYw2kBZ4yK3iZ1sh1W59xQKhcIAjFxZOwJb6v28tfa9+vQCegkhvhdCLBRCjI5hX1PQzBlQUgqFQqHQFyPHbYTyeckQ5+8JjAQ6AfOFEP007osQ4hrgGoAuXbokImtCKPeeQqFQ6I+RFtRWoHO9nzsB20Js85GU0iOl/AVYQ0BhadkXKeXLUspCKWVhXl6ersIrFAqFIrUYqaAWAz2FEN2FEA5gPDCrwTYzgVEAQog2BFx+G4AvgVOFEC2FEC2BU2vfUygUCkUjwTAXn5TSK4S4noBisQL/kVKuFEI8ABRLKWfxuyIqBXzA7VLK3QBCiAcJKDmAB6SUe4ySVaFQKBTmQ0h5SGgnLSksLJTFxcWpFkOhUCgUERBClEgpC7Vsq/KjFQqFQmFKlIJSKBQKhSlRCkqhUCgUpkQpKIVCoVCYEqWgFAqFQmFKMiaLTwixE9hU+2MbYFcKxTEz6tpERl2fyKjrEx51bSITvD5dpZSaOitkjIKqjxCiWGsaY2NDXZvIqOsTGXV9wqOuTWTiuT7KxadQKBQKU6IUlEKhUChMSaYqqJdTLYCJUdcmMur6REZdn/CoaxOZmK9PRsagFAqFQpH+/H97dx8jV1XGcfz7c8FUKzEokRREW5D1BSoF3BrTCi2+oTYWX6rWDbQGEbWt1ajBl6hAgiliDSSKaBUpBNQKKBhioJpC8R+60C52bSlgqQg0XRITtUarZX/+cc7o3dmZ3Z1SuJc7zyfZzJ0zZ+48c3Jmnj33zj2nriOoEEIIz3GRoEIIIVRSrRKUpDMl7ZD0sKQvlh1P1UjaJWmrpEFJXT/1u6SrJQ1LGiqUvUTSekkP5dvDy4yxLG3a5kJJj+f+MyjpXWXGWCZJx0jaIGm7pD9IWpnLu77/jNM2Hfef2pyDktQDPAi8jbQi7wCw2Pa2UgOrEEm7gDfYjosJAUmnAXuBa22fmMu+CfzF9qr8T87hti8oM84ytGmbC4G9tr9VZmxVIGkaMM32ZkmHAfcBZwFL6fL+M07bfJAO+0+dRlCzgYdt77T9b+CnwMKSYwoVZnsj0LwQ5kJgbd5eS/pgdZ02bRMy27ttb87bfwe2A0cT/We8tulYnRLU0cCfC/cf4wAbpcYM3CHpPkkfLzuYijrS9m5IHzTgZSXHUzXLJf0+HwLsusNXrUiaDpwM3EP0n1Ga2gY67D91SlBqUVaP45cHzxzbpwDvBJblwzghTNb3gOOAWcBuYHW54ZRP0ouAm4DP2P5b2fFUSYu26bj/1ClBPQYcU7j/cuCJkmKpJNtP5Nth4Bekw6JhtD35GHrjWPpwyfFUhu09tp+yPQKsocv7j6RDSV/A19u+ORdH/6F12xxI/6lTghoAjpc0Q9LzgQ8Dt5YcU2VImppPWCJpKvB2YGj8Z3WlW4EleXsJcEuJsVRK44s3ey9d3H8kCfgRsN32twsPdX3/adc2B9J/avMrPoD8s8XLgR7gatuXlBxSZUg6ljRqAjgEuKHb20fST4B5pGUA9gBfB34JrANeATwKLLLddT8WaNM280iHZwzsAs5vnG/pNpLmAncDW4GRXPxl0rmWru4/47TNYjrsP7VKUCGEEOqjTof4Qggh1EgkqBBCCJUUCSqEEEIlRYIKIYRQSZGgQgghVFIkqFAbkixpdeH+5/MEpwdj39dI+sDB2NcEr7MozwK9oal8en5/Kwpl35G0dIL9PVtx35gvZWjMmn9E3j5V0iOSTpa0QNJFz3QsoT4iQYU62Qe8r/HlWBV5pv3JOhf4lO35LR4bBlbmC9GfcZIOmWS9E4Ae2zubyl8P3Ah8yPYW4DbgPZJeeNCDDbUUCSrUyX7gB8Bnmx9oHklI2ptv50m6S9I6SQ9KWiWpX9ImpbWzjivs5q2S7s71FuTn90i6TNJAngTz/MJ+N0i6gXTBYnM8i/P+hyRdmsu+BswFrpJ0WYv39yTwW/4/U0Fxf+flGO6XdFNTEmgV9xRJP84xbJE0P5cvlfRzSb8iTSw8TdJGpfV7hiS9uUVc/YydMeG1pIuez7a9CcDposs7gQUt9hHCGJGgQt18F+iX9OIOnnMSsBKYCZwN9NqeDfwQWFGoNx04HXg3KYlMIY14/mq7D+gDzpM0I9efDXzF9uuKLybpKOBS4AzSlfV9ks6yfTFwL9Bv+wttYl0FfK7FqOxm2322TyItb3DuBHEvA7A9k3SF/9pcDvAmYIntM4CPALfbnpXbabBFTHNIa/4U3QIst/27pvJ7gVZJLoQxIkGFWsmzJl8LfLqDpw3kNWz2AX8E7sjlW0lf7g3rbI/YfgjYCbyGNKfhOZIGSdPcvBQ4PtffZPuRFq/XB9xp+0nb+4HrgUnNLJ/3t4mUOIpOzKOkraQRzQkTxD0XuC7v8wHgT0Bvrr++MD3PAPDRfC5vZl7fp9k00uiu6DfAx1ok0mHgqMm81xAiQYU6upw0gphaKNtP7u95MsvieZx9he2Rwv0R0ryFDc3zgpm0zMsK27Py3wzbjQT3jzbxtVoaphPfAC5g9Of3GtKIZSZwETCl8Fi7uNv5X9x54cLTgMeB6ySd06L+P5teD2B5vr2yqXxKrh/ChCJBhdrJ//2vY/Rhrl3AqXl7IXDoAex6kaTn5fNSxwI7gNuBT+blBZDUm2eLH889wOmSjsgjjMXAXZMNIo94tjH6XM5hwO4cR/8k4t7YqCeplzS56Y7m15L0SmDY9hrSDNWntAhpO/CqprKR/L5eLeniQnkvXTwLeuhMJKhQV6tJM3E3rCElhU3AG2k/uhnPDlIi+TXwCdv/Ip2n2gZsljQEfJ/Ro64x8gzOXwI2APcDm213uizDJaQ1zxq+Skp864EHJhH3lUBPPiT4M2BpPsTZbB4wKGkL8H7gihZ1bsv1Rsn7W0j65d6yXDw/1w9hQjGbeQjhaZH0AlKynWP7qXHqHUla5uUtz1pw4TktElQI4WmT9A7SAnWPjlOnD/iP7Va/BAxhjEhQIYQQKinOQYUQQqikSFAhhBAqKRJUCCGESooEFUIIoZIiQYUQQqik/wKC6WZHRmFYZgAAAABJRU5ErkJggg==\n",
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "plt.plot(range(1,Ks),mean_acc,'g')\nplt.fill_between(range(1,Ks),mean_acc - 1 * std_acc,mean_acc + 1 * std_acc, alpha=0.10)\nplt.legend(('Accuracy ', '+/- 3xstd'))\nplt.ylabel('Accuracy ')\nplt.xlabel('Number of Nabors (K)')\nplt.tight_layout()\nplt.show()"
        },
        {
            "cell_type": "code",
            "execution_count": 26,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "The best accuracy was with 0.7857142857142857 with k= 7\n"
                }
            ],
            "source": "print( \"The best accuracy was with\", mean_acc.max(), \"with k=\", mean_acc.argmax()+1) "
        },
        {
            "cell_type": "code",
            "execution_count": 66,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "0.7857142857142857"
                    },
                    "execution_count": 66,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "neigh = KNeighborsClassifier(n_neighbors = 7).fit(X_train,y_train)\nyhat = neigh.predict(X_test)\nmetrics.accuracy_score(y_test, yhat)\n#metrics.log_loss(y_test, yhat)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Decision Tree"
        },
        {
            "cell_type": "code",
            "execution_count": 28,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.tree import DecisionTreeClassifier"
        },
        {
            "cell_type": "code",
            "execution_count": 36,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "0.7714285714285715"
                    },
                    "execution_count": 36,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "drugTree = DecisionTreeClassifier(criterion=\"entropy\", max_depth = 6)\ndrugTree.fit(X_train,y_train)\npredTree = drugTree.predict(X_test)\nmetrics.accuracy_score(y_test, predTree)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Support Vector Machine"
        },
        {
            "cell_type": "code",
            "execution_count": 37,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn import svm"
        },
        {
            "cell_type": "code",
            "execution_count": 44,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "0.7857142857142857"
                    },
                    "execution_count": 44,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "clf = svm.SVC(kernel='linear')\nclf.fit(X_train, y_train)\nyhat_ = clf.predict(X_test)\nmetrics.accuracy_score(y_test, yhat_)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Logistic Regression"
        },
        {
            "cell_type": "code",
            "execution_count": 46,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.linear_model import LogisticRegression"
        },
        {
            "cell_type": "code",
            "execution_count": 67,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "0.7857142857142857"
                    },
                    "execution_count": 67,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "LR = LogisticRegression(C=0.01, solver='sag').fit(X_train,y_train)\nyhatLR = LR.predict(X_test)\nmetrics.accuracy_score(y_test, yhatLR)\n#metrics.log_loss(y_test, yhatLR)"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": ""
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Model Evaluation using Test set"
        },
        {
            "cell_type": "code",
            "execution_count": 68,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.metrics import jaccard_similarity_score\nfrom sklearn.metrics import f1_score\nfrom sklearn.metrics import log_loss"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "First, download and load the test set:"
        },
        {
            "cell_type": "code",
            "execution_count": 69,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "--2020-07-03 13:09:13--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 3642 (3.6K) [text/csv]\nSaving to: \u2018loan_test.csv\u2019\n\n100%[======================================>] 3,642       --.-K/s   in 0s      \n\n2020-07-03 13:09:13 (393 MB/s) - \u2018loan_test.csv\u2019 saved [3642/3642]\n\n"
                }
            ],
            "source": "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Load Test set for evaluation "
        },
        {
            "cell_type": "code",
            "execution_count": 70,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>9/9/2016</td>\n      <td>9/15/2016</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>9/11/2016</td>\n      <td>9/25/2016</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30       9/8/2016   \n1           5             5     PAIDOFF        300      7       9/9/2016   \n2          21            21     PAIDOFF       1000     30      9/10/2016   \n3          24            24     PAIDOFF       1000     30      9/10/2016   \n4          35            35     PAIDOFF        800     15      9/11/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   50              Bechalor  female  \n1  9/15/2016   35       Master or Above    male  \n2  10/9/2016   43  High School or Below  female  \n3  10/9/2016   26               college    male  \n4  9/25/2016   29              Bechalor    male  "
                    },
                    "execution_count": 70,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df = pd.read_csv('loan_test.csv')\ntest_df.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 72,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "(54, 10)"
                    },
                    "execution_count": 72,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df.shape"
        },
        {
            "cell_type": "code",
            "execution_count": 71,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>2016-09-09</td>\n      <td>2016-09-15</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>2016-09-11</td>\n      <td>2016-09-25</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30     2016-09-08   \n1           5             5     PAIDOFF        300      7     2016-09-09   \n2          21            21     PAIDOFF       1000     30     2016-09-10   \n3          24            24     PAIDOFF       1000     30     2016-09-10   \n4          35            35     PAIDOFF        800     15     2016-09-11   \n\n    due_date  age             education  Gender  \n0 2016-10-07   50              Bechalor  female  \n1 2016-09-15   35       Master or Above    male  \n2 2016-10-09   43  High School or Below  female  \n3 2016-10-09   26               college    male  \n4 2016-09-25   29              Bechalor    male  "
                    },
                    "execution_count": 71,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df['due_date'] = pd.to_datetime(test_df['due_date'])\ntest_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\ntest_df.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 73,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>2016-09-09</td>\n      <td>2016-09-15</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>2016-09-11</td>\n      <td>2016-09-25</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n      <td>6</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30     2016-09-08   \n1           5             5     PAIDOFF        300      7     2016-09-09   \n2          21            21     PAIDOFF       1000     30     2016-09-10   \n3          24            24     PAIDOFF       1000     30     2016-09-10   \n4          35            35     PAIDOFF        800     15     2016-09-11   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   50              Bechalor  female          3        0  \n1 2016-09-15   35       Master or Above    male          4        1  \n2 2016-10-09   43  High School or Below  female          5        1  \n3 2016-10-09   26               college    male          5        1  \n4 2016-09-25   29              Bechalor    male          6        1  "
                    },
                    "execution_count": 73,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\ntest_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ntest_df.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 74,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>2016-09-09</td>\n      <td>2016-09-15</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>1</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>26</td>\n      <td>college</td>\n      <td>0</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>2016-09-11</td>\n      <td>2016-09-25</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>0</td>\n      <td>6</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30     2016-09-08   \n1           5             5     PAIDOFF        300      7     2016-09-09   \n2          21            21     PAIDOFF       1000     30     2016-09-10   \n3          24            24     PAIDOFF       1000     30     2016-09-10   \n4          35            35     PAIDOFF        800     15     2016-09-11   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   50              Bechalor       1          3        0  \n1 2016-09-15   35       Master or Above       0          4        1  \n2 2016-10-09   43  High School or Below       1          5        1  \n3 2016-10-09   26               college       0          5        1  \n4 2016-09-25   29              Bechalor       0          6        1  "
                    },
                    "execution_count": 74,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ntest_df.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 75,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>50</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300</td>\n      <td>7</td>\n      <td>35</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>43</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>26</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800</td>\n      <td>15</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   50       1        0         1                     0   \n1        300      7   35       0        1         0                     0   \n2       1000     30   43       1        1         0                     1   \n3       1000     30   26       0        1         0                     0   \n4        800     15   29       0        1         1                     0   \n\n   college  \n0        0  \n1        0  \n2        0  \n3        1  \n4        0  "
                    },
                    "execution_count": 75,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "Feature_test = test_df[['Principal','terms','age','Gender','weekend']]\nFeature_test = pd.concat([Feature_test,pd.get_dummies(test_df['education'])], axis=1)\nFeature_test.drop(['Master or Above'], axis = 1,inplace=True)\nFeature_test.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 76,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "(54, 8)"
                    },
                    "execution_count": 76,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "Feature_test.shape"
        },
        {
            "cell_type": "code",
            "execution_count": 78,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'COLLECTION',\n       'COLLECTION', 'COLLECTION', 'COLLECTION', 'COLLECTION',\n       'COLLECTION', 'COLLECTION', 'COLLECTION', 'COLLECTION',\n       'COLLECTION', 'COLLECTION', 'COLLECTION', 'COLLECTION',\n       'COLLECTION'], dtype=object)"
                    },
                    "execution_count": 78,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "target_test = test_df['loan_status'].values\ntarget_test"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# KNN"
        },
        {
            "cell_type": "code",
            "execution_count": 80,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n  'precision', 'predicted', average, warn_for)\n"
                },
                {
                    "data": {
                        "text/plain": "(0.7407407407407407, 0.6304176516942475)"
                    },
                    "execution_count": 80,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "neigh = KNeighborsClassifier(n_neighbors = 7).fit(X,y)\nyhat = neigh.predict(Feature_test)\njaccardknn = jaccard_similarity_score(target_test, yhat)\nf1knn = f1_score(target_test, yhat, average='weighted')\njaccardknn, f1knn"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Decision Tree"
        },
        {
            "cell_type": "code",
            "execution_count": 81,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n  'precision', 'predicted', average, warn_for)\n"
                },
                {
                    "data": {
                        "text/plain": "(0.7407407407407407, 0.6304176516942475)"
                    },
                    "execution_count": 81,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "drugTree = DecisionTreeClassifier(criterion=\"entropy\", max_depth = 6)\ndrugTree.fit(X,y)\npredTree = drugTree.predict(Feature_test)\njaccardTree = jaccard_similarity_score(target_test, predTree)\nf1Tree = f1_score(target_test, predTree, average='weighted')\njaccardTree, f1Tree"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# SVM"
        },
        {
            "cell_type": "code",
            "execution_count": 82,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n  'precision', 'predicted', average, warn_for)\n"
                },
                {
                    "data": {
                        "text/plain": "(0.7407407407407407, 0.6304176516942475)"
                    },
                    "execution_count": 82,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "clf = svm.SVC(kernel='linear')\nclf.fit(X, y)\nyhat_ = clf.predict(Feature_test)\njaccardsvm = jaccard_similarity_score(target_test, yhat_)\nf1svm = f1_score(target_test, yhat_, average='weighted')\njaccardsvm, f1svm"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Logistic Regression"
        },
        {
            "cell_type": "code",
            "execution_count": 85,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n  'precision', 'predicted', average, warn_for)\n"
                },
                {
                    "data": {
                        "text/plain": "(24.22071960543458, 0.25925925925925924, 0.10675381263616558)"
                    },
                    "execution_count": 85,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "LR = LogisticRegression(C=0.01, solver='sag').fit(X,y)\nyhatLR = LR.predict(Feature_test)\nyhatLR_prob = LR.predict_proba(Feature_test)\nloglossLR = metrics.log_loss(target_test, yhatLR_prob)\njaccardLR = jaccard_similarity_score(target_test, yhatLR)\nf1LR = f1_score(target_test, yhatLR, average='weighted')\nloglossLR, jaccardLR, f1LR"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# RESULTS"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "tab = {'Algorithm':['KNN', 'Decision Tree', 'SVM', 'LogisticRegression'], 'Jaccard':[jaccardknn,jaccardTree,jaccardsvm,jaccardLR ], \n       'F1-score':[f1knn,f1Tree,f1svm,f1LR], 'LogLoss':['NA','NA','NA',loglossLR]}\nResult = pd.DataFrame(tab)\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": ""
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Report\nYou should be able to report the accuracy of the built model using different evaluation metrics:"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "| Algorithm          | Jaccard | F1-score | LogLoss |\n|--------------------|---------|----------|---------|\n| KNN                | ?       | ?        | NA      |\n| Decision Tree      | ?       | ?        | NA      |\n| SVM                | ?       | ?        | NA      |\n| LogisticRegression | ?       | ?        | ?       |"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
             "source": "<h2>Want to learn more?</h2>\n\nIBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems \u2013 by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"http://cocl.us/ML0101EN-SPSSModeler\">SPSS Modeler</a>\n\nAlso, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://cocl.us/ML0101EN_DSX\">Watson Studio</a>\n\n<h3>Thanks for completing this lesson!</h3>\n\n<h4>Author:  <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a></h4>\n<p><a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients\u2019 ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.</p>\n\n<hr>\n\n<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>"
        }

    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.6",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",


            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.6.9"

        }

    ],

@@ -1209,4 +1234,4 @@
        }
    },

}
}