Machine_Learning_Autism_Spectrum_Disorder_.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Machine Learning - Autism Spectrum Disorder .ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyO8QjS876q/IbcYpuq3qeBW"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MT8LPhh5stUi"
      },
      "source": [
        "# **Machine Learning Algorithms** - Autism Spectrum Disorder\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hOMHUIFgtF3x"
      },
      "source": [
        "## What is Machine Learning?\n",
        "\n",
        "Machine Learning is a branch of AI which uses many algorithms to improve its performance on analyzing the data and also on making intelligent decisions automatically from the previous experiences. It relies on defining behavioral rules by examining and comparing lage datasets to find common patterns. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hd_LPUI5G-wn"
      },
      "source": [
        "# Types of Machine Learning:\n",
        "\n",
        "* Supervised Learning\n",
        "* Unsupervised Learning\n",
        "* Reinforcement Learning"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aeW1UbKNZl7H"
      },
      "source": [
        "# Types of ML Algorithms:\n",
        "\n",
        "* **Regression** - The output that we want to predict will be a continuous variable. Eg: Score of student on a subject.\n",
        "* **Classification** - The output that we want to predict will be a categorical variable. Eg: Classifying the emails spam (or) ham.\n",
        "* **Clustering** - The output will be groups or clusters. No predefined notion of the label is allocated. Eg: Customer Segmentation.\n",
        "\n",
        "\n",
        "\n",
        "Regression analysis is a form of predictive modelling technique which investigates the relationship between a dependent and independent variable"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WPgPLsNIpJEo"
      },
      "source": [
        "### Variables\n",
        "\n",
        "* **Continuous Variable** - The continuous variables can take any two numbers and also all the set of real values between that two numbers.\n",
        "\n",
        "* **Categorical Variable** - The Categorical Variables can take two or more categories (values) and assign each of the individual into any one of the category and form a group. Eg: yes/no , healthy/unhealthy.\n",
        "\n",
        "\n",
        "* **Dependent Variable** - It is a variable that depends on the other factors or variables and it will change according to the according to the factors. In ML it is nothing but the data that we want to predict.\n",
        "\n",
        "* **Independent Variable** - It ia a variable that does not depend on others factors and it will not change by the impact of the other variables we are trying to measure. In ML it is nothing but the data that is given.\n",
        "\n",
        "* ***Easy Way to Remember:***\n",
        "    \n",
        "    * **Independent Variable** causes an change in the **Dependent Variable** but it is not possible that **Dependent Variable** causes a change in the **Independent Variable**.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "quK-r76x3zSN"
      },
      "source": [
        "### Supervised Learning\n",
        "\n",
        "* Which is  nothing that there is a supervisor. Supervised Learning is that we train and teach or build the models with the well labelled dataset (or) input-output pairs. It has labels to say what the data represents.\n",
        "\n",
        "* Both Regression and Classification Algorithms comes under the Supervised Learning."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DXJbfPXi6ddt"
      },
      "source": [
        "### Unsupervised Learning\n",
        "\n",
        "* which implies that there is no supervisor. In Unsupervised Learning we will train and teach (or) build the models on the unstructured data. There are no labels or correct outputs, so our task will be to find the structure of the data (i.e. we want to find the class labels).\n",
        "\n",
        "* Clustering will come Unsupervised Learning."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JTzD0Yol8Qjt"
      },
      "source": [
        "### Reinforcement Learning\n",
        "\n",
        "* Reinforcement Learning uses a reward function which will penalize bad actions and reward the good actions. It is all about making decisions sequentially. The output depends on the state of current input and the next input depends on the state of the previous output.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R1Z3KyS0upqO"
      },
      "source": [
        "# Steps Involved In Machine Learning:\n",
        "\n",
        "* STEP 0 - GET THE REQUIRED DATASET\n",
        "* STEP 1 - DATA PREPROCESSING\n",
        "* STEP 2 - BUILD THE MODEL\n",
        "* STEP 3 - TRAIN THE MODEL\n",
        "* STEP 4 - TEST THE MODEL\n",
        "* STEP 5 - DEPLOY THE MODEL"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Be_OOCjxvrrC"
      },
      "source": [
        "## STEP 0 - GET THE REQUIRED DATASET\n",
        "\n",
        "WE GOT THE DATASET FROM KAGGLE WEBSITE\n",
        "Autism Screening data for Toddlers. (Autism data for infants’ classification). July 2018.\n",
        "https://www.kaggle.com/fabdelja/autism-screening-for-toddlers/version/1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N5bL-bdewQWD"
      },
      "source": [
        "## STEP 1 - DATA PREPROCESSING\n",
        "\n",
        "What is Data Preprocessing?\n",
        "\n",
        "Data Preprocessing is Data Mining Technique that used to convert the obtained raw data into an understandable/data driven format. It will help us to remove the missing values, remove noisy data and outliers, removes each and every discrepancies in the data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jFToavdIx7i8"
      },
      "source": [
        "### Steps Involved In Data Preprocessing:\n",
        "\n",
        "* Import the Libraries\n",
        "* Import the Dataset\n",
        "* Checking for Missing Values\n",
        "* Encoding the Categorical Variables\n",
        "* Spliting dataset into Training Set and Test Set\n",
        "* Feature Scaling"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cJMlmRf3q1yT"
      },
      "source": [
        "# Importing the Libraries\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt \n",
        "import pandas as pd"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qk-aqEDZzYcx"
      },
      "source": [
        "# Import the Dataset\n",
        "dataset = pd.read_csv(\"data.csv\")\n",
        "\n",
        "# Knowing the dependent and independent variables\n",
        "x = dataset.iloc[:, 1:18].values\n",
        "y = dataset.iloc[:, -1].values"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lghEuVfa0HMm",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 313
        },
        "outputId": "8db365f7-ed2d-498d-af8b-87b5c2c68eee"
      },
      "source": [
        "dataset.head()"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Case_No</th>\n",
              "      <th>A1</th>\n",
              "      <th>A2</th>\n",
              "      <th>A3</th>\n",
              "      <th>A4</th>\n",
              "      <th>A5</th>\n",
              "      <th>A6</th>\n",
              "      <th>A7</th>\n",
              "      <th>A8</th>\n",
              "      <th>A9</th>\n",
              "      <th>A10</th>\n",
              "      <th>Age_Mons</th>\n",
              "      <th>Qchat-10-Score</th>\n",
              "      <th>Sex</th>\n",
              "      <th>Ethnicity</th>\n",
              "      <th>Jaundice</th>\n",
              "      <th>Family_mem_with_ASD</th>\n",
              "      <th>Who completed the test</th>\n",
              "      <th>Class/ASD Traits</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>28</td>\n",
              "      <td>3</td>\n",
              "      <td>f</td>\n",
              "      <td>middle eastern</td>\n",
              "      <td>yes</td>\n",
              "      <td>no</td>\n",
              "      <td>family member</td>\n",
              "      <td>No</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>36</td>\n",
              "      <td>4</td>\n",
              "      <td>m</td>\n",
              "      <td>White European</td>\n",
              "      <td>yes</td>\n",
              "      <td>no</td>\n",
              "      <td>family member</td>\n",
              "      <td>Yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>36</td>\n",
              "      <td>4</td>\n",
              "      <td>m</td>\n",
              "      <td>middle eastern</td>\n",
              "      <td>yes</td>\n",
              "      <td>no</td>\n",
              "      <td>family member</td>\n",
              "      <td>Yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>24</td>\n",
              "      <td>10</td>\n",
              "      <td>m</td>\n",
              "      <td>Hispanic</td>\n",
              "      <td>no</td>\n",
              "      <td>no</td>\n",
              "      <td>family member</td>\n",
              "      <td>Yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>20</td>\n",
              "      <td>9</td>\n",
              "      <td>f</td>\n",
              "      <td>White European</td>\n",
              "      <td>no</td>\n",
              "      <td>yes</td>\n",
              "      <td>family member</td>\n",
              "      <td>Yes</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Case_No  A1  A2  ...  Family_mem_with_ASD  Who completed the test  Class/ASD Traits \n",
              "0        1   0   0  ...                   no           family member                 No\n",
              "1        2   1   1  ...                   no           family member                Yes\n",
              "2        3   1   0  ...                   no           family member                Yes\n",
              "3        4   1   1  ...                   no           family member                Yes\n",
              "4        5   1   1  ...                  yes           family member                Yes\n",
              "\n",
              "[5 rows x 19 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V6rrLIqa07K9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b4b1eecf-c4f1-4537-ce80-35f696fc5a86"
      },
      "source": [
        "print(\"These are the independent variables\\n\")\n",
        "print(x)\n",
        "\n",
        "print(\"\\n\\nThese are the dependent variable\\n\")\n",
        "print(y)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "These are the independent variables\n",
            "\n",
            "[[0 0 0 ... 'yes' 'no' 'family member']\n",
            " [1 1 0 ... 'yes' 'no' 'family member']\n",
            " [1 0 0 ... 'yes' 'no' 'family member']\n",
            " ...\n",
            " [1 0 1 ... 'yes' 'no' 'family member']\n",
            " [1 0 0 ... 'no' 'yes' 'family member']\n",
            " [1 1 0 ... 'yes' 'yes' 'family member']]\n",
            "\n",
            "\n",
            "These are the dependent variable\n",
            "\n",
            "['No' 'Yes' 'Yes' ... 'Yes' 'No' 'Yes']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Lu-9RH-S0_Su",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "eba830ee-8786-41c7-86d4-82fb47b95aec"
      },
      "source": [
        "# Checking for missing values\n",
        "from sklearn.impute import SimpleImputer\n",
        "imputer = SimpleImputer(missing_values = np.nan, strategy = \"mean\")\n",
        "imputer.fit(x[:, 10:12])\n",
        "x[:, 10:12] = imputer.transform(x[:, 10:12])\n",
        "print(x)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0 0 0 ... 'yes' 'no' 'family member']\n",
            " [1 1 0 ... 'yes' 'no' 'family member']\n",
            " [1 0 0 ... 'yes' 'no' 'family member']\n",
            " ...\n",
            " [1 0 1 ... 'yes' 'no' 'family member']\n",
            " [1 0 0 ... 'no' 'yes' 'family member']\n",
            " [1 1 0 ... 'yes' 'yes' 'family member']]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DYNaK66L1Vhf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a3eef892-cf47-4588-a4a5-c9b1d630f949"
      },
      "source": [
        "# Encoding the Categorical variables\n",
        "from sklearn.preprocessing import OneHotEncoder\n",
        "from sklearn.compose import ColumnTransformer\n",
        "columntrans = ColumnTransformer(transformers = [(\"encoder\", OneHotEncoder(), [12,13,14,15,16])], remainder=\"passthrough\")\n",
        "x = np.array(columntrans.fit_transform(x))\n",
        "print(x)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[1.0 0.0 0.0 ... 1 28.0 3.0]\n",
            " [0.0 1.0 0.0 ... 0 36.0 4.0]\n",
            " [0.0 1.0 0.0 ... 1 36.0 4.0]\n",
            " ...\n",
            " [0.0 1.0 0.0 ... 1 18.0 9.0]\n",
            " [0.0 1.0 0.0 ... 1 19.0 3.0]\n",
            " [0.0 1.0 0.0 ... 0 24.0 6.0]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a0Nu4u0B_rA2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0af939a7-bf94-49ce-b048-4606b806caa0"
      },
      "source": [
        "from sklearn.preprocessing import LabelEncoder\n",
        "le = LabelEncoder()\n",
        "y = le.fit_transform(y)\n",
        "print(y)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0 1 1 ... 1 0 1]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "85azZRXA7-IF"
      },
      "source": [
        "# Splitting dataset into Training set and Test Set\n",
        "from sklearn.model_selection import train_test_split\n",
        "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.35, random_state = 1)"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PojWIwrNBPCv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "921907c2-c704-4a91-a4a0-838300f0d943"
      },
      "source": [
        "# Feature Scaling\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "ss = MinMaxScaler()\n",
        "x_train = ss.fit_transform(x_train)\n",
        "x_test = ss.transform(x_test)\n",
        "print(x_train)\n",
        "print()\n",
        "print(x_test)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0.         1.         0.         ... 1.         0.875      0.4       ]\n",
            " [0.         1.         0.         ... 0.         0.125      0.7       ]\n",
            " [0.         1.         0.         ... 1.         0.66666667 1.        ]\n",
            " ...\n",
            " [0.         1.         0.         ... 1.         0.125      0.1       ]\n",
            " [0.         1.         0.         ... 1.         0.58333333 0.6       ]\n",
            " [1.         0.         0.         ... 1.         0.         0.2       ]]\n",
            "\n",
            "[[0.         1.         0.         ... 1.         1.         0.9       ]\n",
            " [0.         1.         0.         ... 1.         0.25       1.        ]\n",
            " [0.         1.         0.         ... 1.         0.75       0.2       ]\n",
            " ...\n",
            " [1.         0.         0.         ... 1.         0.66666667 1.        ]\n",
            " [0.         1.         0.         ... 1.         1.         0.4       ]\n",
            " [0.         1.         0.         ... 1.         0.41666667 0.9       ]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8LFqQ2wvsqrS"
      },
      "source": [
        "## STEP 2 - BUILD THE MODEL\n",
        "\n",
        "With the help of the ML Algorithms we can build our own model to predict the outcome of the data. Since there are so many algorithms in which will come under in any one type of the Machine Learning Type."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tf7eje0xuJwi"
      },
      "source": [
        "### Supervised Learning Techniques\n",
        "\n",
        "* Regression\n",
        "        * Simple Linear Regression\n",
        "        * Multiple Linear Regression\n",
        "        * Polynomial Linear Regression\n",
        "        * Support Vector Regression\n",
        "        * Decision Tree Regression\n",
        "        * Random Forest Regression\n",
        "        * Ridge Regression\n",
        "        * Lasso Regression\n",
        "        * ElasticNet Regression\n",
        "\n",
        "* Classification\n",
        "        * Logistic Regression\n",
        "        * K-Nearest Neighbors (KNN)\n",
        "        * Support Vector Machine (SVM)\n",
        "        * Kernel SVM\n",
        "        * Naive Bayes\n",
        "        * Decision Tree Classification\n",
        "        * Random Forest Classification\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SXH9zvH3BIGz"
      },
      "source": [
        "## STEP 3 - TRAIN THE MODEL\n",
        "\n",
        "From the above mentioned models select the model which you want to do and then build and then train the model with the training set that we have derived from the dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oBXrKWpG3D5a"
      },
      "source": [
        "###Supervised Learning - Regression\n",
        "\n",
        "* **Simple Linear Regression** - It explains us the relationship between the dependent variable and the independent variaable using a straight line. It tries to approximate the relationship between dependent and independent variables in a straight line."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XsGV9z2wFLX6"
      },
      "source": [
        "* **Multiple Linear Regrssion** - It tells us the relationship between two or more independent input variables and a dependent (response) variable. The more information we put into the model, the better chances of explaining the item and make accurate predictions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GOxyALEl41gY",
        "outputId": "299d3ef5-4378-4e79-bc7a-98f9efc26988"
      },
      "source": [
        "# we have done multiple linear regression\n",
        "from sklearn.linear_model import LinearRegression\n",
        "LR = LinearRegression()\n",
        "LR.fit(x_train, y_train)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vQy_M0jd5ywc",
        "outputId": "2c5ed31c-1872-412b-de03-5f7eb0defc80"
      },
      "source": [
        "from sklearn.metrics import r2_score\n",
        "from sklearn.metrics import mean_squared_error\n",
        "from sklearn.metrics import mean_absolute_error\n",
        "from math import sqrt\n",
        "y_pred = LR.predict(x_test)\n",
        "R2 = r2_score(y_test, y_pred)\n",
        "print(\"R^2: \"+str(R2*100))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE*100))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE*100))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE*100))"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "R^2: 60.81153733491011\n",
            "MEAN SQUARED ERROR: 7.742082135787179\n",
            "MEAN ABSOLUTE ERROR: 23.161204268292682\n",
            "ROOT MEAN SQUARED ERROR: 27.82459727612815\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8mwVvKDmIs0_"
      },
      "source": [
        "* **Polynomial Linear Regression** - It tells us the relationship between independent variable and the dependent variable is modelled as an nth degree polynomial. It fits a non-linear relationship between the dependent and independent variable."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XGes5QYkGxO2",
        "outputId": "9407f1f0-d8d4-4a56-b47f-9c9fe4da7e4f"
      },
      "source": [
        "# Since the PLR is similar to SLR we need to use them when have single independent variable and dependent variable.\n",
        "# But this is how we want to implement ( just an outline )\n",
        "# we cannot do since we many independent variables\n",
        "from sklearn.preprocessing import PolynomialFeatures\n",
        "PR = PolynomialFeatures(degree = 2)\n",
        "x_poly = PR.fit_transform(x)\n",
        "lin_reg_2 = LinearRegression()\n",
        "lin_reg_2.fit(x_poly, y)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l36SK4YgOsvS"
      },
      "source": [
        "* **Support Vector Regression** - In this we will try to fit the error within a certain threshold. In SVR We are basically considering the points within the boundary line. Our best fit line is the line hyperplane that has maximum number of points."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ICW4bH6jTa6P",
        "outputId": "4b66cd9f-50d0-46f7-b0b4-453233a8f0ca"
      },
      "source": [
        "from sklearn.svm import SVR\n",
        "SVR = SVR(kernel = 'rbf')\n",
        "SVR.fit(x_train, y_train)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SVR(C=1.0, cache_size=200, coef0=0.0, degree=3, epsilon=0.1, gamma='scale',\n",
              "    kernel='rbf', max_iter=-1, shrinking=True, tol=0.001, verbose=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ap5pxbgxUQGf",
        "outputId": "3a746740-ec61-4f08-da0e-b61c464795ab"
      },
      "source": [
        "y_pred = SVR.predict(x_test)\n",
        "R2 = r2_score(y_test, y_pred)\n",
        "print(\"R^2: \"+str(R2*100))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE*100))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE*100))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE*100))"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "R^2: 80.50570219706609\n",
            "MEAN SQUARED ERROR: 3.851298175681159\n",
            "MEAN ABSOLUTE ERROR: 14.524946110755163\n",
            "ROOT MEAN SQUARED ERROR: 19.624724649485298\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1zyXQu9jUhZb"
      },
      "source": [
        "* **Decision Tree Algorithm** - It is decision making tool which uses a flowchart like tree structure. It is mainly focuses on all the decisions and their outputs and possible results. This will comes under both regression and classification.\n",
        "\n",
        "* **Decision Tree Regression** - It will observe the features and will train the model in a tree structure to predict the data and to produce a continuous output."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "84tfqYG8YLu-",
        "outputId": "2d5d1feb-e753-4e20-b02a-d01ca1fc325c"
      },
      "source": [
        "from sklearn.tree import DecisionTreeRegressor \n",
        "DTR = DecisionTreeRegressor(random_state = 0)\n",
        "DTR.fit(x_train, y_train)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "DecisionTreeRegressor(ccp_alpha=0.0, criterion='mse', max_depth=None,\n",
              "                      max_features=None, max_leaf_nodes=None,\n",
              "                      min_impurity_decrease=0.0, min_impurity_split=None,\n",
              "                      min_samples_leaf=1, min_samples_split=2,\n",
              "                      min_weight_fraction_leaf=0.0, presort='deprecated',\n",
              "                      random_state=0, splitter='best')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4JUEeeNlYugo",
        "outputId": "deb71b3a-fd47-4aee-b39b-f2e1257a5964"
      },
      "source": [
        "y_pred = DTR.predict(x_test)\n",
        "R2 = r2_score(y_test, y_pred)\n",
        "print(\"R^2: \"+str(R2*100))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE*100))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE*100))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE*100))"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "R^2: 100.0\n",
            "MEAN SQUARED ERROR: 0.0\n",
            "MEAN ABSOLUTE ERROR: 0.0\n",
            "ROOT MEAN SQUARED ERROR: 0.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jnCcPlymZCCQ"
      },
      "source": [
        "* **Random Forest Algorithm** - It is an ensemble technique which performs both the classification and regression tasks with the use of the multiple decision trees. The main idea behind this is to combine multiple decision trees for predicting the final output rather than dependind on a single decision tree. \n",
        "\n",
        "* **Ensemble Learning** - It is technique of combining multiple machine learning models together to obtain more accurate predictions than any of the individual models.\n",
        "\n",
        "* **Random Forest Regression** - It operates by by constructing a crowd of decision trees at the training time and outputing the mean prediction of the individual trees."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LBH1vG-SA85t",
        "outputId": "776c8471-73ec-45df-bac7-01654e6aac4e"
      },
      "source": [
        "from sklearn.ensemble import RandomForestRegressor\n",
        "RFR = RandomForestRegressor(n_estimators=100, random_state = 0)\n",
        "RFR.fit(x_train, y_train)"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "RandomForestRegressor(bootstrap=True, ccp_alpha=0.0, criterion='mse',\n",
              "                      max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
              "                      max_samples=None, min_impurity_decrease=0.0,\n",
              "                      min_impurity_split=None, min_samples_leaf=1,\n",
              "                      min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
              "                      n_estimators=100, n_jobs=None, oob_score=False,\n",
              "                      random_state=0, verbose=0, warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EzOudORECE6n",
        "outputId": "32eabd6e-3bdd-45b3-93e1-fa7834e4b50a"
      },
      "source": [
        "y_pred = RFR.predict(x_test)\n",
        "R2 = r2_score(y_test, y_pred)\n",
        "print(\"R^2: \"+str(R2*100))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE*100))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE*100))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE*100))"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "R^2: 100.0\n",
            "MEAN SQUARED ERROR: 0.0\n",
            "MEAN ABSOLUTE ERROR: 0.0\n",
            "ROOT MEAN SQUARED ERROR: 0.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dT42J5IRZBNl"
      },
      "source": [
        "* In order to create a less complex (or) parsimonious model when we have more number of features in our dataset we use some regularization techniques to overcome over-fitting, They are **L1 and L2 regularization techniques**. The key difference between them is the penalty term.\n",
        "\n",
        "* **Ridge Regression** - A regression model which uses L2 regularization technique is called Ridge Regression Model. It adds squared magnitude of coefficients as penalty term to the loss function."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cY1uIrTsI7b9",
        "outputId": "c31b6b93-dd96-40f8-ce30-bf64d5a42060"
      },
      "source": [
        "from sklearn.linear_model import Ridge\n",
        "RR = Ridge(alpha = 1.0)\n",
        "RR.fit(x_train, y_train)"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None,\n",
              "      normalize=False, random_state=None, solver='auto', tol=0.001)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kCmNjNrkJo2W",
        "outputId": "72d7dcb1-1d20-467f-ea78-924bb08702dc"
      },
      "source": [
        "y_pred = RR.predict(x_test)\n",
        "R2 = r2_score(y_test, y_pred)\n",
        "print(\"R^2: \"+str(R2*100))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE*100))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE*100))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE*100))"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "R^2: 61.09284503081555\n",
            "MEAN SQUARED ERROR: 7.686506919536883\n",
            "MEAN ABSOLUTE ERROR: 23.061449780306397\n",
            "ROOT MEAN SQUARED ERROR: 27.724550347186668\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4vRczaKZUiAm"
      },
      "source": [
        "* **Lasso Regression** - A regression model which uses L1 regularization technique is called Lasso Regression. Lasso Stands for Least Absolute Shrinkage and Selection Operator. It adds absolute value of magnitude of coefficients as penalty term to loss function.\n",
        "\n",
        "* **For More Information:**  for regularization and its techniques\n",
        "* https://towardsdatascience.com/over-fitting-and-regularization-64d16100f45c \n",
        "* https://towardsdatascience.com/l1-and-l2-regularization-methods-ce25e7fc831c"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lqtxri5QLqk2",
        "outputId": "0176f736-2b7a-4240-b850-c6d8ff3bb61b"
      },
      "source": [
        "from sklearn.linear_model import Lasso\n",
        "LAR = Lasso(alpha = 1.0)\n",
        "LAR.fit(x_train, y_train)"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Lasso(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=1000,\n",
              "      normalize=False, positive=False, precompute=False, random_state=None,\n",
              "      selection='cyclic', tol=0.0001, warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2NKvOeNLMT_N",
        "outputId": "c500696c-9d6e-457d-b9e4-2edc3b3b531c"
      },
      "source": [
        "y_pred = LAR.predict(x_test)\n",
        "R2 = r2_score(y_test, y_pred)\n",
        "print(\"R^2: \"+str(R2*100))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE*100))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE*100))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE*100))"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "R^2: -1.7574754860551822\n",
            "MEAN SQUARED ERROR: 20.10323139940867\n",
            "MEAN ABSOLUTE ERROR: 42.210749114790424\n",
            "ROOT MEAN SQUARED ERROR: 44.83662721415235\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FxMyqZoKNSTD"
      },
      "source": [
        "* **ElasticNet Regression** - A regression model which uses both L1 and L2 regularization techniques is called elasticNet Regression. It linearly combines both the L1 and L2 regularization techinques from Lasso and Ridge Regression Models."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T_GznoBpPPbm",
        "outputId": "bf1600b2-23b6-4bbd-f328-8741885ed792"
      },
      "source": [
        "from sklearn.linear_model import ElasticNet\n",
        "ENR = ElasticNet(alpha=1.0, l1_ratio = 0.5)\n",
        "ENR.fit(x_train, y_train)"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "ElasticNet(alpha=1.0, copy_X=True, fit_intercept=True, l1_ratio=0.5,\n",
              "           max_iter=1000, normalize=False, positive=False, precompute=False,\n",
              "           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "D6-4EdXzPQVr",
        "outputId": "c7ef8618-c11d-4aea-b717-561c5a1ec992"
      },
      "source": [
        "y_pred = ENR.predict(x_test)\n",
        "R2 = r2_score(y_test, y_pred)\n",
        "print(\"R^2: \"+str(R2*100))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE*100))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE*100))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE*100))"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "R^2: -1.7574754860551822\n",
            "MEAN SQUARED ERROR: 20.10323139940867\n",
            "MEAN ABSOLUTE ERROR: 42.210749114790424\n",
            "ROOT MEAN SQUARED ERROR: 44.83662721415235\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nVoIyFngPRKK"
      },
      "source": [
        "* The **J48 algorithm** is similar to the CART algorithm that we use in the Scikit learn which is nothing but the normal Decision Tree Classifier."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lpa35VNrPRxf"
      },
      "source": [
        "### Supervised Learning - Classification\n",
        "\n",
        "* **Logistic Regression** - It is used when the dependent variable is a categorical variable. It is used to predict the probability of the target variable, since it does predictive analysis. It is used when the nature of the target variable is dichotomous, which means they will have two possible classes. In this we are applying sigmoid function.\n",
        "\n",
        "* **Types of Logistic Regression**\n",
        "* **Binary Logistic Regression** - The categorical response has only two classes or outputs. Eg. spam or not.\n",
        "* **Multinomial Logistic Regression** - The categorical response has three or more classes or outputs without ordering. Eg. Vegan, Non-Veg, Veg.\n",
        "* **Ordinal Logistic Regression** - The categorical response has three or more classes or outputs with ordering. Eg. Movie Rating 1 to 5.\n",
        "\n",
        "* For More: https://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JYRQbb4bDEtQ",
        "outputId": "a087e17e-ce51-4d73-8fd4-c8d0976b1075"
      },
      "source": [
        "from sklearn.linear_model import LogisticRegression\n",
        "LR = LogisticRegression()\n",
        "LR.fit(x_train, y_train)"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
              "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
              "                   multi_class='auto', n_jobs=None, penalty='l2',\n",
              "                   random_state=None, solver='lbfgs', tol=0.0001, verbose=0,\n",
              "                   warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "34BiY9SGFsmb",
        "outputId": "96a10c6c-25ee-4ae0-8f60-2adf25bd8fab"
      },
      "source": [
        "from sklearn.metrics import accuracy_score\n",
        "from sklearn.metrics import confusion_matrix\n",
        "from sklearn.metrics import f1_score\n",
        "from sklearn.metrics import precision_score\n",
        "from sklearn.metrics import recall_score\n",
        "from sklearn.metrics import mean_squared_error\n",
        "from sklearn.metrics import mean_absolute_error\n",
        "from math import sqrt\n",
        "y_pred = LR.predict(x_test)\n",
        "CM = confusion_matrix(y_test, y_pred)\n",
        "print(\"CONFUSION MATRIX:\")\n",
        "print(CM)\n",
        "ACC = accuracy_score(y_test, y_pred)\n",
        "print(\"ACCURACY: \"+str(ACC))\n",
        "PRE = precision_score(y_test, y_pred)\n",
        "print(\"PRECISION: \"+str(PRE))\n",
        "REC = recall_score(y_test, y_pred)\n",
        "print(\"RECALL: \"+str(REC))\n",
        "F1 = f1_score(y_test, y_pred)\n",
        "print(\"F1: \"+str(F1))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE))"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CONFUSION MATRIX:\n",
            "[[100   0]\n",
            " [  0 269]]\n",
            "ACCURACY: 1.0\n",
            "PRECISION: 1.0\n",
            "RECALL: 1.0\n",
            "F1: 1.0\n",
            "MEAN SQUARED ERROR: 0.0\n",
            "MEAN ABSOLUTE ERROR: 0.0\n",
            "ROOT MEAN SQUARED ERROR: 0.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C2gSaZueS0cF"
      },
      "source": [
        "* **K-Nearest Neighbors(KNN)** - This algorithm finds the nearest neighbor datapoints from the datasetthat we have plotted and assigns the value to it. In other words similar things that are near to each other. it uses the feature similarity to predict the values."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OB1q0BdQGa7m",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1c8b0bcc-17cf-4c56-e204-1b44d3562ad0"
      },
      "source": [
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "KNN = KNeighborsClassifier(n_neighbors = 5, metric = \"minkowski\", p = 2)\n",
        "KNN.fit(x_train,y_train)"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
              "                     metric_params=None, n_jobs=None, n_neighbors=5, p=2,\n",
              "                     weights='uniform')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "noeoFCXhP_FI",
        "outputId": "e03dc79d-e8ae-444d-e109-ffb672847086"
      },
      "source": [
        "y_pred = KNN.predict(x_test)\n",
        "CM = confusion_matrix(y_test, y_pred)\n",
        "print(\"CONFUSION MATRIX:\")\n",
        "print(CM)\n",
        "ACC = accuracy_score(y_test, y_pred)\n",
        "print(\"ACCURACY: \"+str(ACC))\n",
        "PRE = precision_score(y_test, y_pred)\n",
        "print(\"PRECISION: \"+str(PRE))\n",
        "REC = recall_score(y_test, y_pred)\n",
        "print(\"RECALL: \"+str(REC))\n",
        "F1 = f1_score(y_test, y_pred.round())\n",
        "print(\"F1: \"+str(F1))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE))"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CONFUSION MATRIX:\n",
            "[[ 96   4]\n",
            " [ 19 250]]\n",
            "ACCURACY: 0.9376693766937669\n",
            "PRECISION: 0.984251968503937\n",
            "RECALL: 0.929368029739777\n",
            "F1: 0.9560229445506692\n",
            "MEAN SQUARED ERROR: 0.06233062330623306\n",
            "MEAN ABSOLUTE ERROR: 0.06233062330623306\n",
            "ROOT MEAN SQUARED ERROR: 0.24966101679323718\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZaFi2mdmPQ5x"
      },
      "source": [
        "* **Support Vector Machine(SVM)** - An Svm model is basically a representation of different classes in a hyperplane with a high N-dimesional space(N - the number of features). The objective is to find a plane that has a maximum margin (i.e the maximum distance between the data points of both classes)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zVrpBkVdR4C7",
        "outputId": "6d3a60e4-b128-452c-e522-81db6b589030"
      },
      "source": [
        "from sklearn.svm import SVC\n",
        "SVM = SVC(kernel = \"linear\", random_state = 0)\n",
        "SVM.fit(x_train, y_train)"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SVC(C=1.0, break_ties=False, cache_size=200, class_weight=None, coef0=0.0,\n",
              "    decision_function_shape='ovr', degree=3, gamma='scale', kernel='linear',\n",
              "    max_iter=-1, probability=False, random_state=0, shrinking=True, tol=0.001,\n",
              "    verbose=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1UybWYXaS0rO",
        "outputId": "bcdf0ae1-4659-4fd0-c972-4656c8c155ff"
      },
      "source": [
        "y_pred = SVM.predict(x_test)\n",
        "CM = confusion_matrix(y_test, y_pred)\n",
        "print(\"CONFUSION MATRIX:\")\n",
        "print(CM)\n",
        "ACC = accuracy_score(y_test, y_pred)\n",
        "print(\"ACCURACY: \"+str(ACC))\n",
        "PRE = precision_score(y_test, y_pred)\n",
        "print(\"PRECISION: \"+str(PRE))\n",
        "REC = recall_score(y_test, y_pred)\n",
        "print(\"RECALL: \"+str(REC))\n",
        "F1 = f1_score(y_test, y_pred.round())\n",
        "print(\"F1: \"+str(F1))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE))"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CONFUSION MATRIX:\n",
            "[[100   0]\n",
            " [  0 269]]\n",
            "ACCURACY: 1.0\n",
            "PRECISION: 1.0\n",
            "RECALL: 1.0\n",
            "F1: 1.0\n",
            "MEAN SQUARED ERROR: 0.0\n",
            "MEAN ABSOLUTE ERROR: 0.0\n",
            "ROOT MEAN SQUARED ERROR: 0.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q7omSq9sS6zY"
      },
      "source": [
        "* **Kernel SVM** - The SVM algorithm uses a set of mathematical functions that are defined as kernel. Th kernel will take the dataas input and will transform it into required format. There are many different kernel functions like linear, nonlinear, polynomial, radial basis function(rbf), and sigmoid. Kernel rbf is the most commonly used kernel."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YJxDR_pGVV78",
        "outputId": "c11bed7b-c69e-4fd8-8814-9af24d058e24"
      },
      "source": [
        "from sklearn.svm import SVC\n",
        "KSVM = SVC(kernel = \"rbf\", random_state = 0)\n",
        "KSVM.fit(x_train, y_train)"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SVC(C=1.0, break_ties=False, cache_size=200, class_weight=None, coef0=0.0,\n",
              "    decision_function_shape='ovr', degree=3, gamma='scale', kernel='rbf',\n",
              "    max_iter=-1, probability=False, random_state=0, shrinking=True, tol=0.001,\n",
              "    verbose=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mr9i5QnpVcS6",
        "outputId": "0d040d9b-9f3f-42c4-c290-dca2f305f29b"
      },
      "source": [
        "y_pred = KSVM.predict(x_test)\n",
        "CM = confusion_matrix(y_test, y_pred)\n",
        "print(\"CONFUSION MATRIX:\")\n",
        "print(CM)\n",
        "ACC = accuracy_score(y_test, y_pred)\n",
        "print(\"ACCURACY: \"+str(ACC))\n",
        "PRE = precision_score(y_test, y_pred)\n",
        "print(\"PRECISION: \"+str(PRE))\n",
        "REC = recall_score(y_test, y_pred)\n",
        "print(\"RECALL: \"+str(REC))\n",
        "F1 = f1_score(y_test, y_pred.round())\n",
        "print(\"F1: \"+str(F1))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE))"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CONFUSION MATRIX:\n",
            "[[ 96   4]\n",
            " [  1 268]]\n",
            "ACCURACY: 0.986449864498645\n",
            "PRECISION: 0.9852941176470589\n",
            "RECALL: 0.9962825278810409\n",
            "F1: 0.9907578558225507\n",
            "MEAN SQUARED ERROR: 0.013550135501355014\n",
            "MEAN ABSOLUTE ERROR: 0.013550135501355014\n",
            "ROOT MEAN SQUARED ERROR: 0.1164050492949297\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "psRriJrtVct7"
      },
      "source": [
        "* **Decision Tree Classification** - It breaks down a dataset into a smaller and smaller subsets while in the mean time an associated decision tree is incrementally developed. It has two main entities Decision Nodes(where the data is splitted) and Leave Nodes(where we get the output), and the top node is called the Root Node(Decision node which corresponds to best predictor attribute). It is done on the catergorical variable."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XPye_EhIY8Nu",
        "outputId": "cc77bc24-4cd1-4736-e899-9a98b5d2c322"
      },
      "source": [
        "from sklearn.tree import DecisionTreeClassifier\n",
        "DTC = DecisionTreeClassifier(criterion = \"entropy\", random_state = 0)\n",
        "DTC.fit(x_train, y_train)"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "DecisionTreeClassifier(ccp_alpha=0.0, class_weight=None, criterion='entropy',\n",
              "                       max_depth=None, max_features=None, max_leaf_nodes=None,\n",
              "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
              "                       min_samples_leaf=1, min_samples_split=2,\n",
              "                       min_weight_fraction_leaf=0.0, presort='deprecated',\n",
              "                       random_state=0, splitter='best')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zU1HKWc1Y8wC",
        "outputId": "68b5f938-4724-4e47-9e38-d1756ac79ee1"
      },
      "source": [
        "y_pred = DTC.predict(x_test)\n",
        "CM = confusion_matrix(y_test, y_pred)\n",
        "print(\"CONFUSION MATRIX:\")\n",
        "print(CM)\n",
        "ACC = accuracy_score(y_test, y_pred)\n",
        "print(\"ACCURACY: \"+str(ACC))\n",
        "PRE = precision_score(y_test, y_pred)\n",
        "print(\"PRECISION: \"+str(PRE))\n",
        "REC = recall_score(y_test, y_pred)\n",
        "print(\"RECALL: \"+str(REC))\n",
        "F1 = f1_score(y_test, y_pred.round())\n",
        "print(\"F1: \"+str(F1))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE))"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CONFUSION MATRIX:\n",
            "[[100   0]\n",
            " [  0 269]]\n",
            "ACCURACY: 1.0\n",
            "PRECISION: 1.0\n",
            "RECALL: 1.0\n",
            "F1: 1.0\n",
            "MEAN SQUARED ERROR: 0.0\n",
            "MEAN ABSOLUTE ERROR: 0.0\n",
            "ROOT MEAN SQUARED ERROR: 0.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gJoTy-H-Z6wF"
      },
      "source": [
        "* **Random Forest Classification** - It is an ensemble method which combines a lot of individual decision trees and then runs them for multiple times. Each single tree in the random forest will produce a class prediction and the class with the most votes becomes our model's prediction. The low correlation between the models is the key. A large number relatively uncorrelated models(trees) operating as a committee will outperform any individual constituent models. \n",
        "\n",
        "* For More: https://towardsdatascience.com/understanding-random-forest-58381e0602d2\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kRLtxfHFFhze",
        "outputId": "7464a1da-33fc-4c47-bc74-d5736319bdde"
      },
      "source": [
        "from sklearn.ensemble import RandomForestClassifier\n",
        "RFC = RandomForestClassifier(n_estimators = 10, criterion = 'entropy', random_state = 0)\n",
        "RFC.fit(x_train, y_train)"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,\n",
              "                       criterion='entropy', max_depth=None, max_features='auto',\n",
              "                       max_leaf_nodes=None, max_samples=None,\n",
              "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
              "                       min_samples_leaf=1, min_samples_split=2,\n",
              "                       min_weight_fraction_leaf=0.0, n_estimators=10,\n",
              "                       n_jobs=None, oob_score=False, random_state=0, verbose=0,\n",
              "                       warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "swFTWYLu0byQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a036faf0-5bff-4375-e1de-9604916b66cd"
      },
      "source": [
        "y_pred = RFC.predict(x_test)\n",
        "CM = confusion_matrix(y_test, y_pred)\n",
        "print(\"CONFUSION MATRIX:\")\n",
        "print(CM)\n",
        "ACC = accuracy_score(y_test, y_pred)\n",
        "print(\"ACCURACY: \"+str(ACC))\n",
        "PRE = precision_score(y_test, y_pred)\n",
        "print(\"PRECISION: \"+str(PRE))\n",
        "REC = recall_score(y_test, y_pred)\n",
        "print(\"RECALL: \"+str(REC))\n",
        "F1 = f1_score(y_test, y_pred.round())\n",
        "print(\"F1: \"+str(F1))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE))"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CONFUSION MATRIX:\n",
            "[[ 99   1]\n",
            " [  2 267]]\n",
            "ACCURACY: 0.991869918699187\n",
            "PRECISION: 0.996268656716418\n",
            "RECALL: 0.9925650557620818\n",
            "F1: 0.994413407821229\n",
            "MEAN SQUARED ERROR: 0.008130081300813009\n",
            "MEAN ABSOLUTE ERROR: 0.008130081300813009\n",
            "ROOT MEAN SQUARED ERROR: 0.09016696346674323\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0FZlWiPcdu_8"
      },
      "source": [
        "* **Naive Bayes Classification** - It is a probabilistic machine learning model that is used for classification. The crux of the classifier is based on the Bayes Theorem. The assumption is that the presence of a feature in a class is independent to the presence of any other features in the same class. The presence of one particular feature does affect the others that's why called as naive.\n",
        "\n",
        "* **Types of Naive Bayes Classification:**\n",
        "    * Multinomial Naive Bayes \n",
        "    * Bernoulli Naive Bayes\n",
        "    * Gaussian Naive Bayes\n",
        "\n",
        "* For More: https://towardsdatascience.com/naive-bayes-classifier-81d512f50a7c "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j5-ke2baLZs1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6e013b68-46d9-4361-cdbe-cee50ada38b7"
      },
      "source": [
        "# Here we have done Gaussian Naive Bayes\n",
        "from sklearn.naive_bayes import GaussianNB\n",
        "NBC = GaussianNB()\n",
        "NBC.fit(x_train, y_train)"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GaussianNB(priors=None, var_smoothing=1e-09)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UCSTN96-kqEo",
        "outputId": "28c79604-5fb9-4ac8-b086-6aac978bddbb"
      },
      "source": [
        "y_pred = NBC.predict(x_test)\n",
        "CM = confusion_matrix(y_test, y_pred)\n",
        "print(\"CONFUSION MATRIX:\")\n",
        "print(CM)\n",
        "ACC = accuracy_score(y_test, y_pred)\n",
        "print(\"ACCURACY: \"+str(ACC))\n",
        "PRE = precision_score(y_test, y_pred)\n",
        "print(\"PRECISION: \"+str(PRE))\n",
        "REC = recall_score(y_test, y_pred)\n",
        "print(\"RECALL: \"+str(REC))\n",
        "F1 = f1_score(y_test, y_pred.round())\n",
        "print(\"F1: \"+str(F1))\n",
        "MSE = mean_squared_error(y_test, y_pred)\n",
        "print(\"MEAN SQUARED ERROR: \"+str(MSE))\n",
        "MAE = mean_absolute_error(y_test, y_pred)\n",
        "print(\"MEAN ABSOLUTE ERROR: \"+str(MAE))\n",
        "RMSE = sqrt(mean_squared_error(y_test, y_pred))\n",
        "print(\"ROOT MEAN SQUARED ERROR: \"+str(RMSE))"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CONFUSION MATRIX:\n",
            "[[ 99   1]\n",
            " [130 139]]\n",
            "ACCURACY: 0.6449864498644986\n",
            "PRECISION: 0.9928571428571429\n",
            "RECALL: 0.516728624535316\n",
            "F1: 0.6797066014669927\n",
            "MEAN SQUARED ERROR: 0.35501355013550134\n",
            "MEAN ABSOLUTE ERROR: 0.35501355013550134\n",
            "ROOT MEAN SQUARED ERROR: 0.5958301353032602\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}