Mod3_lab3.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Harsha-070/FMML_IIITH/blob/main/Mod3_lab3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kAKrrEmgLZ_J"
      },
      "source": [
        "# **FOUNDATIONS OF MODERN MACHINE LEARNING, IIIT Hyderabad**\n",
        "### MODULE: CLASSIFICATION-1\n",
        "### LAB-3 : Using KNN for Text Classification\n",
        "#### Module Coordinator: Jashn Arora\n",
        "\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Section 1: Understanding NLP tools**"
      ],
      "metadata": {
        "id": "_jeRAsurrdLY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this lab we will be using KNN on a real world NLP application i.e. is text classification. But first look at some NLP techniques for text classification and tools that we use when we want to use python for NLP."
      ],
      "metadata": {
        "id": "iB9N2m-HrgWY"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WMJhTF89MmT_"
      },
      "source": [
        "## Section 1.2: Data Cleaning and Preprocessing step"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nRXVjJHk3urf"
      },
      "source": [
        "Raw text must be processed and converted into a form so that it is suitable to use with various machine-learning algorithms.  \n",
        "In case of text, there are lots of things that need to be taken into account.  \n",
        "\n",
        "\n",
        "1.   Removing numbers from the text\n",
        "2.   Handling capitalization and punctuation.\n",
        "3.   Stemming and Lemmatizing text.  \n",
        "\n",
        "And most importantly, one can't just use words or images directly in algorithms; they need to be converted into vectors- a form that algorithms can understand.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **NLTK**\n",
        "NLTK (or Natural Language Tool Kit) is a commonly used library for processing text. We will use this tool in this lab. Lets first install it.\n"
      ],
      "metadata": {
        "id": "IYq4np0xqUBr"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wFOZHWV3_ABt",
        "outputId": "817ccc08-e9aa-41e8-fbb6-cb1fb473d0bb"
      },
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "nltk.download('punkt')\n",
        "nltk.download('averaged_perceptron_tagger')\n",
        "nltk.download('wordnet')\n",
        "nltk.download('omw-1.4')"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n",
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n",
            "[nltk_data] Downloading package averaged_perceptron_tagger to\n",
            "[nltk_data]     /root/nltk_data...\n",
            "[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.\n",
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n",
            "[nltk_data] Downloading package omw-1.4 to /root/nltk_data...\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gpgq3SQK5IOr"
      },
      "source": [
        "import re\n",
        "import numpy\n",
        "from nltk.corpus import stopwords\n",
        "from nltk.stem import WordNetLemmatizer\n",
        "from nltk.stem import SnowballStemmer\n",
        "from nltk.tokenize import word_tokenize\n",
        "from nltk import pos_tag\n",
        "from nltk.corpus import wordnet\n",
        "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
        "from bs4 import BeautifulSoup\n",
        "\n",
        "def cleanText(text, lemmatize, stemmer):\n",
        "    \"\"\"Method for cleaning text from train and test data. Removes numbers, punctuation, and capitalization. Stems or lemmatizes text.\"\"\"\n",
        "\n",
        "    if isinstance(text, float):\n",
        "        text = str(text)\n",
        "    if isinstance(text, numpy.int64):\n",
        "        text = str(text)\n",
        "    try:\n",
        "        text = text.decode()\n",
        "    except AttributeError:\n",
        "        pass\n",
        "\n",
        "    soup = BeautifulSoup(text, \"lxml\")\n",
        "    text = soup.get_text()\n",
        "    text = re.sub(r\"[^A-Za-z]\", \" \", text)\n",
        "    text = text.lower()\n",
        "\n",
        "\n",
        "    if lemmatize:\n",
        "        wordnet_lemmatizer = WordNetLemmatizer()\n",
        "\n",
        "        def get_tag(tag):\n",
        "            if tag.startswith('J'):\n",
        "                return wordnet.ADJ\n",
        "            elif tag.startswith('V'):\n",
        "                return wordnet.VERB\n",
        "            elif tag.startswith('N'):\n",
        "                return wordnet.NOUN\n",
        "            elif tag.startswith('R'):\n",
        "                return wordnet.ADV\n",
        "            else:\n",
        "                return ''\n",
        "\n",
        "        text_result = []\n",
        "        tokens = word_tokenize(text)  # Generate list of tokens\n",
        "        tagged = pos_tag(tokens)\n",
        "        for t in tagged:\n",
        "            try:\n",
        "                text_result.append(wordnet_lemmatizer.lemmatize(t[0], get_tag(t[1][:2])))\n",
        "            except:\n",
        "                text_result.append(wordnet_lemmatizer.lemmatize(t[0]))\n",
        "        return text_result\n",
        "\n",
        "    if stemmer:\n",
        "        text_result = []\n",
        "        tokens = word_tokenize(text)\n",
        "        snowball_stemmer = SnowballStemmer('english')\n",
        "        for t in tokens:\n",
        "            text_result.append(snowball_stemmer.stem(t))\n",
        "        return text_result"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "sample_text = \"Troubling\"\n",
        "sample_text_result = cleanText(sample_text, lemmatize=False, stemmer=True)\n",
        "sample_text_result = \" \".join(str(x) for x in sample_text_result)\n",
        "print(sample_text)\n",
        "print(sample_text_result)\n",
        "sample_text_result = cleanText(sample_text, lemmatize=True, stemmer=False)\n",
        "sample_text_result = \" \".join(str(x) for x in sample_text_result)\n",
        "print(sample_text_result)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZuBxfzbuzrs1",
        "outputId": "265278f8-7448-41ac-e746-38827fb01e4f"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Troubling\n",
            "troubl\n",
            "trouble\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QcqQi34UoPvq"
      },
      "source": [
        "## Section 1.2: BAG OF WORDS\n",
        "\n",
        "A bag-of-words model, or BoW for short, is a way of extracting features from text for use in modeling, such as with machine learning algorithms.\n",
        "\n",
        "The approach is very simple and flexible, and can be used in many ways for extracting features from documents.\n",
        "\n",
        "A bag-of-words is a representation of text that describes the occurrence of words within a document.\n",
        "It is called a “bag” of words, because any information about the order or structure of words in the document is discarded. The model is only concerned with whether known words occur in the document, not where in the document."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "5*12"
      ],
      "metadata": {
        "id": "FN8sZqXaNLe7",
        "outputId": "795af180-5d1a-4913-83c9-1bb96c22cd8f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "60"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3YYSpQzIM05l"
      },
      "source": [
        "# Functions to convert document(s) to a list of words, with the option of removing stopwords. Returns document-term matrix.\n",
        "\n",
        "def createBagOfWords(train, test, remove_stopwords, lemmatize, stemmer):\n",
        "    if remove_stopwords:\n",
        "        vectorizer = CountVectorizer(analyzer='word', input='content', stop_words=stopwords.words('english'))\n",
        "    else:\n",
        "        vectorizer = CountVectorizer(analyzer='word', input='content')\n",
        "\n",
        "    clean_train = []\n",
        "    for paragraph in train:\n",
        "        paragraph_result = cleanText(paragraph, lemmatize, stemmer)\n",
        "        paragraph = \" \".join(str(x) for x in paragraph_result)\n",
        "        clean_train.append(paragraph)\n",
        "\n",
        "    clean_test = []\n",
        "    for paragraph in test:\n",
        "        paragraph_result = cleanText(paragraph, lemmatize, stemmer)\n",
        "        paragraph = \" \".join(str(x) for x in paragraph_result)\n",
        "        clean_test.append(paragraph)\n",
        "\n",
        "    bag_of_words_train = vectorizer.fit_transform(clean_train).toarray()\n",
        "    bag_of_words_test = vectorizer.transform(clean_test).toarray()\n",
        "    return bag_of_words_train, bag_of_words_test\n"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-v0iCUBqoX82"
      },
      "source": [
        "## Section 1.3: TF-IDF\n",
        "TF-IDF technique is used to find meaning of sentences consisting of words and cancels out the incapabilities of Bag of Words technique which is good for text classification or for helping a machine read words in numbers.\n",
        "\n",
        "The number of times a term occurs in a document is called its Term frequency (TF).\n",
        "\n",
        " Document frequency is the number of documents in which the word is present.  Inverse DF (IDF) is the inverse of the document frequency which measures the informativeness of term *t*.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L6_-DDMQoXEX"
      },
      "source": [
        "def createTFIDF(train, test, remove_stopwords, lemmatize, stemmer):\n",
        "    if remove_stopwords:\n",
        "        vectorizer = TfidfVectorizer(analyzer='word', input='content', stop_words=stopwords.words('english'))\n",
        "    else:\n",
        "        vectorizer =  TfidfVectorizer(analyzer='word', input='content')\n",
        "\n",
        "    clean_train = []\n",
        "    for paragraph in train:\n",
        "        paragraph_result = cleanText(paragraph, lemmatize, stemmer)\n",
        "        paragraph = \" \".join(str(x) for x in paragraph_result)\n",
        "        clean_train.append(paragraph)\n",
        "\n",
        "    clean_test = []\n",
        "    for paragraph in test:\n",
        "        paragraph_result = cleanText(paragraph, lemmatize, stemmer)\n",
        "        paragraph = \" \".join(str(x) for x in paragraph_result)\n",
        "        clean_test.append(paragraph)\n",
        "\n",
        "    tfidf_train = vectorizer.fit_transform(clean_train).toarray()\n",
        "    tfidf_test = vectorizer.transform(clean_test).toarray()\n",
        "    return tfidf_train, tfidf_test"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7g0jS45epcC5"
      },
      "source": [
        "# **Section 2: UNDERSTANDING THE DATA : A REVIEWS DATASET**\n",
        "\n",
        "Sentiment analysis is the interpretation and classification of emotions (such as positive, negative and neutral) within text data using text analysis techniques.  \n",
        "Given below is a dataset consisting of reviews along with sentiment class (positive or negative)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jU6875-NxrHw",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "outputId": "c1605ff7-9348-47b9-a0f0-32824d2abbf3"
      },
      "source": [
        "# Upload the Reviews CSV file that has been shared with you.\n",
        "# Run this cell, click on the 'Choose files' button and upload the file.\n",
        "from google.colab import files\n",
        "uploaded = files.upload()"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-ec753e2b-7260-4cd1-af7f-d2c5b69790ea\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-ec753e2b-7260-4cd1-af7f-d2c5b69790ea\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script>// Copyright 2017 Google LLC\n",
              "//\n",
              "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
              "// you may not use this file except in compliance with the License.\n",
              "// You may obtain a copy of the License at\n",
              "//\n",
              "//      http://www.apache.org/licenses/LICENSE-2.0\n",
              "//\n",
              "// Unless required by applicable law or agreed to in writing, software\n",
              "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
              "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
              "// See the License for the specific language governing permissions and\n",
              "// limitations under the License.\n",
              "\n",
              "/**\n",
              " * @fileoverview Helpers for google.colab Python module.\n",
              " */\n",
              "(function(scope) {\n",
              "function span(text, styleAttributes = {}) {\n",
              "  const element = document.createElement('span');\n",
              "  element.textContent = text;\n",
              "  for (const key of Object.keys(styleAttributes)) {\n",
              "    element.style[key] = styleAttributes[key];\n",
              "  }\n",
              "  return element;\n",
              "}\n",
              "\n",
              "// Max number of bytes which will be uploaded at a time.\n",
              "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
              "\n",
              "function _uploadFiles(inputId, outputId) {\n",
              "  const steps = uploadFilesStep(inputId, outputId);\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  // Cache steps on the outputElement to make it available for the next call\n",
              "  // to uploadFilesContinue from Python.\n",
              "  outputElement.steps = steps;\n",
              "\n",
              "  return _uploadFilesContinue(outputId);\n",
              "}\n",
              "\n",
              "// This is roughly an async generator (not supported in the browser yet),\n",
              "// where there are multiple asynchronous steps and the Python side is going\n",
              "// to poll for completion of each step.\n",
              "// This uses a Promise to block the python side on completion of each step,\n",
              "// then passes the result of the previous step as the input to the next step.\n",
              "function _uploadFilesContinue(outputId) {\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  const steps = outputElement.steps;\n",
              "\n",
              "  const next = steps.next(outputElement.lastPromiseValue);\n",
              "  return Promise.resolve(next.value.promise).then((value) => {\n",
              "    // Cache the last promise value to make it available to the next\n",
              "    // step of the generator.\n",
              "    outputElement.lastPromiseValue = value;\n",
              "    return next.value.response;\n",
              "  });\n",
              "}\n",
              "\n",
              "/**\n",
              " * Generator function which is called between each async step of the upload\n",
              " * process.\n",
              " * @param {string} inputId Element ID of the input file picker element.\n",
              " * @param {string} outputId Element ID of the output display.\n",
              " * @return {!Iterable<!Object>} Iterable of next steps.\n",
              " */\n",
              "function* uploadFilesStep(inputId, outputId) {\n",
              "  const inputElement = document.getElementById(inputId);\n",
              "  inputElement.disabled = false;\n",
              "\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  outputElement.innerHTML = '';\n",
              "\n",
              "  const pickedPromise = new Promise((resolve) => {\n",
              "    inputElement.addEventListener('change', (e) => {\n",
              "      resolve(e.target.files);\n",
              "    });\n",
              "  });\n",
              "\n",
              "  const cancel = document.createElement('button');\n",
              "  inputElement.parentElement.appendChild(cancel);\n",
              "  cancel.textContent = 'Cancel upload';\n",
              "  const cancelPromise = new Promise((resolve) => {\n",
              "    cancel.onclick = () => {\n",
              "      resolve(null);\n",
              "    };\n",
              "  });\n",
              "\n",
              "  // Wait for the user to pick the files.\n",
              "  const files = yield {\n",
              "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
              "    response: {\n",
              "      action: 'starting',\n",
              "    }\n",
              "  };\n",
              "\n",
              "  cancel.remove();\n",
              "\n",
              "  // Disable the input element since further picks are not allowed.\n",
              "  inputElement.disabled = true;\n",
              "\n",
              "  if (!files) {\n",
              "    return {\n",
              "      response: {\n",
              "        action: 'complete',\n",
              "      }\n",
              "    };\n",
              "  }\n",
              "\n",
              "  for (const file of files) {\n",
              "    const li = document.createElement('li');\n",
              "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
              "    li.append(span(\n",
              "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
              "        `last modified: ${\n",
              "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
              "                                    'n/a'} - `));\n",
              "    const percent = span('0% done');\n",
              "    li.appendChild(percent);\n",
              "\n",
              "    outputElement.appendChild(li);\n",
              "\n",
              "    const fileDataPromise = new Promise((resolve) => {\n",
              "      const reader = new FileReader();\n",
              "      reader.onload = (e) => {\n",
              "        resolve(e.target.result);\n",
              "      };\n",
              "      reader.readAsArrayBuffer(file);\n",
              "    });\n",
              "    // Wait for the data to be ready.\n",
              "    let fileData = yield {\n",
              "      promise: fileDataPromise,\n",
              "      response: {\n",
              "        action: 'continue',\n",
              "      }\n",
              "    };\n",
              "\n",
              "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
              "    let position = 0;\n",
              "    do {\n",
              "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
              "      const chunk = new Uint8Array(fileData, position, length);\n",
              "      position += length;\n",
              "\n",
              "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
              "      yield {\n",
              "        response: {\n",
              "          action: 'append',\n",
              "          file: file.name,\n",
              "          data: base64,\n",
              "        },\n",
              "      };\n",
              "\n",
              "      let percentDone = fileData.byteLength === 0 ?\n",
              "          100 :\n",
              "          Math.round((position / fileData.byteLength) * 100);\n",
              "      percent.textContent = `${percentDone}% done`;\n",
              "\n",
              "    } while (position < fileData.byteLength);\n",
              "  }\n",
              "\n",
              "  // All done.\n",
              "  yield {\n",
              "    response: {\n",
              "      action: 'complete',\n",
              "    }\n",
              "  };\n",
              "}\n",
              "\n",
              "scope.google = scope.google || {};\n",
              "scope.google.colab = scope.google.colab || {};\n",
              "scope.google.colab._files = {\n",
              "  _uploadFiles,\n",
              "  _uploadFilesContinue,\n",
              "};\n",
              "})(self);\n",
              "</script> "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Saving reviews.csv to reviews.csv\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv('reviews.csv')"
      ],
      "metadata": {
        "id": "HILJMpa_y26e"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df = df.dropna()"
      ],
      "metadata": {
        "id": "WXD0iAR5k62v"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df"
      ],
      "metadata": {
        "id": "MTCScT7uOdUy",
        "outputId": "9e192dc3-e443-4857-b45a-213b390d350d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        }
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                              sentence sentiment\n",
              "0    Not sure who was more lost - the flat characte...         0\n",
              "1    Attempting artiness with black & white and cle...         0\n",
              "2           Very little music or anything to speak of.         0\n",
              "3    The best scene in the movie was when Gerardo i...         1\n",
              "4    The rest of the movie lacks art, charm, meanin...         0\n",
              "..                                                 ...       ...\n",
              "994  I just got bored watching Jessice Lange take h...         0\n",
              "995  Unfortunately, any virtue in this film's produ...         0\n",
              "996                     In a word, it is embarrassing.         0\n",
              "997                                 Exceptionally bad!         0\n",
              "998  All in all its an insult to one's intelligence...         0\n",
              "\n",
              "[955 rows x 2 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-5f777b11-0862-4b99-9b6e-e9e785509c59\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentence</th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Not sure who was more lost - the flat characte...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Attempting artiness with black &amp; white and cle...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Very little music or anything to speak of.</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>The best scene in the movie was when Gerardo i...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>The rest of the movie lacks art, charm, meanin...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>994</th>\n",
              "      <td>I just got bored watching Jessice Lange take h...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>995</th>\n",
              "      <td>Unfortunately, any virtue in this film's produ...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>996</th>\n",
              "      <td>In a word, it is embarrassing.</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>997</th>\n",
              "      <td>Exceptionally bad!</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>998</th>\n",
              "      <td>All in all its an insult to one's intelligence...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>955 rows × 2 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5f777b11-0862-4b99-9b6e-e9e785509c59')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-5f777b11-0862-4b99-9b6e-e9e785509c59 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-5f777b11-0862-4b99-9b6e-e9e785509c59');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-dd2e5e29-c0d3-4bb4-996a-ca9c91066655\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-dd2e5e29-c0d3-4bb4-996a-ca9c91066655')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-dd2e5e29-c0d3-4bb4-996a-ca9c91066655 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_21263fda-dd4a-4a01-a045-32f5b51d7cb2\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_21263fda-dd4a-4a01-a045-32f5b51d7cb2 button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.to_csv('reviews.csv', index=False)"
      ],
      "metadata": {
        "id": "7j9rNGkQpn7y"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-QXo11Bxvytu"
      },
      "source": [
        "# **Section 3: KNN MODEL**\n",
        "\n",
        "Given below are two KNN models; in the first case we are using Bag-of-Words and in the second case we are using TF-IDF.\n",
        "Note the different metrics and parameters used in each."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R5yZXboZ92Z4"
      },
      "source": [
        "from sklearn import metrics, neighbors\n",
        "from sklearn.model_selection import train_test_split, cross_val_score, cross_val_predict\n",
        "\n",
        "## TASK - 1: Tweak the models below and see results with different parameters and distance metrics.\n",
        "\n",
        "def bow_knn():\n",
        "    \"\"\"Method for determining nearest neighbors using bag-of-words and K-Nearest Neighbor algorithm\"\"\"\n",
        "\n",
        "    training_data = pd.read_csv('reviews.csv')\n",
        "    X_train, X_test, y_train, y_test = train_test_split(training_data[\"sentence\"], training_data[\"sentiment\"], test_size=0.2, random_state=5)\n",
        "    X_train, X_test = createBagOfWords(X_train, X_test, remove_stopwords=True, lemmatize=True, stemmer=False)\n",
        "    # print(X_train)\n",
        "    knn = neighbors.KNeighborsClassifier(n_neighbors=5, weights='uniform', algorithm='auto', leaf_size=30, p=2, metric='euclidean', metric_params=None, n_jobs=1)\n",
        "\n",
        "    knn.fit(X_train, y_train)\n",
        "    predicted = knn.predict(X_test)\n",
        "    acc = metrics.accuracy_score(y_test, predicted)\n",
        "    print('KNN with BOW accuracy = ' + str(acc * 100) + '%')\n",
        "\n",
        "    scores = cross_val_score(knn, X_train, y_train, cv=3)\n",
        "    print(\"Cross Validation Accuracy: %0.2f\" % (scores.mean()))\n",
        "    print(scores)\n",
        "    print('\\n')\n",
        "    return predicted, y_test\n",
        "\n",
        "\n",
        "def tfidf_knn():\n",
        "    \"\"\"Method for determining nearest neighbors using tf-idf and K-Nearest Neighbor algorithm\"\"\"\n",
        "\n",
        "    training_data = pd.read_csv('reviews.csv')\n",
        "    X_train, X_test, y_train, y_test = train_test_split(training_data[\"sentence\"], training_data[\"sentiment\"],\n",
        "                                                        test_size=0.2, random_state=5)\n",
        "    X_train, X_test = createTFIDF(X_train, X_test, remove_stopwords=True, lemmatize=True, stemmer=False)\n",
        "    # print(X_train)\n",
        "    knn = neighbors.KNeighborsClassifier(n_neighbors=5, weights='distance', algorithm='brute', leaf_size=30, p=2,\n",
        "                                         metric='cosine', metric_params=None, n_jobs=1)\n",
        "\n",
        "    knn.fit(X_train, y_train)\n",
        "    predicted = knn.predict(X_test)\n",
        "    acc = metrics.accuracy_score(y_test, predicted)\n",
        "    print('KNN with TFIDF accuracy = ' + str(acc * 100) + '%')\n",
        "\n",
        "    scores = cross_val_score(knn, X_train, y_train, cv=3)\n",
        "    print(\"Cross Validation Accuracy: %0.2f\" % (scores.mean()))\n",
        "    print(scores)\n",
        "    return predicted, y_test"
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Note: Cross-validation will be discussed in detail in the upcoming lab session."
      ],
      "metadata": {
        "id": "uPuI3wrL-8ZJ"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Xzno1rRNHoFT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a6fe7518-9d6d-43e7-c1f4-dfa25492e647"
      },
      "source": [
        "## KNN accuracy after using BoW\n",
        "predicted, y_test = bow_knn()"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-2-3fc32437a98d>:24: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
            "  soup = BeautifulSoup(text, \"lxml\")\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "KNN with BOW accuracy = 62.30366492146597%\n",
            "Cross Validation Accuracy: 0.62\n",
            "[0.60784314 0.58431373 0.66141732]\n",
            "\n",
            "\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/model_selection/_split.py:700: UserWarning: The least populated class in y has only 1 members, which is less than n_splits=3.\n",
            "  warnings.warn(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FI5NMP8L-4eW",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3c59e769-a462-4045-e490-3a28bbec8e7b"
      },
      "source": [
        "## KNN accuracy after using TFIDF\n",
        "predicted, y_test = tfidf_knn()"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-2-3fc32437a98d>:24: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
            "  soup = BeautifulSoup(text, \"lxml\")\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "KNN with TFIDF accuracy = 70.15706806282722%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/model_selection/_split.py:700: UserWarning: The least populated class in y has only 1 members, which is less than n_splits=3.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cross Validation Accuracy: 0.73\n",
            "[0.7254902  0.74117647 0.72834646]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vm2vYDVfa5AP"
      },
      "source": [
        "# Section 4: SPAM TEXT DATASET\n",
        "Now let's use what we've learnt to classify texts as spam or not spam."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "id": "c1GZgvXCsKX1",
        "outputId": "122f50dc-c0e3-4d76-9695-0cb04246a5fc"
      },
      "source": [
        "# Upload the spam text data CSV file that has been shared with you. You can also download the file from https://www.kaggle.com/datasets/uciml/sms-spam-collection-dataset\n",
        "# Run this cell, click on the 'Choose files' button and upload the file.\n",
        "from google.colab import files\n",
        "uploaded = files.upload()"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-15887424-f2e6-4955-8072-79e444475c69\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-15887424-f2e6-4955-8072-79e444475c69\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script>// Copyright 2017 Google LLC\n",
              "//\n",
              "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
              "// you may not use this file except in compliance with the License.\n",
              "// You may obtain a copy of the License at\n",
              "//\n",
              "//      http://www.apache.org/licenses/LICENSE-2.0\n",
              "//\n",
              "// Unless required by applicable law or agreed to in writing, software\n",
              "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
              "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
              "// See the License for the specific language governing permissions and\n",
              "// limitations under the License.\n",
              "\n",
              "/**\n",
              " * @fileoverview Helpers for google.colab Python module.\n",
              " */\n",
              "(function(scope) {\n",
              "function span(text, styleAttributes = {}) {\n",
              "  const element = document.createElement('span');\n",
              "  element.textContent = text;\n",
              "  for (const key of Object.keys(styleAttributes)) {\n",
              "    element.style[key] = styleAttributes[key];\n",
              "  }\n",
              "  return element;\n",
              "}\n",
              "\n",
              "// Max number of bytes which will be uploaded at a time.\n",
              "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
              "\n",
              "function _uploadFiles(inputId, outputId) {\n",
              "  const steps = uploadFilesStep(inputId, outputId);\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  // Cache steps on the outputElement to make it available for the next call\n",
              "  // to uploadFilesContinue from Python.\n",
              "  outputElement.steps = steps;\n",
              "\n",
              "  return _uploadFilesContinue(outputId);\n",
              "}\n",
              "\n",
              "// This is roughly an async generator (not supported in the browser yet),\n",
              "// where there are multiple asynchronous steps and the Python side is going\n",
              "// to poll for completion of each step.\n",
              "// This uses a Promise to block the python side on completion of each step,\n",
              "// then passes the result of the previous step as the input to the next step.\n",
              "function _uploadFilesContinue(outputId) {\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  const steps = outputElement.steps;\n",
              "\n",
              "  const next = steps.next(outputElement.lastPromiseValue);\n",
              "  return Promise.resolve(next.value.promise).then((value) => {\n",
              "    // Cache the last promise value to make it available to the next\n",
              "    // step of the generator.\n",
              "    outputElement.lastPromiseValue = value;\n",
              "    return next.value.response;\n",
              "  });\n",
              "}\n",
              "\n",
              "/**\n",
              " * Generator function which is called between each async step of the upload\n",
              " * process.\n",
              " * @param {string} inputId Element ID of the input file picker element.\n",
              " * @param {string} outputId Element ID of the output display.\n",
              " * @return {!Iterable<!Object>} Iterable of next steps.\n",
              " */\n",
              "function* uploadFilesStep(inputId, outputId) {\n",
              "  const inputElement = document.getElementById(inputId);\n",
              "  inputElement.disabled = false;\n",
              "\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  outputElement.innerHTML = '';\n",
              "\n",
              "  const pickedPromise = new Promise((resolve) => {\n",
              "    inputElement.addEventListener('change', (e) => {\n",
              "      resolve(e.target.files);\n",
              "    });\n",
              "  });\n",
              "\n",
              "  const cancel = document.createElement('button');\n",
              "  inputElement.parentElement.appendChild(cancel);\n",
              "  cancel.textContent = 'Cancel upload';\n",
              "  const cancelPromise = new Promise((resolve) => {\n",
              "    cancel.onclick = () => {\n",
              "      resolve(null);\n",
              "    };\n",
              "  });\n",
              "\n",
              "  // Wait for the user to pick the files.\n",
              "  const files = yield {\n",
              "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
              "    response: {\n",
              "      action: 'starting',\n",
              "    }\n",
              "  };\n",
              "\n",
              "  cancel.remove();\n",
              "\n",
              "  // Disable the input element since further picks are not allowed.\n",
              "  inputElement.disabled = true;\n",
              "\n",
              "  if (!files) {\n",
              "    return {\n",
              "      response: {\n",
              "        action: 'complete',\n",
              "      }\n",
              "    };\n",
              "  }\n",
              "\n",
              "  for (const file of files) {\n",
              "    const li = document.createElement('li');\n",
              "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
              "    li.append(span(\n",
              "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
              "        `last modified: ${\n",
              "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
              "                                    'n/a'} - `));\n",
              "    const percent = span('0% done');\n",
              "    li.appendChild(percent);\n",
              "\n",
              "    outputElement.appendChild(li);\n",
              "\n",
              "    const fileDataPromise = new Promise((resolve) => {\n",
              "      const reader = new FileReader();\n",
              "      reader.onload = (e) => {\n",
              "        resolve(e.target.result);\n",
              "      };\n",
              "      reader.readAsArrayBuffer(file);\n",
              "    });\n",
              "    // Wait for the data to be ready.\n",
              "    let fileData = yield {\n",
              "      promise: fileDataPromise,\n",
              "      response: {\n",
              "        action: 'continue',\n",
              "      }\n",
              "    };\n",
              "\n",
              "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
              "    let position = 0;\n",
              "    do {\n",
              "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
              "      const chunk = new Uint8Array(fileData, position, length);\n",
              "      position += length;\n",
              "\n",
              "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
              "      yield {\n",
              "        response: {\n",
              "          action: 'append',\n",
              "          file: file.name,\n",
              "          data: base64,\n",
              "        },\n",
              "      };\n",
              "\n",
              "      let percentDone = fileData.byteLength === 0 ?\n",
              "          100 :\n",
              "          Math.round((position / fileData.byteLength) * 100);\n",
              "      percent.textContent = `${percentDone}% done`;\n",
              "\n",
              "    } while (position < fileData.byteLength);\n",
              "  }\n",
              "\n",
              "  // All done.\n",
              "  yield {\n",
              "    response: {\n",
              "      action: 'complete',\n",
              "    }\n",
              "  };\n",
              "}\n",
              "\n",
              "scope.google = scope.google || {};\n",
              "scope.google.colab = scope.google.colab || {};\n",
              "scope.google.colab._files = {\n",
              "  _uploadFiles,\n",
              "  _uploadFilesContinue,\n",
              "};\n",
              "})(self);\n",
              "</script> "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Saving spam.csv to spam.csv\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qRiS7dT7piTE",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 513
        },
        "outputId": "e18e5222-1d96-4fe9-eebd-339e5591eb48"
      },
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv('spam.csv', error_bad_lines=False)\n",
        "df"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-16-3b52591eaf2b>:2: FutureWarning: The error_bad_lines argument has been deprecated and will be removed in a future version. Use on_bad_lines in the future.\n",
            "\n",
            "\n",
            "  df = pd.read_csv('spam.csv', error_bad_lines=False)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     Category                                            Message\n",
              "0         ham  Go until jurong point, crazy.. Available only ...\n",
              "1         ham                      Ok lar... Joking wif u oni...\n",
              "2        spam  Free entry in 2 a wkly comp to win FA Cup fina...\n",
              "3         ham  U dun say so early hor... U c already then say...\n",
              "4         ham  Nah I don't think he goes to usf, he lives aro...\n",
              "...       ...                                                ...\n",
              "5567     spam  This is the 2nd time we have tried 2 contact u...\n",
              "5568      ham              Will Ã¼ b going to esplanade fr home?\n",
              "5569      ham  Pity, * was in mood for that. So...any other s...\n",
              "5570      ham  The guy did some bitching but I acted like i'd...\n",
              "5571      ham                         Rofl. Its true to its name\n",
              "\n",
              "[5572 rows x 2 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-36661ca1-a801-4c62-adcd-5f91062b3770\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Category</th>\n",
              "      <th>Message</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>ham</td>\n",
              "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ham</td>\n",
              "      <td>Ok lar... Joking wif u oni...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>spam</td>\n",
              "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ham</td>\n",
              "      <td>U dun say so early hor... U c already then say...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ham</td>\n",
              "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5567</th>\n",
              "      <td>spam</td>\n",
              "      <td>This is the 2nd time we have tried 2 contact u...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5568</th>\n",
              "      <td>ham</td>\n",
              "      <td>Will Ã¼ b going to esplanade fr home?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5569</th>\n",
              "      <td>ham</td>\n",
              "      <td>Pity, * was in mood for that. So...any other s...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5570</th>\n",
              "      <td>ham</td>\n",
              "      <td>The guy did some bitching but I acted like i'd...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5571</th>\n",
              "      <td>ham</td>\n",
              "      <td>Rofl. Its true to its name</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5572 rows × 2 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-36661ca1-a801-4c62-adcd-5f91062b3770')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-36661ca1-a801-4c62-adcd-5f91062b3770 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-36661ca1-a801-4c62-adcd-5f91062b3770');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-18d1871c-d6fd-45d2-9d75-5cbb1052bb40\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-18d1871c-d6fd-45d2-9d75-5cbb1052bb40')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-18d1871c-d6fd-45d2-9d75-5cbb1052bb40 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_5bd7d6c3-6127-4ea9-bb9b-7079bce3bbab\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_5bd7d6c3-6127-4ea9-bb9b-7079bce3bbab button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OsvBm4luNCME"
      },
      "source": [
        "df['Category'] = df['Category'].map({'ham': 0, 'spam': 1})"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KiEHuMypWyb6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "outputId": "64f57fd5-c13c-4c7f-a19e-be579b7a4cb9"
      },
      "source": [
        "df.head(5)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   Category                                            Message\n",
              "0         0  Go until jurong point, crazy.. Available only ...\n",
              "1         0                      Ok lar... Joking wif u oni...\n",
              "2         1  Free entry in 2 a wkly comp to win FA Cup fina...\n",
              "3         0  U dun say so early hor... U c already then say...\n",
              "4         0  Nah I don't think he goes to usf, he lives aro..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-42a8cd12-cb18-442d-a31e-9deeb6cdbd35\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Category</th>\n",
              "      <th>Message</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>Ok lar... Joking wif u oni...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>U dun say so early hor... U c already then say...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-42a8cd12-cb18-442d-a31e-9deeb6cdbd35')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-42a8cd12-cb18-442d-a31e-9deeb6cdbd35 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-42a8cd12-cb18-442d-a31e-9deeb6cdbd35');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-2bdf4a36-ef6f-4711-90a6-9ed9ddcda007\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-2bdf4a36-ef6f-4711-90a6-9ed9ddcda007')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-2bdf4a36-ef6f-4711-90a6-9ed9ddcda007 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "len(df)"
      ],
      "metadata": {
        "id": "WRJU9rFy1XQR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f812565f-9ae0-45f1-eb6f-d655d88b7107"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "5572"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wnv0v4T6sqJQ"
      },
      "source": [
        "from sklearn import metrics, neighbors\n",
        "from sklearn.model_selection import train_test_split, cross_val_score, cross_val_predict\n",
        "\n",
        "## TASK - 2: Tweak the models below and see results with different parameters and distance metrics.\n",
        "\n",
        "def bow_knn():\n",
        "    \"\"\"Method for determining nearest neighbors using bag-of-words and K-Nearest Neighbor algorithm\"\"\"\n",
        "\n",
        "    training_data = pd.read_csv('spam.csv')\n",
        "    training_data['Category'] = training_data['Category'].map({'ham': 0, 'spam': 1})\n",
        "    X_train, X_test, y_train, y_test = train_test_split(training_data[\"Message\"], training_data[\"Category\"], test_size=0.2, random_state=5)\n",
        "    X_train, X_test = createBagOfWords(X_train, X_test, remove_stopwords=True, lemmatize=True, stemmer=False)\n",
        "    knn = neighbors.KNeighborsClassifier(n_neighbors=5, weights='uniform', algorithm='auto', leaf_size=30, p=2, metric='euclidean', metric_params=None, n_jobs=1)\n",
        "\n",
        "    knn.fit(X_train, y_train)\n",
        "    predicted = knn.predict(X_test)\n",
        "    acc = metrics.accuracy_score(y_test, predicted)\n",
        "    print('KNN with BOW accuracy = ' + str(acc * 100) + '%')\n",
        "\n",
        "    scores = cross_val_score(knn, X_train, y_train, cv=3)\n",
        "    print(\"Cross Validation Accuracy: %0.2f\" % (scores.mean()))\n",
        "    print(scores)\n",
        "    print('\\n')\n",
        "    return predicted, y_test\n",
        "\n",
        "\n",
        "def tfidf_knn():\n",
        "    \"\"\"Method for determining nearest neighbors using tf-idf and K-Nearest Neighbor algorithm\"\"\"\n",
        "\n",
        "    training_data = pd.read_csv('spam.csv')\n",
        "    training_data['Category'] = training_data['Category'].map({'ham': 0, 'spam': 1})\n",
        "    X_train, X_test, y_train, y_test = train_test_split(training_data[\"Message\"], training_data[\"Category\"], test_size=0.2, random_state=5)\n",
        "    X_train, X_test = createTFIDF(X_train, X_test, remove_stopwords=True, lemmatize=True, stemmer=False)\n",
        "    knn = neighbors.KNeighborsClassifier(n_neighbors=5, weights='distance', algorithm='brute', leaf_size=30, p=2, metric='cosine', metric_params=None, n_jobs=1)\n",
        "\n",
        "    knn.fit(X_train, y_train)\n",
        "    predicted = knn.predict(X_test)\n",
        "    acc = metrics.accuracy_score(y_test, predicted)\n",
        "    print('KNN with TFIDF accuracy = ' + str(acc * 100) + '%')\n",
        "\n",
        "    scores = cross_val_score(knn, X_train, y_train, cv=3)\n",
        "    print(\"Cross Validation Accuracy: %0.2f\" % (scores.mean()))\n",
        "    print(scores)\n",
        "    return predicted, y_test"
      ],
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W8PwydHYs1h_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1d68d0f2-8d9c-41d6-94a7-8d81d8a5e1f3"
      },
      "source": [
        "# This cell may take some time to run\n",
        "predicted, y_test = bow_knn()"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-2-3fc32437a98d>:24: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
            "  soup = BeautifulSoup(text, \"lxml\")\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "KNN with BOW accuracy = 92.19730941704036%\n",
            "Cross Validation Accuracy: 0.91\n",
            "[0.90713324 0.90040377 0.91245791]\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# This cell may take some time to run\n",
        "predicted, y_test = tfidf_knn()"
      ],
      "metadata": {
        "id": "zf8i2P1nxpl8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2f0a3884-e8eb-419b-f66f-bcf1681cf2a4"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-2-3fc32437a98d>:24: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
            "  soup = BeautifulSoup(text, \"lxml\")\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "KNN with TFIDF accuracy = 98.56502242152466%\n",
            "Cross Validation Accuracy: 0.97\n",
            "[0.96837147 0.96769852 0.96363636]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Questions to Think About and Answer\n",
        "1. Why does the TF-IDF approach generally result in a better accuracy than Bag-of-Words ?\n",
        "2. Can you think of techniques that are better than both BoW and TF-IDF ?\n",
        "3. Read about Stemming and Lemmatization from the resources given below. Think about the pros/cons of each."
      ],
      "metadata": {
        "id": "f9xvbzS4yLTr"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#1.Why does the TF-IDF approach generally result in a better accuracy than Bag-of-Words ?\n",
        "Ans:-**The TF-IDF (Term Frequency-Inverse Document Frequency) approach and the Bag-of-Words (BoW) model are both techniques used in natural language processing (NLP) for representing and analyzing text data. While the effectiveness of these approaches can depend on the specific task and dataset, TF-IDF often outperforms Bag-of-Words in certain scenarios. Here are some reasons why TF-IDF can result in better accuracy than Bag-of-Words:**\n",
        "\n",
        "1. **Term Importance:**\n",
        "   - In Bag-of-Words, each term in the document is treated equally, without considering its importance in the context of the entire corpus. TF-IDF, on the other hand, takes into account not only the term frequency in a document (TF) but also the inverse document frequency (IDF), which measures how important a term is across the entire corpus. Terms that are common in many documents receive lower IDF scores and are therefore downweighted, allowing more emphasis on terms that are specific to a particular document.\n",
        "\n",
        "2. **Discriminative Power:**\n",
        "   - TF-IDF emphasizes words that are unique or discriminative to a document. Terms that appear frequently in a particular document but rarely in the rest of the corpus are considered important. This helps in capturing the distinctive characteristics of each document and can be particularly useful in tasks such as text classification.\n",
        "\n",
        "3. **Noise Reduction:**\n",
        "   - Bag-of-Words can be sensitive to noise, as it includes all terms present in a document. TF-IDF reduces the impact of common terms that may not carry much information about the content of the document. By assigning lower weights to terms that are common across many documents, TF-IDF helps in reducing noise and focusing on more informative features.\n",
        "\n",
        "4. **Dimensionality Reduction:**\n",
        "   - TF-IDF implicitly performs a form of dimensionality reduction by assigning lower weights to less important terms. This can be beneficial in reducing the dimensionality of the feature space, which is important for computational efficiency and mitigating the curse of dimensionality.\n",
        "\n",
        "5. **Semantic Understanding:**\n",
        "   - TF-IDF captures some level of semantic understanding by considering the importance of terms in the context of the entire corpus. This can help in capturing the meaning of words beyond their simple frequency of occurrence.\n"
      ],
      "metadata": {
        "id": "JXsTNXcYNavF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#2.Can you think of techniques that are better than both BoW and TF-IDF ?\n",
        "\n",
        "Ans:- While Bag-of-Words (BoW) and TF-IDF are traditional and widely used text representation techniques, several advanced methods have been developed that often outperform them in various natural language processing (NLP) tasks. Some of these techniques include:\n",
        "\n",
        "1. **Word Embeddings:**\n",
        "   - Word embeddings, such as Word2Vec, GloVe, and FastText, represent words as dense vectors in a continuous vector space. These embeddings capture semantic relationships between words and can capture contextual information. They are often pre-trained on large corpora and can be fine-tuned for specific tasks.\n",
        "\n",
        "2. **Doc2Vec (Paragraph Vectors):**\n",
        "   - Doc2Vec extends the concept of Word2Vec to entire documents. It represents documents as vectors in a continuous space, allowing for the capture of document-level semantics. This can be particularly useful for tasks like document similarity and clustering.\n",
        "\n",
        "3. **BERT (Bidirectional Encoder Representations from Transformers):**\n",
        "   - BERT is a transformer-based model that has achieved state-of-the-art results in various NLP tasks. It considers the bidirectional context of words in a sentence, capturing intricate relationships and dependencies. BERT embeddings can be used as feature vectors or fine-tuned for specific tasks.\n",
        "\n",
        "4. **ELMo (Embeddings from Language Models):**\n",
        "   - ELMo produces contextualized word embeddings by considering the context in which a word appears in a sentence. It uses a deep, bi-directional LSTM network to generate embeddings that capture the meaning of words in context.\n",
        "\n",
        "5. **ULMFiT (Universal Language Model Fine-tuning):**\n",
        "   - ULMFiT is a transfer learning approach for NLP tasks. It involves pre-training a language model on a large corpus and then fine-tuning it for a specific task with a smaller dataset. This has been successful in achieving good results with limited labeled data.\n",
        "\n",
        "6. **Transformer Models (e.g., GPT, T5):**\n",
        "   - Large transformer-based models like GPT (Generative Pre-trained Transformer) and T5 (Text-to-Text Transfer Transformer) have shown exceptional performance across a wide range of NLP tasks. They are pre-trained on massive amounts of data and fine-tuned for specific tasks, enabling them to learn intricate language patterns.\n",
        "\n",
        "7. **Ensemble Models:**\n",
        "   - Combining predictions from multiple models, either with different architectures or trained on different data subsets, can often lead to improved performance. Ensemble methods can help mitigate the weaknesses of individual models.\n",
        "\n"
      ],
      "metadata": {
        "id": "dwBNzWDmN0dI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#3.Read about Stemming and Lemmatization from the resources given below. Think about the pros/cons of each.\n",
        "\n",
        "\n",
        "\n",
        "Ans:-  Stemming and lemmatization are both techniques used in natural language processing to reduce words to their base or root forms. Here's a brief overview of each, along with their pros and cons:\n",
        "\n",
        "### Stemming:\n",
        "\n",
        "**Definition:**\n",
        "Stemming involves removing or replacing suffixes from words to obtain their root or base form. It operates by heuristically chopping off the ends of words.\n",
        "\n",
        "**Pros:**\n",
        "1. **Simplicity and Speed:** Stemming is computationally less expensive and faster than lemmatization. It is simpler as it applies heuristic rules to truncate word endings.\n",
        "2. **Reduction of Variations:** Stemming reduces words to their base or root forms, helping to reduce the variation in words and group similar terms together.\n",
        "\n",
        "**Cons:**\n",
        "1. **Lack of Accuracy:** Stemming may not always produce a valid word or the actual root form. It can result in stems that are not semantically correct or meaningful.\n",
        "2. **Over-stemming or Under-stemming:** Stemming algorithms might over-stem (remove too many letters) or under-stem (remove too few letters), leading to inaccurate representations.\n",
        "3. **Language Dependency:** Stemming algorithms are often language-dependent, meaning they may not work well for languages with complex morphologies.\n",
        "\n",
        "### Lemmatization:\n",
        "\n",
        "**Definition:**\n",
        "Lemmatization involves reducing words to their base or dictionary forms (lemmas) by considering the context and meaning of the words. It typically requires a lexical resource like a dictionary or a morphological analysis.\n",
        "\n",
        "**Pros:**\n",
        "1. **Semantic Accuracy:** Lemmatization provides a more accurate representation of a word's base form, considering its semantics and context.\n",
        "2. **Improved Interpretability:** Since lemmatization produces valid words, the output is more interpretable and understandable compared to stemming.\n",
        "3. **Language Independence:** Lemmatization can be more language-independent as it relies on a dictionary or language rules.\n",
        "\n",
        "**Cons:**\n",
        "1. **Computational Complexity:** Lemmatization is computationally more expensive than stemming, as it involves dictionary lookups or morphological analysis.\n",
        "2. **Dependency on Lexical Resources:** Lemmatization requires access to a comprehensive lexical resource, which may not be available for all languages or domains.\n",
        "3. **Potential for Over-generation:** In some cases, lemmatization might over-generate candidates for a word, especially if there are multiple possible lemmas.\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "wi_NAXLAOL_j"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Useful Resources for further reading\n",
        "1. Stemming and Lemmatization: https://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization-1.html\n",
        "2. TF-IDF and BoW : https://www.analyticsvidhya.com/blog/2020/02/quick-introduction-bag-of-words-bow-tf-idf/\n",
        "3. TF-IDF: https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfTransformer.html\n"
      ],
      "metadata": {
        "id": "T6xPL6smyWG-"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "-uUw16_tkGdV"
      },
      "execution_count": 22,
      "outputs": []
    }
  ]
}