diff --git a/Email Spam Classification/Datasets/Readme.md b/Email Spam Classification/Datasets/Readme.md new file mode 100644 index 000000000..e96d45b86 --- /dev/null +++ b/Email Spam Classification/Datasets/Readme.md @@ -0,0 +1,2 @@ +https://www.kaggle.com/datasets/uciml/sms-spam-collection-dataset/data +*Dataset from Kaggle* \ No newline at end of file diff --git a/Email Spam Classification/Datasets/archive (3).zip b/Email Spam Classification/Datasets/archive (3).zip new file mode 100644 index 000000000..57efc7aa4 Binary files /dev/null and b/Email Spam Classification/Datasets/archive (3).zip differ diff --git a/Email Spam Classification/Images/Readme.md b/Email Spam Classification/Images/Readme.md new file mode 100644 index 000000000..5b0cf37b6 --- /dev/null +++ b/Email Spam Classification/Images/Readme.md @@ -0,0 +1,9 @@ +This folder contains jpeg files of EDA results + + If there is the very high different between the positive values and negative values, then we can say our dataset in Imbalance Dataset, as is case here + + The wordclouds show more popular word-stems in spam and ham mail + + The ham message length on average is less than spam message length. + + diff --git a/Email Spam Classification/Images/Screenshot (237).png b/Email Spam Classification/Images/Screenshot (237).png new file mode 100644 index 000000000..09053e382 Binary files /dev/null and b/Email Spam Classification/Images/Screenshot (237).png differ diff --git a/Email Spam Classification/Images/Screenshot (238).png b/Email Spam Classification/Images/Screenshot (238).png new file mode 100644 index 000000000..060cd5745 Binary files /dev/null and b/Email Spam Classification/Images/Screenshot (238).png differ diff --git a/Email Spam Classification/Images/Screenshot (239).png b/Email Spam Classification/Images/Screenshot (239).png new file mode 100644 index 000000000..ed47ed860 Binary files /dev/null and b/Email Spam Classification/Images/Screenshot (239).png differ diff --git a/Email Spam Classification/Images/Screenshot (240).png b/Email Spam Classification/Images/Screenshot (240).png new file mode 100644 index 000000000..5be78dfac Binary files /dev/null and b/Email Spam Classification/Images/Screenshot (240).png differ diff --git a/Email Spam Classification/Models/Readme.md b/Email Spam Classification/Models/Readme.md new file mode 100644 index 000000000..91067b8dc --- /dev/null +++ b/Email Spam Classification/Models/Readme.md @@ -0,0 +1,29 @@ +# Spam Email Classification https://github.com/World-of-ML/DL-Simplified/issues/340 + +Full name : Aindree Chatterjee + +GitHub Profile Link : https://github.com/aindree-2005 + +Email ID : aindree2005@gmail.com + +Program : CodePeak + +Approach for this Project : + + +**LSTM (Long Short-Term Memory):** + +**Type:** LSTM is a type of recurrent neural network (RNN). +**Usage:** It is used for sequential data and is particularly effective in tasks where context or order of the data is important, such as time series prediction or natural language processing. +**Strengths:** LSTMs are designed to capture long-term dependencies and can be effective in handling sequences of variable length. +**Limitations:** LSTMs may struggle with capturing very long-term dependencies, and they can be computationally expensive. + +**BERT (Bidirectional Encoder Representations from Transformers):** + +**Type:** BERT is based on the transformer architecture. +**Usage:** BERT is specifically designed for natural language understanding tasks, such as question answering, sentiment analysis, and text classification. It has been pre-trained on large amounts of text data and can be fine-tuned for specific tasks. +**Strengths:** BERT models excel in capturing contextual information and have achieved state-of-the-art results in a wide range of NLP tasks. They can understand the meaning of words in context and handle bidirectional dependencies well. +**Limitations:** BERT models are computationally expensive, and they require a large amount of pre-training data. Fine-tuning on specific tasks is necessary for optimal performance. + +**My Conclusion** +LSTM is preferred over BERT for spam email detection here as it can capture sequential patterns in text. Spam emails often exhibit specific linguistic structures and patterns, and LSTMs excel in modeling sequential dependencies. BERT, designed for contextual understanding, might be overkill for simpler spam detection tasks, where the order of words matters less than overall patterns diff --git a/Email Spam Classification/Models/email-spam-classification (1).ipynb b/Email Spam Classification/Models/email-spam-classification (1).ipynb new file mode 100644 index 000000000..111cb63e5 --- /dev/null +++ b/Email Spam Classification/Models/email-spam-classification (1).ipynb @@ -0,0 +1 @@ +{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"tpu1vmV38","dataSources":[{"sourceId":982,"sourceType":"datasetVersion","datasetId":483}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n for filename in filenames:\n print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-12-10T08:28:08.307009Z","iopub.execute_input":"2023-12-10T08:28:08.307329Z","iopub.status.idle":"2023-12-10T08:28:08.315540Z","shell.execute_reply.started":"2023-12-10T08:28:08.307299Z","shell.execute_reply":"2023-12-10T08:28:08.314949Z"},"trusted":true},"execution_count":3,"outputs":[{"name":"stdout","text":"/kaggle/input/sms-spam-collection-dataset/spam.csv\n","output_type":"stream"}]},{"cell_type":"code","source":"! pip install nltk\n! pip install spacy","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:28:08.316900Z","iopub.execute_input":"2023-12-10T08:28:08.317218Z","iopub.status.idle":"2023-12-10T08:28:15.423772Z","shell.execute_reply.started":"2023-12-10T08:28:08.317191Z","shell.execute_reply":"2023-12-10T08:28:15.422839Z"},"trusted":true},"execution_count":4,"outputs":[{"name":"stdout","text":"Requirement already satisfied: nltk in /usr/local/lib/python3.10/site-packages (3.8.1)\nRequirement already satisfied: tqdm in /usr/local/lib/python3.10/site-packages (from nltk) (4.66.1)\nRequirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/site-packages (from nltk) (2023.10.3)\nRequirement already satisfied: click in /usr/local/lib/python3.10/site-packages (from nltk) (8.1.7)\nRequirement already satisfied: joblib in /usr/local/lib/python3.10/site-packages (from nltk) (1.3.2)\n\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n\u001b[0m\n\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\nRequirement already satisfied: spacy in /usr/local/lib/python3.10/site-packages (3.7.2)\nRequirement already satisfied: jinja2 in /usr/local/lib/python3.10/site-packages (from spacy) (3.1.2)\nRequirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /usr/local/lib/python3.10/site-packages (from spacy) (3.0.12)\nRequirement already satisfied: wasabi<1.2.0,>=0.9.1 in /usr/local/lib/python3.10/site-packages (from spacy) (1.1.2)\nRequirement already satisfied: numpy>=1.19.0 in /usr/local/lib/python3.10/site-packages (from spacy) (1.26.2)\nRequirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in /usr/local/lib/python3.10/site-packages (from spacy) (2.5.2)\nRequirement already satisfied: typer<0.10.0,>=0.3.0 in /usr/local/lib/python3.10/site-packages (from spacy) (0.9.0)\nRequirement already satisfied: setuptools in /usr/local/lib/python3.10/site-packages (from spacy) (65.5.1)\nRequirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.10/site-packages (from spacy) (3.3.0)\nRequirement already satisfied: smart-open<7.0.0,>=5.2.1 in /usr/local/lib/python3.10/site-packages (from spacy) (6.4.0)\nRequirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.10/site-packages (from spacy) (4.66.1)\nRequirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.10/site-packages (from spacy) (1.0.10)\nRequirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/site-packages (from spacy) (2.31.0)\nRequirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.10/site-packages (from spacy) (3.0.9)\nRequirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.10/site-packages (from spacy) (2.0.10)\nRequirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.10/site-packages (from spacy) (1.0.5)\nRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/site-packages (from spacy) (23.2)\nRequirement already satisfied: srsly<3.0.0,>=2.4.3 in /usr/local/lib/python3.10/site-packages (from spacy) (2.4.8)\nRequirement already satisfied: weasel<0.4.0,>=0.1.0 in /usr/local/lib/python3.10/site-packages (from spacy) (0.3.4)\nRequirement already satisfied: thinc<8.3.0,>=8.1.8 in /usr/local/lib/python3.10/site-packages (from spacy) (8.2.1)\nRequirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.10/site-packages (from spacy) (2.0.8)\nRequirement already satisfied: typing-extensions>=4.6.1 in /usr/local/lib/python3.10/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy) (4.8.0)\nRequirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy) (0.6.0)\nRequirement already satisfied: pydantic-core==2.14.5 in /usr/local/lib/python3.10/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy) (2.14.5)\nRequirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2.1.0)\nRequirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy) (3.3.2)\nRequirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy) (3.6)\nRequirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2023.11.17)\nRequirement already satisfied: blis<0.8.0,>=0.7.8 in /usr/local/lib/python3.10/site-packages (from thinc<8.3.0,>=8.1.8->spacy) (0.7.11)\nRequirement already satisfied: confection<1.0.0,>=0.0.1 in /usr/local/lib/python3.10/site-packages (from thinc<8.3.0,>=8.1.8->spacy) (0.1.4)\nRequirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.10/site-packages (from typer<0.10.0,>=0.3.0->spacy) (8.1.7)\nRequirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in /usr/local/lib/python3.10/site-packages (from weasel<0.4.0,>=0.1.0->spacy) (0.16.0)\nRequirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/site-packages (from jinja2->spacy) (2.1.3)\n\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n\u001b[0m\n\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n","output_type":"stream"}]},{"cell_type":"code","source":"import re\nimport string\nimport numpy as np \nimport random\nimport pandas as pd \nimport os\nimport nltk\nimport spacy\nimport random\nimport matplotlib.pyplot as plt\n%matplotlib inline\nfrom collections import Counter\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.feature_extraction.text import CountVectorizer\nfrom sklearn.feature_extraction.text import TfidfTransformer\nimport nltk as nltk\nfrom nltk.corpus import stopwords\nfrom nltk.tokenize import word_tokenize\nfrom spacy.util import compounding\nfrom spacy.util import minibatch\n\nimport keras\nfrom keras.models import Sequential\nfrom keras.initializers import Constant\nfrom keras.layers import (LSTM, \n Embedding, \n BatchNormalization,\n Dense, \n TimeDistributed, \n Dropout, \n Bidirectional,\n Flatten, \n GlobalMaxPool1D)\nfrom keras.preprocessing.text import Tokenizer\nfrom keras.preprocessing.sequence import pad_sequences\nfrom keras.callbacks import ModelCheckpoint, ReduceLROnPlateau\nfrom keras.optimizers import Adam\n\nfrom sklearn.metrics import (\n precision_score, \n recall_score, \n f1_score, \n classification_report,\n accuracy_score\n)","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:28:58.585230Z","iopub.execute_input":"2023-12-10T08:28:58.585631Z","iopub.status.idle":"2023-12-10T08:28:58.595770Z","shell.execute_reply.started":"2023-12-10T08:28:58.585599Z","shell.execute_reply":"2023-12-10T08:28:58.595010Z"},"trusted":true},"execution_count":11,"outputs":[]},{"cell_type":"markdown","source":"# Email Spam Classification\n### Aindree Chatterjee","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(\"/kaggle/input/sms-spam-collection-dataset/spam.csv\", encoding=\"latin-1\")\ndf = df.dropna(how=\"any\", axis=1)\ndf.columns = ['target', 'message']\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:29:01.091180Z","iopub.execute_input":"2023-12-10T08:29:01.091575Z","iopub.status.idle":"2023-12-10T08:29:01.114319Z","shell.execute_reply.started":"2023-12-10T08:29:01.091543Z","shell.execute_reply":"2023-12-10T08:29:01.113422Z"},"trusted":true},"execution_count":12,"outputs":[{"execution_count":12,"output_type":"execute_result","data":{"text/plain":" target message\n0 ham Go until jurong point, crazy.. Available only ...\n1 ham Ok lar... Joking wif u oni...\n2 spam Free entry in 2 a wkly comp to win FA Cup fina...\n3 ham U dun say so early hor... U c already then say...\n4 ham Nah I don't think he goes to usf, he lives aro...","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
targetmessage
0hamGo until jurong point, crazy.. Available only ...
1hamOk lar... Joking wif u oni...
2spamFree entry in 2 a wkly comp to win FA Cup fina...
3hamU dun say so early hor... U c already then say...
4hamNah I don't think he goes to usf, he lives aro...
\n
"},"metadata":{}}]},{"cell_type":"code","source":"df['message_len'] = df['message'].apply(lambda x: len(x.split(' ')))\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:28:53.918732Z","iopub.execute_input":"2023-12-10T08:28:53.918968Z","iopub.status.idle":"2023-12-10T08:28:53.934956Z","shell.execute_reply.started":"2023-12-10T08:28:53.918943Z","shell.execute_reply":"2023-12-10T08:28:53.934205Z"},"trusted":true},"execution_count":7,"outputs":[{"execution_count":7,"output_type":"execute_result","data":{"text/plain":" target message message_len\n0 ham Go until jurong point, crazy.. Available only ... 20\n1 ham Ok lar... Joking wif u oni... 6\n2 spam Free entry in 2 a wkly comp to win FA Cup fina... 28\n3 ham U dun say so early hor... U c already then say... 11\n4 ham Nah I don't think he goes to usf, he lives aro... 13","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
targetmessagemessage_len
0hamGo until jurong point, crazy.. Available only ...20
1hamOk lar... Joking wif u oni...6
2spamFree entry in 2 a wkly comp to win FA Cup fina...28
3hamU dun say so early hor... U c already then say...11
4hamNah I don't think he goes to usf, he lives aro...13
\n
"},"metadata":{}}]},{"cell_type":"code","source":"new=df.groupby('target')['target'].value_counts()\nnew\nnew.plot(kind='bar',title='Count (target)');","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:28:53.935813Z","iopub.execute_input":"2023-12-10T08:28:53.936054Z","iopub.status.idle":"2023-12-10T08:28:54.193058Z","shell.execute_reply.started":"2023-12-10T08:28:53.936029Z","shell.execute_reply":"2023-12-10T08:28:54.192231Z"},"trusted":true},"execution_count":8,"outputs":[{"output_type":"display_data","data":{"text/plain":"
","image/png":"iVBORw0KGgoAAAANSUhEUgAAAjAAAAHfCAYAAABOC+KlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAws0lEQVR4nO3de3xNd77/8XcuskNkJ3HLZYQELVK3Nqakpa02I9VwipiWdlRb6jCJQaqMU+PWix6dFr2gRlt6cZRW9ZCSatxmSNG0cdBSTIiOJlGabAwJyfr90UfWz25QIex84/V8PPbj0az13SufncfgNStrr+1lWZYlAAAAg3h7egAAAIDKImAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgABjl0KFD8vf316ZNmzw9SpX75ptv5Ovrq507d3p6FKDaI2CAGmr//v36z//8TzVr1kz+/v5yOp26/fbbNWvWLJ06dcrT40mSZs+erQULFlTqOVOnTlWnTp10++2329sWLVqkmTNnVu1wV9GF5o2JiVFiYqImTpx47YcCDOPFZyEBNU9aWpp+//vfy+Fw6JFHHlGbNm1UUlKif/zjH/roo4/06KOPat68eZ4eU23atFGDBg20fv36S1p/5MgR/eY3v9HChQs1YMAAe3vPnj21c+dOHThw4OoMWsUuNu+qVat03333ad++fWrevPm1Hw4whK+nBwBQtXJyctS/f381bdpUa9euVXh4uL0vOTlZ+/btU1pamgcnvHzvvfeefH191atXr6v+vc6ePauysjL5+fld9e91rvj4eIWEhGjhwoWaOnXqNf3egEn4FRJQw0yfPl0nTpzQm2++6RYv5Vq0aKGRI0faX589e1bPPPOMmjdvLofDoaioKP3Xf/2XiouL3Z7n5eWlyZMnVzheVFSUHn30UfvrBQsWyMvLS5s2bVJqaqoaNmyogIAA9enTR0eOHHF73q5du7RhwwZ5eXnJy8tLd91110Vf2/Lly9WpUyfVrVvX3nbXXXcpLS1NBw8etI8TFRUlSSopKdHEiRMVGxuroKAgBQQEqGvXrlq3bp3bcQ8cOCAvLy/99a9/1cyZM+2fxTfffCNJWr9+vTp27Ch/f381b95cb7zxhiZPniwvL68KM7733nuKjY1V7dq1Va9ePfXv31+HDh26pHklqVatWrrrrrv0ySefXPRnAVzvOAMD1DArVqxQs2bNdNttt13S+iFDhmjhwoXq16+fnnzySW3ZskXTpk3Tt99+q48//viy5xgxYoRCQkI0adIkHThwQDNnzlRKSoo++OADSdLMmTM1YsQI1a1bV08//bQkKTQ09ILHO3PmjLZt26bhw4e7bX/66adVVFSk77//XjNmzJAkO3BcLpfmz5+vAQMG6IknntDx48f15ptvKiEhQVu3blWHDh3cjvX222/r9OnTGjp0qBwOh+rVq6evv/5a9957r8LDwzVlyhSVlpZq6tSpatiwYYUZn3vuOf3lL3/RAw88oCFDhujIkSN69dVXdccdd+jrr79WcHDwRectFxsbq08++UQul0tOp7MSP3XgOmIBqDGKioosSdb9999/Seuzs7MtSdaQIUPcto8ZM8aSZK1du9beJsmaNGlShWM0bdrUGjRokP3122+/bUmy4uPjrbKyMnv76NGjLR8fH6uwsNDedtNNN1l33nnnJc26b98+S5L16quvVtiXmJhoNW3atML2s2fPWsXFxW7bfvrpJys0NNR6/PHH7W05OTmWJMvpdFoFBQVu63v16mXVqVPH+te//mVv27t3r+Xr62ud+1fogQMHLB8fH+u5555ze/6OHTssX19ft+0XmrfcokWLLEnWli1bLrgGuN7xKySgBnG5XJKkwMDAS1r/6aefSpJSU1Pdtj/55JOSdEXXygwdOtTtVyxdu3ZVaWmpDh48eFnHO3r0qCQpJCTkkp/j4+NjX8NSVlamY8eO6ezZs+rYsaO++uqrCuuTkpLczqyUlpbq888/V+/evRUREWFvb9GihXr06OH23GXLlqmsrEwPPPCAfvzxR/sRFhamG264ocKvrS6m/DX++OOPl/wc4HrDr5CAGqT81w3Hjx+/pPUHDx6Ut7e3WrRo4bY9LCxMwcHBlx0bktSkSRO3r8v/Uf7pp58u+5iSZFXyjZMLFy7USy+9pN27d+vMmTP29ujo6Aprf7mtoKBAp06dqvDzkVRh2969e2VZlm644YbzzlGrVq1Lnrn8NZ7vGhsAPyNggBrE6XQqIiKi0jdCu5J/KEtLS8+73cfH57zbKxsg5erXry+pcgH03nvv6dFHH1Xv3r311FNPqVGjRvLx8dG0adO0f//+Cutr1659WbNJP5/h8fLy0qpVq8772n95ncvFlL/GBg0aXPY8QE1HwAA1TM+ePTVv3jxlZmYqLi7uomubNm2qsrIy7d27V61bt7a35+fnq7CwUE2bNrW3hYSEqLCw0O35JSUl+uGHHy571sqEU5MmTVS7dm3l5ORc8nE+/PBDNWvWTMuWLXNbM2nSpEv6no0aNZK/v7/27dtXYd8vtzVv3lyWZSk6Olo33njjRY/7a687JydH3t7ev3oc4HrGNTBADTN27FgFBARoyJAhys/Pr7B///79mjVrliTpvvvuk6QKd4V9+eWXJUmJiYn2tubNm2vjxo1u6+bNm3fBMzCXIiAgoEIUXUitWrXUsWNHffnll+c9TlFRUYXt5WdCzj3rs2XLFmVmZl7S9/Tx8VF8fLyWL1+uw4cP29v37dunVatWua3t27evfHx8NGXKlApnmSzLsq/hudi85bKysnTTTTcpKCjokuYErkecgQFqmObNm2vRokV68MEH1bp1a7c78W7evFlLly6179vSvn17DRo0SPPmzVNhYaHuvPNObd26VQsXLlTv3r3VrVs3+7hDhgzRsGHDlJSUpN/97nfavn270tPTr+jXHLGxsZozZ46effZZtWjRQo0aNdLdd999wfX333+/nn766QpvL46NjdUHH3yg1NRU/fa3v1XdunXVq1cv9ezZU8uWLVOfPn2UmJionJwczZ07VzExMTpx4sQlzTh58mR99tlnuv322zV8+HCVlpbqtddeU5s2bZSdnW2va968uZ599lmNHz9eBw4cUO/evRUYGKicnBx9/PHHGjp0qMaMGXPReaWf3y6+YcMG/fGPf7yMnyhwHfHcG6AAXE3fffed9cQTT1hRUVGWn5+fFRgYaN1+++3Wq6++ap0+fdped+bMGWvKlClWdHS0VatWLSsyMtIaP3682xrLsqzS0lJr3LhxVoMGDaw6depYCQkJ1r59+y74Nupt27a5PX/dunWWJGvdunX2try8PCsxMdEKDAy0JP3qW6rz8/MtX19f691333XbfuLECeuhhx6ygoODLUn2W5TLysqs559/3mratKnlcDism2++2Vq5cqU1aNAgt7cxl7+N+sUXXzzv983IyLBuvvlmy8/Pz2revLk1f/5868knn7T8/f0rrP3oo4+sLl26WAEBAVZAQIDVqlUrKzk52dqzZ8+vzmtZlrVq1SpLkrV3796L/iyA6x2fhQTAKIMHD9Z3332nv//97x6do3fv3tq1a5f27t1b5cf18vK6opsIAtcDroEBYJRJkyZp27Zt2rRp0zX7nr/89O69e/fq008//dWPPqisb7/9VitXrtQzzzxTpccFaiLOwADArwgPD9ejjz6qZs2a6eDBg5ozZ46Ki4v19ddfX/C+LwCuLi7iBYBfce+99+p//ud/lJeXJ4fDobi4OD3//PPEC+BBnIEBAADG4RoYAABgHAIGAAAYp8ZeA1NWVqbDhw8rMDCQD0QDAMAQlmXp+PHjioiIkLf3hc+z1NiAOXz4sCIjIz09BgAAuAyHDh1S48aNL7i/UgEzefJkTZkyxW1by5YttXv3bknS6dOn9eSTT2rx4sUqLi5WQkKCZs+erdDQUHt9bm6uhg8frnXr1qlu3boaNGiQpk2bJl/f/z/K+vXrlZqaql27dikyMlITJkywb31+qQIDAyX9/AM495bjAACg+nK5XIqMjLT/Hb+QSp+Buemmm/T555///wOcEx6jR49WWlqali5dqqCgIKWkpKhv3772DadKS0uVmJiosLAwbd68WT/88IMeeeQR1apVS88//7yknz+FNTExUcOGDdP777+vjIwMDRkyROHh4UpISLjkOct/beR0OgkYAAAM82uXf1TqbdSTJ0/W8uXL3T7ArFxRUZEaNmyoRYsWqV+/fpKk3bt3q3Xr1srMzFTnzp21atUq9ezZU4cPH7bPysydO1fjxo3TkSNH5Ofnp3HjxiktLU07d+60j92/f38VFhZq9erVlzqqXC6XgoKCVFRURMAAAGCIS/33u9LvQtq7d68iIiLUrFkzPfzww8rNzZX088e/nzlzRvHx8fbaVq1aqUmTJvZH12dmZqpt27Zuv1JKSEiQy+XSrl277DXnHqN8TfkxLqS4uFgul8vtAQAAaqZKBUynTp20YMECrV69WnPmzFFOTo66du2q48ePKy8vT35+fgoODnZ7TmhoqPLy8iRJeXl5bvFSvr9838XWuFyuCp9Hcq5p06YpKCjIfnABLwAANVelroHp0aOH/d/t2rVTp06d1LRpUy1ZskS1a9eu8uEqY/z48UpNTbW/Lr8ICAAA1DxXdCO74OBg3Xjjjdq3b5/CwsJUUlKiwsJCtzX5+fkKCwuTJIWFhSk/P7/C/vJ9F1vjdDovGkkOh8O+YJcLdwEAqNmuKGBOnDih/fv3Kzw8XLGxsapVq5YyMjLs/Xv27FFubq7i4uIkSXFxcdqxY4cKCgrsNWvWrJHT6VRMTIy95txjlK8pPwYAAEClAmbMmDHasGGDDhw4oM2bN6tPnz7y8fHRgAEDFBQUpMGDBys1NVXr1q1TVlaWHnvsMcXFxalz586SpO7duysmJkYDBw7U9u3blZ6ergkTJig5OVkOh0OSNGzYMP3zn//U2LFjtXv3bs2ePVtLlizR6NGjq/7VAwAAI1XqGpjvv/9eAwYM0NGjR9WwYUN16dJFX3zxhRo2bChJmjFjhry9vZWUlOR2I7tyPj4+WrlypYYPH664uDgFBARo0KBBmjp1qr0mOjpaaWlpGj16tGbNmqXGjRtr/vz5lboHDAAAqNkqdR8Yk3AfGAAAzHPV7gMDAADgaQQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADBOpe4DAzNE/TnN0yPgGjrwQqKnRwCAa44zMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMc0UB88ILL8jLy0ujRo2yt50+fVrJycmqX7++6tatq6SkJOXn57s9Lzc3V4mJiapTp44aNWqkp556SmfPnnVbs379et1yyy1yOBxq0aKFFixYcCWjAgCAGuSyA2bbtm1644031K5dO7fto0eP1ooVK7R06VJt2LBBhw8fVt++fe39paWlSkxMVElJiTZv3qyFCxdqwYIFmjhxor0mJydHiYmJ6tatm7KzszVq1CgNGTJE6enplzsuAACoQS4rYE6cOKGHH35Yf/vb3xQSEmJvLyoq0ptvvqmXX35Zd999t2JjY/X2229r8+bN+uKLLyRJn332mb755hu999576tChg3r06KFnnnlGr7/+ukpKSiRJc+fOVXR0tF566SW1bt1aKSkp6tevn2bMmFEFLxkAAJjusgImOTlZiYmJio+Pd9uelZWlM2fOuG1v1aqVmjRposzMTElSZmam2rZtq9DQUHtNQkKCXC6Xdu3aZa/55bETEhLsY5xPcXGxXC6X2wMAANRMvpV9wuLFi/XVV19p27ZtFfbl5eXJz89PwcHBbttDQ0OVl5dnrzk3Xsr3l++72BqXy6VTp06pdu3aFb73tGnTNGXKlMq+HAAAYKBKnYE5dOiQRo4cqffff1/+/v5Xa6bLMn78eBUVFdmPQ4cOeXokAABwlVQqYLKyslRQUKBbbrlFvr6+8vX11YYNG/TKK6/I19dXoaGhKikpUWFhodvz8vPzFRYWJkkKCwur8K6k8q9/bY3T6Tzv2RdJcjgccjqdbg8AAFAzVSpg7rnnHu3YsUPZ2dn2o2PHjnr44Yft/65Vq5YyMjLs5+zZs0e5ubmKi4uTJMXFxWnHjh0qKCiw16xZs0ZOp1MxMTH2mnOPUb6m/BgAAOD6VqlrYAIDA9WmTRu3bQEBAapfv769ffDgwUpNTVW9evXkdDo1YsQIxcXFqXPnzpKk7t27KyYmRgMHDtT06dOVl5enCRMmKDk5WQ6HQ5I0bNgwvfbaaxo7dqwef/xxrV27VkuWLFFaWlpVvGYAAGC4Sl/E+2tmzJghb29vJSUlqbi4WAkJCZo9e7a938fHRytXrtTw4cMVFxengIAADRo0SFOnTrXXREdHKy0tTaNHj9asWbPUuHFjzZ8/XwkJCVU9LgAAMJCXZVmWp4e4Glwul4KCglRUVHTdXQ8T9WfOVF1PDryQ6OkRAKDKXOq/33wWEgAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjFOpgJkzZ47atWsnp9Mpp9OpuLg4rVq1yt5/+vRpJScnq379+qpbt66SkpKUn5/vdozc3FwlJiaqTp06atSokZ566imdPXvWbc369et1yy23yOFwqEWLFlqwYMHlv0IAAFDjVCpgGjdurBdeeEFZWVn68ssvdffdd+v+++/Xrl27JEmjR4/WihUrtHTpUm3YsEGHDx9W37597eeXlpYqMTFRJSUl2rx5sxYuXKgFCxZo4sSJ9pqcnBwlJiaqW7duys7O1qhRozRkyBClp6dX0UsGAACm87Isy7qSA9SrV08vvvii+vXrp4YNG2rRokXq16+fJGn37t1q3bq1MjMz1blzZ61atUo9e/bU4cOHFRoaKkmaO3euxo0bpyNHjsjPz0/jxo1TWlqadu7caX+P/v37q7CwUKtXr77kuVwul4KCglRUVCSn03klL9E4UX9O8/QIuIYOvJDo6REAoMpc6r/fl30NTGlpqRYvXqyTJ08qLi5OWVlZOnPmjOLj4+01rVq1UpMmTZSZmSlJyszMVNu2be14kaSEhAS5XC77LE5mZqbbMcrXlB/jQoqLi+VyudweAACgZqp0wOzYsUN169aVw+HQsGHD9PHHHysmJkZ5eXny8/NTcHCw2/rQ0FDl5eVJkvLy8tzipXx/+b6LrXG5XDp16tQF55o2bZqCgoLsR2RkZGVfGgAAMESlA6Zly5bKzs7Wli1bNHz4cA0aNEjffPPN1ZitUsaPH6+ioiL7cejQIU+PBAAArhLfyj7Bz89PLVq0kCTFxsZq27ZtmjVrlh588EGVlJSosLDQ7SxMfn6+wsLCJElhYWHaunWr2/HK36V07ppfvnMpPz9fTqdTtWvXvuBcDodDDoejsi8HAAAY6IrvA1NWVqbi4mLFxsaqVq1aysjIsPft2bNHubm5iouLkyTFxcVpx44dKigosNesWbNGTqdTMTEx9ppzj1G+pvwYAAAAlToDM378ePXo0UNNmjTR8ePHtWjRIq1fv17p6ekKCgrS4MGDlZqaqnr16snpdGrEiBGKi4tT586dJUndu3dXTEyMBg4cqOnTpysvL08TJkxQcnKyffZk2LBheu211zR27Fg9/vjjWrt2rZYsWaK0NN5ZAwAAflapgCkoKNAjjzyiH374QUFBQWrXrp3S09P1u9/9TpI0Y8YMeXt7KykpScXFxUpISNDs2bPt5/v4+GjlypUaPny44uLiFBAQoEGDBmnq1Kn2mujoaKWlpWn06NGaNWuWGjdurPnz5yshIaGKXjIAADDdFd8HprriPjC4XnAfGAA1yVW/DwwAAICnEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4lQqYadOm6be//a0CAwPVqFEj9e7dW3v27HFbc/r0aSUnJ6t+/fqqW7eukpKSlJ+f77YmNzdXiYmJqlOnjho1aqSnnnpKZ8+edVuzfv163XLLLXI4HGrRooUWLFhwea8QAADUOJUKmA0bNig5OVlffPGF1qxZozNnzqh79+46efKkvWb06NFasWKFli5dqg0bNujw4cPq27evvb+0tFSJiYkqKSnR5s2btXDhQi1YsEATJ0601+Tk5CgxMVHdunVTdna2Ro0apSFDhig9Pb0KXjIAADCdl2VZ1uU++ciRI2rUqJE2bNigO+64Q0VFRWrYsKEWLVqkfv36SZJ2796t1q1bKzMzU507d9aqVavUs2dPHT58WKGhoZKkuXPnaty4cTpy5Ij8/Pw0btw4paWlaefOnfb36t+/vwoLC7V69epLms3lcikoKEhFRUVyOp2X+xKNFPXnNE+PgGvowAuJnh4BAKrMpf77fUXXwBQVFUmS6tWrJ0nKysrSmTNnFB8fb69p1aqVmjRposzMTElSZmam2rZta8eLJCUkJMjlcmnXrl32mnOPUb6m/BjnU1xcLJfL5fYAAAA102UHTFlZmUaNGqXbb79dbdq0kSTl5eXJz89PwcHBbmtDQ0OVl5dnrzk3Xsr3l++72BqXy6VTp06dd55p06YpKCjIfkRGRl7uSwMAANXcZQdMcnKydu7cqcWLF1flPJdt/PjxKioqsh+HDh3y9EgAAOAq8b2cJ6WkpGjlypXauHGjGjdubG8PCwtTSUmJCgsL3c7C5OfnKywszF6zdetWt+OVv0vp3DW/fOdSfn6+nE6nateufd6ZHA6HHA7H5bwcAABgmEqdgbEsSykpKfr444+1du1aRUdHu+2PjY1VrVq1lJGRYW/bs2ePcnNzFRcXJ0mKi4vTjh07VFBQYK9Zs2aNnE6nYmJi7DXnHqN8TfkxAADA9a1SZ2CSk5O1aNEiffLJJwoMDLSvWQkKClLt2rUVFBSkwYMHKzU1VfXq1ZPT6dSIESMUFxenzp07S5K6d++umJgYDRw4UNOnT1deXp4mTJig5ORk+wzKsGHD9Nprr2ns2LF6/PHHtXbtWi1ZskRpaby7BgAAVPIMzJw5c1RUVKS77rpL4eHh9uODDz6w18yYMUM9e/ZUUlKS7rjjDoWFhWnZsmX2fh8fH61cuVI+Pj6Ki4vTH/7wBz3yyCOaOnWqvSY6OlppaWlas2aN2rdvr5deeknz589XQkJCFbxkAABguiu6D0x1xn1gcL3gPjAAapJrch8YAAAATyBgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxql0wGzcuFG9evVSRESEvLy8tHz5crf9lmVp4sSJCg8PV+3atRUfH6+9e/e6rTl27JgefvhhOZ1OBQcHa/DgwTpx4oTbmv/7v/9T165d5e/vr8jISE2fPr3yrw4AANRIlQ6YkydPqn379nr99dfPu3/69Ol65ZVXNHfuXG3ZskUBAQFKSEjQ6dOn7TUPP/ywdu3apTVr1mjlypXauHGjhg4dau93uVzq3r27mjZtqqysLL344ouaPHmy5s2bdxkvEQAA1DRelmVZl/1kLy99/PHH6t27t6Sfz75EREToySef1JgxYyRJRUVFCg0N1YIFC9S/f399++23iomJ0bZt29SxY0dJ0urVq3Xffffp+++/V0REhObMmaOnn35aeXl58vPzkyT9+c9/1vLly7V79+5Lms3lcikoKEhFRUVyOp2X+xKNFPXnNE+PgGvowAuJnh4BAKrMpf77XaXXwOTk5CgvL0/x8fH2tqCgIHXq1EmZmZmSpMzMTAUHB9vxIknx8fHy9vbWli1b7DV33HGHHS+SlJCQoD179uinn36qypEBAICBfKvyYHl5eZKk0NBQt+2hoaH2vry8PDVq1Mh9CF9f1atXz21NdHR0hWOU7wsJCanwvYuLi1VcXGx/7XK5rvDVAACA6qrGvAtp2rRpCgoKsh+RkZGeHgkAAFwlVRowYWFhkqT8/Hy37fn5+fa+sLAwFRQUuO0/e/asjh075rbmfMc493v80vjx41VUVGQ/Dh06dOUvCAAAVEtVGjDR0dEKCwtTRkaGvc3lcmnLli2Ki4uTJMXFxamwsFBZWVn2mrVr16qsrEydOnWy12zcuFFnzpyx16xZs0YtW7Y876+PJMnhcMjpdLo9AABAzVTpgDlx4oSys7OVnZ0t6ecLd7Ozs5WbmysvLy+NGjVKzz77rP73f/9XO3bs0COPPKKIiAj7nUqtW7fWvffeqyeeeEJbt27Vpk2blJKSov79+ysiIkKS9NBDD8nPz0+DBw/Wrl279MEHH2jWrFlKTU2tshcOAADMVemLeL/88kt169bN/ro8KgYNGqQFCxZo7NixOnnypIYOHarCwkJ16dJFq1evlr+/v/2c999/XykpKbrnnnvk7e2tpKQkvfLKK/b+oKAgffbZZ0pOTlZsbKwaNGigiRMnut0rBgAAXL+u6D4w1Rn3gcH1gvvAAKhJPHIfGAAAgGuBgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGCcSn8aNQDAc/iw1usLH9Z6YZyBAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxqnWAfP6668rKipK/v7+6tSpk7Zu3erpkQAAQDVQbQPmgw8+UGpqqiZNmqSvvvpK7du3V0JCggoKCjw9GgAA8LBqGzAvv/yynnjiCT322GOKiYnR3LlzVadOHb311lueHg0AAHhYtQyYkpISZWVlKT4+3t7m7e2t+Ph4ZWZmenAyAABQHfh6eoDz+fHHH1VaWqrQ0FC37aGhodq9e/d5n1NcXKzi4mL766KiIkmSy+W6eoNWU2XF//b0CLiGrsf/jV/P+PN9fbke/3yXv2bLsi66rloGzOWYNm2apkyZUmF7ZGSkB6YBrp2gmZ6eAMDVcj3/+T5+/LiCgoIuuL9aBkyDBg3k4+Oj/Px8t+35+fkKCws773PGjx+v1NRU++uysjIdO3ZM9evXl5eX11WdF57ncrkUGRmpQ4cOyel0enocAFWIP9/XF8uydPz4cUVERFx0XbUMGD8/P8XGxiojI0O9e/eW9HOQZGRkKCUl5bzPcTgccjgcbtuCg4Ov8qSobpxOJ3/BATUUf76vHxc781KuWgaMJKWmpmrQoEHq2LGjbr31Vs2cOVMnT57UY4895unRAACAh1XbgHnwwQd15MgRTZw4UXl5eerQoYNWr15d4cJeAABw/am2ASNJKSkpF/yVEXAuh8OhSZMmVfg1IgDz8ecb5+Nl/dr7lAAAAKqZankjOwAAgIshYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcar1fWAAANeno0ePauLEiVq3bp0KCgpUVlbmtv/YsWMemgzVBQEDY1mWpQ8//PCCf8EtW7bMQ5MBuFIDBw7Uvn37NHjwYIWGhvKhvKiAgIGxRo0apTfeeEPdunXjLzighvn73/+uf/zjH2rfvr2nR0E1RcDAWO+++66WLVum++67z9OjAKhirVq10qlTpzw9BqoxLuKFsYKCgtSsWTNPjwHgKpg9e7aefvppbdiwQUePHpXL5XJ7AAQMjDV58mRNmTKF/5cG1EDBwcFyuVy6++671ahRI4WEhCgkJETBwcEKCQnx9HioBvgwRxjr1KlT6tOnjzZt2qSoqCjVqlXLbf9XX33lockAXKlbb71Vvr6+Gjly5Hmvcbvzzjs9NBmqC66BgbEGDRqkrKws/eEPf+AiXqCG2blzp77++mu1bNnS06OgmiJgYKy0tDSlp6erS5cunh4FQBXr2LGjDh06RMDggggYGCsyMlJOp9PTYwC4CkaMGKGRI0fqqaeeUtu2bSv8irhdu3YemgzVBdfAwFhpaWl69dVXNXfuXEVFRXl6HABVyNu74ntMvLy8ZFmWvLy8VFpa6oGpUJ0QMDBWSEiI/v3vf+vs2bOqU6dOhf+Hxq3GAXMdPHjwovubNm16jSZBdcWvkGCsmTNnenoEAFcJgYJfwxkYAEC19c033yg3N1clJSVu2//jP/7DQxOhuuAMDGqE06dPV/gLjgt8AXP985//VJ8+fbRjxw772hdJ9u0SuAYG3IkXxjp58qRSUlLUqFEjBQQE2HfqLH8AMNfIkSMVHR2tgoIC1alTR7t27dLGjRvVsWNHrV+/3tPjoRogYGCssWPHau3atZozZ44cDofmz5+vKVOmKCIiQu+8846nxwNwBTIzMzV16lQ1aNBA3t7e8vb2VpcuXTRt2jT96U9/8vR4qAYIGBhrxYoVmj17tpKSkuTr66uuXbtqwoQJev755/X+++97ejwAV6C0tFSBgYGSpAYNGujw4cOSfr64d8+ePZ4cDdUE18DAWMeOHbM/jdrpdNpvm+7SpYuGDx/uydEAXKE2bdpo+/btio6OVqdOnTR9+nT5+flp3rx5fAo9JHEGBgZr1qyZcnJyJEmtWrXSkiVLJP18ZiY4ONiDkwG4UhMmTFBZWZkkaerUqcrJyVHXrl316aef6pVXXvHwdKgOeBs1jDVjxgz5+PjoT3/6kz7//HP16tVLlmXpzJkzevnllzVy5EhPjwigCh07dkwhISF8cCskETCoQQ4ePKisrCy1aNGCz0kBapBDhw5J+vnzz4ByXAMDo2VkZCgjI0MFBQX26eZyb731loemAnClzp49qylTpuiVV17RiRMnJEl169bViBEjNGnSpAofHYLrDwEDY02ZMkVTp05Vx44dFR4ezmlloAYZMWKEli1bpunTpysuLk7Sz2+tnjx5so4ePao5c+Z4eEJ4Gr9CgrHCw8M1ffp0DRw40NOjAKhiQUFBWrx4sXr06OG2/dNPP9WAAQNUVFTkoclQXfAuJBirpKREt912m6fHAHAVOBwORUVFVdgeHR0tPz+/az8Qqh0CBsYaMmSIFi1a5OkxAFwFKSkpeuaZZ1RcXGxvKy4u1nPPPaeUlBQPTobqgl8hwSipqan2f5eVlWnhwoVq166d2rVrV+GivpdffvlajwegivTp00cZGRlyOBxq3769JGn79u0qKSnRPffc47Z22bJlnhgRHsZFvDDK119/7fZ1hw4dJEk7d+50284FvYDZgoODlZSU5LaNt1HjXJyBAQBUO6dOnVJZWZkCAgIkSQcOHNDy5cvVunVrJSQkeHg6VAdcAwMAqHbuv/9+vfvuu5KkwsJCde7cWS+99JJ69+7NW6ghiYABAFRDX331lbp27SpJ+vDDDxUaGqqDBw/qnXfe4bOQIImAAQBUQ//+978VGBgoSfrss8/Ut29feXt7q3Pnzjp48KCHp0N1QMAAAKqdFi1aaPny5Tp06JDS09PVvXt3SVJBQYGcTqeHp0N1QMAAAKqdiRMnasyYMYqKilKnTp3sjxP47LPPdPPNN3t4OlQHvAsJAFAt5eXl6YcfflD79u3l7f3z/9/eunWrnE6nWrVq5eHp4GkEDAAAMA6/QgIAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAXBN3HXXXRo1apSnx7BVt3kAVA4BA8AYJSUlnh4BQDVBwAC46h599FFt2LBBs2bNkpeXl7y8vLR//34NHjxY0dHRql27tlq2bKlZs2ZVeF7v3r313HPPKSIiQi1btpQkbd68WR06dJC/v786duyo5cuXy8vLS9nZ2fZzd+7cqR49eqhu3boKDQ3VwIED9eOPP15wngMHDlyrHweAKuDr6QEA1HyzZs3Sd999pzZt2mjq1KmSpJCQEDVu3FhLly5V/fr1tXnzZg0dOlTh4eF64IEH7OdmZGTI6XRqzZo1kiSXy6VevXrpvvvu06JFi3Tw4MEKvwoqLCzU3XffrSFDhmjGjBk6deqUxo0bpwceeEBr16497zwNGza8Nj8MAFWCgAFw1QUFBcnPz0916tRRWFiYvX3KlCn2f0dHRyszM1NLlixxC5iAgADNnz9ffn5+kqS5c+fKy8tLf/vb3+Tv76+YmBj961//0hNPPGE/57XXXtPNN9+s559/3t721ltvKTIyUt99951uvPHG884DwBwEDACPef311/XWW28pNzdXp06dUklJiTp06OC2pm3btna8SNKePXvUrl07+fv729tuvfVWt+ds375d69atU926dSt8z/379+vGG2+s2hcC4JojYAB4xOLFizVmzBi99NJLiouLU2BgoF588UVt2bLFbV1AQEClj33ixAn16tVL//3f/11hX3h4+GXPDKD6IGAAXBN+fn4qLS21v960aZNuu+02/fGPf7S37d+//1eP07JlS7333nsqLi6Ww+GQJG3bts1tzS233KKPPvpIUVFR8vU9/19zv5wHgFl4FxKAayIqKkpbtmzRgQMH9OOPP+qGG27Ql19+qfT0dH333Xf6y1/+UiFEzuehhx5SWVmZhg4dqm+//Vbp6en661//Kkny8vKSJCUnJ+vYsWMaMGCAtm3bpv379ys9PV2PPfaYHS2/nKesrOzqvXgAVY6AAXBNjBkzRj4+PoqJiVHDhg2VkJCgvn376sEHH1SnTp109OhRt7MxF+J0OrVixQplZ2erQ4cOevrppzVx4kRJsq+LiYiI0KZNm1RaWqru3burbdu2GjVqlIKDg+Xt7X3eeXJzc6/eiwdQ5bwsy7I8PQQAXIn3339fjz32mIqKilS7dm1PjwPgGuAaGADGeeedd9SsWTP95je/0fbt2+17vBAvwPWDgAFgnLy8PE2cOFF5eXkKDw/X73//ez333HOeHgvANcSvkAAAgHG4iBcAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAY5/8BeRZUfkCceasAAAAASUVORK5CYII="},"metadata":{}}]},{"cell_type":"code","source":"ham_df = df[df['target'] == 'ham']['message_len'].value_counts().sort_index()\nspam_df = df[df['target'] == 'spam']['message_len'].value_counts().sort_index()","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:28:54.194065Z","iopub.execute_input":"2023-12-10T08:28:54.194330Z","iopub.status.idle":"2023-12-10T08:28:54.202279Z","shell.execute_reply.started":"2023-12-10T08:28:54.194303Z","shell.execute_reply":"2023-12-10T08:28:54.201479Z"},"trusted":true},"execution_count":9,"outputs":[]},{"cell_type":"code","source":"nltk.download('stopwords')","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:29:31.397139Z","iopub.execute_input":"2023-12-10T08:29:31.397580Z","iopub.status.idle":"2023-12-10T08:29:31.611873Z","shell.execute_reply.started":"2023-12-10T08:29:31.397544Z","shell.execute_reply":"2023-12-10T08:29:31.611127Z"},"trusted":true},"execution_count":14,"outputs":[{"name":"stderr","text":"[nltk_data] Downloading package stopwords to /root/nltk_data...\n[nltk_data] Unzipping corpora/stopwords.zip.\n","output_type":"stream"},{"execution_count":14,"output_type":"execute_result","data":{"text/plain":"True"},"metadata":{}}]},{"cell_type":"code","source":"def clean_text(text):\n text = str(text).lower()\n text = re.sub('\\[.*?\\]', '', text)\n text = re.sub('https?://\\S+|www\\.\\S+', '', text)\n text = re.sub('<.*?>+', '', text)\n text = re.sub('[%s]' % re.escape(string.punctuation), '', text)\n text = re.sub('\\n', '', text)\n text = re.sub('\\w*\\d\\w*', '', text)\n return text\nstop_words = stopwords.words('english')\nmore_stopwords = ['u', 'im', 'c']\nstop_words = stop_words + more_stopwords\n\ndef remove_stopwords(text):\n text = ' '.join(word for word in text.split(' ') if word not in stop_words)\n return text\n\nstemmer = nltk.SnowballStemmer(\"english\")\ndef stemm_text(text):\n text = ' '.join(stemmer.stem(word) for word in text.split(' '))\n return text\n","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:29:39.356201Z","iopub.execute_input":"2023-12-10T08:29:39.356577Z","iopub.status.idle":"2023-12-10T08:29:39.363877Z","shell.execute_reply.started":"2023-12-10T08:29:39.356544Z","shell.execute_reply":"2023-12-10T08:29:39.363236Z"},"trusted":true},"execution_count":16,"outputs":[]},{"cell_type":"code","source":"df['cleantxt'] = df['message'].apply(clean_text)\ndf['cleantxt'] = df['cleantxt'].apply(clean_text).apply(remove_stopwords)\ndf['cleantxt'] = df['cleantxt'].apply(stemm_text)\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:29:44.087169Z","iopub.execute_input":"2023-12-10T08:29:44.087887Z","iopub.status.idle":"2023-12-10T08:29:45.053945Z","shell.execute_reply.started":"2023-12-10T08:29:44.087850Z","shell.execute_reply":"2023-12-10T08:29:45.053255Z"},"trusted":true},"execution_count":17,"outputs":[{"execution_count":17,"output_type":"execute_result","data":{"text/plain":" target message \\\n0 ham Go until jurong point, crazy.. Available only ... \n1 ham Ok lar... Joking wif u oni... \n2 spam Free entry in 2 a wkly comp to win FA Cup fina... \n3 ham U dun say so early hor... U c already then say... \n4 ham Nah I don't think he goes to usf, he lives aro... \n\n cleantxt \n0 go jurong point crazi avail bugi n great world... \n1 ok lar joke wif oni \n2 free entri wkli comp win fa cup final tkts m... \n3 dun say earli hor alreadi say \n4 nah dont think goe usf live around though ","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
targetmessagecleantxt
0hamGo until jurong point, crazy.. Available only ...go jurong point crazi avail bugi n great world...
1hamOk lar... Joking wif u oni...ok lar joke wif oni
2spamFree entry in 2 a wkly comp to win FA Cup fina...free entri wkli comp win fa cup final tkts m...
3hamU dun say so early hor... U c already then say...dun say earli hor alreadi say
4hamNah I don't think he goes to usf, he lives aro...nah dont think goe usf live around though
\n
"},"metadata":{}}]},{"cell_type":"code","source":"labelencoder= LabelEncoder()\nlabelencoder.fit(df['target'])\ndf['target_num'] = labelencoder.transform(df['target'])\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:29:50.228785Z","iopub.execute_input":"2023-12-10T08:29:50.229489Z","iopub.status.idle":"2023-12-10T08:29:50.240948Z","shell.execute_reply.started":"2023-12-10T08:29:50.229453Z","shell.execute_reply":"2023-12-10T08:29:50.240221Z"},"trusted":true},"execution_count":18,"outputs":[{"execution_count":18,"output_type":"execute_result","data":{"text/plain":" target message \\\n0 ham Go until jurong point, crazy.. Available only ... \n1 ham Ok lar... Joking wif u oni... \n2 spam Free entry in 2 a wkly comp to win FA Cup fina... \n3 ham U dun say so early hor... U c already then say... \n4 ham Nah I don't think he goes to usf, he lives aro... \n\n cleantxt target_num \n0 go jurong point crazi avail bugi n great world... 0 \n1 ok lar joke wif oni 0 \n2 free entri wkli comp win fa cup final tkts m... 1 \n3 dun say earli hor alreadi say 0 \n4 nah dont think goe usf live around though 0 ","text/html":"
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
targetmessagecleantxttarget_num
0hamGo until jurong point, crazy.. Available only ...go jurong point crazi avail bugi n great world...0
1hamOk lar... Joking wif u oni...ok lar joke wif oni0
2spamFree entry in 2 a wkly comp to win FA Cup fina...free entri wkli comp win fa cup final tkts m...1
3hamU dun say so early hor... U c already then say...dun say earli hor alreadi say0
4hamNah I don't think he goes to usf, he lives aro...nah dont think goe usf live around though0
\n
"},"metadata":{}}]},{"cell_type":"code","source":"X = df['cleantxt']\ny= df['target_num']\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=40,test_size=0.2)","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:29:55.687455Z","iopub.execute_input":"2023-12-10T08:29:55.688203Z","iopub.status.idle":"2023-12-10T08:29:55.695842Z","shell.execute_reply.started":"2023-12-10T08:29:55.688165Z","shell.execute_reply":"2023-12-10T08:29:55.695178Z"},"trusted":true},"execution_count":19,"outputs":[]},{"cell_type":"markdown","source":"## LSTM Approach","metadata":{}},{"cell_type":"code","source":"from keras.preprocessing import sequence","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:29:58.226008Z","iopub.execute_input":"2023-12-10T08:29:58.226641Z","iopub.status.idle":"2023-12-10T08:29:58.230175Z","shell.execute_reply.started":"2023-12-10T08:29:58.226610Z","shell.execute_reply":"2023-12-10T08:29:58.229467Z"},"trusted":true},"execution_count":20,"outputs":[]},{"cell_type":"code","source":"max_words = 400\nmax_len = 150\ntok = Tokenizer(num_words=400)\ntok.fit_on_texts(X_train)\nsequences = tok.texts_to_sequences(X_train)\nsequences_train = sequence.pad_sequences(sequences,maxlen=150)","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:30:00.053097Z","iopub.execute_input":"2023-12-10T08:30:00.053421Z","iopub.status.idle":"2023-12-10T08:30:00.174123Z","shell.execute_reply.started":"2023-12-10T08:30:00.053393Z","shell.execute_reply":"2023-12-10T08:30:00.173408Z"},"trusted":true},"execution_count":21,"outputs":[]},{"cell_type":"code","source":"from keras.models import Model\nfrom keras.layers import LSTM, Activation, Dense, Dropout, Input, Embedding\nfrom keras.optimizers import RMSprop","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:30:41.913952Z","iopub.execute_input":"2023-12-10T08:30:41.914729Z","iopub.status.idle":"2023-12-10T08:30:41.918089Z","shell.execute_reply.started":"2023-12-10T08:30:41.914695Z","shell.execute_reply":"2023-12-10T08:30:41.917475Z"},"trusted":true},"execution_count":23,"outputs":[]},{"cell_type":"code","source":"def RNN():\n inputs = Input(name='inputs',shape=[max_len])\n layer = Embedding(max_words,50,input_length=max_len)(inputs)\n layer = LSTM(64)(layer)\n layer = Dense(256,name='FC1')(layer)\n layer = Activation('relu')(layer)\n layer = Dropout(0.5)(layer)\n layer = Dense(1,name='out_layer')(layer)\n layer = Activation('sigmoid')(layer)\n model = Model(inputs=inputs,outputs=layer)\n return model","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:30:46.704976Z","iopub.execute_input":"2023-12-10T08:30:46.705775Z","iopub.status.idle":"2023-12-10T08:30:46.710300Z","shell.execute_reply.started":"2023-12-10T08:30:46.705742Z","shell.execute_reply":"2023-12-10T08:30:46.709655Z"},"trusted":true},"execution_count":24,"outputs":[]},{"cell_type":"code","source":"model = RNN()\nmodel.summary()\nmodel.compile(loss='binary_crossentropy',optimizer=RMSprop(),metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:30:49.234326Z","iopub.execute_input":"2023-12-10T08:30:49.235087Z","iopub.status.idle":"2023-12-10T08:30:53.533957Z","shell.execute_reply.started":"2023-12-10T08:30:49.235053Z","shell.execute_reply":"2023-12-10T08:30:53.533096Z"},"trusted":true},"execution_count":25,"outputs":[{"name":"stderr","text":"2023-12-10 08:30:53.199015: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.199129: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.199230: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.199302: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.199369: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.199565: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.199665: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.199778: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.199864: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.199939: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.200135: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.200247: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.200336: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.200400: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.200506: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.200701: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.200805: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.200891: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.200979: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.201043: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.201318: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.201410: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.201493: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.201571: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.201657: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.201950: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.202038: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.202120: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.202221: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.202300: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.202605: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.202733: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.202826: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.202898: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.202992: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.203234: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.203328: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.203407: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.203502: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n2023-12-10 08:30:53.203585: E ./tensorflow/compiler/xla/stream_executor/stream_executor_internal.h:124] SetPriority unimplemented for this stream.\n","output_type":"stream"},{"name":"stdout","text":"Model: \"model\"\n_________________________________________________________________\n Layer (type) Output Shape Param # \n=================================================================\n inputs (InputLayer) [(None, 150)] 0 \n \n embedding (Embedding) (None, 150, 50) 20000 \n \n lstm (LSTM) (None, 64) 29440 \n \n FC1 (Dense) (None, 256) 16640 \n \n activation (Activation) (None, 256) 0 \n \n dropout (Dropout) (None, 256) 0 \n \n out_layer (Dense) (None, 1) 257 \n \n activation_1 (Activation) (None, 1) 0 \n \n=================================================================\nTotal params: 66337 (259.13 KB)\nTrainable params: 66337 (259.13 KB)\nNon-trainable params: 0 (0.00 Byte)\n_________________________________________________________________\n","output_type":"stream"}]},{"cell_type":"code","source":"from keras.callbacks import EarlyStopping\nmodel.fit(sequences_train,y_train,batch_size=128,epochs=10,\n validation_split=0.2,callbacks=[EarlyStopping(monitor='val_loss',min_delta=0.005)])","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:30:58.718072Z","iopub.execute_input":"2023-12-10T08:30:58.719117Z","iopub.status.idle":"2023-12-10T08:31:25.219659Z","shell.execute_reply.started":"2023-12-10T08:30:58.719078Z","shell.execute_reply":"2023-12-10T08:31:25.218528Z"},"trusted":true},"execution_count":26,"outputs":[{"name":"stdout","text":"Epoch 1/10\n28/28 [==============================] - 6s 160ms/step - loss: 0.4174 - accuracy: 0.8550 - val_loss: 0.2778 - val_accuracy: 0.8823\nEpoch 2/10\n28/28 [==============================] - 4s 143ms/step - loss: 0.2314 - accuracy: 0.9156 - val_loss: 0.1489 - val_accuracy: 0.9641\nEpoch 3/10\n28/28 [==============================] - 4s 145ms/step - loss: 0.1204 - accuracy: 0.9655 - val_loss: 0.0951 - val_accuracy: 0.9686\nEpoch 4/10\n28/28 [==============================] - 4s 142ms/step - loss: 0.0796 - accuracy: 0.9784 - val_loss: 0.0800 - val_accuracy: 0.9776\nEpoch 5/10\n28/28 [==============================] - 4s 143ms/step - loss: 0.0630 - accuracy: 0.9801 - val_loss: 0.0742 - val_accuracy: 0.9821\nEpoch 6/10\n28/28 [==============================] - 4s 142ms/step - loss: 0.0521 - accuracy: 0.9837 - val_loss: 0.0755 - val_accuracy: 0.9821\n","output_type":"stream"},{"execution_count":26,"output_type":"execute_result","data":{"text/plain":""},"metadata":{}}]},{"cell_type":"code","source":"test_sequences = tok.texts_to_sequences(X_test)\ntest_sequences_matrix = sequence.pad_sequences(test_sequences,maxlen=max_len)","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:31:34.125142Z","iopub.execute_input":"2023-12-10T08:31:34.125544Z","iopub.status.idle":"2023-12-10T08:31:34.153639Z","shell.execute_reply.started":"2023-12-10T08:31:34.125512Z","shell.execute_reply":"2023-12-10T08:31:34.152503Z"},"trusted":true},"execution_count":27,"outputs":[]},{"cell_type":"code","source":"accr = model.evaluate(test_sequences_matrix,y_test)","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:31:36.946648Z","iopub.execute_input":"2023-12-10T08:31:36.947101Z","iopub.status.idle":"2023-12-10T08:31:37.672753Z","shell.execute_reply.started":"2023-12-10T08:31:36.947066Z","shell.execute_reply":"2023-12-10T08:31:37.671734Z"},"trusted":true},"execution_count":28,"outputs":[{"name":"stdout","text":"35/35 [==============================] - 1s 18ms/step - loss: 0.0775 - accuracy: 0.9785\n","output_type":"stream"}]},{"cell_type":"code","source":"print('Test set\\n Loss: {:0.2f}\\n Accuracy: {:0.2f}'.format(accr[0],accr[1]))","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:31:40.644967Z","iopub.execute_input":"2023-12-10T08:31:40.645366Z","iopub.status.idle":"2023-12-10T08:31:40.650579Z","shell.execute_reply.started":"2023-12-10T08:31:40.645335Z","shell.execute_reply":"2023-12-10T08:31:40.649607Z"},"trusted":true},"execution_count":29,"outputs":[{"name":"stdout","text":"Test set\n Loss: 0.08\n Accuracy: 0.98\n","output_type":"stream"}]},{"cell_type":"markdown","source":"## USING BERT","metadata":{}},{"cell_type":"code","source":"\n!pip install transformers","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:31:46.320920Z","iopub.execute_input":"2023-12-10T08:31:46.321405Z","iopub.status.idle":"2023-12-10T08:31:50.071438Z","shell.execute_reply.started":"2023-12-10T08:31:46.321358Z","shell.execute_reply":"2023-12-10T08:31:50.070499Z"},"trusted":true},"execution_count":30,"outputs":[{"name":"stdout","text":"Requirement already satisfied: transformers in /usr/local/lib/python3.10/site-packages (4.35.2)\nRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/site-packages (from transformers) (23.2)\nRequirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/site-packages (from transformers) (6.0.1)\nRequirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/site-packages (from transformers) (2023.10.3)\nRequirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/site-packages (from transformers) (0.19.4)\nRequirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/site-packages (from transformers) (0.15.0)\nRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/site-packages (from transformers) (1.26.2)\nRequirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/site-packages (from transformers) (0.4.1)\nRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/site-packages (from transformers) (4.66.1)\nRequirement already satisfied: filelock in /usr/local/lib/python3.10/site-packages (from transformers) (3.13.1)\nRequirement already satisfied: requests in /usr/local/lib/python3.10/site-packages (from transformers) (2.31.0)\nRequirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.16.4->transformers) (4.8.0)\nRequirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.16.4->transformers) (2023.12.1)\nRequirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/site-packages (from requests->transformers) (2023.11.17)\nRequirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/site-packages (from requests->transformers) (2.1.0)\nRequirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/site-packages (from requests->transformers) (3.3.2)\nRequirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/site-packages (from requests->transformers) (3.6)\n\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n\u001b[0m\n\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n","output_type":"stream"}]},{"cell_type":"code","source":"from transformers import BertTokenizer\ntokenizer = BertTokenizer.from_pretrained('bert-large-uncased')\n\ndef bert_encode(data, maximum_length) :\n input_ids = []\n attention_masks = []\n\n for text in data:\n encoded = tokenizer.encode_plus(\n text, \n add_special_tokens=True,\n max_length=maximum_length,\n pad_to_max_length=True,\n\n return_attention_mask=True,\n )\n input_ids.append(encoded['input_ids'])\n attention_masks.append(encoded['attention_mask'])\n \n return np.array(input_ids),np.array(attention_masks)","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:31:50.073203Z","iopub.execute_input":"2023-12-10T08:31:50.073481Z","iopub.status.idle":"2023-12-10T08:31:51.485706Z","shell.execute_reply.started":"2023-12-10T08:31:50.073453Z","shell.execute_reply":"2023-12-10T08:31:51.484732Z"},"trusted":true},"execution_count":31,"outputs":[{"name":"stderr","text":"/usr/local/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n from .autonotebook import tqdm as notebook_tqdm\ntokenizer_config.json: 100%|██████████| 28.0/28.0 [00:00<00:00, 116kB/s]\nvocab.txt: 100%|██████████| 232k/232k [00:00<00:00, 5.99MB/s]\ntokenizer.json: 100%|██████████| 466k/466k [00:00<00:00, 12.4MB/s]\nconfig.json: 100%|██████████| 571/571 [00:00<00:00, 3.04MB/s]\n","output_type":"stream"}]},{"cell_type":"code","source":"# Detect hardware, return appropriate distribution strategy\nimport tensorflow as tf\ntry:\n # TPU detection. No parameters necessary if TPU_NAME environment variable is\n # set: this is always the case on Kaggle.\n tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n print('Running on TPU ', tpu.master())\nexcept ValueError:\n tpu = None\n\nif tpu:\n tf.config.experimental_connect_to_cluster(tpu)\n tf.tpu.experimental.initialize_tpu_system(tpu)\n strategy = tf.distribute.experimental.TPUStrategy(tpu)\nelse:\n # Default distribution strategy in Tensorflow. Works on CPU and single GPU.\n strategy = tf.distribute.get_strategy()\n\nprint(\"REPLICAS: \", strategy.num_replicas_in_sync)","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:32:11.255996Z","iopub.execute_input":"2023-12-10T08:32:11.256457Z","iopub.status.idle":"2023-12-10T08:32:16.032297Z","shell.execute_reply.started":"2023-12-10T08:32:11.256421Z","shell.execute_reply":"2023-12-10T08:32:16.029388Z"},"trusted":true},"execution_count":32,"outputs":[{"name":"stdout","text":"Running on TPU \nINFO:tensorflow:Deallocate tpu buffers before initializing tpu system.\nINFO:tensorflow:Initializing the TPU system: local\nINFO:tensorflow:Finished initializing TPU system.\n","output_type":"stream"},{"name":"stderr","text":"WARNING:absl:`tf.distribute.experimental.TPUStrategy` is deprecated, please use the non-experimental symbol `tf.distribute.TPUStrategy` instead.\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:Found TPU system:\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:Found TPU system:\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Num TPU Cores: 8\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Num TPU Cores: 8\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Num TPU Workers: 1\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Num TPU Workers: 1\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Num TPU Cores Per Worker: 8\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Num TPU Cores Per Worker: 8\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 0, 0)\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 0, 0)\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:0, TPU, 0, 0)\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:0, TPU, 0, 0)\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:1, TPU, 0, 0)\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:1, TPU, 0, 0)\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:2, TPU, 0, 0)\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:2, TPU, 0, 0)\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:3, TPU, 0, 0)\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:3, TPU, 0, 0)\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:4, TPU, 0, 0)\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:4, TPU, 0, 0)\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:5, TPU, 0, 0)\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:5, TPU, 0, 0)\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:6, TPU, 0, 0)\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:6, TPU, 0, 0)\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:7, TPU, 0, 0)\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU:7, TPU, 0, 0)\n","output_type":"stream"},{"name":"stdout","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 0, 0)\n","output_type":"stream"},{"name":"stderr","text":"INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 0, 0)\n","output_type":"stream"},{"name":"stdout","text":"REPLICAS: 8\n","output_type":"stream"}]},{"cell_type":"code","source":"import pandas as pd\nsms = df['cleantxt']\ntarget = df['target_num']\n\ntrain_input_ids, train_attention_masks = bert_encode(sms,60)","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:32:19.018250Z","iopub.execute_input":"2023-12-10T08:32:19.018647Z","iopub.status.idle":"2023-12-10T08:32:20.673552Z","shell.execute_reply.started":"2023-12-10T08:32:19.018615Z","shell.execute_reply":"2023-12-10T08:32:20.672256Z"},"trusted":true},"execution_count":33,"outputs":[{"name":"stderr","text":"Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n/usr/local/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:2614: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n warnings.warn(\n","output_type":"stream"}]},{"cell_type":"code","source":"def bertie(bert_model):\n \n input_ids = tf.keras.Input(shape=(60,),dtype='int32')\n attention_masks = tf.keras.Input(shape=(60,),dtype='int32')\n\n output = bert_model([input_ids,attention_masks])\n output = output[1]\n output = tf.keras.layers.Dense(32,activation='relu')(output)\n output = tf.keras.layers.Dropout(0.2)(output)\n output = tf.keras.layers.Dense(1,activation='sigmoid')(output)\n \n model = tf.keras.models.Model(inputs = [input_ids,attention_masks],outputs = output)\n model.compile(Adam(lr=1e-5), loss='binary_crossentropy', metrics=['accuracy'])\n return model","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:32:28.990126Z","iopub.execute_input":"2023-12-10T08:32:28.990520Z","iopub.status.idle":"2023-12-10T08:32:28.997328Z","shell.execute_reply.started":"2023-12-10T08:32:28.990489Z","shell.execute_reply":"2023-12-10T08:32:28.996444Z"},"trusted":true},"execution_count":35,"outputs":[]},{"cell_type":"code","source":"from transformers import TFBertModel\nbert_model = TFBertModel.from_pretrained('bert-base-uncased')","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:32:34.350133Z","iopub.execute_input":"2023-12-10T08:32:34.350528Z","iopub.status.idle":"2023-12-10T08:32:38.655280Z","shell.execute_reply.started":"2023-12-10T08:32:34.350498Z","shell.execute_reply":"2023-12-10T08:32:38.654225Z"},"trusted":true},"execution_count":36,"outputs":[{"name":"stderr","text":"config.json: 100%|██████████| 570/570 [00:00<00:00, 2.15MB/s]\nmodel.safetensors: 100%|██████████| 440M/440M [00:02<00:00, 213MB/s] \nSome weights of the PyTorch model were not used when initializing the TF 2.0 model TFBertModel: ['cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.bias']\n- This IS expected if you are initializing TFBertModel from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).\n- This IS NOT expected if you are initializing TFBertModel from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).\nAll the weights of TFBertModel were initialized from the PyTorch model.\nIf your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertModel for predictions without further training.\n","output_type":"stream"}]},{"cell_type":"code","source":"import tensorflow as tf","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:32:40.600906Z","iopub.execute_input":"2023-12-10T08:32:40.601300Z","iopub.status.idle":"2023-12-10T08:32:41.718038Z","shell.execute_reply.started":"2023-12-10T08:32:40.601269Z","shell.execute_reply":"2023-12-10T08:32:41.716736Z"},"trusted":true},"execution_count":38,"outputs":[]},{"cell_type":"code","source":"model=bertie(bert_model)\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:32:42.939181Z","iopub.execute_input":"2023-12-10T08:32:42.939576Z","iopub.status.idle":"2023-12-10T08:32:47.855018Z","shell.execute_reply.started":"2023-12-10T08:32:42.939545Z","shell.execute_reply":"2023-12-10T08:32:47.853857Z"},"trusted":true},"execution_count":39,"outputs":[{"name":"stderr","text":"WARNING:absl:`lr` is deprecated in Keras optimizer, please use `learning_rate` or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.\n","output_type":"stream"},{"name":"stdout","text":"Model: \"model_1\"\n__________________________________________________________________________________________________\n Layer (type) Output Shape Param # Connected to \n==================================================================================================\n input_1 (InputLayer) [(None, 60)] 0 [] \n \n input_2 (InputLayer) [(None, 60)] 0 [] \n \n tf_bert_model (TFBertModel TFBaseModelOutputWithPooli 1094822 ['input_1[0][0]', \n ) ngAndCrossAttentions(last_ 40 'input_2[0][0]'] \n hidden_state=(None, 60, 76 \n 8), \n pooler_output=(None, 768) \n , past_key_values=None, hi \n dden_states=None, attentio \n ns=None, cross_attentions= \n None) \n \n dense (Dense) (None, 32) 24608 ['tf_bert_model[0][1]'] \n \n dropout_38 (Dropout) (None, 32) 0 ['dense[0][0]'] \n \n dense_1 (Dense) (None, 1) 33 ['dropout_38[0][0]'] \n \n==================================================================================================\nTotal params: 109506881 (417.74 MB)\nTrainable params: 109506881 (417.74 MB)\nNon-trainable params: 0 (0.00 Byte)\n__________________________________________________________________________________________________\n","output_type":"stream"}]},{"cell_type":"code","source":"history = model.fit(\n [train_input_ids, train_attention_masks],\n target,\n validation_split=0.2, \n epochs=3,\n batch_size=10\n)","metadata":{"execution":{"iopub.status.busy":"2023-12-10T08:32:53.650878Z","iopub.execute_input":"2023-12-10T08:32:53.651280Z","iopub.status.idle":"2023-12-10T08:57:06.722297Z","shell.execute_reply.started":"2023-12-10T08:32:53.651248Z","shell.execute_reply":"2023-12-10T08:57:06.721239Z"},"trusted":true},"execution_count":40,"outputs":[{"name":"stdout","text":"Epoch 1/3\n446/446 [==============================] - 522s 1s/step - loss: 0.4315 - accuracy: 0.8600 - val_loss: 0.4368 - val_accuracy: 0.8700\nEpoch 2/3\n446/446 [==============================] - 468s 1s/step - loss: 0.4182 - accuracy: 0.8649 - val_loss: 0.3889 - val_accuracy: 0.8700\nEpoch 3/3\n446/446 [==============================] - 463s 1s/step - loss: 0.4216 - accuracy: 0.8620 - val_loss: 0.3865 - val_accuracy: 0.8700\n","output_type":"stream"}]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]} \ No newline at end of file diff --git a/Email Spam Classification/requirements.txt b/Email Spam Classification/requirements.txt new file mode 100644 index 000000000..fecf21dae --- /dev/null +++ b/Email Spam Classification/requirements.txt @@ -0,0 +1,9 @@ +nltk +spacy +keras +tensorflow +pandas +sklearn +matplotlib + +These are all the libraries. Steps to install and import are given in the notebook \ No newline at end of file