ar-prediction-imbalanced-and-balanced-datasets.ipynb

{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"source":"<a href=\"https://www.kaggle.com/code/volkanatalay/ar-prediction-imbalanced-and-balanced-datasets?scriptVersionId=136327615\" target=\"_blank\"><img align=\"left\" alt=\"Kaggle\" title=\"Open in Kaggle\" src=\"https://kaggle.com/static/images/open-in-kaggle.svg\"></a>","metadata":{},"cell_type":"markdown"},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-07-10T18:21:03.060756Z","iopub.execute_input":"2023-07-10T18:21:03.061866Z","iopub.status.idle":"2023-07-10T18:21:03.104441Z","shell.execute_reply.started":"2023-07-10T18:21:03.061828Z","shell.execute_reply":"2023-07-10T18:21:03.103442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" ## **Abstract** \n\nQSAR Androgen Receptor Data Set contains binder/positive (199) and non-binder/negative (1488) molecules (data instances). Therefore, the dataset is imbalanced (with a ratio of 7.5). Here, I use shallow binary classifiers and evaluate their performances on two configurations of the dataset:  (Configuration I) stratified 5-fold cross-validation (as mentioned in the original paper) and (Configuration II) balanced dataset obtained by using Butina clustering on the molecules. \n\n ## **1. PROBLEM DEFINITION**\n\nThis is a binary classification problem: we need to predict whether a given molecule (data instance) is a binder to the Androgen Receptor or not. ","metadata":{}},{"cell_type":"markdown","source":" ## **2. DATASET**\n\nEach data instance corresponds to a molecule. A molecule is represented by molecular fingerprints consisting of 1024 bits. The last bit in each row is the target label: binder/positive or non-binder/negative.","metadata":{}},{"cell_type":"code","source":"qsar_data = pd.read_csv('/kaggle/input/qsar-androgen-receptor-data-set/qsar_androgen_receptor.csv', header=None, sep=';')\nprint(qsar_data.shape)\nqsar_data.head()","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:21:07.633076Z","iopub.execute_input":"2023-07-10T18:21:07.633553Z","iopub.status.idle":"2023-07-10T18:21:07.945489Z","shell.execute_reply.started":"2023-07-10T18:21:07.633516Z","shell.execute_reply":"2023-07-10T18:21:07.94436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import sklearn # learning algorithms\nfrom sklearn.model_selection import train_test_split \n# use this to split input dataset into training dataset and independent test dataset ","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:21:11.229243Z","iopub.execute_input":"2023-07-10T18:21:11.229596Z","iopub.status.idle":"2023-07-10T18:21:11.886358Z","shell.execute_reply.started":"2023-07-10T18:21:11.22957Z","shell.execute_reply":"2023-07-10T18:21:11.885181Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" ## **3. <u>CONFIGURATION I</u>: STRATIFIED 5-FOLD CROSS-VALIDATION**\n\nThis is the configuration where a training, validation, and  test dataset splits are generated by keeping the same percentages of data instances of the two classes (binders/positive and non-binders/negative) in each split. However, in this case, since dataset is initially imbalanced, this imbalance between two classes will be kept in the splits as well.\n\nA. Data is split into training dataset and independent test dataset. Independent test dataset is only used for tests while 5-fold cross-validation is applied with the training dataset.\n\nB. Since the attributes (fingerprints) are in the first 1024 column of a row (data instance/molecule) and the last column indicates the label (positive or negative), these are separated and stored in different lists.\n\nC. The labels indicated as positive or negative are then encoded by a bit (0 or 1).\n\nD. Support Vector Machine (SVM) is employed as a shallow classifier. The hyperparameter values of the SVM are tuned by using 5-fold cross-validation and grid searh on a selected set of hyperparameters. \n\nE. The SVM-based classifier model is then evaluated in terms of a few metrics by using 5-fold cross-validation. \n\nF. An SVM-based classifier model is subsequently obtained using the tuned hyperparameter values and the whole training dataset. The classifier model is used to get predictions for the independent test dataset. The performance of the classifier model is evaluated both on the training dataset and on the independent test dataset.","metadata":{}},{"cell_type":"code","source":"# A. Data is split into training dataset and independent test dataset\n\n# last column of data indicates the desired output\ndesired_outputs = qsar_data.iloc[:,-1]\n\n# split dataset into training 0.75 and test 0.25 parts\n# stratified split for training dataset and test dataset according to the labels, that is \"desired outputs\"\ntraining_dataset, test_dataset = train_test_split(qsar_data, test_size=0.25, stratify=desired_outputs)","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:21:16.13141Z","iopub.execute_input":"2023-07-10T18:21:16.131794Z","iopub.status.idle":"2023-07-10T18:21:16.154968Z","shell.execute_reply.started":"2023-07-10T18:21:16.131768Z","shell.execute_reply":"2023-07-10T18:21:16.153896Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# B. Attributes and labels are separated and stored in different lists\n\n# last column indicate the corresponding data instance (row) is positive or negative\n# first 1024 columns are feature values ECFP4\n\n# for training dataset\ntraining_desired_outputs = training_dataset.iloc[:,-1]\ntraining_input_features = training_dataset.iloc[:, 0:1024]\n\n# for test dataset\ntest_desired_outputs = test_dataset.iloc[:,-1]\ntest_input_features = test_dataset.iloc[:, 0:1024]","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:21:19.175433Z","iopub.execute_input":"2023-07-10T18:21:19.176544Z","iopub.status.idle":"2023-07-10T18:21:19.191721Z","shell.execute_reply.started":"2023-07-10T18:21:19.176508Z","shell.execute_reply":"2023-07-10T18:21:19.190563Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# C. Labels indicated as positive or negative are then encoded by a bit (0 or 1)\n\nfrom sklearn import preprocessing\n\n# encode positive and negative as 0 and 1\nlabel_encoder = preprocessing.LabelEncoder()\ntraining_desired_outputs = label_encoder.fit_transform(training_desired_outputs)\ntest_desired_outputs = label_encoder.fit_transform(test_desired_outputs)\n#training_desired_outputs","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:21:22.390814Z","iopub.execute_input":"2023-07-10T18:21:22.391236Z","iopub.status.idle":"2023-07-10T18:21:22.39752Z","shell.execute_reply.started":"2023-07-10T18:21:22.391208Z","shell.execute_reply":"2023-07-10T18:21:22.396397Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# D. Tune hyperparameter values of the SVM by using 5-fold cross-validation and grid search. \n\n# Evaluation metric values are displayed for each of the 5 folds. \n\nfrom sklearn import svm\nfrom sklearn.model_selection import GridSearchCV\n\nsvc=svm.SVC()\n# declare parameters for hyperparameter tuning\nparameters = [ {'C':[1, 10, 100, 1000], 'kernel':['linear']},\n               {'C':[1, 10, 100, 1000], 'kernel':['rbf'], 'gamma':[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]},\n               {'C':[1, 10, 100, 1000], 'kernel':['poly'], 'degree': [2,3,4] ,'gamma':[0.01,0.02,0.03,0.04,0.05]} \n              ]\n\ngrid_search = GridSearchCV(estimator = svc,  \n                           param_grid = parameters,\n                           scoring = 'recall',\n                           cv = 5,\n                           verbose=1)\ngrid_search.fit(training_input_features, training_desired_outputs)\n\nprint('GridSearch CV best score : {:.4f}\\n\\n'.format(grid_search.best_score_))\n# print parameters that give the best results\nprint('Parameters that give the best results :','\\n\\n', (grid_search.best_params_))","metadata":{"execution":{"iopub.status.busy":"2023-07-07T21:41:52.539087Z","iopub.execute_input":"2023-07-07T21:41:52.53989Z","iopub.status.idle":"2023-07-07T21:45:38.586037Z","shell.execute_reply.started":"2023-07-07T21:41:52.539854Z","shell.execute_reply":"2023-07-07T21:45:38.584617Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# E. Evaluate SVM-based classifier model by using 5-fold cross-validation.\n\nfrom sklearn import svm\nfrom sklearn import metrics\nfrom sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, matthews_corrcoef\nfrom sklearn.model_selection import cross_val_score, cross_validate\n\n# best parameters for SVM\nlinear_svc=svm.SVC(kernel='linear', C=1.0) \n\nscoring = ['precision', 'recall', 'f1', 'matthews_corrcoef']\nscores = cross_validate(linear_svc, training_input_features, training_desired_outputs, cv=5, scoring=scoring)\nprint(\"precision \", scores['test_precision'])\nprint(\"recall \", scores['test_recall'])\nprint(\"f1 score\", scores['test_f1'])\nprint(\"matthews corr coef \", scores['test_matthews_corrcoef'])","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:22:14.421318Z","iopub.execute_input":"2023-07-10T18:22:14.421737Z","iopub.status.idle":"2023-07-10T18:22:15.575336Z","shell.execute_reply.started":"2023-07-10T18:22:14.421706Z","shell.execute_reply":"2023-07-10T18:22:15.574261Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"When we look at the precision, F1-score and MCC values, there are significant variations among these 5 folds and this is indeed not a good sign. These variations have been most probably caused by the imbalanced data.","metadata":{}},{"cell_type":"code","source":"# F. Obtain SVM-based classifier model using the tuned hyperparameter values and the whole training dataset. \n# Performance of the classifier model is evaluated both on the training dataset and on the independent test dataset.\n\n\nfrom sklearn.metrics import confusion_matrix\nfrom sklearn.metrics import ConfusionMatrixDisplay\nfrom sklearn.metrics import classification_report\nimport matplotlib.pyplot as plt\n\n\n# svm trained by the whole training set\nlinear_svc.fit(training_input_features, training_desired_outputs)\n\n# predictions for the training set\ntraining_predicted_outputs = linear_svc.predict(training_input_features)\n# predictions for the independent test set \ntest_predicted_outputs = linear_svc.predict(test_input_features)\n\nprint(\"performance metrics for training\")\nprint(classification_report(training_desired_outputs, training_predicted_outputs))\nprint()\nprint()\n\n# performance metrics for test    \nprint(\"performance metrics for test\")\nprint(classification_report(test_desired_outputs, test_predicted_outputs))\n# confusion matrix for test set\nconf_matrix = confusion_matrix(test_desired_outputs, test_predicted_outputs)\nConfusionMatrixDisplay(conf_matrix).plot(cmap='GnBu')    ","metadata":{"execution":{"iopub.status.busy":"2023-07-07T21:51:47.150904Z","iopub.execute_input":"2023-07-07T21:51:47.151388Z","iopub.status.idle":"2023-07-07T21:51:47.986691Z","shell.execute_reply.started":"2023-07-07T21:51:47.151356Z","shell.execute_reply":"2023-07-07T21:51:47.985597Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Although the overall test results seem to be fine, there is a problem with Class 1 (as can be observed from the confusion matrix): 29 (out of 50) instances of Class 1 are predicted to be Class 0. This is most probably caused by the imbalance in the training dataset. ","metadata":{}},{"cell_type":"markdown","source":" ## **4. <u>CONFIGURATION II</u>: BALANCED DATASET BY USING BUTINA CLUSTERING**\n\nThis is the configuration where a training, validation, and  test dataset splits are generated by keeping the <u>same number of data instances</u> of the two classes (binders/positive and non-binders/negative) in each split. In this case, since dataset is initially balanced, this balance between two classes will be kept in the splits as well.\n\nA. rdkit package is used for Butina clustering algorithm. rdkit and some other packages are imported.\n\nB. Butina clustering algorithm is implemented as a function that inputs directly the fingerprints of a molecule. Butina clustering algorithm uses Tanimoto similarity between two fingerprints.\n\nC. The binders/posiitive and non-binders/negative compounds are grouped separately. \n\nD. Preprocessing is applied both on binders/posiitive and non-binders/negative data instances.\n\nE. Butina clustering algorithm is applied separately on the posiitive instance group and negative instance group. \nThe labels indicated as positive or negative are then encoded by a bit (0 or 1).\n\nF. Cluster centers and their corresponding molecules' figerprints are stored.\n\nG. Data instances belonging to positive set is split into training dataset and test dataset. Same procedure is applied for the data instances belonging to negative set. From these, final training dataset ans final test dataset is formed.\n\nH. Output labels (0 and 1) are associated.\n\nI. The SVM-based classifier model is evaluated in terms of a few metrics by using 5-fold cross-validation. \n\nJ. The SVM-based classifier model is subsequently obtained using the whole training dataset. The classifier model is used to get predictions for the independent test dataset. The performance of the classifier model is evaluated both on the training dataset and on the independent test dataset.","metadata":{}},{"cell_type":"code","source":"# A. rdkit and some other packages are imported\nimport sys\n!{sys.executable} -m pip install rdkit\n\nimport rdkit\nimport pandas as pd\n\nfrom rdkit import DataStructs\nfrom rdkit.ML.Cluster import Butina\nfrom rdkit.Chem import rdMolDescriptors as rdmd\nfrom rdkit.Chem import Descriptors\nfrom rdkit.Chem import PandasTools, Draw","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:48:15.645105Z","iopub.execute_input":"2023-07-10T18:48:15.64551Z","iopub.status.idle":"2023-07-10T18:48:27.293502Z","shell.execute_reply.started":"2023-07-10T18:48:15.645478Z","shell.execute_reply":"2023-07-10T18:48:27.29202Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# B. Butina clustering algorithm is implemented as a function\n\n# Butina clustering function that input directly the fingerprints\ndef butina_cluster(fp_list,cutoff=0.35):\n    dists = []\n    nfps = len(fp_list)\n    for i in range(1,nfps):\n        sims = DataStructs.BulkTanimotoSimilarity(fp_list[i],fp_list[:i])\n        dists.extend([1-x for x in sims])\n    mol_clusters = Butina.ClusterData(dists,nfps,cutoff,isDistData=True)\n   \n    cluster_id_list = [0]*nfps\n    for idx,cluster in enumerate(mol_clusters,1):\n        for member in cluster:\n            cluster_id_list[member] = idx\n            \n    return mol_clusters, cluster_id_list","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:48:27.296696Z","iopub.execute_input":"2023-07-10T18:48:27.297103Z","iopub.status.idle":"2023-07-10T18:48:27.306291Z","shell.execute_reply.started":"2023-07-10T18:48:27.297066Z","shell.execute_reply":"2023-07-10T18:48:27.305112Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# C. The binders/posiitive and non-binders/negative compounds are grouped separately. \n\n\n# group rows in terms of being positive or negative\ngrouped = qsar_data.groupby(qsar_data.iloc[:,-1])\ndf_positive = grouped.get_group(\"positive\")\ndf_negative = grouped.get_group(\"negative\")","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:48:27.308156Z","iopub.execute_input":"2023-07-10T18:48:27.308607Z","iopub.status.idle":"2023-07-10T18:48:27.335221Z","shell.execute_reply.started":"2023-07-10T18:48:27.308575Z","shell.execute_reply":"2023-07-10T18:48:27.334285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# D. Preprocessing is applied both on binders/posiitive and non-binders/negative data instances.\n\n# convert positive group into ExplicitBitVect\ndf_pos_fp_int = df_positive.iloc[:, 0:1024]\n\n# convert dataframe into a list of ExplicitBitVect\npos_fp_int_list = df_pos_fp_int.values.tolist() # these should be converted into ExplicitBitVect\npos_fp_list =[]\nfor mol in pos_fp_int_list:\n    temp_str = ''.join(str(x) for x in mol)\n    pos_fp_list.append(DataStructs.CreateFromBitString(temp_str))\n    \n# convert negative group into ExplicitBitVect\ndf_neg_fp_int = df_negative.iloc[:, 0:1024]\n\n# convert dataframe into a list of ExplicitBitVect\nneg_fp_int_list = df_neg_fp_int.values.tolist() # these should be converted into ExplicitBitVect\nneg_fp_list =[]\nfor mol in neg_fp_int_list:\n    temp_str = ''.join(str(x) for x in mol)\n    neg_fp_list.append(DataStructs.CreateFromBitString(temp_str))\n    \nprint(\"number of positives \", len(pos_fp_list))\nprint(\"number of negatives \", len(neg_fp_list))","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:48:27.338498Z","iopub.execute_input":"2023-07-10T18:48:27.338893Z","iopub.status.idle":"2023-07-10T18:48:27.764145Z","shell.execute_reply.started":"2023-07-10T18:48:27.338852Z","shell.execute_reply":"2023-07-10T18:48:27.763139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# E. Butina clustering algorithm is applied separately on the posiitive instance group and negative instance group. \n\n# cluster the negative compounds\nneg_clusters, neg_cluster_idList = butina_cluster(neg_fp_list, cutoff=0.5)\n# cluster the positive compounds\npos_clusters, pos_cluster_idList = butina_cluster(pos_fp_list, cutoff=0.5)","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:48:27.765386Z","iopub.execute_input":"2023-07-10T18:48:27.765736Z","iopub.status.idle":"2023-07-10T18:48:28.322736Z","shell.execute_reply.started":"2023-07-10T18:48:27.765708Z","shell.execute_reply":"2023-07-10T18:48:28.3217Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# F. Cluster centers and their sorresponding molecules' figerprints are stored.\n\n# Get the cluster center of each cluster (first molecule in each cluster)\npos_cluster_centers = [c[0] for c in pos_clusters]\n# How many cluster centers/clusters do we have?\nprint(\"Number of cluster centers:\", len(pos_cluster_centers))\n\n# Get the cluster center of each cluster (first molecule in each cluster)\nneg_cluster_centers = [c[0] for c in neg_clusters]\n# How many cluster centers/clusters do we have?\nprint(\"Number of cluster centers:\", len(neg_cluster_centers))\n\n# data from pos_fp_int_list for pos cluster centers\n# data from neg_fp_int_list for neg cluster centers\npos_molecules = [ pos_fp_int_list[m] for m in pos_cluster_centers]\nneg_molecules = [ neg_fp_int_list[m] for m in neg_cluster_centers]","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:48:28.324263Z","iopub.execute_input":"2023-07-10T18:48:28.324594Z","iopub.status.idle":"2023-07-10T18:48:28.331945Z","shell.execute_reply.started":"2023-07-10T18:48:28.324566Z","shell.execute_reply":"2023-07-10T18:48:28.33082Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# G. Training dataset and test dataset splits are obtained. \n#    From these, final training dataset ans final test dataset is formed.\n\nimport sklearn\nfrom sklearn.model_selection import train_test_split\n\n\npos_training_dataset, pos_test_dataset = train_test_split(pos_molecules, test_size=0.25, random_state=43)\nneg_training_dataset, neg_test_dataset = train_test_split(neg_molecules, train_size=len(pos_training_dataset), random_state=43)\n\nprint(\"pos training test\", len(pos_training_dataset), len(pos_test_dataset))\nprint(\"neg training test\", len(neg_training_dataset), len(neg_test_dataset))\n\ntraining_dataset = pos_training_dataset + neg_training_dataset\ntest_dataset = pos_test_dataset + neg_test_dataset","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:48:28.333541Z","iopub.execute_input":"2023-07-10T18:48:28.333921Z","iopub.status.idle":"2023-07-10T18:48:28.354271Z","shell.execute_reply.started":"2023-07-10T18:48:28.333886Z","shell.execute_reply":"2023-07-10T18:48:28.353262Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# H. Generate output lists positive 1, negative 0\n\ntraining_output = [1] * len(pos_training_dataset)\ntemp = [0] * len(neg_training_dataset)\ntraining_output += temp\n\ntest_output = [1] * len(pos_test_dataset)\ntemp = [0] * len(neg_test_dataset)\ntest_output += temp","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:48:28.355688Z","iopub.execute_input":"2023-07-10T18:48:28.356102Z","iopub.status.idle":"2023-07-10T18:48:28.365426Z","shell.execute_reply.started":"2023-07-10T18:48:28.356066Z","shell.execute_reply":"2023-07-10T18:48:28.364669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# I. Evaluate SVM-based classifier model by using 5-fold cross-validation.\n\nfrom sklearn import svm\nfrom sklearn import metrics\nfrom sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, matthews_corrcoef\nfrom sklearn.model_selection import cross_val_score, cross_validate\n\n# best parameters for SVM\nclf=svm.SVC(kernel='linear', C=10) \n\nscoring = ['precision', 'recall', 'f1', 'matthews_corrcoef']\nscores = cross_validate(clf, training_dataset, training_output, cv=5, scoring=scoring)\nprint(\"precision \", scores['test_precision'])\nprint(\"recall \", scores['test_recall'])\nprint(\"f1 score\", scores['test_f1'])\nprint(\"matthews corr coef \", scores['test_matthews_corrcoef'])","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:48:28.366197Z","iopub.execute_input":"2023-07-10T18:48:28.366498Z","iopub.status.idle":"2023-07-10T18:48:28.547417Z","shell.execute_reply.started":"2023-07-10T18:48:28.36646Z","shell.execute_reply":"2023-07-10T18:48:28.546574Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# J. An SVM-based classifier model is subsequently obtained using the whole training dataset. \n\nimport matplotlib.pyplot as plt\nfrom sklearn.metrics import confusion_matrix\nfrom sklearn.metrics import ConfusionMatrixDisplay\nfrom sklearn.metrics import classification_report\n\n\n# support vector classifier\n \nclf.fit(training_dataset, training_output)\n\n\n# predictions for training dataset\ntraining_predicted_outputs = clf.predict(training_dataset)\n# predictions for test dataset\ntest_predicted_outputs = clf.predict(test_dataset)\n\nprint(\"for training\")\nprint(classification_report(training_output, training_predicted_outputs))\n \nprint(\"for test\")\nprint(classification_report(test_output, test_predicted_outputs))\n\nconf_matrix = confusion_matrix(test_output, test_predicted_outputs)\nConfusionMatrixDisplay(conf_matrix).plot(cmap='GnBu')   \n","metadata":{"execution":{"iopub.status.busy":"2023-07-10T18:48:28.549878Z","iopub.execute_input":"2023-07-10T18:48:28.550459Z","iopub.status.idle":"2023-07-10T18:48:29.014343Z","shell.execute_reply.started":"2023-07-10T18:48:28.550421Z","shell.execute_reply":"2023-07-10T18:48:29.013606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n","metadata":{}}]}