DL_Lab7a_J050.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "DL_LAB7a_J050.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "machine_shape": "hm"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1EKwXmlxj0WY",
        "colab_type": "text"
      },
      "source": [
        "#How to get Images from ImageNet with Python in Google Colaboratory\n",
        "\n",
        "The first step to train a model for image recognition is finding images that belong to the desired class (or classes), and ImageNet is very useful for this because it currently has 14,197,122 images with 21841 synsets indexed. ImageNet aims to provide on average 1000 images to illustrate each one of their 100,000 synsets, the majority of the synsets are nouns (80.000+).\n",
        "\n",
        "For instance if the synset needed is pictures of ships it can be found by searching for ship on the imagenet website and the result will be the following page which has the wnid: n04194289"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lz42zfTtkKKQ",
        "colab_type": "text"
      },
      "source": [
        "#Get the list of URLs for the images of the synset:\n",
        "\n",
        "Said list of URLs can be downloaded from the URL http://www.image-net.org/api/text/imagenet.synset.geturls?wnid= followed by the wnid so in the case of ships it would be “http://www.image-net.org/api/text/imagenet.synset.geturls?wnid=n04194289\" this can be done with the Python library BeautifulSoup:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4LtZJ6sPGzQM",
        "colab_type": "text"
      },
      "source": [
        "#Data augmentation - Flow from dataframe"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oP2R9Ou5G5cf",
        "colab_type": "code",
        "outputId": "b3c45a03-fc66-41a8-b5ae-70c3a5c4eb24",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 319
        }
      },
      "source": [
        "#code part 1\n",
        "from bs4 import BeautifulSoup\n",
        "import numpy as np\n",
        "import requests\n",
        "import cv2\n",
        "import PIL.Image\n",
        "import urllib\n",
        "\n",
        "ie_page = requests.get(\"http://www.image-net.org/api/text/imagenet.synset.geturls?wnid=n02504013\")#indian elephant synset\n",
        "# print(ie_page.content)\n",
        "ie_soup = BeautifulSoup(ie_page.content, 'html.parser')#puts the content of the website into the soup variable, each url on a different line\n",
        "\n",
        "ae_page = requests.get(\"http://www.image-net.org/api/text/imagenet.synset.geturls?wnid=n02504458\")#african elephant synset\n",
        "# print(ae_page.content)\n",
        "ae_soup = BeautifulSoup(ae_page.content, 'html.parser')#puts the content of the website into the soup variable, each url on a different line\n",
        "\n",
        "\n",
        "#code part 2\n",
        "ie_str_soup=str(ie_soup)#convert soup to string so it can be split\n",
        "type(ie_str_soup)\n",
        "ie_split_urls=ie_str_soup.split('\\r\\n')#split so each url is a different possition on a list\n",
        "print(len(ie_split_urls))#print the length of the list so you know how many urls you have\n",
        "\n",
        "#code part 2.2\n",
        "ae_str_soup=str(ae_soup)#convert soup to string so it can be split\n",
        "type(ae_str_soup)\n",
        "ae_split_urls=ae_str_soup.split('\\r\\n')#split so each url is a different possition on a list\n",
        "print(len(ae_split_urls))\n",
        "\n",
        "\n",
        "!mkdir /content/elephants_train/ \n",
        "!mkdir /content/elephants_test/ \n",
        "\n",
        "img_rows, img_cols = 32, 32 #number of rows and columns to convert the images to\n",
        "input_shape = (img_rows, img_cols, 3)#format to store the images (rows, columns,channels) called channels last\n",
        "\n",
        "def url_to_image(url):\n",
        "\t# download the image, convert it to a NumPy array, and then read\n",
        "\t# it into OpenCV format\n",
        "\tresp = urllib.request.urlopen(url)\n",
        "\timage = np.asarray(bytearray(resp.read()), dtype=\"uint8\")\n",
        "\timage = cv2.imdecode(image, cv2.IMREAD_COLOR)\n",
        " \n",
        "\t# return the image\n",
        "\treturn image\n",
        "\n",
        "n_of_training_images=150#the number of training images to use\n",
        "for progress in range(n_of_training_images):#store all the images on a directory\n",
        " \n",
        "    if(progress%20==0):\n",
        "        print(progress)\n",
        "    if not ie_split_urls[progress] == None:\n",
        "      try:\n",
        "        I = url_to_image(ie_split_urls[progress])\n",
        "        if (len(I.shape))==3: #check if the image has width, length and channels\n",
        "          save_path = '/content/elephants_train/ie.'+str(progress)+'.jpg'#create a name of each image\n",
        "          cv2.imwrite(save_path,I)\n",
        "\n",
        "      except:\n",
        "        None\n",
        "\n",
        "for progress in range(n_of_training_images):#store all the images on a directory\n",
        "  \n",
        "    if(progress%20==0):\n",
        "        print(progress)\n",
        "    if not ae_split_urls[progress] == None:\n",
        "      try:\n",
        "        I = url_to_image(ae_split_urls[progress])\n",
        "        if (len(I.shape))==3: #check if the image has width, length and channels\n",
        "          save_path = '/content/elephants_train/ae.'+str(progress)+'.jpg'#create a name of each image\n",
        "          cv2.imwrite(save_path,I)\n",
        "\n",
        "      except:\n",
        "        None"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1650\n",
            "2278\n",
            "0\n",
            "20\n",
            "40\n",
            "60\n",
            "80\n",
            "100\n",
            "120\n",
            "140\n",
            "0\n",
            "20\n",
            "40\n",
            "60\n",
            "80\n",
            "100\n",
            "120\n",
            "140\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_D6S7Ma5l0QL",
        "colab_type": "text"
      },
      "source": [
        "#Test data images"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B7DHRnzLl2JE",
        "colab_type": "code",
        "outputId": "840a01b1-efaa-43cf-f6a9-4900ad629f82",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 118
        }
      },
      "source": [
        "#Validation data:\n",
        "\n",
        "for progress in range(50):#store all the images on a directory\n",
        "    # Print out progress whenever progress is a multiple of 20 so we can follow the\n",
        "    # (relatively slow) progress\n",
        "    if(progress%20==0):\n",
        "        print(progress)\n",
        "    if not ie_split_urls[progress] == None:\n",
        "      try:\n",
        "        I = url_to_image(ie_split_urls[n_of_training_images+progress])#get images that are different from the ones used for training\n",
        "        if (len(I.shape))==3: #check if the image has width, length and channels\n",
        "          save_path = '/content/elephants_test/ie.'+str(progress)+'.jpg'#create a name of each image\n",
        "          cv2.imwrite(save_path,I)\n",
        "\n",
        "      except:\n",
        "        None\n",
        "\n",
        "#do the same for bikes:\n",
        "for progress in range(50):#store all the images on a directory\n",
        "    # Print out progress whenever progress is a multiple of 20 so we can follow the\n",
        "    # (relatively slow) progress\n",
        "    if(progress%20==0):\n",
        "        print(progress)\n",
        "    if not ae_split_urls[progress] == None:\n",
        "      try:\n",
        "        I = url_to_image(ae_split_urls[n_of_training_images+progress])#get images that are different from the ones used for training\n",
        "        if (len(I.shape))==3: #check if the image has width, length and channels\n",
        "          save_path = '/content/elephants_test/ae.'+str(progress)+'.jpg'#create a name of each image\n",
        "          cv2.imwrite(save_path,I)\n",
        "\n",
        "      except:\n",
        "        None"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0\n",
            "20\n",
            "40\n",
            "0\n",
            "20\n",
            "40\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yy4gvz0UktHZ",
        "colab_type": "text"
      },
      "source": [
        "#Save images to folder"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q_xEzEEsG90r",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Path to the train folder\n",
        "import os\n",
        "original_train = '/content/elephants_train/'\n",
        " \n",
        "filenames = os.listdir(original_train)\n",
        "categories = []\n",
        "for filename in filenames:\n",
        "    category = filename.split('.')[0]\n",
        "    if category == 'ie':\n",
        "        categories.append('0')\n",
        "    else:\n",
        "        categories.append('1')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jt2YSDOskwuE",
        "colab_type": "text"
      },
      "source": [
        "#Create dataframe from folder"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QALGG9VZHAos",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "data = pd.DataFrame({'filename':filenames,'label':categories})\n",
        "\n",
        "data.to_csv(\"original_elephants.csv\",index=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qT-mfohJuUDm",
        "colab_type": "text"
      },
      "source": [
        "#Create dataframe for test "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5ErOV_4VuWUq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Path to the train folder\n",
        "import os\n",
        "original_test = '/content/elephants_test/'\n",
        " \n",
        "filenames = os.listdir(original_test)\n",
        "categories = []\n",
        "for filename in filenames:\n",
        "    category = filename.split('.')[0]\n",
        "    if category == 'ie':\n",
        "        categories.append('0')\n",
        "    else:\n",
        "        categories.append('1')\n",
        "\n",
        "import pandas as pd\n",
        "data_test = pd.DataFrame({'filename':filenames,'label':categories})\n",
        "\n",
        "data_test.to_csv(\"original_elephants_test.csv\",index=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Bk-iwznOHtn6",
        "colab_type": "code",
        "outputId": "0aff9f96-829a-442b-c30d-31a835a86180",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 67
        }
      },
      "source": [
        "from keras.models import Sequential\n",
        "from keras.layers import Conv2D, MaxPooling2D\n",
        "from keras.layers import Dropout, Flatten, Dense\n",
        "from keras.optimizers import SGD\n",
        "from keras.preprocessing.image import ImageDataGenerator\n",
        "\n",
        "datagen = ImageDataGenerator(\n",
        "        rescale=1./255,\n",
        "        shear_range=0.2,\n",
        "        zoom_range=0.2,\n",
        "        horizontal_flip=True,\n",
        "        validation_split=0.2)\n",
        "\n",
        "\n",
        "test_datagen = ImageDataGenerator(rescale=1./255)\n",
        "\n",
        "\n",
        "\n",
        "train_generator = datagen.flow_from_dataframe(dataframe=data, directory=original_train,\n",
        "                                             x_col='filename',\n",
        "                                             y_col='label',\n",
        "                                             target_size=(150,150),\n",
        "                                             class_mode='binary',\n",
        "                                             batch_size=10,\n",
        "                                             subset='training',\n",
        "                                             shuffle=True,\n",
        "                                             seed=7)\n",
        " \n",
        "validation_generator = datagen.flow_from_dataframe(dataframe=data, directory=original_train,\n",
        "                                             x_col='filename',\n",
        "                                             y_col='label',\n",
        "                                             target_size=(150,150),\n",
        "                                             class_mode='binary',\n",
        "                                             batch_size=10,\n",
        "                                             subset='validation',\n",
        "                                             shuffle=True,\n",
        "                                             seed=7)\n",
        "\n",
        "test_datagen=ImageDataGenerator(rescale=1./255.)\n",
        "\n",
        "test_generator=test_datagen.flow_from_dataframe(\n",
        "dataframe=data_test,\n",
        "directory=\"/content/elephants_test/\",\n",
        "x_col=\"filename\",\n",
        "y_col=None,\n",
        "batch_size=10,\n",
        "seed=42,\n",
        "shuffle=False,\n",
        "class_mode=None,\n",
        "target_size=(150,150))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found 148 validated image filenames belonging to 2 classes.\n",
            "Found 37 validated image filenames belonging to 2 classes.\n",
            "Found 66 validated image filenames.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Taw_PmJbminf",
        "colab_type": "text"
      },
      "source": [
        "#Model building"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "twev8tYZmkX1",
        "colab_type": "code",
        "outputId": "a6e0e5c6-e143-4124-8600-463f16c1e550",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 877
        }
      },
      "source": [
        "from keras.utils.np_utils import to_categorical\n",
        "from keras import models\n",
        "from keras.datasets import cifar10\n",
        "from keras.layers import Dense, Dropout\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Activation, Flatten, Conv2D\n",
        "from keras.utils import to_categorical\n",
        "from keras.datasets import mnist\n",
        "from keras.utils.vis_utils import model_to_dot\n",
        "from IPython.display import SVG\n",
        "from keras.preprocessing.image import ImageDataGenerator\n",
        "\n",
        "STEP_SIZE_TRAIN=train_generator.n//train_generator.batch_size\n",
        "STEP_SIZE_VALID=validation_generator.n//validation_generator.batch_size\n",
        "STEP_SIZE_TEST=test_generator.n//test_generator.batch_size\n",
        "\n",
        "\n",
        "\n",
        "model = Sequential()\n",
        "#We need to use a Conv2D layer at start of the neural network \n",
        "#the syntax is Conv2D(1, (image_width,image_height), padding=\"valid\", activation=\"relu\", input_shape=X_train.shape[1:])\n",
        "#the we add a flatten layer\n",
        "model.add(Conv2D(512, (150, 150), padding=\"valid\", activation=\"relu\", input_shape=(150, 150, 3)))\n",
        "model.add(Flatten())\n",
        "model.add(Dense(512))\n",
        "model.add(Activation('relu'))\n",
        "model.add(Dense(1))\n",
        "model.add(Activation('sigmoid'))\n",
        "\n",
        "model.compile(loss='binary_crossentropy',\n",
        "                  optimizer='adam',\n",
        "                  metrics=['accuracy'])\n",
        "\n",
        "\n",
        "model.fit_generator(generator=train_generator,\n",
        "                    steps_per_epoch=STEP_SIZE_TRAIN,\n",
        "                    validation_data=validation_generator,\n",
        "                    validation_steps=STEP_SIZE_VALID,\n",
        "                    epochs=10)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:66: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:541: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4432: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/optimizers.py:793: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3657: The name tf.log is deprecated. Please use tf.math.log instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/nn_impl.py:183: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1033: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1020: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3005: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
            "\n",
            "Epoch 1/10\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:190: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:197: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:207: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:216: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:223: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.\n",
            "\n",
            "14/14 [==============================] - 15s 1s/step - loss: 9.5451 - acc: 0.3894 - val_loss: 10.2081 - val_acc: 0.3667\n",
            "Epoch 2/10\n",
            "14/14 [==============================] - 1s 100ms/step - loss: 9.3536 - acc: 0.4197 - val_loss: 9.5515 - val_acc: 0.4074\n",
            "Epoch 3/10\n",
            "14/14 [==============================] - 1s 86ms/step - loss: 9.8414 - acc: 0.3894 - val_loss: 8.9545 - val_acc: 0.4444\n",
            "Epoch 4/10\n",
            "14/14 [==============================] - 1s 78ms/step - loss: 9.5303 - acc: 0.4087 - val_loss: 9.5515 - val_acc: 0.4074\n",
            "Epoch 5/10\n",
            "14/14 [==============================] - 1s 93ms/step - loss: 9.5557 - acc: 0.4071 - val_loss: 8.5963 - val_acc: 0.4667\n",
            "Epoch 6/10\n",
            "14/14 [==============================] - 1s 100ms/step - loss: 9.6661 - acc: 0.4003 - val_loss: 10.1484 - val_acc: 0.3704\n",
            "Epoch 7/10\n",
            "14/14 [==============================] - 1s 73ms/step - loss: 9.1231 - acc: 0.4340 - val_loss: 11.3424 - val_acc: 0.2963\n",
            "Epoch 8/10\n",
            "14/14 [==============================] - 1s 82ms/step - loss: 10.0181 - acc: 0.3785 - val_loss: 8.3575 - val_acc: 0.4815\n",
            "Epoch 9/10\n",
            "14/14 [==============================] - 1s 92ms/step - loss: 9.0952 - acc: 0.4357 - val_loss: 10.2081 - val_acc: 0.3667\n",
            "Epoch 10/10\n",
            "14/14 [==============================] - 1s 96ms/step - loss: 10.0987 - acc: 0.3735 - val_loss: 8.9545 - val_acc: 0.4444\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7fb68097c780>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RdJaUvqVm0tn",
        "colab_type": "text"
      },
      "source": [
        "#Evaluate the model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DnVOz-K5m17H",
        "colab_type": "code",
        "outputId": "690e362a-4ef1-408f-c62a-2066bcd418f1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "model.evaluate_generator(generator=validation_generator,steps=STEP_SIZE_TEST)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[9.89707627212792, 0.3859649190777226]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E0B7s-NSnKg_",
        "colab_type": "text"
      },
      "source": [
        "#Measure the performance on validation set"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xJsMJIb2nLS0",
        "colab_type": "code",
        "outputId": "07e90112-35b9-4b5a-cc1e-e8cb8f512b54",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 84
        }
      },
      "source": [
        "from sklearn.metrics import roc_auc_score\n",
        "\n",
        "# AUC for prediction on validation sample\n",
        "X_val_sample, val_labels = next(validation_generator)\n",
        "val_pred = model.predict_proba(X_val_sample)\n",
        "val_pred = np.reshape(val_pred, val_labels.shape)\n",
        "val_score_auc = roc_auc_score(val_labels, val_pred)\n",
        "print (\"AUC validation score\")\n",
        "print (val_score_auc)\n",
        "print ('\\n')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "AUC validation score\n",
            "0.5\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FAU7xrc6m5gn",
        "colab_type": "text"
      },
      "source": [
        "#Predict the output"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DfUjO9mam6jY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "test_generator.reset()\n",
        "pred=model.predict_generator(test_generator)\n",
        "\n",
        "predictions = []\n",
        "for i in pred:\n",
        "    if i >=0.5:\n",
        "        predictions.append('1')\n",
        "    else:\n",
        "        predictions.append('0')\n",
        " \n",
        "filenames=test_generator.filenames\n",
        "results=pd.DataFrame({\"Filename\":filenames,\n",
        "                      \"Predictions\":predictions})\n",
        "\n",
        "\n",
        "results.to_csv(\"results_elephants_test.csv\",index=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cLvXacXK_7iP",
        "colab_type": "code",
        "outputId": "37e89ac2-71be-4a43-d2ac-1beadd4973be",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "pred"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.],\n",
              "       [0.]], dtype=float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ncYBMddc-Qxq",
        "colab_type": "text"
      },
      "source": [
        "#Confusion matrix"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "70xW8TGA-Sns",
        "colab_type": "code",
        "outputId": "745999e1-38f4-46eb-ccdf-f367a630959e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 273
        }
      },
      "source": [
        "from sklearn.metrics import confusion_matrix\n",
        "import seaborn as sn\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "cm=confusion_matrix(data_test['label'],results['Predictions'])\n",
        "\n",
        "sn.set(font_scale=1.4) # for label size\n",
        "sn.heatmap(cm, annot=True, annot_kws={\"size\": 16}) # font size\n",
        "\n",
        "plt.show()\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWsAAAEACAYAAAB1dVfhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAYMUlEQVR4nO3de3xU9ZnH8W8SEi7C5MLNQposuUhJ\njApqE21NKkbcQiIGaoEWS1nELQFESwWpbLRWQxXFWgl0vdGUWl1xKUUoCAU3gBZFgi23IiRAAiIi\nhEwgCbnM7B8uWePEZEKSOfPL+bx9zR85c+acR1/my8NzfudMgNvtdgsA4NcCrS4AANAywhoADEBY\nA4ABCGsAMABhDQAGIKwBwABdrC6gculMq0uAn3HMWml1CfBTdTXH2/T52s+Kvd43uE9Mm87V3iwP\nawDwGVe91RVcMsIagH24XVZXcMkIawD24SKsAcDvuemsAcAA9XVWV3DJCGsA9sEFRgAwAGMQADAA\nFxgBwP9xgREATEBnDQAGqK+1uoJLRlgDsA/GIABgAMYgAGAAOmsAMACdNQD4P7eLC4wA4P/orAHA\nAMysAcAAPMgJAAxAZw0ABmBmDQAG4MsHAMAAdNYA4P/cbi4wAoD/o7MGAAOwGgQADEBnDQAGYDUI\nABiAMQgAGIAxCAAYgLAGAAMwBgEAA3CBEQAMYPAYJNDqAgDAZ9wu71+tsGHDBk2YMEHJyclKSkpS\nenq6nnjiCVVUVDTar6CgQFlZWQ37LF++3Otz0FkDsI8O6qzLy8t1/fXXa/LkyQoNDdWBAwe0ePFi\nHThwQC+//LIkadeuXcrOztbo0aM1d+5cFRYWKjc3V126dNGECRNaPAdhDcA+Oiis77zzzkY/Jycn\nq2vXrsrJydHJkyfVv39/5eXlKSEhQbm5uZKklJQUnThxQnl5eRo3bpwCA5sfdDAGAWAfbrf3rzYK\nDw+XJNXW1qqmpkbbt2/XyJEjG+2TkZGhU6dOae/evS0ej7AGYB91dd6/LkF9fb0uXLigPXv2KC8v\nT8OHD1dkZKRKSkpUW1ur2NjYRvvHx8dLkoqLi1s8NmMQAPbRiguHTqdTTqfTY7vD4ZDD4WjyM8nJ\nyQ0XFW+66SY9/fTTkj6faV/87JeP9cX3m0NYA7CPVsys8/PztXjxYo/tM2bM0MyZM5v8zPLly1VV\nVaWDBw9q6dKl+slPfqJly5ZdcrlfRFgDsI9WzKInTZqkrKwsj+1f1VVL0pAhQyRJw4YNU2JiosaO\nHauNGzcqLi5Okjw69Ys/h4aGtlgPYQ3APlrRWTc37vDGkCFDFBgYqJKSEg0fPlzBwcEqLi5Wampq\nwz6HDh2SJMXExLR4PC4wArAPl8v7Vxvt2rVLLpdLkZGRCgkJUUpKitatW9donzVr1qhv375KTExs\n8Xh01gBsw13fMV+YO2XKFKWkpCg+Pl5du3bV/v379dJLL2nw4MFKT0+XJE2fPl0TJ07U/PnzlZmZ\nqcLCQq1YsUI5OTktrrGWCGsAdtJBN8UkJSVp9erVOnbsmCQpMjJS48eP1+TJkxUSEiJJGjp0qJYs\nWaJFixZp1apV6tevn+bNm+fV3YuSFOB2t8Pq7zaoXNr0VdXOZuPBT7T+wCfad9KpssoaXe7opuGx\n/TXlm4N0Wcj//5l54FOnfvPOQe36+KwCA6RrIyM0O3WwosJ6WFi9bzlmrbS6BMtFRg7Q0089ovRb\nblJAQIA2bd6qn85+WKWlH1tdmqXqao636fOtyZse055r07naGzNrH1m+86iCAgI081vxWpw1THcm\nfV0r/lGqaSt3yvV/f14eLTuvf1uxQxUX6vT4vybpkVuv1Alnlaas2KEzlRcs/jeAr3Tv3k0b33pd\ngwfHavKU+zRp8r2Kixukv25YoR49ultdntlcbu9ffoYxiI/8+vahiugR0vDzdZERcnQLVs6GPfrg\n2Bl98+u99bsPjigwIEB5dwxTr27BkqSky0N1+++26fc7j+q+m66wqnz40N1TfqiYmCglXJmqoqIj\nkqTdu/frn/u26Z6pd+nXzz5vbYEm4xGpaMkXg/qixMs/Xxb06bnPu+bdn5zVVV8LbQhqSerfq5ti\ne/fU5qJPfVMoLJeZMULvvVfYENSSdORIqd59d4duzxxhXWGdQX299y8/41VnXVRUpC1btqi4uLjh\ntsjQ0FDFxMQoNTXV4353eGfnsTJJ0qCIyyRJgQEBCg7y/PMzJChQx85W6kJdvbp2CfJpjfC9hIQr\ntPrNDR7b9+77SN8bm2FBRZ2IwZ11s2FdXV2thx56SH/5y18UHBysqKiohkXixcXF+vOf/6wnn3xS\nI0eOVG5urrp27eqTojuDT89Va+nfDik5KkKJ/T+/e+lfwi/T30+cVW29qyG0z9fUqej0ObklOS/U\nqS9h3elFRITp7NmzHtvLys4qPLzlO93QDD+cRXur2bB+6qmn9M4772jhwoUaMWJEwxKUi2pqarRx\n40Y99thjWrhwoebPn9+hxXYWlTV1um/1hwoKDNQvbr2yYfuEa6K08eBJPb55v7JviFWdy61FWw6o\nqvbzv5IxswLayOAvzG3293/t2rWaN2+eMjIyPIJakkJCQjRq1CjNnTtXa9eu7bAiO5PqunrNWr1L\nx8srtSRrmPr36tbw3tCB4Zp38ze06eBJ3fbiFo16eavO1dQpI2GAgoMC5PjCLBudV1lZucLCwjy2\nh4eHqays5aezoRmddTVIdXW1+vTp0+JB+vTpo+rq6nYrqrOqrXfpgTV/176TTi0dc63i+/Ty2Of7\nV0fpjsRIlZZX6rKQLrq8VzdN/9NOXXl5aJPzbHQ++/Z9pMQEz5U/CUPitX//RxZU1Hm4DZ5ZN/vb\nP2zYMOXl5TX7rNXy8nItWbJE1113XbsX15m43G49tH63dpSe0aLMa3TV1zw7p4tCugQqtndPXd6r\nmw5+VqH3S8/ozqSv+7BaWOnNNRuUnDxMgwZFNWyLjo7UjTderzfXbLSwsk7A4NUgzd7BePToUd11\n112qqKjQDTfcoLi4OPXq9Xk3WFFRoaKiIv3tb3+Tw+FQfn6+oqOjW12AXe5gfHzTPr2x+5ju/uYg\n3TSob6P3+vfspv69uulkRbVW/KNUVw8IU3BQoPaddGrZjsO6Mbq3nhh1tUWV+57d72Ds0aO7Cj/Y\nqKrqauU8/KTcbrd+8cgc9ep5mYZem67z5yutLtEybb2D8fyjP/R638tyXmnTudpbs2OQ6OhorV27\nVq+++qq2bt2qN954o+H5qw6HQ7GxsZo2bZrGjx/fEOJo2jtHPpMkvfj+Yb34/uFG7/17cox+ckOc\nugQGaPcn5frv3cd0vrZOkaE9NDU5Rj8YGtXUIdFJVVZW6dbbvq+nn3pE+ct+o4CAAG1+e5t+Ovth\nWwd1uzB4DMKzQeB37N5Z46u1ubPOGe/1vpc9+lqbztXeuN0cgH0YvHSPsAZgH364JM9bhDUA23DX\n+d8qD28R1gDsg84aAAzAzBoADEBnDQD+z01YA4ABuMAIAAagswYAAxDWAOD/LH66RpsQ1gDsg84a\nAAxAWAOA/3PXcVMMAPg/c7OasAZgH9wUAwAmIKwBwACMQQDA/zEGAQADuOsIawDwf4xBAMD/Gfzd\nA4Q1ABshrAHA/9FZA4AB3HVWV3DpCGsAtkFnDQAGIKwBwATuAKsruGSENQDboLMGAAO4XXTWAOD3\nXPWENQD4PcYgAGAAxiAAYAC3uQ/dU6DVBQCAr7hdAV6/WmPdunXKzs5WWlqarrnmGmVmZuqPf/yj\nXK7Gc5eCggJlZWUpKSlJ6enpWr58udfnoLMGYBsddYFx2bJlGjBggObMmaPevXvrvffe0+OPP67S\n0lLNnTtXkrRr1y5lZ2dr9OjRmjt3rgoLC5Wbm6suXbpowoQJLZ6DsAZgGx01s/7tb3+riIiIhp9T\nUlJUWVmpV155Rffff79CQkKUl5enhIQE5ebmNuxz4sQJ5eXlady4cQoMbH7QwRgEgG243QFev1rj\ni0F90ZAhQ3ThwgWdPXtWNTU12r59u0aOHNlon4yMDJ06dUp79+5t8RyENQDbcLu8f7XVzp07FRYW\npt69e6ukpES1tbWKjY1ttE98fLwkqbi4uMXjMQYBYBuuVnTMTqdTTqfTY7vD4ZDD4Wj2s7t379bK\nlSs1ffp0BQUFqby8vOGzXz6WpIb3m0NYA7CN1ow38vPztXjxYo/tM2bM0MyZM7/yc6dOndK9996r\npKQkTZ069ZLqbAphDcA2WrMaZNKkScrKyvLY3lxXXVFRoalTp6pbt25aunSpgoODJUmhoaGS5NGp\nX/z54vvNIawB2EZrVoN4M+74ogsXLmjatGk6ffq0XnvtNYWHhze8FxUVpeDgYBUXFys1NbVh+6FD\nhyRJMTExLR6fC4wAbMPlDvD61Rp1dXWaNWuWDhw4oBdeeEEDBw5s9H5ISIhSUlK0bt26RtvXrFmj\nvn37KjExscVz0FkDsI3WLsnz1qOPPqq3335bDzzwgKqrq/Xhhx82vBcXF6eePXtq+vTpmjhxoubP\nn6/MzEwVFhZqxYoVysnJaXGNtURYA7CRjno2yLZt2yRJCxcu9Hjv97//vZKTkzV06FAtWbJEixYt\n0qpVq9SvXz/NmzfPq7sXJcIagI20drzhrc2bN3u1X1pamtLS0i7pHIQ1ANtw8YjUSxd85/1WlwB/\nM2ul1RWgk+qoztoXLA9rAPCVjrrA6AuENQDboLMGAAMY/EUxhDUA+6h3mXsfIGENwDYM/nJzwhqA\nfbjFzBoA/J7L4KE1YQ3ANlx01gDg/xiDAIAB6glrAPB/rAYBAAMQ1gBgAGbWAGAAg5+QSlgDsA+W\n7gGAAeqtLqANCGsAtuEKoLMGAL9n8N3mhDUA+2DpHgAYgNUgAGAAbjcHAAPQWQOAAZhZA4ABWA0C\nAAZgDAIABmAMAgAGqKezBgD/R2cNAAYgrAHAAKwGAQADsBoEAAzAGAQADMCXDwCAARiDAIABGIMA\ngAFYDQIABnAZHNeENQDb4AIjABiAmTUAGIDVIABgAGbWAGAAc6OasAZgI8ysAcAA9Qb31oFWFwAA\nvuJqxas1jh49qpycHI0ePVoJCQnKyMhocr+CggJlZWUpKSlJ6enpWr58udfnIKwB2IZLbq9frXHw\n4EEVFBQoOjpasbGxTe6za9cuZWdna8iQIXrhhRc0ZswY5ebm6tVXX/XqHIxBANhGRw1Bhg8frvT0\ndEnSgw8+qD179njsk5eXp4SEBOXm5kqSUlJSdOLECeXl5WncuHEKDGy+dyasfeSd93bqpT+sUNGR\nEjkrKhQRFqprkhKU/W8/VOygaEnSj2fM0Qe7djf5+W8lX6v/XPSYL0uGhSIjB+jppx5R+i03KSAg\nQJs2b9VPZz+s0tKPrS7NaB11gbGloK2pqdH27ds1e/bsRtszMjL0+uuva+/evUpKSmr2GIS1j5Q7\nK5QwOE7jx4xSeFioTpw8pZeWv64f3HO//rR8qQZc3l//MXu6zp2vbPS5v+/5p5587nnd/O0UiyqH\nr3Xv3k0b33pdF2ouaPKU++R2u/XoL+borxtWaOi16aqsrLK6RGNZdYGxpKREtbW1HiOS+Ph4SVJx\ncTFh7S9G3vodjbz1O422JQ0ZrMwfTNWGt7fpxxPGNnTYX/TGm+sVHNxF301P81GlsNrdU36omJgo\nJVyZqqKiI5Kk3bv365/7tumeqXfp188+b22BBmvNLNrpdMrpdHpsdzgccjgcrTpveXl5w2e/fKwv\nvt8cLjBaKCy0lyQpKCioyferqqu1YfNWfedbyQp19PJlabBQZsYIvfdeYUNQS9KRI6V6990duj1z\nhHWFdQLuVrzy8/N1yy23eLzy8/Mtqb3dOuuPP/5Y77//vu644472OmSnVF9fL5fLpY8/+VTPLF2m\nPr3DPTruizYVvKvzlVUa/d103xYJSyUkXKHVb27w2L5330f63timl4TBO63prCdNmqSsrCyP7a3t\nqiUpNDRUkjw69Ys/X3y/Oe0W1rt379a8efMI6xZMmHq/9h04KEmKihygl37zK/UOD2ty39XrNyki\nPEzfTrnelyXCYhERYTp79qzH9rKyswoPb/mXGl+tNRcYL2Xc8VWioqIUHBys4uJipaamNmw/dOiQ\nJCkmJqbFYzAG8bEFOT/TH59/Rk8+Mlc9L+uhe+77uY6fOOmx36enTmv7Bx8qY8TN6tKl6TEJgNZx\nt+Kf9hQSEqKUlBStW7eu0fY1a9aob9++SkxMbPEYLXbWmZmZXhVz/vx5r/azu9h/iZIkXZX4DX07\n5Trd9r0f68Xlr+vhOTMb7bdmw2a5XC7dzgjEdsrKyhUW5vm3rfDwMJWVtXwhCl+to1aDVFVVqaCg\nQJJ0/PhxnTt3TuvXr5ckJSUlaeDAgZo+fbomTpyo+fPnKzMzU4WFhVqxYoVycnJaXPoneRHWxcXF\niouLU0JCQrP7HT9+XCdOnPDm3wv/x9Grp74+cIBKj3uunf3zX/6qwXEx+kZ8y389Queyb99HSky4\nwmN7wpB47d//kQUVdR4dtc769OnTmjVrVqNtF39esGCBxowZo6FDh2rJkiVatGiRVq1apX79+mne\nvHmaMGGCV+doMazj4+MVHR2tBQsWNLvfW2+9pR07dnh1UnzuszNlOlxSqowRNzfavmf/Ryo6UqI5\nM++xqDJY6c01G/TkE/+hQYOidPhwiSQpOjpSN954vX7+UPO/h2iey90xnXVkZKQOHDjQ4n5paWlK\nS7u0ZbgthvVVV12lrVu3enUwdwf9h+gM7p33qBKuiNMVcYPUs0cPHSk9ruX/9Sd1CQrSpPFjGu27\nev0mdQkK0qjbbv6Ko6Eze/GlV5Q97cda+d8vK+fhJ+V2u/WLR+aotPRjPf+C9w/+gSeTE6rFsL77\n7ru9+pMgLS1NmzZtapeiOqOrE7+htzZvVf5rK1VbW6fL+/XR9cOu0t13jdPAr/Vv2K+2rk7r/vo/\n+lbKtV+5SgSdW2VllW697ft6+qlHlL/sNwoICNDmt7fpp7Mf1vkv3eGK1jH5m2IC3Ba3w7WfFVt5\nevih7gNusroE+Km6muNt+vyEaO+XFr96dFWbztXeuN0cgG3UGdxZE9YAbKO910/7EmENwDb4DkYA\nMIDJK9YIawC2YfJqEMIagG2Y/O3mhDUA26CzBgADMLMGAAOwGgQADMA6awAwADNrADBAvdvcQQhh\nDcA2GIMAgAE66ssHfIGwBmAb5kY1YQ3ARrjACAAGIKwBwACsBgEAA7AaBAAMwLNBAMAAzKwBwAB0\n1gBggHqDn7tHWAOwDe5gBAADsBoEAAxAZw0ABqCzBgAD0FkDgAG43RwADMAYBAAM4KazBgD/x+3m\nAGAAbjcHAAPQWQOAAepdzKwBwO+xGgQADMDMGgAMwMwaAAxAZw0ABuACIwAYgDEIABiAMQgAGMDk\nR6QGWl0AAPiKuxX/tNaRI0c0ZcoUDR06VCkpKfrlL3+pqqqqdqudzhqAbXRUZ+10OvWjH/1IAwYM\n0LPPPqszZ85owYIFOnPmjJ555pl2OQdhDcA2XB30iNTXXntNTqdTq1atUkREhCQpKChIP/vZz5Sd\nna34+Pg2n4MxCADbcLvdXr9aY8uWLUpJSWkIakm67bbbFBISoi1btrRL7XTWAGyjNSHsdDrldDo9\ntjscDjkcjkbbioqKNHbs2EbbQkJCFBUVpeLi4ksr9kssD+vgPjFWlwA/U1dz3OoS0EnVtuL/reee\ne06LFy/22D5jxgzNnDmz0Tan0+kR4NLnwV5eXt76QptgeVgDgD+aNGmSsrKyPLY3Fcq+QFgDQBOa\nGnc0t29TIxOn06mYmPaZHnCBEQDaKDY2VkVFRY221dTUqKSkhLAGAH+Rmpqq7du3q6ysrGHbxo0b\nVVNTo7S0tHY5R4Db5JvlAcAPOJ1OZWRkaODAgcrOztbp06f1q1/9SjfccEO73RRDWANAOzh8+LAe\ne+wx7dy5U127dtWoUaP0wAMPqHv37u1yfMIaAAzAzBoADEBYA4ABCGuLdfRjFWGeo0ePKicnR6NH\nj1ZCQoIyMjKsLgl+gJtiLOSLxyrCPAcPHlRBQYGuvvpquVwuo7/dBO2HsLaQLx6rCPMMHz5c6enp\nkqQHH3xQe/bssbgi+APGIBbyxWMVYZ7AQH4t4Yn/KyxUVFSkuLi4Rtva+7GKADoHwtpCvnisIoDO\ngbAGAAMQ1hZq7rGKoaGhFlQEwF8R1hbyxWMVAXQOhLWFfPFYRQCdA+usLTR+/Hj94Q9/UHZ2dqPH\nKo4cOdJjlQjso6qqSgUFBZKk48eP69y5c1q/fr0kKSkpSQMHDrSyPFiEp+5ZrKMfqwjzHDt2TLfc\nckuT7y1YsEBjxozxcUXwB4Q1ABiAmTUAGICwBgADENYAYADCGgAMQFgDgAEIawAwAGENAAYgrAHA\nAIQ1ABjgfwGBxlMlnx4Q7AAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YFVzBJEj5sBO",
        "colab_type": "text"
      },
      "source": [
        "#Thank you for completing the notebook"
      ]
    }
  ]
}