keras/Keras_First_Neural_Net_101.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Keras First Neural Net 101",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/manishiitg/ML_Experments/blob/master/keras/Keras_First_Neural_Net_101.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VCwvU8keNlO5",
        "colab_type": "text"
      },
      "source": [
        "**This is a very simple neural network. This was the first NN developed in Keras by me. This is just pay around and learn keras**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Kdzi3iCfv8rv",
        "colab_type": "code",
        "outputId": "297167ae-6861-411c-f0a0-479360a20f25",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "from __future__ import print_function\n",
        "import random \n",
        "import numpy as np\n",
        "import keras\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Dense, Activation\n",
        "from keras.optimizers import Adam"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using TensorFlow backend.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rTyPDgo0OJ7n",
        "colab_type": "text"
      },
      "source": [
        "**Sequential** this means that we can different NN layers in sequence\n",
        " \n",
        "**Dense** is a standard full connected layer or called \"Feed Forward\" (FF) or \"Fully Connected\" FC\n",
        "\n",
        "**Activation** this is layer activation function\n",
        "\n",
        "**Adam** is an optmizer for the NN to optmize loss\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xBT7ClTpwBR-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "training_data = []\n",
        "training_labels = []\n",
        "\n",
        "\n",
        "# actual data\n",
        "# i.e no from 1 to 2000 denotes 1 which means MEN\n",
        "for i in range(1,2000):\n",
        "  training_data.append(i)\n",
        "  training_labels.append(1)\n",
        "\n",
        "# no from 2001 to 4000 denotes 0 which means WOMEN\n",
        "for i in range(2001,4000):\n",
        "  training_data.append(i)\n",
        "  training_labels.append(0)\n",
        "\n",
        "# let's add some noise or wrong data\n",
        "for i in range(0,100):\n",
        "  data = random.randint(1,2000)\n",
        "  training_data.append(data)\n",
        "  training_labels.append(0)\n",
        "\n",
        "for i in range(0,100):\n",
        "  data = random.randint(2001,4000)\n",
        "  training_data.append(data)\n",
        "  training_labels.append(1)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D8sOn6lgO56R",
        "colab_type": "text"
      },
      "source": [
        "Here, just generating a some numbers from 1,2000 which denote MEN. and 2001-4000 denote women. \n",
        "\n",
        "Then adding some noise later on.\n",
        "\n",
        "Purpose of the NN is to simply be able to identify numbers 1-2000 as MEN and numbers 2001-4000 as WOMEN. \n",
        "\n",
        "*Thats it!*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1P539r0DSp5d",
        "colab_type": "code",
        "outputId": "7a482844-9fc8-47bb-a39e-ddc8ae1dcc98",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 418
        }
      },
      "source": [
        "# creating our NN network\n",
        "\n",
        "model = Sequential()\n",
        "model.add(Dense(8, input_shape=(1,)))\n",
        "model.add(Activation('relu'))\n",
        "model.add(Dense(2))\n",
        "model.add(Activation(\"softmax\"))\n",
        "\n",
        "model.summary()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:66: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:541: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4432: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n",
            "\n",
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_1 (Dense)              (None, 8)                 16        \n",
            "_________________________________________________________________\n",
            "activation_1 (Activation)    (None, 8)                 0         \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 2)                 18        \n",
            "_________________________________________________________________\n",
            "activation_2 (Activation)    (None, 2)                 0         \n",
            "=================================================================\n",
            "Total params: 34\n",
            "Trainable params: 34\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "meAt_tRvPVEX",
        "colab_type": "text"
      },
      "source": [
        "Above, is a very simple Feed Forward NN without 2 layers. First layer has 8 nodes or (neurons) and second layer has 2. \n",
        "\n",
        "The second layer has 2 nodes because our output has two labels i.e MEN/WOMEN.\n",
        "\n",
        "The first layer uses \"relu\" as activation function, which is quite standard.\n",
        "\n",
        "The second layer uses \"softmax\" to predict the output. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c9NqGUkgS2fL",
        "colab_type": "code",
        "outputId": "2d2c1d0f-244f-41ed-853d-c2b393f10f4b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 106
        }
      },
      "source": [
        "model.compile(Adam(lr=.001),loss=\"sparse_categorical_crossentropy\",metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/optimizers.py:793: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3622: The name tf.log is deprecated. Please use tf.math.log instead.\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hAY77IHLP-9d",
        "colab_type": "text"
      },
      "source": [
        "Here we setup Adas as the optmizer with learning rate of .001 and loss function as \"sparse_categorical_crossentropy\". There are again standards there no specific reason to choose any over other avaiable methods"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GOLz6t2WQEWs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4aRu2ICQS7wE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# data preprosessing will see this in detail later\n",
        "\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "scalar = MinMaxScaler()\n",
        "scaled_train_samples = scalar.fit_transform(np.array(training_data).reshape(-1, 1))\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rs1XCsMBQNap",
        "colab_type": "text"
      },
      "source": [
        "This uses the skikit library mainly to scale values between 0 to 1 from 0,4000. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jOB9nxLMTQ0o",
        "colab_type": "code",
        "outputId": "36c32477-af62-43f2-dd1e-f40d5763eaa7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 851
        }
      },
      "source": [
        "model.fit(scaled_train_samples,np.array(training_labels), validation_split=.1 ,batch_size=20, epochs=20,verbose=2, shuffle=True)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:1250: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1033: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead.\n",
            "\n",
            "Train on 3778 samples, validate on 420 samples\n",
            "Epoch 1/20\n",
            " - 0s - loss: 0.6130 - acc: 0.4979 - val_loss: 0.5674 - val_acc: 0.7357\n",
            "Epoch 2/20\n",
            " - 0s - loss: 0.6009 - acc: 0.5564 - val_loss: 0.5696 - val_acc: 0.7262\n",
            "Epoch 3/20\n",
            " - 0s - loss: 0.5897 - acc: 0.6053 - val_loss: 0.5715 - val_acc: 0.7048\n",
            "Epoch 4/20\n",
            " - 0s - loss: 0.5789 - acc: 0.6443 - val_loss: 0.5732 - val_acc: 0.6786\n",
            "Epoch 5/20\n",
            " - 0s - loss: 0.5684 - acc: 0.6805 - val_loss: 0.5748 - val_acc: 0.6762\n",
            "Epoch 6/20\n",
            " - 0s - loss: 0.5581 - acc: 0.7099 - val_loss: 0.5763 - val_acc: 0.6643\n",
            "Epoch 7/20\n",
            " - 0s - loss: 0.5479 - acc: 0.7327 - val_loss: 0.5780 - val_acc: 0.6524\n",
            "Epoch 8/20\n",
            " - 0s - loss: 0.5378 - acc: 0.7560 - val_loss: 0.5799 - val_acc: 0.6500\n",
            "Epoch 9/20\n",
            " - 0s - loss: 0.5279 - acc: 0.7755 - val_loss: 0.5821 - val_acc: 0.6429\n",
            "Epoch 10/20\n",
            " - 0s - loss: 0.5182 - acc: 0.7896 - val_loss: 0.5845 - val_acc: 0.6405\n",
            "Epoch 11/20\n",
            " - 0s - loss: 0.5085 - acc: 0.8052 - val_loss: 0.5873 - val_acc: 0.6381\n",
            "Epoch 12/20\n",
            " - 0s - loss: 0.4990 - acc: 0.8176 - val_loss: 0.5904 - val_acc: 0.6262\n",
            "Epoch 13/20\n",
            " - 0s - loss: 0.4895 - acc: 0.8293 - val_loss: 0.5940 - val_acc: 0.6167\n",
            "Epoch 14/20\n",
            " - 0s - loss: 0.4802 - acc: 0.8404 - val_loss: 0.5980 - val_acc: 0.6119\n",
            "Epoch 15/20\n",
            " - 0s - loss: 0.4710 - acc: 0.8494 - val_loss: 0.6022 - val_acc: 0.6048\n",
            "Epoch 16/20\n",
            " - 0s - loss: 0.4619 - acc: 0.8573 - val_loss: 0.6071 - val_acc: 0.6000\n",
            "Epoch 17/20\n",
            " - 0s - loss: 0.4530 - acc: 0.8655 - val_loss: 0.6123 - val_acc: 0.5952\n",
            "Epoch 18/20\n",
            " - 0s - loss: 0.4443 - acc: 0.8714 - val_loss: 0.6177 - val_acc: 0.5905\n",
            "Epoch 19/20\n",
            " - 0s - loss: 0.4358 - acc: 0.8785 - val_loss: 0.6236 - val_acc: 0.5881\n",
            "Epoch 20/20\n",
            " - 0s - loss: 0.4274 - acc: 0.8835 - val_loss: 0.6296 - val_acc: 0.5810\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6be923cac8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3enbL_83Rd9X",
        "colab_type": "text"
      },
      "source": [
        "FIT basically means to \"train\" the neural network on the data and also in the same time, looking at loss, accuracy and validation loss/accuracy "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z0Hi0owIUop2",
        "colab_type": "code",
        "outputId": "ccddd8b6-f592-4e0d-99d0-71bdbc58b7e6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 987
        }
      },
      "source": [
        "control_data = np.random.randint(low = 1, high = 4000, size = 50)\n",
        "\n",
        "print(control_data)\n",
        "\n",
        "\n",
        "final_data = scalar.fit_transform(control_data.reshape(-1,1))\n",
        "\n",
        "\n",
        "predictions = model.predict_classes(final_data, batch_size=10)\n",
        "print(predictions)\n",
        "\n",
        "\n",
        "predictions = model.predict(final_data, batch_size=10)\n",
        "print(predictions)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[ 870 2712  465 3541 1495 3300 2715 2805 1032 1802 3066 3276 2952 1611\n",
            " 2256  453 1378 1906 1068 2956 3772 2486  584 2935 3050 2619 2737 3609\n",
            " 2088  547 2541 1838 2886 1326 2435 1882  522 3958 2252  755 3477 2081\n",
            " 3377 1940  682 2974 1593  352 3716 1350]\n",
            "[1 0 1 0 1 0 0 0 1 0 0 0 0 1 0 1 1 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 1\n",
            " 0 0 1 0 0 0 0 1 0 1 1 0 1]\n",
            "[[0.35593158 0.64406836]\n",
            " [0.72720337 0.27279666]\n",
            " [0.35593158 0.64406836]\n",
            " [0.86328363 0.1367164 ]\n",
            " [0.429109   0.57089096]\n",
            " [0.83091664 0.16908333]\n",
            " [0.72782195 0.272178  ]\n",
            " [0.7459678  0.25403222]\n",
            " [0.35593158 0.64406836]\n",
            " [0.50846547 0.49153456]\n",
            " [0.79392076 0.20607924]\n",
            " [0.8273802  0.17261988]\n",
            " [0.7738424  0.22615762]\n",
            " [0.45888805 0.541112  ]\n",
            " [0.6238993  0.3761007 ]\n",
            " [0.35593158 0.64406836]\n",
            " [0.3995849  0.60041505]\n",
            " [0.5354528  0.46454722]\n",
            " [0.35593158 0.64406836]\n",
            " [0.77456975 0.22543028]\n",
            " [0.8892526  0.11074743]\n",
            " [0.6781726  0.32182747]\n",
            " [0.35593158 0.64406836]\n",
            " [0.7707325  0.22926751]\n",
            " [0.79118437 0.20881571]\n",
            " [0.7075961  0.29240388]\n",
            " [0.7323318  0.26766822]\n",
            " [0.8714197  0.12858024]\n",
            " [0.5820933  0.41790673]\n",
            " [0.35593158 0.64406836]\n",
            " [0.6905302  0.30946988]\n",
            " [0.51782084 0.48217916]\n",
            " [0.7616016  0.2383984 ]\n",
            " [0.3866805  0.61331946]\n",
            " [0.66648597 0.33351406]\n",
            " [0.52923757 0.4707624 ]\n",
            " [0.35593158 0.64406836]\n",
            " [0.90692157 0.09307848]\n",
            " [0.6229225  0.37707755]\n",
            " [0.35593158 0.64406836]\n",
            " [0.8552343  0.14476562]\n",
            " [0.5803209  0.41967914]\n",
            " [0.8418737  0.15812628]\n",
            " [0.54423845 0.45576164]\n",
            " [0.35593158 0.64406836]\n",
            " [0.77782243 0.22217762]\n",
            " [0.45424232 0.54575765]\n",
            " [0.35593158 0.64406836]\n",
            " [0.8833842  0.11661572]\n",
            " [0.39261782 0.6073821 ]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5mnLwnHyRt4r",
        "colab_type": "text"
      },
      "source": [
        "Any finally once our network is trained, we are able to predict the data from the network and see the output. Output is a percentage based probablity distribution.\n",
        "\n",
        "**Cool! thats it for the first NN**"
      ]
    }
  ]
}