tutorial_timegan.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.4"
    },
    "colab": {
      "name": "tutorial_timegan.ipynb",
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/HarisNaveed17/TimeGAN/blob/master/tutorial_timegan.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vybrqja6MSqF",
        "outputId": "8bc026e5-cfca-4f28-cacd-4bbb2a25202d"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kwhLnyXRMU8P",
        "outputId": "dbe02c87-16a5-43f3-8cfc-d7bb2f3e795b"
      },
      "source": [
        "cd drive/MyDrive/FYP"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[Errno 2] No such file or directory: 'drive/MyDrive/FYP'\n",
            "/content/drive/MyDrive/FYP\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Aab0HnK2MZgm",
        "outputId": "7583cf71-b955-4ed8-8dce-ddc0194b8478"
      },
      "source": [
        "!git clone https://github.com/HarisNaveed17/TimeGAN.git"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'TimeGAN'...\n",
            "remote: Enumerating objects: 119, done.\u001b[K\n",
            "remote: Counting objects: 100% (119/119), done.\u001b[K\n",
            "remote: Compressing objects: 100% (74/74), done.\u001b[K\n",
            "remote: Total 163 (delta 63), reused 98 (delta 43), pack-reused 44\u001b[K\n",
            "Receiving objects: 100% (163/163), 2.08 MiB | 12.26 MiB/s, done.\n",
            "Resolving deltas: 100% (74/74), done.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8skAv2OzMhRh",
        "outputId": "89f6b616-a145-4bf0-d57f-bbefbf67d83c"
      },
      "source": [
        "cd TimeGAN"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/drive/MyDrive/FYP/TimeGAN\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0YmKsL1pOHAP",
        "outputId": "75fc5310-e387-4332-8279-81d7737cfb95",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "%tensorflow_version 1.x"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "TensorFlow is already loaded. Please restart the runtime to change versions.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GTxjib46L-3J"
      },
      "source": [
        "# TimeGAN Tutorial\n",
        "\n",
        "## Time-series Generative Adversarial Networks\n",
        "\n",
        "- Paper: Jinsung Yoon, Daniel Jarrett, Mihaela van der Schaar, \"Time-series Generative Adversarial Networks,\" Neural Information Processing Systems (NeurIPS), 2019.\n",
        "\n",
        "- Paper link: https://papers.nips.cc/paper/8789-time-series-generative-adversarial-networks\n",
        "\n",
        "- Last updated Date: April 24th 2020\n",
        "\n",
        "- Code author: Jinsung Yoon (jsyoon0823@gmail.com)\n",
        "\n",
        "This notebook describes the user-guide of a time-series synthetic data generation application using timeGAN framework. We use Stock, Energy, and Sine dataset as examples.\n",
        "\n",
        "### Prerequisite\n",
        "Clone https://github.com/jsyoon0823/timeGAN.git to the current directory."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xuXj1h_KL-3R"
      },
      "source": [
        "## Necessary packages and functions call\n",
        "\n",
        "- timegan: Synthetic time-series data generation module\n",
        "- data_loading: 2 real datasets and 1 synthetic datasets loading and preprocessing\n",
        "- metrics: \n",
        "    - discriminative_metrics: classify real data from synthetic data\n",
        "    - predictive_metrics: train on synthetic, test on real\n",
        "    - visualization: PCA and tSNE analyses"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "haa7kELbL-3U",
        "outputId": "14ef464a-82d5-49e7-959d-a1e3da710204"
      },
      "source": [
        "## Necessary packages\n",
        "from __future__ import absolute_import\n",
        "from __future__ import division\n",
        "from __future__ import print_function\n",
        "\n",
        "import numpy as np\n",
        "import warnings\n",
        "import pandas as pd\n",
        "warnings.filterwarnings(\"ignore\")\n",
        "\n",
        "# 1. TimeGAN model\n",
        "from timegan import timegan\n",
        "# 2. Data loading\n",
        "from data_loading import real_data_loading, sine_data_generation\n",
        "from utils import denormalizer\n",
        "# 3. Metrics\n",
        "from metrics.discriminative_metrics import discriminative_score_metrics\n",
        "from metrics.predictive_metrics import predictive_score_metrics\n",
        "from metrics.visualization_metrics import visualization"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Num GPUs Available:  1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zHVe90iiL-3U"
      },
      "source": [
        "## Data Loading\n",
        "\n",
        "Load original dataset and preprocess the loaded data.\n",
        "\n",
        "- data_name: stock, energy, or sine\n",
        "- seq_len: sequence length of the time-series data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0WGynKqML-3V",
        "outputId": "2f608251-8281-4168-e35e-86e244514a40"
      },
      "source": [
        "## Data loading\n",
        "data_name = 'telecomm'\n",
        "seq_len = 24\n",
        "\n",
        "if data_name in ['stock', 'energy','telecomm']:\n",
        "  ori_data, dat_min, dat_max = real_data_loading(data_name, seq_len)\n",
        "elif data_name == 'sine':\n",
        "  # Set number of samples and its dimensions\n",
        "  no, dim = 10000, 5\n",
        "  ori_data = sine_data_generation(no, seq_len, dim)\n",
        "    \n",
        "print(data_name + ' dataset is ready.')"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "telecomm dataset is ready.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kfS_whepL-3V"
      },
      "source": [
        "## Set network parameters\n",
        "\n",
        "TimeGAN network parameters should be optimized for different datasets.\n",
        "\n",
        "- module: gru, lstm, or lstmLN\n",
        "- hidden_dim: hidden dimensions\n",
        "- num_layer: number of layers\n",
        "- iteration: number of training iterations\n",
        "- batch_size: the number of samples in each batch"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7cCvCuqfL-3V"
      },
      "source": [
        "## Newtork parameters\n",
        "parameters = dict()\n",
        "\n",
        "parameters['module'] = 'gru' \n",
        "parameters['hidden_dim'] = 24\n",
        "parameters['num_layer'] = 3\n",
        "parameters['iterations'] = 5000\n",
        "parameters['batch_size'] = 64"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nH170h6XL-3W"
      },
      "source": [
        "## Run TimeGAN for synthetic time-series data generation\n",
        "\n",
        "TimeGAN uses the original data and network parameters to return the generated synthetic data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 326
        },
        "id": "y_wpT_KlL-3W",
        "outputId": "2f940054-d1be-4afa-d8d8-6b89ffb246fd"
      },
      "source": [
        "# Run TimeGAN\n",
        "generated_data = timegan(ori_data, parameters)   \n",
        "print('Finish Synthetic Data Generation')"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "error",
          "ename": "AttributeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-15-80dade3fc85f>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Run TimeGAN\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mgenerated_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtimegan\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mori_data\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Finish Synthetic Data Generation'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/content/drive/MyDrive/FYP/TimeGAN/timegan.py\u001b[0m in \u001b[0;36mtimegan\u001b[0;34m(ori_data, parameters)\u001b[0m\n\u001b[1;32m     38\u001b[0m   \"\"\"\n\u001b[1;32m     39\u001b[0m   \u001b[0;31m# Initialization on the Graph\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 40\u001b[0;31m   \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreset_default_graph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     41\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     42\u001b[0m   \u001b[0;31m# Basic Parameters\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mAttributeError\u001b[0m: module 'tensorflow' has no attribute 'reset_default_graph'"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RMXA5jvBL-3X"
      },
      "source": [
        "denorm_data = denormalizer(dat_max, dat_min, generated_data, columns=[['internet', 'tweets',\r\n",
        "                                                                            'coverage', 'conditions', 'From Milan',\r\n",
        "                                                                            'To Milan', 'strength', 'Days', 'Hours',\r\n",
        "                                                                            'dayofyear']])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HWvNbY0JNPAp"
      },
      "source": [
        "denorm_data.to_csv('internet4259_gputest.csv')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hwfvygiGL-3X"
      },
      "source": [
        "## Evaluate the generated data\n",
        "\n",
        "### 1. Discriminative score\n",
        "\n",
        "To evaluate the classification accuracy between original and synthetic data using post-hoc RNN network. The output is |classification accuracy - 0.5|.\n",
        "\n",
        "- metric_iteration: the number of iterations for metric computation."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CvfUtDPyL-3X"
      },
      "source": [
        "metric_iteration = 5\n",
        "\n",
        "discriminative_score = list()\n",
        "for _ in range(metric_iteration):\n",
        "  temp_disc = discriminative_score_metrics(ori_data, generated_data)\n",
        "  discriminative_score.append(temp_disc)\n",
        "\n",
        "print('Discriminative score: ' + str(np.round(np.mean(discriminative_score), 4)))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kIYo5oH3L-3Y"
      },
      "source": [
        "## Evaluate the generated data\n",
        "\n",
        "### 2. Predictive score\n",
        "\n",
        "To evaluate the prediction performance on train on synthetic, test on real setting. More specifically, we use Post-hoc RNN architecture to predict one-step ahead and report the performance in terms of MAE."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YN7kscbPL-3Y"
      },
      "source": [
        "predictive_score = list()\n",
        "for tt in range(metric_iteration):\n",
        "  temp_pred = predictive_score_metrics(ori_data, generated_data)\n",
        "  predictive_score.append(temp_pred)   \n",
        "    \n",
        "print('Predictive score: ' + str(np.round(np.mean(predictive_score), 4)))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ipiBIq9iL-3Y"
      },
      "source": [
        "## Evaluate the generated data\n",
        "\n",
        "### 3. Visualization\n",
        "\n",
        "We visualize the original and synthetic data distributions using PCA and tSNE analysis."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aNWjl35aL-3Y"
      },
      "source": [
        "visualization(ori_data, generated_data, 'pca')\n",
        "visualization(ori_data, generated_data, 'tsne')"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}