notebooks/membership_inference_label_only_decision_boundary_attack.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Rf_QO8flWx6_"
   },
   "source": [
    "# Membership Inference Label Only Decision-Boundary Attack Examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WKAYn2M4Wx7F"
   },
   "source": [
    "<table align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/privML/privacy-evaluator/blob/main/notebooks/membership_inference_label_only_decision_boundary_attack.ipynb\"><img src=\"https://raw.githubusercontent.com/privML/privacy-evaluator/main/notebooks/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://github.com/privML/privacy-evaluator/blob/main/notebooks/membership_inference_abel_only_decision_boundary_attack.ipynb\"><img src=\"https://raw.githubusercontent.com/privML/privacy-evaluator/main/notebooks/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ia0VmlzeWx7H"
   },
   "source": [
    "\n",
    "## Overview\n",
    "\n",
    "In this notebook, we want to show you how to use the `privacy-evaluator` tool to perform the Membership Inference Attacks Black Box Label Only Decision-Boundary Attack on both a provided PyTorch and a provided Tensorflow model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Enable Logging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "logging.basicConfig(level = logging.DEBUG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DH6rB9KSWx7J"
   },
   "source": [
    "## Setup\n",
    "\n",
    "First, you should set the notebook's runtime to use a GPU (e.g. if Colab is used go to ***Runtime > Change runtime type > Hardware accelerator***). Now we can install the `privacy-evaluator` package and import all needed modules."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lNU7L3GCWx7M",
    "outputId": "d22beea4-0303-4691-f169-a703679212e4"
   },
   "outputs": [],
   "source": [
    "!pip3 install git+https://github.com/privML/privacy-evaluator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9d7ab468"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import torch\n",
    "import numpy as np\n",
    "\n",
    "import tensorflow.python.ops.numpy_ops.np_config as np_config\n",
    "np_config.enable_numpy_behavior()\n",
    "\n",
    "import privacy_evaluator.models.torch.dcti.dcti as torch_dcti\n",
    "import privacy_evaluator.models.tf.dcti.dcti as tf_dcti \n",
    "\n",
    "from privacy_evaluator.datasets.tf.cifar10 import TFCIFAR10\n",
    "from privacy_evaluator.datasets.torch.cifar10 import TorchCIFAR10\n",
    "\n",
    "from privacy_evaluator.classifiers.classifier import Classifier\n",
    "\n",
    "from privacy_evaluator.attacks.membership_inference.label_only_decision_boundary import MembershipInferenceLabelOnlyDecisionBoundaryAttack\n",
    "from privacy_evaluator.attacks.membership_inference import MembershipInferenceAttackAnalysis\n",
    "\n",
    "from privacy_evaluator.attacks.membership_inference import MembershipInferenceAttackAnalysis\n",
    "\n",
    "from privacy_evaluator.attacks.membership_inference.data_structures.attack_input_data import AttackInputData\n",
    "from privacy_evaluator.attacks.membership_inference.data_structures.slicing import Slicing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mUH4YQ4tWx7Q"
   },
   "source": [
    "## Conduct Membership Inference Attacks\n",
    "\n",
    "Now we can start with conducting the Membership Inference Label Only Decision-Boundary Attacks. Therefore, we prepared two instances of the attack: one attacking a PyTorch model and attacking a TensorFlow model. For both attacks, we implemented a simple neural network trained on the CIFAR-10 dataset. For details about the provided network have a look at the following paper: https://www.scitepress.org/Papers/2018/67520/67520.pdf)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b2da0876"
   },
   "source": [
    "### PyTorch\n",
    "\n",
    "We start the evaluation of the PyTorch version of the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7aQIN6UMWx7W"
   },
   "source": [
    "#### Prepare target model\n",
    "\n",
    "Now, we need to initialize our pre-trained Lightweight Deep Convolutional Neural Network (short DCTI) as a generic `Classifier`. Therefore we need to specify the loss function used to train the model (in our case the `torch.nn.CrossEntropyLoss`), the number of classes and the input shape of our CIFAR-10 dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "fz4UuSxMWx7Y"
   },
   "outputs": [],
   "source": [
    "# Initalize PyTorch model as a Classifier\n",
    "target_model = Classifier(\n",
    "    torch_dcti.load_dcti(), # PyTorch DCTI \n",
    "    loss=torch.nn.CrossEntropyLoss(reduction=\"none\"), # Loss function of the PyTorch model\n",
    "    nb_classes=TorchCIFAR10.N_CLASSES, # Number of classes of the CIFAR10 dataset\n",
    "    input_shape=TorchCIFAR10.INPUT_SHAPE # Input shape of the CIFAR10 dataset\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sv_fGg8jWx7S"
   },
   "source": [
    "#### Load CIFAR10 Dataset\n",
    "\n",
    "Before we can start to conduct the membership inference attacks, we need to load the dataset. The CIFAR10 dataset needs to be preprocessed in a specific manner to work for the PyTorch model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 181,
     "referenced_widgets": [
      "43dcea0052a344e7964235becfada993",
      "93e35f58af654d3faa275847d09e4273",
      "1638393248744194a0be271eeac4c5e2",
      "df9ab6dd7720452d828c90561a7015ca",
      "f6d4e15c3e8a4afab0cbba23a173de83",
      "d3c4f999107a46d580ed4a3a629ae82a",
      "04eb0901586e488a8eef8d8c7b485ed8",
      "5019997b07ad406abc065edf7d41649b"
     ]
    },
    "id": "dcae11d8",
    "outputId": "dbeddf6c-9542-4964-d760-592e3971cfb9"
   },
   "outputs": [],
   "source": [
    "# Load CIFAR10 dataset as numpy array\n",
    "x_train, y_train, x_test, y_test = TorchCIFAR10.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ee10af5d"
   },
   "source": [
    "#### Perform Membership Inference Label Only Decision Boundary Attack\n",
    "\n",
    "Next, we want to perform the `MembershipInferenceLabelOnlyDecisionBoundaryAttack`. First, we initialize the attack with our respective `target_model`. But before we can attack our `target_model`, we need to fit the newly initialized attack model. Therefore, we call the `fit` method with the corresponding training dataset. The training dataset of the attack model consists of two different datasets. The first contains the data (`x_train`) and its corresponding labels (`y_train`) that were used to train the target model. The second contains the data (`x_test`) and its corresponding labels (`y_test`) that were not part of the training process of the target model. To optimize the attack model fitting, consider providing a suitable training set. The better the provided training dataset is, the better the attack model performs, and the better you can assess how well your target model is protected against `MembershipInferenceLabelOnlyDecisionBoundaryAttack`s. Additionally, to optimize the attack model, multiple parameters, such as `max_iter`, `max_eval` and `init_eval`, are provided. More precise information about which parameters can be used and how they are used can be found below or in the `fit` method documentation. Following these steps, we can now run the attack via the `attack` method. The `attack` method takes a datasets as parameter. This dataset contains the data (`x_train`) and its corresponding labels (`y_train`) which we want to attack. As a result, we get an array holding the inferred membership status for each data point, whereas 1 indicates a member and 0 indicates non-member.\n",
    "\n",
    "The following parameters can optionally be set, which is highly suggested, as the default values of the underlying ART framework may result in extremely long run times:\n",
    "\n",
    "* **max_iter:** Maximum number of iterations for HopSkipJump.\n",
    "* **max_eval:** Maximum number of evaluations for estimating gradient for HopSkipJump.\n",
    "* **init_eval:** Initial number of evaluations for estimating gradient for HopSkipJump.\n",
    "* **init_size:** Maximum number of trials for initial generation of adversarial examples for HopSkipJump.\n",
    "\n",
    "(For more details about the HopSkipJump parameters, please read the following paper: https://arxiv.org/pdf/1904.02144.pdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 245,
     "referenced_widgets": [
      "2e13ff6e78114d358ed2ac4e293b0df5",
      "0dd19a7aafb8407d86cee7cb7332928d",
      "6a13201b387445fcbf48f1197cd63d4f",
      "0b081b61e4884485b507960efa27ad2c",
      "ad0a09fe598841a5a7ba7a5a121d449d",
      "d2ef9ac5009346e2a3e37341f63dc460",
      "e359ca8a35124cf183a7f583ef846aed",
      "dc405031fd474035bb91ada99441a89f",
      "2ed121c1a5fd4c0a9ed226cae0667f5a",
      "7e9b8f3191c845349985959ab2d1a473",
      "a57b9ce5cb7f4eedadc21f38074a9d96",
      "1ad93432843e499d863fb0b5ffc7c44b",
      "8fc4c59bcbef47369d63f4d35fde6f4a",
      "db2c6484fe0f4115a4f764b2f5b51976",
      "91175da0615744649ce8c39d59f37ecd",
      "a55bb1bebf2847fda4ac6c0673b67d51",
      "7c923b12d7364f1eaa310b8960e2e841",
      "41fe6d3d924d47c9a0fae7caa3bf434e",
      "1cb69a016a90495e93fef1b409aab8bc",
      "da73c3f5610749e0b0813dd562d55801",
      "d5457dc3a67d42718946b4eeff47289c",
      "cca5563026ee4aec90e865bd9699cc59",
      "28fff2a46c2348efb2c96a74a0f2b4dc",
      "5f5cf08ed09c461f8665296bc08c0d95"
     ]
    },
    "id": "73278928",
    "outputId": "d5aaccb2-9c7c-462f-eae9-01d3c72b331d"
   },
   "outputs": [],
   "source": [
    "attack = MembershipInferenceLabelOnlyDecisionBoundaryAttack(target_model)\n",
    "attack.fit(x_train[:100], y_train[:100], x_test[:20], y_test[:20], max_iter=5, max_eval=5, init_eval=5)\n",
    "attack.attack(x_train[:20], y_train[:20])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0e0b5639"
   },
   "source": [
    "#### Get human-readable attack statistics\n",
    "\n",
    "For this attack we want a human-readable output, thus we convert it to a sting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 87,
     "referenced_widgets": [
      "f9a5818caf31424a9cd302a647602241",
      "d255b3cf6f3b46a29085c21db11b1056",
      "788d4b8604674525a762d22b535a9b48",
      "feb97483de0647d58db0c1b6fcadc48d",
      "9fe3e52ad03b40ac9b41de471407d44e",
      "29fd72c6528f45a1af87af5d70a4ca3b",
      "e2fe2f67d8de425f8e7ca6029816747d",
      "cf55222977e94299a39d4632bee76e14"
     ]
    },
    "id": "ec3551f5",
    "outputId": "1189ecfa-57c6-4975-e08b-662d7e82c161"
   },
   "outputs": [],
   "source": [
    "output = attack.attack_output(\n",
    "    x_train[:20], \n",
    "    y_train[:20], \n",
    "    x_train, \n",
    "    y_train, \n",
    "    x_test, \n",
    "    y_test,\n",
    "    np.ones((20,))\n",
    ")\n",
    "\n",
    "str(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VcyzBkd39Y-a"
   },
   "source": [
    "#### Explanation of the outcome:\n",
    "\n",
    "##### Attack Model Accuracy:\n",
    "The attack model accuracy specifies how well the membership attack model performs in predicting if a given data point was used for training the target model. Since we have a two-class classification problem that the attack model solves (member or non-member), the baseline accuracy is 50% (random guessing for each sample). The best accuracy is at 100% if the model predicts every data point it sees right as member or non-member.\n",
    "\n",
    "##### Train-Test-Gap (difference):\n",
    "If your model has a train-test-gap larger than 5%, this could be a sign that your model overfits. Overfitting can be beneficial for successful membership inference attacks [1]. Therefore, you might want to reduce it by introducing regularization methods in your training, or using specific privacy methods [2,3], such as Differential Privacy [4].\n",
    "\n",
    "(For References, please see last box)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wg91lAQ5Kqru"
   },
   "source": [
    "#### Perpare attack analysis\n",
    "\n",
    "Next, we prepare our attack analysis. To initialize our attack analysis we define the Membership Inference Attack method we want to perform (in this case we use the `MembershipInferenceLabelOnlyDecisionBoundaryAttack`) and the Attack Input Data. The Attack Input Data consists of two different sets. The first contains the data (`x_train`) and its corresponding labels (`y_train`) which were used to train the target model. The second contains the data (`x_test`) and its corresponding labels (`y_test`) which were not part of the training process of the target model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cIFO6E8JLKZq"
   },
   "outputs": [],
   "source": [
    "attack_analysis = MembershipInferenceAttackAnalysis(\n",
    "    MembershipInferenceLabelOnlyDecisionBoundaryAttack, \n",
    "    AttackInputData(\n",
    "        x_train[:100], \n",
    "        y_train[:100], \n",
    "        x_test[:20], \n",
    "        y_test[:20]\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HNAKAwE1LVrK"
   },
   "source": [
    "#### Define the slicing\n",
    "\n",
    "Now we can define the slicing for our analysis. The slicing defines how the data will be sliced. Each slice will then be analysed by the attack separately, so see if the inference works better on certain parts of data than on others.\n",
    "\n",
    "The types of slices can be chosen:\n",
    "##### entire_dataset:\n",
    "This options allows to pick the whole dataset as a single slice.\n",
    "##### by_class:\n",
    "If this option is chosen, the analysis will be done for each class individually, resulting in as many slices as there are classes.\n",
    "##### by_classification_correctness:\n",
    "If this option is chosen, two slices will be analyzed. The first slice is the set of all correctly classified samples, and the second slice is the set of all incorrectly classified samples.\n",
    "\n",
    "If all three choices are selected, a total of 3 + n slices will be analyzed, where n is the amount of classes of the dataset.\n",
    "\n",
    "For further information, please see the explanation of the outcome below the analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "KTnov_yTLZ0U"
   },
   "outputs": [],
   "source": [
    "slicing = Slicing(\n",
    "    entire_dataset=True, \n",
    "    by_class=True, \n",
    "    by_classification_correctness=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xiyv0m64Lbcu"
   },
   "source": [
    "#### Perform Membership Inference Attack Analysis\n",
    "\n",
    "Finally, we can perform our Membership Inference Attack Analysis. Therefore, we input the target model, the data that should be analysed, the membership labels (i.e. the labels which correctly describe if a data point is a member of the training dataset or not) and the splicing specification into the `analyse()` method. As a result, we get for each slice the indices of the corresponding data points, a human-readable description of the slice and the advantage score of the Membership Inference Attack (for more details about the advantage score, please read the following paper: https://arxiv.org/abs/1709.01604)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For the advantage score, we need to use data points from both train and test sets.\n",
    "# Otherwise the advantage score cannot be calculated correctly if all data samples are\n",
    "# from the same class (member vs non-member).\n",
    "x = np.concatenate((x_train[:100], x_test[:100]))\n",
    "y = np.concatenate((y_train[:100], y_test[:100]))\n",
    "membership = np.concatenate((np.ones(len(x_train[:100])), np.zeros(len(x_test[:100]))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 150,
     "referenced_widgets": [
      "3cd78a3aa77c4f8baec926c6d4249e84",
      "24c4d293f4ea4716a92c2906671ee28c",
      "11a58548e6e24dfbad59b64349f26e28",
      "db277d05ad3441d4ae66778d56a447d4",
      "7aec12bbe11e483d97c84d13094b4f9e",
      "2d4b00f813b049b49f554f2481964c46",
      "c339f5d42b204b53b9ed785caa0b8849",
      "0c849c1b8c9b42fb8712b48fbd53ec4f",
      "5ce52950b0254bdfb4d584877b601cf8",
      "ff907b52b209496c803f14c2c582dc27",
      "2a3aaecfbf164615bf28afb7d1052b38",
      "8eb569df3b5140dbb7274ec64c1199a1",
      "6129113a083143b1b2584c7ce9b66b89",
      "fd7259d59da14068a5ee2b31692c840e",
      "ebf23f27e6c046059dcee10708750d23",
      "33f23656dfa44165861b49224e342954",
      "fba8b82ad964466e80f2dea1ff357ae5",
      "658eb2ea46e748ec9cc0bd7a4ca669a7",
      "4706064b00614560b6a7f162102df5fe",
      "147c3a4727b042e39223a22162850114",
      "90d123a12b2a4ac19d0ad894cfbea3ad",
      "b0fefcaa018b49088e9e589a23b399d6",
      "eb4d1e07ab564739afc0c9da989be524",
      "aa1d0c46cc34457885b030925dda239a"
     ]
    },
    "id": "3d0wiDYQLegJ",
    "outputId": "298c5c7a-3b25-4d6d-89c0-aca785b59fa9"
   },
   "outputs": [],
   "source": [
    "result = attack_analysis.analyse(\n",
    "    target_model, \n",
    "    x, \n",
    "    y, \n",
    "    membership, \n",
    "    slicing,\n",
    "    max_iter=5, max_eval=5, init_eval=5\n",
    ")\n",
    "\n",
    "print(\"\\n\".join((str(r) for r in result)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I_f_Hk7hLt3v"
   },
   "source": [
    "#### Explanation of the outcome:\n",
    "##### Attacker Advantage:\n",
    "The attacker advantageis a score that relies on comparing the model output on member and non-member data points. The model outputs are probability values over all classes, and they are often different on member and non-member data points. Usually, the model is more confident on member data points, because it has seen them during training. When trying to find a threshold value to tell apart member and non-member samples by their different model outputs, the attacker has interest in finding the best ratio between false positives “fpr” (non-members that are classified as members) and true positives “tpr” (members that are correctly identifies as members). \n",
    "\n",
    "This best ratio is calculated as the max(tpr-fpr) over all threshold values and represents the attacker advantage. \n",
    "\n",
    "##### Slicing: Incorrectly classified:\n",
    "It is normal that the attacker is more successful to deduce membership on incorrectly classified samples than on correctly classified ones. This results from the fact, that model predictions are often better on training than on test data points, whereby your attack model might learn to predict incorrectly classified samples as non-members. If your model overfits the training data, this assumption might hold true often enough to make the attack seem more successful on this slice. If you wish to reduce that, pay attention to reducing your model’s overfitting.\n",
    "\n",
    "##### Slicing: Specific classes more vulnerable: \n",
    "It seems that the membership inference attack is more successful on your class X than on the other classes. Research has shown that the class distribution (and also the distribution of data points within one class) are factors that influence the vulnerability of a class for membership inference attacks [5].\n",
    "\n",
    "Also, small classes (belonging to minority groups) can be more prone to membership inference attacks [6]. One reason for this could be, that there is less data for that class, and therefore, the model overfits within this class. It might make sense to look into the vulnerable classes of your model again, and maybe add more data to them, use private synthetic data, or introduce privacy methods like Differential Privacy [6]. Attention, the use of Differential Privacy could have a negative influence on the performance of your model for the minority classes.\n",
    "\n",
    "\n",
    "(For References, please see last box)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xG4nYKGzWx7i"
   },
   "source": [
    "### TensorFlow\n",
    "\n",
    "Now we do the same with the TensorFlow model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note: to force run on CPU uncomment the line below\n",
    "# os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_7ty54AKWx7k"
   },
   "source": [
    "#### Prepare target model\n",
    "\n",
    "Now, we need to initialize our pre-trained Lightweight Deep Convolutional Neural Network (short DCTI) as a generic `Classifier`. Therefore we need to specify the loss function used to train the model (in our case the `tf.keras.losses.CategoricalCrossentropy`), the number of classes and the input shape of our CIFAR-10 dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Uhrn5znOWx7k"
   },
   "outputs": [],
   "source": [
    "# Initalize TensorFlow target model\n",
    "target_model = Classifier(\n",
    "    tf_dcti.load_dcti(), # TensorFlow DCTI\n",
    "    loss=tf.keras.losses.CategoricalCrossentropy(), # Loss function of the TensorFlow target model\n",
    "    nb_classes=TFCIFAR10.N_CLASSES, # Number of classes of the CIFAR10 dataset\n",
    "    input_shape=TFCIFAR10.INPUT_SHAPE # Input shape of the CIFAR10 dataset\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UBfeudg_Wx7i"
   },
   "source": [
    "#### Load CIFAR10 Dataset\n",
    "\n",
    "Again, before we can start to conduct the membership inference attacks, we need to load the dataset. The CIFAR10 dataset needs to be preprocessed in a specific manner to work for the TensorFlow model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "kAJgfSgoWx7j"
   },
   "outputs": [],
   "source": [
    "# Load CIFAR10 dataset as numpy array\n",
    "x_train, y_train, x_test, y_test = TFCIFAR10.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pCzLLSreWx7q"
   },
   "source": [
    "#### Perform Membership Inference Label Only Decision Boundary Attack\n",
    "\n",
    "Next, we want to perform the `MembershipInferenceLabelOnlyDecisionBoundaryAttack`. First, we initialize the attack with our respective `target_model`. But before we can attack our `target_model`, we need to fit the newly initialized attack model. Therefore, we call the `fit` method with the corresponding training dataset. The training dataset of the attack model consists of two different datasets. The first contains the data (`x_train`) and its corresponding labels (`y_train`) that were used to train the target model. The second contains the data (`x_test`) and its corresponding labels (`y_test`) that were not part of the training process of the target model. To optimize the attack model fitting, consider providing a suitable training set. The better the provided training dataset is, the better the attack model performs, and the better you can assess how well your target model is protected against `MembershipInferenceLabelOnlyDecisionBoundaryAttack`s. Additionally, to optimize the attack model, multiple parameters, such as `max_iter`, `max_eval` and `init_eval`, are provided. More precise information about which parameters can be used and how they are used can be found below or in the `fit` method documentation. Following these steps, we can now run the attack via the `attack` method. The `attack` method takes a datasets as parameter. This dataset contains the data (`x_train`) and its corresponding labels (`y_train`) which we want to attack. As a result, we get an array holding the inferred membership status for each data point, whereas 1 indicates a member and 0 indicates non-member.\n",
    "\n",
    "\n",
    "The following parameters can optionally be set, which is highly suggested, as the default values of the underlying ART framework may result in extremely long run times:\n",
    "\n",
    "* **max_iter:** Maximum number of iterations for HopSkipJump.\n",
    "* **max_eval:** Maximum number of evaluations for estimating gradient for HopSkipJump.\n",
    "* **init_eval:** Initial number of evaluations for estimating gradient for HopSkipJump.\n",
    "* **init_size:** Maximum number of trials for initial generation of adversarial examples for HopSkipJump.\n",
    "\n",
    "(For more details about the HopSkipJump parameters, please read the following paper: https://arxiv.org/pdf/1904.02144.pdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "K9qarWJSWx7r"
   },
   "outputs": [],
   "source": [
    "attack = MembershipInferenceLabelOnlyDecisionBoundaryAttack(target_model)\n",
    "attack.fit(x_train[:100], y_train[:100], x_test[:20], y_test[:20], max_iter=5, max_eval=5, init_eval=5)\n",
    "attack.attack(x_train[:20], y_train[:20])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0I8WspzkWx7r"
   },
   "source": [
    "#### Get human-readable attack statistics\n",
    "\n",
    "For this attack we want a human-readable output, thus we convert it to a sting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8nR1lNg7Wx7s"
   },
   "outputs": [],
   "source": [
    "output = attack.attack_output(\n",
    "    x_train[:20], \n",
    "    y_train[:20], \n",
    "    x_train, \n",
    "    y_train, \n",
    "    x_test, \n",
    "    y_test,\n",
    "    np.ones((20,))\n",
    ")\n",
    "\n",
    "str(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WGG22i6w8Fzf"
   },
   "source": [
    "#### Explanation of the outcome:\n",
    "\n",
    "##### Attack Model Accuracy:\n",
    "The attack model accuracy specifies how well the membership attack model performs in predicting if a given data point was used for training the target model. Since we have a two-class classification problem that the attack model solves (member or non-member), the baseline accuracy is 50% (random guessing for each sample). The best accuracy is at 100% if the model predicts every data point it sees right as member or non-member.\n",
    "\n",
    "##### Train-Test-Gap (difference):\n",
    "If your model has a train-test-gap larger than 5%, this could be a sign that your model overfits. Overfitting can be beneficial for successful membership inference attacks [1]. Therefore, you might want to reduce it by introducing regularization methods in your training, or using specific privacy methods [2,3], such as Differential Privacy [4].\n",
    "\n",
    "(For References, please see last box)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NR2b6KWDMfGp"
   },
   "source": [
    "#### Perpare attack analysis\n",
    "\n",
    "Next, we prepare our attack analysis. To initialize our attack analysis we define the Membership Inference Attack method we want to perform (in this case we use the `MembershipInferenceLabelOnlyDecisionBoundaryAttack`) and the Attack Input Data. The Attack Input Data consists of two different sets. The first contains the data (`x_train`) and its corresponding labels (`y_train`) which were used to train the target model. The second contains the data (`x_test`) and its corresponding labels (`y_test`) which were not part of the training process of the target model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "hnl0pkYAMhtH"
   },
   "outputs": [],
   "source": [
    "analysis = MembershipInferenceAttackAnalysis(\n",
    "    MembershipInferenceLabelOnlyDecisionBoundaryAttack, \n",
    "    AttackInputData(\n",
    "        x_train[:100], \n",
    "        y_train[:100], \n",
    "        x_test[:100], \n",
    "        y_test[:100]\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Cy-3bAKZMmwC"
   },
   "source": [
    "#### Define the slicing\n",
    "\n",
    "Now we can define the slicing for our analysis. The slicing defines how the data will be sliced. Each slice will then be analysed by the attack separately, so see if the inference works better on certain parts of data than on others.\n",
    "\n",
    "The types of slices can be chosen:\n",
    "##### entire_dataset:\n",
    "This options allows to pick the whole dataset as a single slice.\n",
    "##### by_class:\n",
    "If this option is chosen, the analysis will be done for each class individually, resulting in as many slices as there are classes.\n",
    "##### by_classification_correctness:\n",
    "If this option is chosen, two slices will be analyzed. The first slice is the set of all correctly classified samples, and the second slice is the set of all incorrectly classified samples.\n",
    "\n",
    "If all three choices are selected, a total of 3 + n slices will be analyzed, where n is the amount of classes of the dataset.\n",
    "\n",
    "For further information, please see the explanation of the outcome below the analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-M7Dm5YEMnya"
   },
   "outputs": [],
   "source": [
    "slicing = Slicing(\n",
    "    entire_dataset=True, \n",
    "    by_class=True, \n",
    "    by_classification_correctness=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iJvZGh0NMow9"
   },
   "source": [
    "#### Perform Membership Inference Attack Analysis\n",
    "\n",
    "Finally, we can perform our Membership Inference Attack Analysis. Therefore, we input the target model, the data that should be analysed, the membership labels (i.e. the labels which correctly describe if a data point is a member of the training dataset or not) and the splicing specification into the `analyse()` method. As a result, we get for each slice the indices of the corresponding data points, a human-readable description of the slice and the advantage score of the Membership Inference Attack (for more details about the advantage score, please read the following paper: https://arxiv.org/abs/1709.01604)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9J1rEkl-MquD"
   },
   "outputs": [],
   "source": [
    "result = analysis.analyse(\n",
    "    target_model, \n",
    "    np.concatenate((x_train[:100], x_test[:100])), \n",
    "    np.concatenate((y_train[:100], y_test[:100])), \n",
    "    np.concatenate((np.ones(len(x_train[:100])), np.zeros(len(x_test[:100])))), \n",
    "    slicing\n",
    ")\n",
    "\n",
    "print(\"\\n\".join((str(r) for r in result)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qGVKO2IwMqW5"
   },
   "source": [
    "#### Explanation of the outcome:\n",
    "##### Attacker Advantage:\n",
    "The attacker advantageis a score that relies on comparing the model output on member and non-member data points. The model outputs are probability values over all classes, and they are often different on member and non-member data points. Usually, the model is more confident on member data points, because it has seen them during training. When trying to find a threshold value to tell apart member and non-member samples by their different model outputs, the attacker has interest in finding the best ratio between false positives “fpr” (non-members that are classified as members) and true positives “tpr” (members that are correctly identifies as members). \n",
    "\n",
    "This best ratio is calculated as the max(tpr-fpr) over all threshold values and represents the attacker advantage. \n",
    "\n",
    "##### Slicing: Incorrectly classified:\n",
    "It is normal that the attacker is more successful to deduce membership on incorrectly classified samples than on correctly classified ones. This results from the fact, that model predictions are often better on training than on test data points, whereby your attack model might learn to predict incorrectly classified samples as non-members. If your model overfits the training data, this assumption might hold true often enough to make the attack seem more successful on this slice. If you wish to reduce that, pay attention to reducing your model’s overfitting.\n",
    "\n",
    "##### Slicing: Specific classes more vulnerable: \n",
    "It seems that the membership inference attack is more successful on your class X than on the other classes. Research has shown that the class distribution (and also the distribution of data points within one class) are factors that influence the vulnerability of a class for membership inference attacks [5].\n",
    "\n",
    "Also, small classes (belonging to minority groups) can be more prone to membership inference attacks [6]. One reason for this could be, that there is less data for that class, and therefore, the model overfits within this class. It might make sense to look into the vulnerable classes of your model again, and maybe add more data to them, use private synthetic data, or introduce privacy methods like Differential Privacy [6]. Attention, the use of Differential Privacy could have a negative influence on the performance of your model for the minority classes.\n",
    "\n",
    "\n",
    "(For References, please see last box)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9j73m8k-9dDZ"
   },
   "source": [
    "[1]S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha. \\Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting\". In: 2018 IEEE 31st Computer Security Foundations Symposium (CSF). July 2018, pp. 268{282. doi:10.1109/CSF.2018.00027.\n",
    "\n",
    "[2] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Mem-bership Inference Attacks Against Machine Learning Models. In2017 IEEE Sym-posium on Security and Privacy (SP). 3–18.\n",
    "\n",
    "[3] Milad Nasr, Reza Shokri, and Amir Houmansadr. 2018. Machine Learning withMembership Privacy Using Adversarial Regularization. InProceedings of the 2018ACM SIGSAC Conference on Computer and Communications Security(Toronto,Canada)(CCS ’18). Association for Computing Machinery, New York, NY, USA,634–64\n",
    "\n",
    "[4] Cynthia Dwork. 2006.  Differential Privacy. InAutomata, Languages and Pro-gramming, Michele Bugliesi, Bart Preneel, Vladimiro Sassone, and Ingo Wegener(Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg\n",
    "\n",
    "[5] Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei. 2019.Demystifying Membership Inference Attacks in Machine Learning as a Service.IEEE Transactions on Services Computing(2019)\n",
    "\n",
    "[6] Suriyakumar, Vinith M., Nicolas Papernot, Anna Goldenberg, and Marzyeh Ghassemi. \"Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings.\" In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 723-734. 2021."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "membership_inference_abel_only_decision_boundary_attack.ipynb",
   "provenance": []
  },
  "interpreter": {
   "hash": "bdaae9e836e6796a4363c97979c836477756ef9f67b8698af83fef8b5f4ebf2e"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "04eb0901586e488a8eef8d8c7b485ed8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0b081b61e4884485b507960efa27ad2c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_dc405031fd474035bb91ada99441a89f",
      "placeholder": "​",
      "style": "IPY_MODEL_e359ca8a35124cf183a7f583ef846aed",
      "value": " 100/100 [00:20&lt;00:00,  4.80it/s]"
     }
    },
    "0c849c1b8c9b42fb8712b48fbd53ec4f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0dd19a7aafb8407d86cee7cb7332928d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "11a58548e6e24dfbad59b64349f26e28": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "HopSkipJump: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2d4b00f813b049b49f554f2481964c46",
      "max": 100,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7aec12bbe11e483d97c84d13094b4f9e",
      "value": 100
     }
    },
    "147c3a4727b042e39223a22162850114": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_aa1d0c46cc34457885b030925dda239a",
      "placeholder": "​",
      "style": "IPY_MODEL_eb4d1e07ab564739afc0c9da989be524",
      "value": " 124/200 [16:15&lt;09:45,  7.71s/it]"
     }
    },
    "1638393248744194a0be271eeac4c5e2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d3c4f999107a46d580ed4a3a629ae82a",
      "max": 170498071,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f6d4e15c3e8a4afab0cbba23a173de83",
      "value": 170498071
     }
    },
    "1ad93432843e499d863fb0b5ffc7c44b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a55bb1bebf2847fda4ac6c0673b67d51",
      "placeholder": "​",
      "style": "IPY_MODEL_91175da0615744649ce8c39d59f37ecd",
      "value": " 20/20 [00:04&lt;00:00,  4.24it/s]"
     }
    },
    "1cb69a016a90495e93fef1b409aab8bc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "HopSkipJump: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cca5563026ee4aec90e865bd9699cc59",
      "max": 20,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d5457dc3a67d42718946b4eeff47289c",
      "value": 20
     }
    },
    "24c4d293f4ea4716a92c2906671ee28c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "28fff2a46c2348efb2c96a74a0f2b4dc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "29fd72c6528f45a1af87af5d70a4ca3b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2a3aaecfbf164615bf28afb7d1052b38": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "HopSkipJump: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_fd7259d59da14068a5ee2b31692c840e",
      "max": 20,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_6129113a083143b1b2584c7ce9b66b89",
      "value": 20
     }
    },
    "2d4b00f813b049b49f554f2481964c46": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2e13ff6e78114d358ed2ac4e293b0df5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_6a13201b387445fcbf48f1197cd63d4f",
       "IPY_MODEL_0b081b61e4884485b507960efa27ad2c"
      ],
      "layout": "IPY_MODEL_0dd19a7aafb8407d86cee7cb7332928d"
     }
    },
    "2ed121c1a5fd4c0a9ed226cae0667f5a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a57b9ce5cb7f4eedadc21f38074a9d96",
       "IPY_MODEL_1ad93432843e499d863fb0b5ffc7c44b"
      ],
      "layout": "IPY_MODEL_7e9b8f3191c845349985959ab2d1a473"
     }
    },
    "33f23656dfa44165861b49224e342954": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3cd78a3aa77c4f8baec926c6d4249e84": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_11a58548e6e24dfbad59b64349f26e28",
       "IPY_MODEL_db277d05ad3441d4ae66778d56a447d4"
      ],
      "layout": "IPY_MODEL_24c4d293f4ea4716a92c2906671ee28c"
     }
    },
    "41fe6d3d924d47c9a0fae7caa3bf434e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "43dcea0052a344e7964235becfada993": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_1638393248744194a0be271eeac4c5e2",
       "IPY_MODEL_df9ab6dd7720452d828c90561a7015ca"
      ],
      "layout": "IPY_MODEL_93e35f58af654d3faa275847d09e4273"
     }
    },
    "4706064b00614560b6a7f162102df5fe": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "",
      "description": "HopSkipJump:  62%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b0fefcaa018b49088e9e589a23b399d6",
      "max": 200,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_90d123a12b2a4ac19d0ad894cfbea3ad",
      "value": 124
     }
    },
    "5019997b07ad406abc065edf7d41649b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5ce52950b0254bdfb4d584877b601cf8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2a3aaecfbf164615bf28afb7d1052b38",
       "IPY_MODEL_8eb569df3b5140dbb7274ec64c1199a1"
      ],
      "layout": "IPY_MODEL_ff907b52b209496c803f14c2c582dc27"
     }
    },
    "5f5cf08ed09c461f8665296bc08c0d95": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6129113a083143b1b2584c7ce9b66b89": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "658eb2ea46e748ec9cc0bd7a4ca669a7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6a13201b387445fcbf48f1197cd63d4f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "HopSkipJump: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d2ef9ac5009346e2a3e37341f63dc460",
      "max": 100,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_ad0a09fe598841a5a7ba7a5a121d449d",
      "value": 100
     }
    },
    "788d4b8604674525a762d22b535a9b48": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "HopSkipJump: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_29fd72c6528f45a1af87af5d70a4ca3b",
      "max": 20,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9fe3e52ad03b40ac9b41de471407d44e",
      "value": 20
     }
    },
    "7aec12bbe11e483d97c84d13094b4f9e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "7c923b12d7364f1eaa310b8960e2e841": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_1cb69a016a90495e93fef1b409aab8bc",
       "IPY_MODEL_da73c3f5610749e0b0813dd562d55801"
      ],
      "layout": "IPY_MODEL_41fe6d3d924d47c9a0fae7caa3bf434e"
     }
    },
    "7e9b8f3191c845349985959ab2d1a473": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8eb569df3b5140dbb7274ec64c1199a1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_33f23656dfa44165861b49224e342954",
      "placeholder": "​",
      "style": "IPY_MODEL_ebf23f27e6c046059dcee10708750d23",
      "value": " 20/20 [04:24&lt;00:00, 13.21s/it]"
     }
    },
    "8fc4c59bcbef47369d63f4d35fde6f4a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "90d123a12b2a4ac19d0ad894cfbea3ad": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "91175da0615744649ce8c39d59f37ecd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "93e35f58af654d3faa275847d09e4273": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9fe3e52ad03b40ac9b41de471407d44e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "a55bb1bebf2847fda4ac6c0673b67d51": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a57b9ce5cb7f4eedadc21f38074a9d96": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "HopSkipJump: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_db2c6484fe0f4115a4f764b2f5b51976",
      "max": 20,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8fc4c59bcbef47369d63f4d35fde6f4a",
      "value": 20
     }
    },
    "aa1d0c46cc34457885b030925dda239a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ad0a09fe598841a5a7ba7a5a121d449d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "b0fefcaa018b49088e9e589a23b399d6": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c339f5d42b204b53b9ed785caa0b8849": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cca5563026ee4aec90e865bd9699cc59": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cf55222977e94299a39d4632bee76e14": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d255b3cf6f3b46a29085c21db11b1056": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d2ef9ac5009346e2a3e37341f63dc460": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d3c4f999107a46d580ed4a3a629ae82a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d5457dc3a67d42718946b4eeff47289c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "da73c3f5610749e0b0813dd562d55801": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5f5cf08ed09c461f8665296bc08c0d95",
      "placeholder": "​",
      "style": "IPY_MODEL_28fff2a46c2348efb2c96a74a0f2b4dc",
      "value": " 20/20 [02:40&lt;00:00,  8.03s/it]"
     }
    },
    "db277d05ad3441d4ae66778d56a447d4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0c849c1b8c9b42fb8712b48fbd53ec4f",
      "placeholder": "​",
      "style": "IPY_MODEL_c339f5d42b204b53b9ed785caa0b8849",
      "value": " 100/100 [13:36&lt;00:00,  8.17s/it]"
     }
    },
    "db2c6484fe0f4115a4f764b2f5b51976": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dc405031fd474035bb91ada99441a89f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "df9ab6dd7720452d828c90561a7015ca": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5019997b07ad406abc065edf7d41649b",
      "placeholder": "​",
      "style": "IPY_MODEL_04eb0901586e488a8eef8d8c7b485ed8",
      "value": " 170499072/? [00:03&lt;00:00, 45376702.31it/s]"
     }
    },
    "e2fe2f67d8de425f8e7ca6029816747d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e359ca8a35124cf183a7f583ef846aed": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "eb4d1e07ab564739afc0c9da989be524": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ebf23f27e6c046059dcee10708750d23": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f6d4e15c3e8a4afab0cbba23a173de83": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "f9a5818caf31424a9cd302a647602241": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_788d4b8604674525a762d22b535a9b48",
       "IPY_MODEL_feb97483de0647d58db0c1b6fcadc48d"
      ],
      "layout": "IPY_MODEL_d255b3cf6f3b46a29085c21db11b1056"
     }
    },
    "fba8b82ad964466e80f2dea1ff357ae5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4706064b00614560b6a7f162102df5fe",
       "IPY_MODEL_147c3a4727b042e39223a22162850114"
      ],
      "layout": "IPY_MODEL_658eb2ea46e748ec9cc0bd7a4ca669a7"
     }
    },
    "fd7259d59da14068a5ee2b31692c840e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "feb97483de0647d58db0c1b6fcadc48d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cf55222977e94299a39d4632bee76e14",
      "placeholder": "​",
      "style": "IPY_MODEL_e2fe2f67d8de425f8e7ca6029816747d",
      "value": " 20/20 [02:32&lt;00:00,  7.62s/it]"
     }
    },
    "ff907b52b209496c803f14c2c582dc27": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}