deconv_rep.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5ee34dd8-831b-4fcb-b8b0-744b4199960f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Tesla V100-SXM2-16GB'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "torch.cuda.device_count()\n",
    "torch.cuda.get_device_name(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7be50412-6c66-40ee-ba9b-67456f972fba",
   "metadata": {},
   "source": [
    "## Reproducibility Attempts on Network Deconvolution\n",
    "\n",
    "#### Original Study - https://github.com/yechengxi/deconvolution\n",
    "```\n",
    "@inproceedings{\n",
    "Ye2020Network,\n",
    "title={Network Deconvolution},\n",
    "author={Chengxi Ye and Matthew Evanusa and Hua He and Anton Mitrokhin and Tom Goldstein and James A. Yorke and Cornelia Fermuller and Yiannis Aloimonos},\n",
    "booktitle={International Conference on Learning Representations},\n",
    "year={2020},\n",
    "url={https://openreview.net/forum?id=rkeu30EtvS }\n",
    "```\n",
    "\n",
    "#### Reproducibility Study - https://github.com/rochanaro/cs895-deconvolution\n",
    "```\n",
    "Kumushini Thennakoon | Rochana Obadage  \n",
    "04/11/2024\n",
    "```\n",
    "\n",
    "### Parameters\n",
    "* Model Architectures \n",
    "    * VGG-16 --- vgg16\n",
    "    * ResNet-18 --- resnet18d\n",
    "    * Preact-18 --- preact\n",
    "    * DenseNet-121 --- densenet121\n",
    "    * ResNext-29 --- resnext\n",
    "    * MobileNet v2 --- mobilev2\n",
    "    * DPN-92 --- dpn\n",
    "    * PNASNetA --- pnasnetA\n",
    "    * SENet-18 --- senet\n",
    "    * EfficientNet --- efficient\n",
    "* Datasets --- [CIFAR-10, CIFAR-100]\n",
    "* Performance enhancement techniques --- [Batch Normalization, Network Deconvolution]\n",
    "* Epochs --- [1, 20, 100]\n",
    "* Attempts --- [1, 2, 3]\n",
    "* Optimizer --- SGD"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a200f5c-2148-4c1f-b118-390476245aaa",
   "metadata": {},
   "source": [
    "### Hyper-Parameters\n",
    "* batch\\_size = 128 \n",
    "* learning_rate = 0.1\n",
    "* Weight-Decay = 0.001"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "111ffd46",
   "metadata": {},
   "outputs": [],
   "source": [
    "!git clone https://github.com/rochanaro/cs895-deconvolution.git"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9659d4e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "%cd \"cs895-deconvolution\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7fc14cec",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -r requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a09cb9c5-1322-4340-88e5-253776eb71bf",
   "metadata": {},
   "source": [
    "# VGG-16 --- vgg16"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02001197-76b0-4b80-8ba3-2da2d1b86064",
   "metadata": {},
   "source": [
    "# ResNet-18 --- resnet18d"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnet18d --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4ba9ad3-0dc5-4f06-a78e-a4434b5ef98c",
   "metadata": {},
   "source": [
    "# Preact-18 --- preact"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch preact --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ca681f8-f259-4446-a9f7-9cd449964fb5",
   "metadata": {},
   "source": [
    "# DenseNet-121 --- densenet121"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch densenet121 --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c31d961a-3dc3-4eec-a287-5a0bc75086b1",
   "metadata": {},
   "source": [
    "# ResNext-29 --- resnext"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch resnext --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbf33ada-6102-460c-9f82-f26c4c3be5d6",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# MobileNet v2 --- mobilev2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch mobilev2 --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f586bd1c-385b-4cb5-8ca5-33c96613564d",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# DPN-92 --- dpn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch dpn --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e228ce48-88ac-483d-81e5-3e6bb1aea968",
   "metadata": {},
   "source": [
    "# PNASNetA --- pnasnetA"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch pnasnetA --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b4bdbd4-6ea6-4a38-80f2-dfcd50c2fb86",
   "metadata": {},
   "source": [
    "# SENet-18 --- senet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch senet --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "410856f2-83fc-4c36-9c85-7c35a6491c89",
   "metadata": {},
   "source": [
    "# EfficientNet --- efficient"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 1 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 20 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch vgg16 --epochs 100 --dataset cifar10  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f4a5d3",
   "metadata": {},
   "source": [
    "> ## CIFAR-100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a2875e-d8eb-47ba-8232-ad1d264533d8",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "b05bf0b2-89b8-4b4c-9548-5e957cc8df74",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch efficient --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6213acd-8852-4a82-86e9-56336ffd6fd7",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "44f5392a-ded9-4655-87a5-216150e85889",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch efficient --epochs 1 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f50545a6",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "669c19f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch efficient --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff0cde",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c15aa5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch efficient --epochs 20 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f803f37c",
   "metadata": {},
   "source": [
    "#### batch normalization --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25d4107a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch efficient --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv False --block-fc 0 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "867f0d39",
   "metadata": {},
   "source": [
    "#### network deconvolution --- epochs 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b11c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python main.py --lr .1 --optimizer SGD --arch efficient --epochs 100 --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e989bb1-c8eb-4b81-a514-236d9d626aae",
   "metadata": {},
   "source": [
    "### Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "ac19782b-1789-40ae-8c1d-8accbaaa95d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob\n",
    "import os\n",
    "import pandas as pd\n",
    "\n",
    "\n",
    "# file_list = glob.glob('checkpoints/data?.txt')\n",
    "file_list_ = glob.glob('checkpoints/*ep.20*')\n",
    "file_list = sorted(file_list_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "83aa5aa0-8f68-4604-abdb-28e74a15729e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 densenet121---BN\n",
      "Attempt 1\n",
      "densenet121 - ep.20 - mode: BN - Attempt_1 - train - top1_acc: 98.32 - time: 1174.68\n",
      "densenet121 - ep.20 - mode: BN - Attempt_1 - test - top1_acc: 93.1 - time: 111.86\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "densenet121 - ep.20 - mode: BN - Attempt_2 - train - top1_acc: 98.108 - time: 1132.57\n",
      "densenet121 - ep.20 - mode: BN - Attempt_2 - test - top1_acc: 93.02 - time: 91.34\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "densenet121 - ep.20 - mode: BN - Attempt_3 - train - top1_acc: 98.304 - time: 1162.58\n",
      "densenet121 - ep.20 - mode: BN - Attempt_3 - test - top1_acc: 93.21 - time: 97.98\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "2 densenet121---deconv.1\n",
      "Attempt 1\n",
      "densenet121 - ep.20 - mode: deconv.1 - Attempt_1 - train - top1_acc: 99.402 - time: 5028.65\n",
      "densenet121 - ep.20 - mode: deconv.1 - Attempt_1 - test - top1_acc: 94.96 - time: 132.67\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "densenet121 - ep.20 - mode: deconv.1 - Attempt_2 - train - top1_acc: 99.42 - time: 4630.60\n",
      "densenet121 - ep.20 - mode: deconv.1 - Attempt_2 - test - top1_acc: 94.89 - time: 125.93\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "densenet121 - ep.20 - mode: deconv.1 - Attempt_3 - train - top1_acc: 99.416 - time: 4718.71\n",
      "densenet121 - ep.20 - mode: deconv.1 - Attempt_3 - test - top1_acc: 94.75 - time: 126.13\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "3 dpn---BN\n",
      "Attempt 1\n",
      "dpn - ep.20 - mode: BN - Attempt_1 - train - top1_acc: 95.506 - time: 2750.28\n",
      "dpn - ep.20 - mode: BN - Attempt_1 - test - top1_acc: 91.59 - time: 179.18\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "dpn - ep.20 - mode: BN - Attempt_2 - train - top1_acc: 96.052 - time: 2780.88\n",
      "dpn - ep.20 - mode: BN - Attempt_2 - test - top1_acc: 92.19 - time: 178.25\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "dpn - ep.20 - mode: BN - Attempt_3 - train - top1_acc: 95.35 - time: 2772.89\n",
      "dpn - ep.20 - mode: BN - Attempt_3 - test - top1_acc: 91.59 - time: 181.87\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "4 dpn---deconv.1\n",
      "Attempt 1\n",
      "dpn - ep.20 - mode: deconv.1 - Attempt_1 - train - top1_acc: 98.586 - time: 6567.51\n",
      "dpn - ep.20 - mode: deconv.1 - Attempt_1 - test - top1_acc: 93.7 - time: 196.55\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "dpn - ep.20 - mode: deconv.1 - Attempt_2 - train - top1_acc: 98.656 - time: 6534.73\n",
      "dpn - ep.20 - mode: deconv.1 - Attempt_2 - test - top1_acc: 93.47 - time: 176.97\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "dpn - ep.20 - mode: deconv.1 - Attempt_3 - train - top1_acc: 98.584 - time: 6397.82\n",
      "dpn - ep.20 - mode: deconv.1 - Attempt_3 - test - top1_acc: 93.47 - time: 173.28\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "5 efficient---BN\n",
      "Attempt 1\n",
      "efficient - ep.20 - mode: BN - Attempt_1 - train - top1_acc: 86.844 - time: 539.96\n",
      "efficient - ep.20 - mode: BN - Attempt_1 - test - top1_acc: 85.52 - time: 48.22\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "efficient - ep.20 - mode: BN - Attempt_2 - train - top1_acc: 85.498 - time: 541.57\n",
      "efficient - ep.20 - mode: BN - Attempt_2 - test - top1_acc: 84.18 - time: 50.30\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "efficient - ep.20 - mode: BN - Attempt_3 - train - top1_acc: 86.004 - time: 569.22\n",
      "efficient - ep.20 - mode: BN - Attempt_3 - test - top1_acc: 84.34 - time: 48.84\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "6 efficient---deconv.1\n",
      "Attempt 1\n",
      "efficient - ep.20 - mode: deconv.1 - Attempt_1 - train - top1_acc: 88.776 - time: 568.10\n",
      "efficient - ep.20 - mode: deconv.1 - Attempt_1 - test - top1_acc: 87.45 - time: 50.71\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "efficient - ep.20 - mode: deconv.1 - Attempt_2 - train - top1_acc: 88.88 - time: 545.68\n",
      "efficient - ep.20 - mode: deconv.1 - Attempt_2 - test - top1_acc: 87.76 - time: 49.01\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "efficient - ep.20 - mode: deconv.1 - Attempt_3 - train - top1_acc: 89.22 - time: 621.97\n",
      "efficient - ep.20 - mode: deconv.1 - Attempt_3 - test - top1_acc: 88.09 - time: 50.00\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "7 mobilev2---BN\n",
      "Attempt 1\n",
      "mobilev2 - ep.20 - mode: BN - Attempt_1 - train - top1_acc: 93.42 - time: 451.53\n",
      "mobilev2 - ep.20 - mode: BN - Attempt_1 - test - top1_acc: 90.73 - time: 65.72\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "mobilev2 - ep.20 - mode: BN - Attempt_2 - train - top1_acc: 93.356 - time: 521.64\n",
      "mobilev2 - ep.20 - mode: BN - Attempt_2 - test - top1_acc: 90.67 - time: 69.50\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "mobilev2 - ep.20 - mode: BN - Attempt_3 - train - top1_acc: 93.244 - time: 503.55\n",
      "mobilev2 - ep.20 - mode: BN - Attempt_3 - test - top1_acc: 90.71 - time: 76.06\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "8 mobilev2---deconv.1\n",
      "Attempt 1\n",
      "mobilev2 - ep.20 - mode: deconv.1 - Attempt_1 - train - top1_acc: 95.508 - time: 1810.48\n",
      "mobilev2 - ep.20 - mode: deconv.1 - Attempt_1 - test - top1_acc: 92.22 - time: 81.40\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "mobilev2 - ep.20 - mode: deconv.1 - Attempt_2 - train - top1_acc: 95.57 - time: 1766.51\n",
      "mobilev2 - ep.20 - mode: deconv.1 - Attempt_2 - test - top1_acc: 92.07 - time: 61.81\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "mobilev2 - ep.20 - mode: deconv.1 - Attempt_3 - train - top1_acc: 95.71 - time: 1637.50\n",
      "mobilev2 - ep.20 - mode: deconv.1 - Attempt_3 - test - top1_acc: 92.03 - time: 56.76\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "9 pnasnetA---BN\n",
      "Attempt 1\n",
      "pnasnetA - ep.20 - mode: BN - Attempt_1 - train - top1_acc: 65.386 - time: 369.46\n",
      "pnasnetA - ep.20 - mode: BN - Attempt_1 - test - top1_acc: 65.66 - time: 45.06\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "pnasnetA - ep.20 - mode: BN - Attempt_2 - train - top1_acc: 46.346 - time: 370.00\n",
      "pnasnetA - ep.20 - mode: BN - Attempt_2 - test - top1_acc: 46.31 - time: 46.03\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "pnasnetA - ep.20 - mode: BN - Attempt_3 - train - top1_acc: 65.346 - time: 369.91\n",
      "pnasnetA - ep.20 - mode: BN - Attempt_3 - test - top1_acc: 64.56 - time: 46.17\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "10 pnasnetA---deconv.1\n",
      "Attempt 1\n",
      "pnasnetA - ep.20 - mode: deconv.1 - Attempt_1 - train - top1_acc: 83.32 - time: 1595.45\n",
      "pnasnetA - ep.20 - mode: deconv.1 - Attempt_1 - test - top1_acc: 81.94 - time: 42.82\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "pnasnetA - ep.20 - mode: deconv.1 - Attempt_2 - train - top1_acc: 83.348 - time: 1591.27\n",
      "pnasnetA - ep.20 - mode: deconv.1 - Attempt_2 - test - top1_acc: 81.45 - time: 46.74\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "pnasnetA - ep.20 - mode: deconv.1 - Attempt_3 - train - top1_acc: 83.24 - time: 1592.78\n",
      "pnasnetA - ep.20 - mode: deconv.1 - Attempt_3 - test - top1_acc: 81.44 - time: 45.45\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "11 preact---BN\n",
      "Attempt 1\n",
      "preact - ep.20 - mode: BN - Attempt_1 - train - top1_acc: 93.296 - time: 636.06\n",
      "preact - ep.20 - mode: BN - Attempt_1 - test - top1_acc: 89.65 - time: 101.83\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "preact - ep.20 - mode: BN - Attempt_2 - train - top1_acc: 94.27 - time: 592.51\n",
      "preact - ep.20 - mode: BN - Attempt_2 - test - top1_acc: 90.57 - time: 94.32\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "preact - ep.20 - mode: BN - Attempt_3 - train - top1_acc: 94.732 - time: 689.99\n",
      "preact - ep.20 - mode: BN - Attempt_3 - test - top1_acc: 90.74 - time: 97.09\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "12 preact---deconv.1\n",
      "Attempt 1\n",
      "preact - ep.20 - mode: deconv.1 - Attempt_1 - train - top1_acc: 98.868 - time: 1097.81\n",
      "preact - ep.20 - mode: deconv.1 - Attempt_1 - test - top1_acc: 94.3 - time: 63.00\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "preact - ep.20 - mode: deconv.1 - Attempt_2 - train - top1_acc: 98.888 - time: 1048.43\n",
      "preact - ep.20 - mode: deconv.1 - Attempt_2 - test - top1_acc: 94.16 - time: 62.74\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "preact - ep.20 - mode: deconv.1 - Attempt_3 - train - top1_acc: 98.894 - time: 1037.29\n",
      "preact - ep.20 - mode: deconv.1 - Attempt_3 - test - top1_acc: 94.32 - time: 65.84\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "13 resnet---BN\n",
      "Attempt 1\n",
      "resnet - ep.20 - mode: BN - Attempt_1 - train - top1_acc: 97.178 - time: 1237.81\n",
      "resnet - ep.20 - mode: BN - Attempt_1 - test - top1_acc: 92.32 - time: 171.35\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "resnet - ep.20 - mode: BN - Attempt_2 - train - top1_acc: 97.264 - time: 1309.77\n",
      "resnet - ep.20 - mode: BN - Attempt_2 - test - top1_acc: 92.33 - time: 163.28\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "resnet - ep.20 - mode: BN - Attempt_3 - train - top1_acc: 97.252 - time: 1157.50\n",
      "resnet - ep.20 - mode: BN - Attempt_3 - test - top1_acc: 92.39 - time: 159.91\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "14 resnet---deconv.1\n",
      "Attempt 1\n",
      "resnet - ep.20 - mode: deconv.1 - Attempt_1 - train - top1_acc: 99.0 - time: 3559.51\n",
      "resnet - ep.20 - mode: deconv.1 - Attempt_1 - test - top1_acc: 94.2 - time: 127.77\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "resnet - ep.20 - mode: deconv.1 - Attempt_2 - train - top1_acc: 98.924 - time: 3581.87\n",
      "resnet - ep.20 - mode: deconv.1 - Attempt_2 - test - top1_acc: 94.06 - time: 136.87\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "resnet - ep.20 - mode: deconv.1 - Attempt_3 - train - top1_acc: 98.996 - time: 3619.60\n",
      "resnet - ep.20 - mode: deconv.1 - Attempt_3 - test - top1_acc: 94.34 - time: 136.99\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "15 resnext---BN\n",
      "Attempt 1\n",
      "resnext - ep.20 - mode: BN - Attempt_1 - train - top1_acc: 96.72 - time: 904.01\n",
      "resnext - ep.20 - mode: BN - Attempt_1 - test - top1_acc: 92.61 - time: 82.99\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "resnext - ep.20 - mode: BN - Attempt_2 - train - top1_acc: 96.772 - time: 896.13\n",
      "resnext - ep.20 - mode: BN - Attempt_2 - test - top1_acc: 92.37 - time: 78.88\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "resnext - ep.20 - mode: BN - Attempt_3 - train - top1_acc: 96.772 - time: 919.30\n",
      "resnext - ep.20 - mode: BN - Attempt_3 - test - top1_acc: 92.04 - time: 98.90\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "16 resnext---deconv.1\n",
      "Attempt 1\n",
      "resnext - ep.20 - mode: deconv.1 - Attempt_1 - train - top1_acc: 98.558 - time: 2614.81\n",
      "resnext - ep.20 - mode: deconv.1 - Attempt_1 - test - top1_acc: 93.49 - time: 108.24\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "resnext - ep.20 - mode: deconv.1 - Attempt_2 - train - top1_acc: 98.582 - time: 2578.39\n",
      "resnext - ep.20 - mode: deconv.1 - Attempt_2 - test - top1_acc: 93.53 - time: 92.23\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "resnext - ep.20 - mode: deconv.1 - Attempt_3 - train - top1_acc: 98.57 - time: 2594.16\n",
      "resnext - ep.20 - mode: deconv.1 - Attempt_3 - test - top1_acc: 93.69 - time: 86.59\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "17 senet---BN\n",
      "Attempt 1\n",
      "senet - ep.20 - mode: BN - Attempt_1 - train - top1_acc: 97.25 - time: 337.94\n",
      "senet - ep.20 - mode: BN - Attempt_1 - test - top1_acc: 92.89 - time: 48.81\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "senet - ep.20 - mode: BN - Attempt_2 - train - top1_acc: 97.258 - time: 357.15\n",
      "senet - ep.20 - mode: BN - Attempt_2 - test - top1_acc: 92.84 - time: 46.63\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "senet - ep.20 - mode: BN - Attempt_3 - train - top1_acc: 97.252 - time: 337.57\n",
      "senet - ep.20 - mode: BN - Attempt_3 - test - top1_acc: 92.5 - time: 48.70\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "18 senet---deconv.1\n",
      "Attempt 1\n",
      "senet - ep.20 - mode: deconv.1 - Attempt_1 - train - top1_acc: 98.764 - time: 1747.25\n",
      "senet - ep.20 - mode: deconv.1 - Attempt_1 - test - top1_acc: 94.17 - time: 51.92\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "senet - ep.20 - mode: deconv.1 - Attempt_2 - train - top1_acc: 98.814 - time: 1399.06\n",
      "senet - ep.20 - mode: deconv.1 - Attempt_2 - test - top1_acc: 94.44 - time: 49.08\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "senet - ep.20 - mode: deconv.1 - Attempt_3 - train - top1_acc: 98.73 - time: 1407.07\n",
      "senet - ep.20 - mode: deconv.1 - Attempt_3 - test - top1_acc: 94.15 - time: 50.37\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "19 vgg16---BN\n",
      "Attempt 1\n",
      "vgg16 - ep.20 - mode: BN - Attempt_1 - train - top1_acc: 94.872 - time: 414.66\n",
      "vgg16 - ep.20 - mode: BN - Attempt_1 - test - top1_acc: 90.18 - time: 91.41\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "vgg16 - ep.20 - mode: BN - Attempt_2 - train - top1_acc: 95.038 - time: 452.15\n",
      "vgg16 - ep.20 - mode: BN - Attempt_2 - test - top1_acc: 90.6 - time: 100.62\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "vgg16 - ep.20 - mode: BN - Attempt_3 - train - top1_acc: 94.784 - time: 355.24\n",
      "vgg16 - ep.20 - mode: BN - Attempt_3 - test - top1_acc: 90.3 - time: 86.00\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "20 vgg16---deconv.1\n",
      "Attempt 1\n",
      "vgg16 - ep.20 - mode: deconv.1 - Attempt_1 - train - top1_acc: 98.238 - time: 978.15\n",
      "vgg16 - ep.20 - mode: deconv.1 - Attempt_1 - test - top1_acc: 92.96 - time: 69.48\n",
      "\n",
      "\n",
      "Attempt 2\n",
      "vgg16 - ep.20 - mode: deconv.1 - Attempt_2 - train - top1_acc: 98.238 - time: 1325.19\n",
      "vgg16 - ep.20 - mode: deconv.1 - Attempt_2 - test - top1_acc: 92.81 - time: 83.78\n",
      "\n",
      "\n",
      "Attempt 3\n",
      "vgg16 - ep.20 - mode: deconv.1 - Attempt_3 - train - top1_acc: 98.13 - time: 1245.88\n",
      "vgg16 - ep.20 - mode: deconv.1 - Attempt_3 - test - top1_acc: 92.72 - time: 85.62\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import glob\n",
    "import os\n",
    "import pandas as pd\n",
    "\n",
    "\n",
    "# file_list = glob.glob('checkpoints/data?.txt')\n",
    "file_list_ = glob.glob('checkpoints/*ep.20*')\n",
    "file_list = sorted(file_list_)\n",
    "\n",
    "count = 0\n",
    "for num, dir in enumerate(file_list):\n",
    "    three_folders = glob.glob(f'{dir}/*')\n",
    "    print(num+1, dir.split(\",\")[1], end='---')\n",
    "    \n",
    "    arch = dir.split(\",\")[1]\n",
    "    mode_ = dir.split(\",\")[9]\n",
    "    if mode_ == 'deconv.0':\n",
    "        mode_ = 'BN'\n",
    "    epochs_ = dir.split(\",\")[2]\n",
    "    print(mode_)\n",
    "\n",
    "    \n",
    "    for i, folder in enumerate(three_folders):\n",
    "        files = glob.glob(f'{folder}/*.log')\n",
    "        # print(files)\n",
    "        print(f\"Attempt {i+1}\")\n",
    "        for file in files:\n",
    "            if os.path.basename(file) != 'train_batch.log':\n",
    "                df = pd.read_csv(file, sep='\\t', engine='python')\n",
    "                type_ = os.path.basename(file).replace('.log','')\n",
    "                # print(type_)\n",
    "                # print(df)\n",
    "                print(f\"{arch} - {epochs_} - mode: {mode_} - Attempt_{i+1} - {type_} - top1_acc: {max(df['top1'])} - time: {sum(df['time']):.2f}\")\n",
    "        \n",
    "        print('\\n')\n",
    "            \n",
    "\n",
    "    # count+=1\n",
    "    # if count == 3:\n",
    "    #     break\n",
    "    print('\\n\\n')\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "11070ff4-44c7-4c11-9bab-ebb937916a36",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#!/bin/bash\n",
      "\n",
      "#SBATCH -c 8\n",
      "#SBATCH -p gpu\n",
      "#SBATCH --gres gpu:1\n",
      "\n",
      "enable_lmod\n",
      "module load container_env pytorch-gpu/1.13.0\n",
      "\n",
      "export CUDA_HOME=/cm/shared/applications/cuda-toolkit/11.7.1/\n",
      "export XLA_FLAGS=--xla_gpu_cuda_data_dir=$CUDA_HOME\n",
      "\n",
      "# crun.tensorflow-gpu -p ~/envs/cs834_project python lemos_kerasnlp_for_slurm_job_training.py -itr $iteration -ep $epochs\n",
      "\n",
      "crun.pytorch-gpu -p ~/envs/cs834_project python main.py --lr .1 --optimizer SGD --arch $architecture --epochs $epochs --dataset cifar100  --batch-size 128 --msg True --deconv True --block-fc 512 --wd .001"
     ]
    }
   ],
   "source": [
    "!cat single_experiment_net_deconv_cifar100.sh"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10.13 ('lemos_cv')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "98699c61c0c876e75ce040de3f75f19a49466b6e44f74fd8bd638a7e4e681381"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}