Study_Pytorch: model implementation

Outline

1. Goal	2. Using Google colab	3. Install Pytorch - With PyCharm	4. Study Models	5. Author

1. Goal

() model: I plan to study, but there is no plan to implement yet

Classfication & Backbone
LeNet5	VGGNet	(ResNet)	EfficientNet
Object detection
Faster R-CNN	(FPN)	YOLO(V4)
Segmentation
(FCN)	U-Net	(DeepLab V3, V3+)	Mask R-CNN
Generative model
GAN	(DCGAN)	CycleGAN	StarGAN(V1,V2)

Finished

• LeNet(2020.04)
• VGG Net(2020.04)
• (ResNet)(2020.06)
• GAN(2020.05)
• (DCGAN)
• (FCN)(2020.07)
• U-Net(2020.05)
• CycleGAN(2020.05)
• (DeepLabV3)
• StarGAN(V1/V2)
• Faster R-CNN(2021.05)
• (FPN)(2020.04)
• EfficientNet
• Mask R-CNN(2020.07)
• YOLO(V3)

Go Outline

2. Using Google colab

from google.colab import drive
drive.mount('/content/drive/')

!pwd
!ls
!git clone https://github.com/nh9k/pytorch-implementation.git

cd pytorch-implementation/0_basic

!python cifar10_tutorial.py

Go Outline

3. Install Pytorch - With PyCharm

you can see contents about installing pytorch here! Blog Link

Go Outline

4. Study Models

model-Implementation or using
LeNet5	VGGNet	GAN	U-Net	Faster R-CNN & Mask R-CNN

paper-review or study
ResNet	CycleGAN	FCN	FPN

Go Outline

LeNet5

[LeNet paper] [cifar10 information] [model-code] [training-code]

(1) Prerequisites

• numpy>=1.18.1
• torch>=1.4.0
• torchvision>=0.5.0
• Pillow==7.0.0

(2) Usage

Train a model

python train.py

Test the model

python eval.py

(3) Results

Cifar10
GroundTruth: cat ship ship plane
Predicted: cat car ship plane

test

Accuracy of the network on the 10000 test images: 54 %
Accuracy of plane : 61 %
Accuracy of   car : 65 %
Accuracy of  bird : 49 %
Accuracy of   cat : 50 %
Accuracy of  deer : 29 %
Accuracy of   dog : 34 %
Accuracy of  frog : 63 %
Accuracy of horse : 62 %
Accuracy of  ship : 58 %
Accuracy of truck : 67 %

Go 4.Study Models

VGGNet

[VGGNet paper] [torchvision: a model pre-trained on ImageNet] [model-code] [training-code]

[Blog - VGGNet, Shell script]

Dataset: cifar10
32x32 -> 16x16 -> 8x8 -> 4x4 -> 2x2 -> 1x1 with 5 maxpooling and same padding(conv2d)

(1) Prerequisites

• numpy
• torch
• torchvision
• tensorboard
• argparse

(2) Usage

Training

bash -x run_train.sh

Test

bash -x run_eval.sh

Use Tensorboard

tensorboard --logdir=result/tensorboard

Use matplotlib for input image

python3 show_input.py

Use matplotlib for log

python3 log_print.py

(3) Results

tensorboard

matplotlib

vgg11 test

Accuracy of the network on the 10000 test images: 75 %
Accuracy of plane : 72 %
Accuracy of   car : 85 %
Accuracy of  bird : 60 %
Accuracy of   cat : 47 %
Accuracy of  deer : 77 %
Accuracy of   dog : 63 %
Accuracy of  frog : 80 %
Accuracy of horse : 76 %
Accuracy of  ship : 87 %
Accuracy of truck : 84 %

vgg13 test

Accuracy of the network on the 10000 test images: 76 %
Accuracy of plane : 72 %
Accuracy of   car : 89 %
Accuracy of  bird : 72 %
Accuracy of   cat : 55 %
Accuracy of  deer : 70 %
Accuracy of   dog : 66 %
Accuracy of  frog : 75 %
Accuracy of horse : 80 %
Accuracy of  ship : 90 %
Accuracy of truck : 82 %

vgg16_1 test

Accuracy of the network on the 10000 test images: 76 %
Accuracy of plane : 82 %
Accuracy of   car : 92 %
Accuracy of  bird : 66 %
Accuracy of   cat : 47 %
Accuracy of  deer : 74 %
Accuracy of   dog : 78 %
Accuracy of  frog : 72 %
Accuracy of horse : 88 %
Accuracy of  ship : 87 %
Accuracy of truck : 71 %

vgg16 test

Accuracy of the network on the 10000 test images: 77 %
Accuracy of plane : 79 %
Accuracy of   car : 85 %
Accuracy of  bird : 81 %
Accuracy of   cat : 55 %
Accuracy of  deer : 66 %
Accuracy of   dog : 60 %
Accuracy of  frog : 86 %
Accuracy of horse : 92 %
Accuracy of  ship : 90 %
Accuracy of truck : 69 %

vgg19 test

Accuracy of the network on the 10000 test images: 76 %
Accuracy of plane : 65 %
Accuracy of   car : 82 %
Accuracy of  bird : 60 %
Accuracy of   cat : 58 %
Accuracy of  deer : 66 %
Accuracy of   dog : 60 %
Accuracy of  frog : 83 %
Accuracy of horse : 80 %
Accuracy of  ship : 87 %
Accuracy of truck : 84 %

Go 4.Study Models

ResNet

[ResNet paper] [model-code of torchvision]

model is not implemented from my code, but I reviewed paper for my study.
[Blog - ResNet Summary(Korean)]

Go 4.Study Models

GAN

[GAN paper] [model-code] [training-code]

Dataset: MNIST

(1) Prerequisites

• numpy
• torch
• torchvision
• tensorboard
• argparse

(2) Usage

Training

cd GAN
python train.py

(3) Experiments and Issues

1. BatchNorm function applied in model Generator
2. Discriminator layers simplication
3. Model input/output should be same image shape(scale)
4. for training, do care using fake_imgs.detach() or backward(retain_graph=True)

The above issues are very important for training GAN

(4) Results

Base model: using ReLU
Best model: using LeakyReLU

1. Using LeakyReLU instead of ReLU

500	1000	1500	2000	2500
40000	60000	80000	90000	Result(gif)

2. Using ReLU instead of LeakyReLU

500	1000	1500	2000	2500
20000	40000	60000	80000	Result(gif)

3. No BatchNorm in model

500	1000	1500	2000	2500

4. No Simplication of Discriminator layers

500	1000	1500	2000	2500

5. backward(retain_graph=True) instead of fake_imgs.detach()

500	1000	1500	2000	2500

Go 4.Study Models

FCN

[FCN paper] [model-code of torchvision] [other model code]

model is not implemented from my code, but I studyed roughly.
[Blog - FCN Summary(Korean)]

Go 4.Study Models

U-Net

[U-Net Paper] [The full implementation (based on Caffe) and the trained networks] [model-code]

[Blog - UNet, Binary Cross Entropy(Korean)] [Blog - UNet, Metrics(Korean)]

Dataset: isbi_challenge
you can download segmentation dataset after join in the page.

(1) Prerequisites

• numpy
• torch
• torchvision
• Pillow
• matplotlib

(2) Usage

• Training : code

cd U-Net
python train.py

• Test : code

cd U-Net
python eval.py

• Tensorboard

cd U-Net
tensorboard --logdir=runs/no_batch_normalization or add_batch_normalization

• Jupyter Notebook & Colab :
  no batch normalization result & code
  add batch normalization result & code

(3) Experiments and Results

Cross Entropy Loss is correct because the output segmentation map channels of the last layer is 2 in the paper, but i modified it to 1 and used Binary Cross Entropy

1. No batch normalization

Inputs	Labels	Outputs(Test Loss: 0.239)

2. added batch normalization

## model file: UNet.py (line8~12), added batch normalization function
        def DoubleConv(in_channels, out_channels):
            layers = []
            layers += [nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=1, padding=1), nn.BatchNorm2d(num_features=out_channels), nn.ReLU(inplace=True)]
            layers += [nn.Conv2d(out_channels, out_channels, kernel_size=3, stride=1, padding=1), nn.BatchNorm2d(num_features=out_channels), nn.ReLU(inplace=True)]
            return nn.Sequential(*layers)

Inputs	Labels	Outputs(Test Loss: 0.103)

Go 4.Study Models

CycleGAN

[CycleGAN paper]

model is not implemented from my code, but I reviewed paper for my study.
[Blog - Replay Buffer(Korean)] [Blog - Idea(Korean)]

Go 4.Study Models

FPN

[FPN paper]

model is not implemented yet from my code, but I studyed roughly.
[Blog - FPN research(Korean)]

Go 4.Study Models

Faster R-CNN & Mask R-CNN

[Faster R-CNN paper] [Mask R-CNN paper] [model-code of Detectron] [Detectron2 github]

model is not implemented yet from my code, but I studied detectron2.
[Blog - Detection and Segmentation Summary(Korean)]
[Webtoon Segmentation using Detectron2(model: Mask R-CNN)]

If you want to train model: Faster R-CNN, Just change the model

from

cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))

to

cfg.merge_from_file(model_zoo.get_config_file("COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"))

if you don't know how to label the dataset, consider the labeling tool labelme [labelme github]

Go 4.Study Models

5. Author

Nanhee Kim @nh9k / Personal blog