Skip to content

Latest commit

 

History

History
525 lines (487 loc) · 21.5 KB

README-old.md

File metadata and controls

525 lines (487 loc) · 21.5 KB
Raw

Latest-development-of-ISR-VSR

[Updating...] Mainly ICCV, ECCV and CVPR about ISR and VSR, especially lasted two years developments.

Useful repositories：
1、A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow. which has most great papers/models about ISR and VSR. Include some useful tools: some models with pre-trained weights, link of datasets, VSR package which offers a training and data processing framework based on TF or pytorch.

Metrics dispute

Suggestion in SR: CVPR2018 "The Perception-Distortion Tradeoff"

Latest survey

arXiv2019: "Deep Learning for Image Super-resolution: A Survey"

Upscale method

  1. Dconvolution: "Deconvolutional networks"
  2. sub-pixel: "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network"
  3. Unpooling: "Visualizing and understanding convolutional networks"
  4. DUpsample: "Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation"
  5. carafe: "CARAFE- Content-Aware ReAssembly of FEatures"
  6. meta-SR: "Meta-SR-A Magnification-Arbitrary Network for Super-Resolution"

Unsupervised Super-resolution Method

  1. “Zero-Shot” Super-Resolution using Deep Internal Learning, CVPR2018
  2. Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks, CVPRW2018
  3. Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy, Medical image analysis 2019
  4. Self-Supervised Fine-tuning for Image Enhancement of Super-Resolution Deep Neural Networks, arXiv2019
  5. Unsupervised Learning for Real-World Super-Resolution, arXiv2019
  6. Unsupervised Single-Image Super-Resolution with Multi-Gram Loss, MDPI2019

Real-Word Image Super-Resolution

  • Based on the proposed HR-LR Image Pairs
  1. Toward Bridging the Simulated-to-Real Gap: Benchmarking Super-Reslution on Real Data, TPAMI2019
  2. Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model,ICCV2019
  3. Camera Lens Super-Resolution, CVPR2019
  4. Zoom to Learn, Learn to Zoom, CVPR2019
  • Based on the simulated degradation method
  1. Blind Super-Resolution with Iterative Kernel Corrections, CVPR2019
  2. Deep Plug-and-Play Super-Resolution for Arbitrary Blur Kernels, CVPR2019
  3. Blind Super-Resolution Kernel Estimation using an Internal-GAN, NeurIPS2019
  4. Kernel Modeling Super-Resolution on Real Low-Resolution Images, ICCV2019

Stereo Image Super-Resolution

  1. Enhancing the Spatial Resolution of Stereo Images using a Parallax Prior, CVPR2018
  1. Learning Parallax Attention for Stereo Image Super-Resolution, CVPR2019
  1. Stereoscopic Image Super‑Resolution with Stereo Consistent Feature, AAAI2020 oral
  1. A Stereo Attention Module for Stereo Image Super-Resolution, SPL2020

ISR

abbreviation full name published code description keywords in undergraduationt*
SRCNN Image Super-Resolution Using Deep Convlutional Network ECCV2014 keras：https://github.com/qobilidop/srcnn has two version 2014 and ex-2016. Milestone in deep learning about SR.Simple three CNN network：patch extraction and representation, non-linear mapping and reconstraction Loss:MSE CNN *
FSRCNN Accelerating the Super-Resolution Convolutional Neural Network ECCV2016 official:matlab,caffe：http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html Develop SRCNN, add deconv, input image don't need to upsample by bicubic and fine-tune accelerate deconvolution fine-tuninig last deconv *
ESPCN Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network CVPR2016 github(tensorflow): https://github.com/drakelevy/ESPCN-TensorFlowhttps:// github(pytorch): https://github.com/leftthomas/ESPCNhttps:// github(caffe): https://github.com/wangxuewen99/Super-Resolution/tree/master/ESPCNhttps:// A new way to upsamping: sub-pixel sub-pixel Tanh instead Relu Real time *
VDSR Accurate Image Super-Resolution Using Very Deep Convolutional Networks CVPR2016 "code: https://cv.snu.ac.kr/research/VDSR/ github(caffe): https://github.com/huangzehao/caffe-vdsrhttps:// github(tensorflow): https://github.com/Jongchan/tensorflow-vdsrhttps:// github(pytorch): https://github.com/twtygqyy/pytorch-vdsrhttps://" Add residual, padding 0 every layer, scale mixture training "residual network Deep" *
DRCN Deeply-Recursive Convolutional Network for Image Super-Resolution CVPR2016 "code: https://cv.snu.ac.kr/research/DRCN/ github(tensorflow): https://github.com/jiny2001/deeply-recursive-cnn-tfhttps://" "Learn RNN to add recursive and skip input image is interpolation image" "Recursive Neural Network Recursive Neural Network" *
RED Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections NIPS2016 … Encoder-decoder and skip Encoder-decoder *
DRRN Image Super-Resolution via Deep Recursive Residual Network CVPR2017 github(caffe): https://github.com/tyshiwo/DRRN_CVPR17 combine resNet and recursive "residual networkrecursive" *
LapSRN Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution CVPR2017 "github(matconvnet): https://github.com/phoenix104104/LapSRN github(pytorch): https://github.com/twtygqyy/pytorch-LapSRNhttps:/ github(tensorflow): https://github.com/zjuela/LapSRN-tensorflowhttps:/" Pyramid network new loss to constrain "Pyramid networkHuber loss" *
SRDenseNet Image Super-Resolution Using Dense Skip Connections ICCV2017 "pytorch: https://github.com/wxywhu/SRDenseNet-pytorch" add dense block to model dense block *
SRGAN Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network CVPR2017 "github(tensorflow): https://github.com/zsdonghao/SRGANhttps:// github(tensorflow): https://github.com/buriburisuri/SRGANhttps:// github(torch): https://github.com/junhocho/SRGANhttps:/AN github(caffe): https://github.com/ShenghaiRong/caffe_srganhttps:///caffe_srgan github(tensorflow): https://github.com/brade31919/SRGAN-tensorflowhttps://RGAN-tensorflow github(keras): https://github.com/titu1994/Super-Resolution-using-Generative-Adversarial-Networks https://er-Resolution-using-Generative-Adversarial-Networks github(pytorch): https://github.com/ai-tor/PyTorch-SRGAN" 1st proposed GAN GAN *
EDSR(workshop) Enhanced Deep Residual Networks for Single Image Super-Resolution CVPR2017 "github(torch): https://github.com/LimBee/NTIRE2017https://2017 github(tensorflow): https://github.com/jmiller656/EDSR-Tensorflowhttps:// github(pytorch): https://github.com/thstkdgus35/EDSR-PyTorchhttps://" remove BN "no BN MDSR" *
WDSR Wide Activation for Efficient and Accurate Image Super-Resolution arxiv2018 pytorch：https://github.com/JiahuiYu/wdsr_ntire2018 widen feature map and WN weight normalization *
SRMD Learning a Single Convolutional Super-Resolution Network for Multiple Degradations CVPR2018 "matlab： https://github.com/cszn/SRMD" Degraded Fuzzy Kernel and Noise Level Degraded Fuzzy Kernel and Noise Level *
RDN(oral) Residual Dense Network for Image Super-Resolution(CVPR 2018 Spotlight CVPR2018 "official: https://github.com/yulunzhang/RDN" "bicubic downsampling, gaussian kernel feature fusing" local and global Residual *
DBPN Deep Back-Projection Networks For Super-Resolution CVPR2018 "pytorch: https://github.com/alterzero/DBPN-Pytorch" repeat down and up sample a back mechanism Back-Projection *
ZSSR “Zero-Shot” Super-Resolution using Deep Internal Learning(2018 CVPR CVPR2018 "pytorch: https://github.com/jacobgil/pytorch-zssr" re-sample train test internally train *
SFTGAN Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform CVPR2018 "pytorch: https://github.com/xinntao/CVPR18-SFTGAN" semantic probability "semantic SFT" *
EUSR(workshop) Deep Residual Network with Enhanced Upscaling Module for Super-Resolution CVPR2018 … change EDSR to EUSR by adding EUM enhanced upscaling module (EUM) *
CARN Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network  ECCV2018 "pytorch: https://github.com/nmhkahn/CARN-pytorch" cascading block fast *
GAN_degradation To learn image super-resolution, use a GAN to learn how to do image degradation first  ECCV2018 … use GAN to prodecu LR near to nature mainly face test *
RCAN Image Super-Resolution Using Very Deep Residual Channel Attention Networks ECCV2018 pytorch: https://github.com/yulunzhang/RCAN very deep residual block with channel attention using several skip connection and channel weight Deep, Residual, Channel Attention /
EPSR(workshop) Analyzing Perception-Distortion Tradeoff using Enhanced Perceptual Super-resolution Network ECCV2018 ... has a new metrics idea ... *
SRFBN Feedback Network for Image Super-Resolution CVPR2019 pytorch:https://github.com/Paper99/SRFBN_CVPR19 feedback and a lot of comparation feedback /
zoom-learn-zoom Zoom to Learn, Learn to Zoom CVPR2019 tensorflow: https://github.com/ceciliavision/zoom-learn-zoom new direction for SR-RAW datasets and new CoBi loss function for alignment SR-RAW dataset and CoBi loss, real-word /
CameraSR Camera Lens Super-Resolution CVPR2019 tensorflow: https://github.com/ngchc/CameraSR Create City100 Dataset for real-word application real-word, City100 dataset /
RealSR Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model ICCV2019 caffe: https://github.com/csjcai/RealSR New RealSR datasets more flexible and convenient to use RealSR dataset, real-word, LP-KPN /
Simulated-to-Real Gap Toward Bridging the Simulated-to-Real Gap: Benchmarking Super-Reslution on Real Data TPAMI2019 / hardware binning method hardware binning, real-word, maybe the method older for it's journal /
RankSRGAN RankSRGAN: Generative Adversarial Networks with Ranker for Image Super- Resolution ICCV2019 github:https://github.com/WenlongZhang0724/RankSRGANfocus focus on perceptual quality, and new method to use perceptual metrics named Ranker Ranker, GAN /
IMDN Lightweight Image Super-Resolution with Information Multi-distillation Network ACM MM2019 github: https://github.com/Zheng222/IMDN todo: ... /
... ... ... ... ... ... /

VSR

abbreviation full name published code description keywords in undergraduation*
BRCN Bidirectional Recurrent Convolutional Networks for Multi-Frame Super-Resolution NIPS2015 matlab: https://github.com/linan142857/BRCN It has three conv. Feedforward conv, recurrent conv and conditioned conv. And two sub-network: forward and backward sub-network Two sub-network and three kind conv use recurrent *
VESPCN Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation CVPR2017 "pytorch: https://github.com/JuheonYi/VESPCN-PyTorch tensorflow: https://github.com/JuheonYi/VESPCN-tensorflow" compensation transformer: compare early fusion, slow fusion and 3D conv. "sub-pixel for video compensation transformer" *
SPMC Detail-revealing Deep Video Super-resolution ICCV2017 "tensorflow: https://github.com/jiangsutx/SPMC_VideoSR" "show that proper frame alignment and motion compensation is crucial for achieving high quality results It includes motion estimate, SPMC layer and Detail Fusion Net" SPMC: Subpixel Motion Compensation layer *
BRCN Bidirectional Recurrent Convolutional Networks for Multi-Frame Super-Resolution NIPS2015 matlab: https://github.com/linan142857/BRCN It has three conv. Feedforward conv, recurrent conv and conditioned conv. And two sub-network: forward and backward sub-network Two sub-network and three kind conv use recurrent *
FRVSR Frame-Recurrent Video Super-Resolution CVPR2018 "official: https://github.com/msmsajjadi/FRVSR" "we use a recurrent approach that passes the previously estimated HR frame as an input for the following iteration. Model includes Fnet and SRNet" "Flow estimation Upscaling flow Warping previous output Mapping to LR space Super-Resolution Warp" *
DUF Deep Video Super-Resolution Network Using Dynamic Upsampling Filters Without Explicit Motion Compensation CVPR2018 "tensorflow: https://github.com/HymEric/VSR-DUF-Reimplement https://github.com/yhjo09/VSR-DUF" "propose a novel end-to-end deep neural network that generates dynamic upsampling filters and a residual image, which are computed depending on the local spatio-temporal neighborhood of each pixel to avoid explicit motion compensation. The model includes filter generation network and residual generation network" "Dynamic upsampling filter Residual Learning" *
RBPN Recurrent Back-Projection Network for Video Super-Resolution CVPR2019 Pytorch：https://github.com/alterzero/RBPN-PyTorch ... recurrent encoder-decoder module /
EDVR EDVR: Video Restoration with Enhanced Deformable Convolutional Networks CVPR2019 Pytorch: https://github.com/xinntao/EDVR proposed two specify modules: PCD and TSA. PCD is for alignment and STA is for fusion. With deformable convolution, self-ensemble and two-stage redfine, it wins all four tracks in the NTIRE19 Challenges for Video PCD:Pyramid, Cascading and Deformable (PCD) alignment module, TSA:Temporal and Spatial Attention fusion module /
Updating ... ... ... ... ... /

Author

EricHym (Yongming He)
Interests: CV and Deep Learning
If you have or find any problems, this is my email: [email protected]. And I'm glad to reply it. Thanks.

Anyone can make contrbutions!