README.md

State-of-the-art result for all Machine Learning Problems

LAST UPDATE: 20th Februray 2019

NEWS: I am looking for a Collaborator esp who does research in NLP, Computer Vision and Reinforcement learning. If you are not a researcher, but you are willing, contact me. Email me: [email protected]

This repository provides state-of-the-art (SoTA) results for all machine learning problems. We do our best to keep this repository up to date. If you do find a problem's SoTA result is out of date or missing, please raise this as an issue (with this information: research paper name, dataset, metric, source code and year). We will fix it immediately.

You can also submit this Google Form if you are new to Github.

This is an attempt to make one stop for all types of machine learning problems state of the art result. I can not do this alone. I need help from everyone. Please submit the Google form/raise an issue if you find SOTA result for a dataset. Please share this on Twitter, Facebook, and other social media.

This summary is categorized into:

• Supervised Learning
  • Speech
  • Computer Vision
  • NLP
• Semi-supervised Learning
  • Computer Vision
• Unsupervised Learning
  • Speech
  • Computer Vision
  • NLP
• Transfer Learning
• Reinforcement Learning

Supervised Learning

NLP

1. Language Modelling

Research Paper	Datasets	Metric	Source Code	Year
Language Models are Unsupervised Multitask Learners	PTB WikiText-2	Perplexity: 35.76 Perplexity: 18.34	Tensorflow	2019
BREAKING THE SOFTMAX BOTTLENECK: A HIGH-RANK RNN LANGUAGE MODEL	PTB WikiText-2	Perplexity: 47.69 Perplexity: 40.68	Pytorch	2017
DYNAMIC EVALUATION OF NEURAL SEQUENCE MODELS	PTB WikiText-2	Perplexity: 51.1 Perplexity: 44.3	Pytorch	2017
Averaged Stochastic Gradient Descent with Weight Dropped LSTM or QRNN	PTB WikiText-2	Perplexity: 52.8 Perplexity: 52.0	Pytorch	2017
FRATERNAL DROPOUT	PTB WikiText-2	Perplexity: 56.8 Perplexity: 64.1	Pytorch	2017
Factorization tricks for LSTM networks	One Billion Word Benchmark	Perplexity: 23.36	Tensorflow	2017

2. Machine Translation

Research Paper	Datasets	Metric	Source Code	Year
Understanding Back-Translation at Scale	WMT 2014 English-to-French WMT 2014 English-to-German	BLEU: 45.6 BLEU: 35.0	PyTorch	2018
WEIGHTED TRANSFORMER NETWORK FOR MACHINE TRANSLATION	WMT 2014 English-to-French WMT 2014 English-to-German	BLEU: 41.4 BLEU: 28.9	NOT FOUND	2017
Attention Is All You Need	WMT 2014 English-to-French WMT 2014 English-to-German	BLEU: 41.0 BLEU: 28.4	PyTorch Tensorflow	2017
NON-AUTOREGRESSIVE NEURAL MACHINE TRANSLATION	WMT16 Ro→En	BLEU: 31.44	PyTorch	2017
Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets	NIST02 NIST03 NIST04 NIST05	38.74 36.01 37.54 33.76	NMTPY	2017

3. Text Classification

Research Paper	Datasets	Metric	Source Code	Year
Learning Structured Text Representations	Yelp	Accuracy: 68.6	Tensorflow	2017
Attentive Convolution	Yelp	Accuracy: 67.36	Theano	2017

4. Natural Language Inference

Leader board:

Stanford Natural Language Inference (SNLI)

MultiNLI

Research Paper	Datasets	Metric	Source Code	Year
NATURAL LANGUAGE INFERENCE OVER INTERACTION SPACE	Stanford Natural Language Inference (SNLI)	Accuracy: 88.9	Tensorflow	2017
BERT-LARGE (ensemble)	Multi-Genre Natural Language Inference (MNLI)	Matched accuracy: 86.7 Mismatched accuracy: 85.9	Tensorflow PyTorch	2018

5. Question Answering

Leader Board

SQuAD

Research Paper	Datasets	Metric	Source Code	Year
BERT-LARGE (ensemble)	The Stanford Question Answering Dataset	Exact Match: 87.4 F1: 93.2	Tensorflow PyTorch	2018

6. Named entity recognition

Research Paper	Datasets	Metric	Source Code	Year
Named Entity Recognition in Twitter using Images and Text	Ritter	F-measure: 0.59	NOT FOUND	2017

7. Abstractive Summarization

Research Paper	Datasets	Metric	Source Code	Year
Cutting-off redundant repeating generations for neural abstractive summarization	DUC-2004 Gigaword	DUC-2004 ROUGE-1: 32.28 ROUGE-2: 10.54 ROUGE-L: 27.80 Gigaword ROUGE-1: 36.30 ROUGE-2: 17.31 ROUGE-L: 33.88	NOT YET AVAILABLE	2017
Convolutional Sequence to Sequence	DUC-2004 Gigaword	DUC-2004 ROUGE-1: 33.44 ROUGE-2: 10.84 ROUGE-L: 26.90 Gigaword ROUGE-1: 35.88 ROUGE-2: 27.48 ROUGE-L: 33.29	PyTorch	2017

8. Dependency Parsing

Research Paper	Datasets	Metric	Source Code	Year
Globally Normalized Transition-Based Neural Networks	Final CoNLL ’09 dependency parsing	94.08% UAS accurancy 92.15% LAS accurancy	SyntaxNet	2017

Computer Vision

1. Classification

           

Research Paper	Datasets	Metric	Source Code	Year
Dynamic Routing Between Capsules	MNIST	Test Error: 0.25±0.005	Official Implementation PyTorch Tensorflow Keras Chainer List of all implementations	2017
High-Performance Neural Networks for Visual Object Classification	NORB	Test Error: 2.53 ± 0.40	NOT FOUND	2011
Giant AmoebaNet with GPipe	CIFAR-10 CIFAR-100 ImageNet-1k ...	Test Error: 1.0% Test Error: 8.7% Top-1 Error 15.7 ...	NOT FOUND	2018
ShakeDrop regularization	CIFAR-10 CIFAR-100	Test Error: 2.31% Test Error: 12.19%	NOT FOUND	2017
Aggregated Residual Transformations for Deep Neural Networks	CIFAR-10	Test Error: 3.58%	PyTorch	2017
Random Erasing Data Augmentation	CIFAR-10 CIFAR-100 Fashion-MNIST	Test Error: 3.08% Test Error: 17.73% Test Error: 3.65%	Pytorch	2017
EraseReLU: A Simple Way to Ease the Training of Deep Convolution Neural Networks	CIFAR-10 CIFAR-100	Test Error: 3.56% Test Error: 16.53%	Pytorch	2017
Dynamic Routing Between Capsules	MultiMNIST	Test Error: 5%	PyTorch Tensorflow Keras Chainer List of all implementations	2017
Learning Transferable Architectures for Scalable Image Recognition	ImageNet-1k	Top-1 Error:17.3	Tensorflow	2017
Squeeze-and-Excitation Networks	ImageNet-1k	Top-1 Error: 18.68	CAFFE	2017
Aggregated Residual Transformations for Deep Neural Networks	ImageNet-1k	Top-1 Error: 20.4%	Torch	2016

2. Instance Segmentation

Research Paper	Datasets	Metric	Source Code	Year
Mask R-CNN	COCO	Average Precision: 37.1%	Detectron (Official Version) MXNet Keras TensorFlow	2017

3. Visual Question Answering

Research Paper	Datasets	Metric	Source Code	Year
Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge	VQA	Overall score: 69	NOT FOUND	2017

4. Person Re-identification

     

Research Paper	Datasets	Metric	Source Code	Year
Random Erasing Data Augmentation	Market-1501 CUHK03-new-protocol DukeMTMC-reID	Rank-1: 89.13 mAP: 83.93 Rank-1: 84.02 mAP: 78.28 labeled (Rank-1: 63.93 mAP: 65.05) detected (Rank-1: 64.43 mAP: 64.75)	Pytorch	2017

Speech

Speech SOTA

1. ASR

Research Paper	Datasets	Metric	Source Code	Year
The Microsoft 2017 Conversational Speech Recognition System	Switchboard Hub5'00	WER: 5.1	NOT FOUND	2017
The CAPIO 2017 Conversational Speech Recognition System	Switchboard Hub5'00	WER: 5.0	NOT FOUND	2017

Semi-supervised Learning

Computer Vision

     

Research Paper	Datasets	Metric	Source Code	Year
DISTRIBUTIONAL SMOOTHINGWITH VIRTUAL ADVERSARIAL TRAINING	SVHN NORB	Test error: 24.63 Test error: 9.88	Theano	2016
Virtual Adversarial Training: a Regularization Method for Supervised and Semi-supervised Learning	MNIST	Test error: 1.27	NOT FOUND	2017
Few Shot Object Detection	VOC2007 VOC2012	mAP : 41.7 mAP : 35.4	NOT FOUND	2017
Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro	Market-1501 CUHK-03 DukeMTMC-reID CUB-200-2011	Rank-1: 83.97 mAP: 66.07 Rank-1: 84.6 mAP: 87.4 Rank-1: 67.68 mAP: 47.13           Test Accuracy: 84.4	Matconvnet	2017

Unsupervised Learning

Computer Vision

1. Generative Model

Research Paper	Datasets	Metric	Source Code	Year
PROGRESSIVE GROWING OF GANS FOR IMPROVED QUALITY, STABILITY, AND VARIATION	Unsupervised CIFAR 10	Inception score: 8.80	Theano	2017

NLP

Machine Translation

Research Paper	Datasets	Metric	Source Code	Year
UNSUPERVISED MACHINE TRANSLATION USING MONOLINGUAL CORPORA ONLY	Multi30k-Task1(en-fr fr-en de-en en-de)	BLEU:(32.76 32.07 26.26 22.74)	NOT FOUND	2017
Unsupervised Neural Machine Translation with Weight Sharing	WMT14(en-fr fr-en) WMT16 (de-en en-de)	BLEU:(16.97 15.58) BLEU:(14.62 10.86)	NOT FOUND	2018

Transfer Learning

Research Paper	Datasets	Metric	Source Code	Year
One Model To Learn Them All	WMT EN → DE WMT EN → FR (BLEU) ImageNet (top-5 accuracy)	BLEU: 21.2 BLEU:30.5 86%	Tensorflow	2017

Reinforcement Learning

Research Paper	Datasets	Metric	Source Code	Year
Mastering the game of Go without human knowledge	the game of Go	ElO Rating: 5185	C++	2017

Email: [email protected]