迁移学习 Transfer Learning

Everything about Transfer Learning (Probably the most complete repository?). Your contribution is highly valued! If you find this repo helpful, please cite it as follows:

关于迁移学习的所有资料，包括：介绍、综述文章、最新文章、代表工作及其代码、常用数据集、硕博士论文、比赛等等。(可能是目前最全的迁移学习资料库？) 欢迎一起贡献！ 如果认为本仓库有用，请在你的论文和其他出版物中进行引用！

@Misc{transferlearning.xyz,
howpublished = {\url{http://transferlearning.xyz}},   
title = {Everything about Transfer Learning and Domain Adapation},  
author = {Wang, Jindong and others}  
}  

Contents
0.Latest Publications (最新论文)	1.Introduction and Tutorials (简介与教程)
2.Transfer Learning Areas and Papers (研究领域与相关论文)	3.Theory and Survey (理论与综述)
4.Code (代码)	5.Transfer Learning Scholars (著名学者)
6.Transfer Learning Thesis (硕博士论文)	7.Datasets and Benchmarks (数据集与评测结果)
8.Transfer Learning Challenges (迁移学习比赛)	Applications (迁移学习应用)
Other Resources (其他资源)	Contributing (欢迎参与贡献)

关于机器学习和行为识别的资料，请参考：行为识别｜机器学习

---

0.Latest Publications (最新论文)

A good website to see the latest arXiv preprints by search: Transfer learning, Domain adaptation

一个很好的网站，可以直接看到最新的arXiv文章: Transfer learning, Domain adaptation

迁移学习文章汇总 Awesome transfer learning papers

• Weekly latest papers

  • 20200927 Privacy-preserving Transfer Learning via Secure Maximum Mean Discrepancy
    • 加密情况下的MMD用于迁移学习
  • 20200927 EMNLP-20 Feature Adaptation of Pre-Trained Language Models across Languages and Domains for Text Classification
    • 跨语言和领域的预训练模型用于文本分类
  • 20200927 Transfer Learning by Cascaded Network to identify and classify lung nodules for cancer detection
    • 迁移学习用于肺癌检测
  • 20200927 Knowledge Distillation for Multi-task Learning
    • 针对多任务学习的知识蒸馏
  • 20200915 Overcoming Negative Transfer: A Survey
    • 负迁移的综述
  • 20200914 ECML-PKDD-20 Towards Interpretable Multi-Task Learning Using Bilevel Programming
    • 用bilevel programming解释多任务学习

更多 More...

---

1.Introduction and Tutorials (简介与教程)

Want to quickly learn transfer learning？想尽快入门迁移学习？看下面的教程。

• The first transfer learning tutorial 入门教程

  • 《迁移学习简明手册》Transfer Learning Tutorial Read online, PDF
  • Zhihu blogs - 知乎专栏《小王爱迁移》系列文章

• Video tutorials 视频教程

  • Domain adaptation - 迁移学习中的领域自适应方法(中文)
  • Transfer learning by Hung-yi Lee @ NTU - 台湾大学李宏毅的视频讲解(中文视频)
  • Chelsea finn's Stanford CS330 class on multi-task and meta-learning - 2020斯坦福大学多任务与元学习教程CS330

• Brief introduction and slides 简介与ppt资料

  • Chinese introduction
  • PPT (English)
  • PPT (中文)
  • 迁移学习中的领域自适应方法 Domain adaptation: PDF ｜ Video
  • Transfer learning report by Mingsheng Long @ THU - 清华大学龙明盛老师的深度迁移学习报告：PPT(Samsung)、PPT(Google China)

• Talk is cheap, show me the code 动手教程、代码、数据

  • Pytorch官方迁移学习示意代码
  • Pytorch的finetune Fine-tune based on Alexnet and Resnet
  • 用Pytorch进行深度特征提取
  • 基于Tensorflow的几个深度adaptation实现
  • 更多 More...

• Transfer Learning Scholars and Labs - 迁移学习领域的著名学者、代表工作及实验室介绍

• Negative transfer - 负迁移？

---

2.Transfer Learning Areas and Papers (研究领域与相关论文)

Related articles by research areas:

• General Transfer Learning (普通迁移学习)
  • Theory (理论)
  • Others (其他)
• Domain Adaptation (领域自适应)
  • Traditional Methods (传统迁移方法)
  • Deep / Adversarial Methods (深度/对抗迁移方法)
• Domain Generalization
• Multi-source Transfer Learning (多源迁移学习)
• Heterogeneous Transfer Learning (异构迁移学习)
• Online Transfer Learning (在线迁移学习)
• Zero-shot / Few-shot Learning
• Deep Transfer Learning (深度迁移学习)
  • Non-Adversarial Transfer Learning (非对抗深度迁移)
  • Deep Adversarial Transfer Learning (对抗迁移学习)
• Multi-task Learning (多任务学习)
• Transfer Reinforcement Learning (强化迁移学习)
• Transfer Metric Learning (迁移度量学习)
• Transitive Transfer Learning (传递迁移学习)
• Lifelong Learning (终身迁移学习)
• Negative Transfer (负迁移)
• Transfer Learning Applications (应用)

Paperweekly: 一个推荐、分享论文的网站比较好，上面会持续整理相关的文章并分享阅读笔记。

---

3.Theory and Survey (理论与综述)

Here are some articles on transfer learning theory and survey.

Survey (综述文章)：

• The most influential survey on transfer learning （最权威和经典的综述）: A survey on transfer learning.

• Latest survey - 较新的综述：

  • 2020 负迁移的综述：Overcoming Negative Transfer: A Survey
    • 2020 知识蒸馏的综述: Knowledge Distillation: A Survey
    • 用transfer learning进行sentiment classification的综述：A Survey of Sentiment Analysis Based on Transfer Learning
    • 2019 一篇新survey：Transfer Adaptation Learning: A Decade Survey
    • 2018 一篇迁移度量学习的综述: Transfer Metric Learning: Algorithms, Applications and Outlooks
    • 2018 一篇最近的非对称情况下的异构迁移学习综述：Asymmetric Heterogeneous Transfer Learning: A Survey
    • 2018 Neural style transfer的一个survey：Neural Style Transfer: A Review
    • 2018 深度domain adaptation的一个综述：Deep Visual Domain Adaptation: A Survey
    • 2017 多任务学习的综述，来自香港科技大学杨强团队：A survey on multi-task learning
    • 2017 异构迁移学习的综述：A survey on heterogeneous transfer learning
    • 2017 跨领域数据识别的综述：Cross-dataset recognition: a survey
    • 2016 A survey of transfer learning。其中交代了一些比较经典的如同构、异构等学习方法代表性文章。
    • 2015 中文综述：迁移学习研究进展

• Survey on applications - 应用导向的综述：

  • 视觉domain adaptation综述：Visual Domain Adaptation: A Survey of Recent Advances
  • 迁移学习应用于行为识别综述：Transfer Learning for Activity Recognition: A Survey
  • 迁移学习与增强学习：Transfer Learning for Reinforcement Learning Domains: A Survey
  • 多个源域进行迁移的综述：A Survey of Multi-source Domain Adaptation。

Theory （理论文章）:

• Early transfer learning theory papers - 早期迁移学习的理论分析文章：

  • NIPS-06 Analysis of Representations for Domain Adaptation
  • ML-10 A Theory of Learning from Different Domains
  • NIPS-08 Learning Bounds for Domain Adaptation
  • COLT-09 Domain adaptation: Learning bounds and algorithms

• Latest theory papers

  • ICML-20 Few-shot domain adaptation by causal mechanism transfer

    • The first work on causal transfer learning
    • 日本理论组大佬Sugiyama的工作，causal transfer learning

  • CVPR-19 Characterizing and Avoiding Negative Transfer

    • Characterizing and avoid negative transfer
    • 形式化并提出如何避免负迁移

  • ICML-20 On Learning Language-Invariant Representations for Universal Machine Translation

    • Theory for universal machine translation
    • 对统一机器翻译模型进行了理论论证

  • ICML-19 Bridging Theory and Algorithm for Domain Adaptation

  • 最近几年在ICML、NIPS、COLT、ALT上出现了一些理论分析的文章，以domain adaptation为关键字可以搜索到。

• MMD (Maximum mean discrepancy):

  • MMD的提出：A Hilbert Space Embedding for Distributions 以及 A Kernel Two-Sample Test
  • 多核MMD(MK-MMD)：Optimal kernel choice for large-scale two-sample tests
  • MMD及多核MMD代码：Matlab | Python

---

4.Code (代码)

请见这里 | Please see HERE for some popular transfer learning codes.

---

5.Transfer Learning Scholars (著名学者)

Here are some transfer learning scholars and labs.

全部列表以及代表工作性见这里

Please note that this list is far not complete. A full list can be seen in here. Transfer learning is an active field. If you are aware of some scholars, please add them here.

• General transfer learning algorithms and applications:

  • Qiang Yang：中文名杨强。香港科技大学计算机系讲座教授，迁移学习领域世界性专家。IEEE/ACM/AAAI/IAPR/AAAS fellow。[Google scholar]

  • Sinno Jialin Pan：杨强的学生，香港科技大学博士，现任新加坡南洋理工大学助理教授。迁移学习领域代表性综述A survey on transfer learning的第一作者（Qiang Yang是二作）。[Google scholar]

• Transfer learning algorithms:

  • Wenyuan Dai：中文名戴文渊，上海交通大学硕士，现任第四范式人工智能创业公司CEO。迁移学习领域著名的牛人，在顶级会议上发表多篇高水平文章，每篇论文引用量巨大。[Google scholar]

  • Mingsheng Long：中文名龙明盛，清华大学博士，现任清华大学助理教授、博士生导师。[Google scholar]

  • Lixin Duan：中文名段立新，新加坡南洋理工大学博士，现就职于电子科技大学，教授。[Google scholar]

• Transfer learning + computer vision

  • Boqing Gong：南加州大学博士，现就职于腾讯AI Lab(西雅图)。曾任中佛罗里达大学助理教授。[Google scholar]

  • Tatiana Tommasi：Researcher at the Italian Institute of Technology.

  • Vinod K Kurmi[home page]: Researcher at the Indian Institute of Technology Kanpur(India)

• Transfer learning + recommendation systems

  • Weike Pan：中文名潘微科，杨强的学生，现任深圳大学副教授，香港科技大学博士毕业。主要做迁移学习在推荐系统方面的一些工作。 [Google Scholar]

  • Fuzhen Zhuang：中文名庄福振，中科院计算所博士，现任中科院计算所副研究员。[Google scholar]

• Online transfer learning:

  • Qingyao Wu：中文名吴庆耀，现任华南理工大学副教授。主要做在线迁移学习、异构迁移学习方面的研究。[Google scholar]

• Theory:

  • Tongliang Liu：中文名刘同亮，现任悉尼大学助理教授。主要做迁移学习的一些理论方面的工作。[Google scholar]

---

6.Transfer Learning Thesis (硕博士论文)

Here are some popular thesis on transfer learning.

硕博士论文可以让我们很快地对迁移学习的相关领域做一些了解，同时，也能很快地了解概括相关研究者的工作。其中，比较有名的有

• 2016 Baochen Sun的Correlation Alignment for Domain Adaptation

• 2015 南加州大学的Boqing Gong的Kernel Methods for Unsupervised Domain Adaptation

• 2014 清华大学龙明盛的迁移学习问题与方法研究

• 2014 中科院计算所赵中堂的自适应行为识别中的迁移学习方法研究

• 2012 杨强的学生Hao Hu的Learning based Activity Recognition

• 2012 杨强的学生Wencheng Zheng的Learning with Limited Data in Sensor-based Human Behavior Prediction

• 2010 杨强的学生Sinno Jialin Pan的Feature-based Transfer Learning and Its Applications

• 2009 上海交通大学戴文渊的基于实例和特征的迁移学习算法研究

其他的文章，请见完整版。

---

7.Datasets and Benchmarks (数据集与评测结果)

Please see HERE for the popular transfer learning datasets and benchmark results.

这里整理了常用的公开数据集和一些已发表的文章在这些数据集上的实验结果。

---

8.Transfer Learning Challenges (迁移学习比赛)

• Visual Domain Adaptation Challenge (VisDA)

---

Applications (迁移学习应用)

See HERE for transfer learning applications.

迁移学习应用请见这里。

---

Other Resources (其他资源)

• Call for papers:

  • DLKT: Deep Learning for Knowledge Transfer @ ICDM 2020

• Related projects:

  • Salad: A semi-supervised domain adaptation library
  • Dassl: A PyTorch toolbox for domain adaptation and semi-supervised learning

---

Contributing (欢迎参与贡献)

If you are interested in contributing, please refer to HERE for instructions in contribution.

---

Copyright notice

[Notes]This Github repo can be used by following the corresponding licenses. I want to emphasis that it may contain some PDFs or thesis, which were downloaded by me and can only be used for academic purposes. The copyrights of these materials are owned by corresponding publishers or organizations. All this are for better adademic research. If any of the authors or publishers have concerns, please contact me to delete or replace them.

[文章版权声明]这个仓库可以遵守相关的开源协议进行使用。这个仓库中包含有很多研究者的论文、硕博士论文等，都来源于在网上的下载，仅作为学术研究使用。我对其中一些文章都写了自己的浅见，希望能很好地帮助理解。这些文章的版权属于相应的出版社。如果作者或出版社有异议，请联系我进行删除。一切都是为了更好地学术！