Skip to content

Latest commit

 

History

History
134 lines (99 loc) · 11.9 KB

README.md

File metadata and controls

134 lines (99 loc) · 11.9 KB

AIGC Digital Human (Update)

This is a repository for organizing papres, codes and other resources related to AIGC Digital Human.

🔆 This project is still on-going, pull requests are welcomed!!

⭐ If you find this repo useful, please star it!!!

Contents

Introduction

In this documentation, we have collected papers, databases, and code resources related to AIGC Digital Human. Digital Humans refer to virtual entities generated and simulated by computers, possessing human characteristics and behaviors.

2D Digital Human

Large Language Model (LLM)

# LLM Paper Code/Project
1 ChatGPT-3.5 - ChatGPT
2 ChatGLM-6B "GLM-130B: An Open Bilingual Pre-trained Mode" github
3 Qwen (通义千问) "QWEN TECHNICAL REPORT" github

Text2Speech Conversion

# Model Paper Code/Project
1 Espeaker - Web

Speech Clone

# Model Paper Code/Project
1 MockingBird - github
2 Real-Time-Voice-Cloning "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis" github

Face Driving

# Model Paper Code/Project
1 MakeIttalk "MakeItTalk: Speaker-Aware Talking-Head Animation" github
2 Audio2Head "Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion" github
3 Sadtalker "SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation" github
4 Dreamtalk "DreamTalk: When Expressive Talking Head Generation Meets Diffusion Probabilistic Models" github
5 Wav2Lip "A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild" github
6 Video-Retalking "VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild" github
7 DINet "DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Video" github
8 IP-LAP "Identity-Preserving Talking Face Generation with Landmark and Appearance Priors" github

Cloth Modification

# Model Paper Code/Project
1 VITON-HD "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis" github

Style Transfer

# Model Paper Code/Project
1 VToonify "VToonify: Controllable High-Resolution Portrait Video Style Transfer" github
2 DCT-Net "DCT-Net: Domain-Calibrated Translation for Portrait Stylization" gihub

Super Resolution

# Model Paper Code/Project
1 BasicVSR++ "On the Generalization of BasicVSR++ to Video Deblurring and Denoising" github

Quality Assessment

# Model Paper Code/Project
1 VSFA "Quality Assessment of In-the-Wild Videos" github

3D Digital Human

NeRF

# Model Paper Code/Project
1 HumanNeRF "HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video" github

Gaussian

# Model Paper Code/Project
1 HumanGaussian "HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting" github

3D Quality Assessment

# Model Paper Code/Project
1 - "A No-Reference Quality Assessment Method for Digital Human Head" -
2 - "Geometry-Aware Video Quality Assessment for Dynamic Digital Human" -
3 - Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluation -
4 - Perceptual Quality Assessment for Digital Human Heads -

Databases

# Database Name Type Title & Link Database Link
1 NoW Generation "Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision" Link
2 FaceScape Generation "FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction" Link
3 Human3.6M Generation "Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments" Link
4 ZJU-Mocap Generation "Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans" Link
5 BEAT Gesture Synthesis "BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis" Link
6 VOCA Face Driving "Capture, Learning, and Synthesis of 3D Speaking Styles" Link
7 MultiFace Face Driving "Multiface: A Dataset for Neural Face Rendering" Link
8 DHHQA Quality Assessment Perceptual Quality Assessment for Digital Human Heads Link
9 DDH-QA Quality Assessment DDH-QA: A DYNAMIC DIGITAL HUMANS QUALITY ASSESSMENT DATABASE Link
10 SJTU-H3D Quality Assessment Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluation Link
11 6G-DHQA Quality Assessment Quality-of-Experience Evaluation for Digital Twins in 6G Network Environments Link
12 THQA Quality Assessment THQA: A PERCEPTUAL QUALITY ASSESSMENT DATABASE FOR TALKING HEADS Link

Related Reference

Awesome-Talking-Face