README.md

Awesome-Indoor-Scene-Synthesis

A curated list of awesome Indoor Scene Synthesis papers.

Table of Contents

• Papers
• Datasets

Papers

3D Scene synthesis

Single Mesh Generation

• SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generation | arxiv 2024 |github | project

Compositional Mesh Generation

• MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation | arxiv 2024 | github | project
• Disentangled 3D Scene Generation with Layout Learning, Epstein, et al., arxiv 2024 | bibtex
• GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting | arxiv 2024 | project
• SceneWiz3D: Towards Text-guided 3D Scene Composition, Zhang et al., arxiv 2023 | bibtext
• Compositional 3D Scene Generation using Locally Conditioned Diffusion, arxiv 2023 | project
• GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs, Gao et al., arxiv 2023 | github | bibtext

applying LLM

• The Scene Language: Representing Scenes with Programs, Words, and Embeddings | github | project
• Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases | arxiv 2024
• LAYOUTVLM: Differentiable Optimization of 3D Layoutvia Vision-Language Models | arxiv 2024 | github | project
• Chat2Layout: Interactive 3D Furniture Layout with a Multimodal LLM | arxiv 2024
• LLplace: The 3D Indoor Scene Layout Generation and Editing via Large Language Model | arxiv 2024
• SceneCraft: An LLM Agent for Synthesizing 3D Scenes as Blender Code | ICML 2024
• I-Design: Personalized LLM Interior Designer, arxiv 2024 | github | project
• GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting | arxiv 2024 | github | project
• SceneTeller: Language-to-3D Scene Generation | arxiv 2024 | project
• Holodeck: Language Guided Generation of 3D Embodied AI Environments | arxiv 2023| github | project
• AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes | arXiv 2023 | github | project
• LayoutGPT: Compositional Visual Planning and Generation with Large Language Models, arxiv 2023| github | project
• 3D-GPT: PROCEDURAL 3D MODELING WITH LARGE LANGUAGE MODELS | project

Model Training without Scene Graph

• Mixed Diffusion for 3D Indoor Scene Synthesis | arxiv 2024 | github
• RoomDesigner: Encoding Anchor-latents for Style-consistent and Shape-compatible Indoor Scene Generation, Zhao et al., 3DV 2024 | github | bibtext
• Mixed Diffusion for 3D Indoor Scene Synthesis | arxiv 2024 | github
• Mixed Diffusion for 3D Indoor Scene Synthesis | arxiv 2024 | github
• PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI | CVPR 2024 | github |project
• ATISS: Autoregressive Transformers for Indoor Scene Synthesis | NIPS 2021 | github | project
• DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis | arXiv 2024 | CVPR 2024 | github | project
• Learning 3D Scene Priors with 2D Supervision | CVPR 2023 | github | peoject
• LEGO-Net: Learning Regular Rearrangements of Objects in Rooms | CVPR 2023 | github | project
• CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embedding | arXiv 2023
• Learning 3D Scene Priors with 2D Supervision | CVPR 2023 | github | project
• SceneFormer: Indoor Scene Generation with Transformers | 3DV 2021 | github | project
• Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Models | CVPR 2019
• Scene Synthesis via Uncertainty-Driven Attribute Synchronization | ICCV 2021 | github

Model Training with Scene Graph

• EcoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusion | ECCV 2025 | github | project
• InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior | ICLR 2024 | github | project
• CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion | NIPS 2023 | github | project
• 3D Scene Diffusion Guidance using Scene Graphs | arXiv 2023
• SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentation | ICCV 2019

Using 2D Prior

• Lay-A-Scene: Personalized 3D Object Arrangement Using Text-to-Image Priors | arxiv 2024 | project

Procedure / Human Design Rules

• Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation | CVPR 2024 |github | project
• Holodeck: Language Guided Generation of 3D Embodied AI Environments | arxiv 2024 | github | project
• I-Design: Personalized LLM Interior Designer | arxiv 2024 | github | project
• ProcTHOR: Large-Scale Embodied AI Using Procedural Generation | NeurIPS 2022 | github | bibtex

Conditioned on Human Montion

• Human-Aware 3D Scene Generation with Spatially-constrained Diffusion Models | arxiv 2024
• Scene Synthesis from Human Motion | SIGGRAPH Asia 2022 | github | project
• Pose2Room: Understanding 3D Scenes from Human Activities | ECCV 2022 | github | project
• Human-centric Indoor Scene Synthesis Using Stochastic Grammar | CVPR 2018
• Rearrange Indoor Scenes for Human-Robot Co-Activity | ICRA 2023 | github | project
• MIME: Human-Aware 3D Scene Generation | CVPR 2023 | github | project
• Language-driven Scene Synthesis using Multi-conditional Diffusion Model | arXiv 2023 | github | project

Condition on 2D Image

• Automated Creation of Digital Cousins for Robust Policy Learning | github | project
• Holistic 3D Scene Understanding from a Single Image with Implicit Representation | CVPR 2021 | github |project
• Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image | CVPR 2020 | github | project

MCMC

• Configurable 3D Scene Synthesis and 2D Image Rendering with Per-Pixel Ground Truth using Stochastic Grammars | IJCV (2018)

Texture Synthesis

Scene texture generation

• RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpainting | arXiv 2024 | github | project
• Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details | CVPR 2023
• Paint3D: Paint Anything 3D with Lighting-less Texture Diffusion Models | CVPR 2024 | project

Object texture generation

• FlashTex: Fast Relightable Mesh Texturing with LightControlNet | ECCV 2024 Oral | github | project
• BlenderAlchemy: Editing 3D Graphics with Vision-Language Models | ECCV 2024| github | project

Material generation

• MatFuse: Controllable Material Generation with Diffusion Models CVPR 2024 | project
• ControlMat: A Controlled Generative Approach to Material Capture ACM ToG (invited to SIGGRAPH Asia 2024) | project

2D Layout / General layout

• LayoutTransformer: Layout Generation and Completion with Self-attention | ICCV2021 | github | project

Datasets

Scene

• Habitat Synthetic Scenes Dataset (HSSD):An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation | CVPR 2024 | github | project
• 3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics | ICCV 2021 | project
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes | CVPR 2017 | github | project
• ProcTHOR: Large-Scale Embodied AI Using Procedural Generation, Deitke et al., NeurIPS 2022 | github | bibtex

Object

• Objaverse-XL | NeurIPS 2023 | github

How to submit a pull request?

Modify the README.md and follow the format TITLE, AUTHOR, CONFERENCE YEAR | OPTIONAL LINK | BIBTEX. For example:

• PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI | CVPR 2024 | github |project

License

Awesome-Indoor-Scene-Synthesis is released under the MIT license.

Citation

If you find this project useful in your research, please consider citing:

This repo is inspired by awesome-NeRF and Awesome-AIGC-3D.

Contact

contact: [email protected].