Skip to content

Latest commit

 

History

History
81 lines (55 loc) · 3.44 KB

README.md

File metadata and controls

81 lines (55 loc) · 3.44 KB
Raw

SegmentStyler [Project's Report]

Part-aware language-based mesh texture editing

SegmentStyler is a framework for part-aware mesh texture editing that uses natural language and allows for separate customization of different mesh segments based on multiple text prompts, resulting in a coherent texture with the text input.
rainbow pattern arm
glowing iron leg
guy fawkes logo back
camo pattern seat		 persian carpet arm
swiss cheese leg
desert sand back
fleur-de-lis pattern seat		 watermelon pattern arm
red leather leg
graffiti back
colorful crochet seat		 snowflake pattern arm
metallic leg
honeycomb back
wooden seat

Note

This project was implemented during a guided research practical course at TUM in the Winter Semester of 2022/2023 by Maximilian Winter and Murilo Bellatini. For more information, please refer to the project's report, which can be found here.

Getting Started

Installation

  1. Run setup.sh.
sh ./setup.sh

  1. Setup PartGlot

    1. Download data and weights here.
    2. Add data files and weights to data/PartGlotData folder.

  2. Setup Data (Note: this step can be skipped if you only want to try out the pipeline with the one shape that's already given in data/)

    1. Download ShapeNetCore.v2 here
    2. Download the part annotations here
    3. Start pre-processing (remeshing and fixing normals) by running
    python src/helper/get_new_normals_from_sdf.py

    For the pre-processing, make sure to have the following packages installed

    • trimesh
    • mesh_to_sdf
    • skimage To avoid conflicts, it is recommended to install them (especially the last two) in a separate environment.

  3. Activate environment.

conda activate dl3d
  1. Confirm Weights & Biases installation. More info: click here.
pip install wandb
wandb login
  1. Enjoy ;)

Run examples

To see an example, you can run

bash example_run.sh

To gain more insights, you can have a look at

  • evaluation_baseline.sh
  • ablation_study_b_to_g.sh which were used for performing our ablation study.

Miscellaneous

The notebooks folder contains code snippets that we used during our development process. You can find a comprehensive index of these notebooks here. However, please note that the documentation within each notebook is currently limited and may not be up-to-date. We apologize for any inconvenience and appreciate your understanding.

Reference Projects

  1. Text2Mesh
  2. PartGlot