절차적 비디오 이해를 위한 행동 이해 태스크에서의 편향 탐구

Code for Exploring bias in Action Understanding Task for Comprehending Instructional Videos (KSC 2023)

Thesis version

This repository only contains video modification and sample visualization code. Refer to each model repository for train and inference.

Setup

pip install numpy tqdm setuptools

Data

1. Download Features of 50salads, GTEA and Breakfast provided by ASFormer and MS-TCN.
2. Unzip the data, rename it to "data" and put into the current directory

Run

python main.py --dataset {gtea/breakfast/50salads} --mode {blank/mask,all} --seed {seed} --name {name}

Evaluate (Visualize)

Use visualize.ipynb by changing directories in shell 2, and 3

Output

./data_{name}

Result

Modification Example

Quantitative Result

Qualitative Result

Cite

Joochan Kim, Minjoon Jung, & Byoung-Tak Zhang (2023-12-20). Exploring Bias in Action Understanding Task for Comprehending Instructional Videos. 한국정보과학회 학술발표논문집, 부산.

Acknowledgement

Datasets

1. GTEA
2. 50Salads
3. Breakfast

Models

1. ASFormer
2. MS-TCN
3. MS-TCN++
4. DiffAct