Skip to content

Commit

Permalink
remove resulting files, make downloadable from gdrive
Browse files Browse the repository at this point in the history
  • Loading branch information
mertyg committed Jul 8, 2024
1 parent acc28b9 commit e23503b
Show file tree
Hide file tree
Showing 3 changed files with 3 additions and 266,629 deletions.
7 changes: 3 additions & 4 deletions evaluation/results/solution_optimization_mm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

We are pleased to release the results of our solution optimization experiments for multimodal tasks, including [MathVista](https://mathvista.github.io/) and [ScienceQA](https://scienceqa.github.io/) tasks. This release aims to ensure transparency and provide detailed insights into our optimization processes.

The files in this repository contain the solution trajectory, final solution, and the answers for each question in the datasets. Each `.json` file is structured as a dictionary where the key is the question index, and the value is another dictionary with the following possible keys:
The files in this repository contain the solution trajectory, final solution, and the answers for each question in the datasets. Each `.json` file (can be downloaded from google drive) is structured as a dictionary where the key is the question index, and the value is another dictionary with the following possible keys:

- **`question`**: The question text.
- **`answer`**: The ground truth answer.
Expand Down Expand Up @@ -31,7 +31,7 @@ python solution_optimization_mm.py --task mathvista \
--majority_voting
```

The result file is `mathvista_predictions.json`.
The resulting `mathvista_predictions.json` file can be downloaded from [here](https://drive.google.com/drive/u/0/folders/1hJ6kQozkTNiUPxtEhZCgCqnK2EFW5MHY).

## Experiment on ScienceQA

Expand All @@ -49,5 +49,4 @@ python solution_optimization_mm.py --task scienceqa \
--num_threads 20
```

The result file is `scienceqa_predictions.json`.

The resulting `scienceqa_predictions.json` file can be downloaded from [here](https://drive.google.com/file/d/1BkMD3CcaxAUpB-9L0aI8bLEqoo8jHqlZ/view?usp=sharing).
Loading

0 comments on commit e23503b

Please sign in to comment.