Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can't reproduce the paper's performance #18

Open
JYeonKim opened this issue Jan 12, 2023 · 1 comment
Open

can't reproduce the paper's performance #18

JYeonKim opened this issue Jan 12, 2023 · 1 comment

Comments

@JYeonKim
Copy link

JYeonKim commented Jan 12, 2023

Hi, Thank you very much for the code you shared.

I tried to run the code with the following arguments:

python main.py -lr=5e-5 -ep=40 -mod=tav -bs=8 --img-interval=500 --early-stop=6 --loss=bce --cuda=3 --model=mme2e_sparse --num-emotions=6 --trans-dim=64 --trans-nlayers=4 --trans-nheads=4 --text-lr-factor=10 -st=0.7 --text-model-size=base --text-max-len=100 --dataset="iemocap"

The papers syas that we get an accuracy of 84.4% but the best accuracy that I am getting is 83.0% on the IEMOCAP dataset. Could you please put the exact arguments to reproduce the results of the papers? How can I get the results of the papers?

(I think "p" equals "st" argument. So I set "st=0.7")

ACC ang exc fru hap neu sad average
paper(p=0.7) 88.2 88.3 74.9 89.5 77.0 88.6 84.4
me(p=0.7) 88.8 80.6 77.9 90.4 74.5 86.0 83.0
me(p=0.9) 88.3 87.5 76.7 90.2 72.1 90.6 84.2

best regards,
Juyeon Kim

@JYeonKim JYeonKim changed the title accuracy mismatch can't reproduce the paper's performance Jan 12, 2023
@rafiepour
Copy link

rafiepour commented Apr 15, 2023

Hi, same thing is happening for me. Any updates?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants