Can not reproduce result

[ikou-austin]

Hi, I tried normally to reproduce your results on the Anti-Spoofing 2019 LA track, but I found that I could only get EER=1.64% min-tDCF=0.047 when I don't modify any training conditions, I would like to confirm with you if you used data augmentation and if you only used the official train subset for the training set? Thank you!

[Jungjee]

Hi, @ikou-austin.

We did not adopt any data augmentations (considering the tasks' property, we are cautious about adopting any kind of augmentations)

We also used the official train subset only as mentioned in the paper.

For exact results, we recommend to match the environment as similar as possible.
Also note that the performances can vary with different initialisation status (why we reported average and best performance in the paper).
If I recall correctly, even the worst experiment show demonstrate EER no higher than 1.3%

[ikou-austin]

Thank you for your reply. I will try to redeploy the environment and also do a few more trainings based on different random number seeds and I will report back to you as soon as possible if there is any progress.

[chenyihao0905]

Hi, i get EER=1.49% min-tDCF=0.047 when i use torch1.9.0, cuda 10.2, tesla T4 with driver version 440.33. Have you got any promotion based on different random number seeds? I tried several random number seeds and the best one is default.

[JJun-Guo]

Hi, i get EER=1.49% min-tDCF=0.047 when i use torch1.9.0, cuda 10.2, tesla T4 with driver version 440.33. Have you got any promotion based on different random number seeds? I tried several random number seeds and the best one is default.

how about eer and loss of the best model with dev set ?

[JJun-Guo]

hi，how about the newest result?

[LearnNLP]

Hi, I also couldn't reproduce the reported results. I obtained EERs of 1.521% and 1.536% for the two AASIST tests. The best dev EER was 0.305 with a loss of 0.00337, and the dev TDCF was 0.00867.

My environment is pytorch1.7.1, cuda10.1, tesla k80 with driver version: 440.33.01.
Another difference is the batch size, which was set to 20 due to GPU limitations.

How should I proceed? Thank you.