How to reproduce the results on ScanQA datasets?

[hulianyuyy]

Many thanks for the great work! I want to ask how to reproduce the results on ScanQA datasets? Could the authors provide some ideas?

[kellyiss]

We co-train the model on the ScanQA dataset in the second stage. After co-training, you can then evaluate the model directly on the ScanQA benchmark. You can test our released model for inference.

[hulianyuyy]

Thanks for your kind reply. I want to ask if your reop current support evaluating the ScanQA dataset by conducting a script just like sh scripts/streamvln_eval_multi_gpu.sh? Or if we could just modify the existed streamvln_eval_multi_gpu.sh and vln_r2r.yaml to support evaluating the ScanQA datasets?