If you would like to evaluate your results on the FigureQA test sets, please compile your predicted answers in the format documented below and email them to the authors.

Please send your results to Samira and Adam A. at figureqa@microsoft.com.

We provide the same statistics on each of the test1 and test2 sets as those in the FigureQA paper, namely:

  • Overall accuracy.
  • Accuracy by question type.
  • Accuracy by figure type.

Predictions Format

Results must be provided in a CSV format, having the first line as a header and each row like so:


Here  question_index  refers to the sample's index in the  qa_pairs  array within qa_pairs.json. All other fields are the same as in qa_pairs.json, documented here.

Here are example results files for test1 and test2. Simply replace the final column field  <0/1>  with your prediction as a  0  or  1  in each row.

To evaluate your results faster, please ensure that your results file is UTF-8 encoded and can be read using the read_csv function from the Python Pandas library.