Question-Answer Pairs


Average Words per Article


Average Words per Answer


Human Performance (F1)

Reasoning Statistics

Reasoning mechanisms needed to answer questions in NewsQA based on 500 examples. For each type, we show an example question with the text snippet that contains the answer span, with words relevant to the reasoning type in bold.

Reasoning Proportion Example
Word Matching 31.6%

Q: When were the findings published?

T: Both sets of research findings were published Thursday...

Paraphrasing 26.8%

Q: Who is the struggle between in Rwanda?

T: The struggle pits ethnic Tutsis, supported by Rwanda, against ethnic Hut

Synthesis 17.8%

Q: Where is Brittanee Drexel from?

T: The mother of a 17-year-old Rochester, New York high school student ... says she did not give her daughter permission to go on the trip. Brittanee Marie Drexel's mom says...

Inference 14.0%

Q: Who drew inspiration from presidents?

T: Rudy Ruiz says the lives of US presidents can make them positive role models for students.

Ambiguous/ Insufficient 9.8%

Q: Whose mother is moving to the White House?

T: ... Barack Obama's mother-in-law, Marian Robinson will join the Obamas at the family's private quarters at 1600 Pennsylvania Avenue. [Michelle is never mentioned]

Consensus Statistics


With Consensus (Including Validated)



Validated Answers

Only 51,630 needed to be validated because there was agreement during second step of the collection.


Validated Answers with Consensus


Story Length Distribution

Question Length Distribution

Question Type Distribution

Overall Answer Length Distribution

Average Answer Length Distribution per Question