Average Words per Article
Average Words per Answer
Human Performance (F1)
Reasoning mechanisms needed to answer questions in NewsQA based on 500 examples. For each type, we show an example question with the text snippet that contains the answer span, with words relevant to the reasoning type in bold.
Q: When were the findings published?
T: Both sets of research findings were published Thursday...
Q: Who is the struggle between in Rwanda?
T: The struggle pits ethnic Tutsis, supported by Rwanda, against ethnic Hut
Q: Where is Brittanee Drexel from?
T: The mother of a 17-year-old Rochester, New York high school student ... says she did not give her daughter permission to go on the trip. Brittanee Marie Drexel's mom says...
Q: Who drew inspiration from presidents?
T: Rudy Ruiz says the lives of US presidents can make them positive role models for students.
Q: Whose mother is moving to the White House?
T: ... Barack Obama's mother-in-law, Marian Robinson will join the Obamas at the family's private quarters at 1600 Pennsylvania Avenue. [Michelle is never mentioned]
With Consensus (Including Validated)
Only 51,630 needed to be validated because there was agreement during second step of the collection.
Validated Answers with Consensus