AN ASSERTED DATE (WHICH WAS THE TOP CANDIDATE A PRECISE METHOD FOR...

2. An asserted date (which was the top candidate

a precise method for computing the QDC scores.

from its associated question) can get replaced by

One way to formalize this process would be to treat

a candidate date originally in positions 2-6

it as evidence gathering and interpret the results in a

(where sixth place is NIL)

Bayesian-like fashion. The original system confi-

Effect #1 is expected to increase precision at the

dences would represent prior probabilities reflecting

risk of worsening recall; effect #2 can go either way.

the system’s belief that the answers are correct. As

We note that NIL, which is only used for dates, can

more evidence is found, the confidences would be

be the correct answer if the desired date assertion is

updated to reflect the changed likelihood that an an-

absent from the corpus; NIL is considered a “value”

swer is correct.

in this evaluation.

We do not know a priori how much “slop” should

By inspection, performances and other indirect

be allowed in enforcing the constraints, since auxil-

works (discussed in the previous section) were usu-

iary questions are as likely to be answered incor-

rectly as the original ones. A further problem is to