SECTION 3.3. A BENEFIT, SINCE IT GIVES MORE OPPORTUNITY FOR ENFORC-

Table 3. Evaluation on TREC12 Factoids.

5 Discussion

Table 2. Evaluation on AQUAINT and CNS

The experiments reported here pointed out many

corpora.

areas of our system which previous failure analysis

of the basic QA system had not pinpointed as being

On the AQUAINT corpus, four out of seven 2

^nd

too problematic, but for which improvement should

place finishers went to first place. On the CNS cor-

help the Constraints process. In particular, this work

pus 16 out of a possible 26 correct no-answer cases

brought to light a matter of major significance, term

were discovered, at a cost of losing three previously

equivalence, which we had not previously focused

correct answers. The percentage correct score in-

on too much (and neither had the QA community as

creased by a relative 10.3% for AQUAINT and

a whole). We will discuss that in Section 5.4.

186% for CNS. In both cases, the error rate was

reduced by about a third.

Quantitatively, the results are very encouraging, but