3 RESULTS AND DISCUSSION CONTAINING NES OF THE QUESTION EXPECTED ANS...
1.3 Results and discussion
containing NEs of the question Expected Answer
There are many problems involved when trying
Type (EAT). Table 2 presents the results ob-
to perform mixed fact and opinion QA. The first
tained for English and Table 3 for Spanish. We
can be the ambiguity of the questions e.g. ¿De
indicate the id of the question (Q), the question
dónde viene la riqueza de EEUU?. The answer
type (T) and the number of answer of the Gold
can be explicitly stated in one of the blog sen-
Standard (A). We present the number of the re-
tences, or a system might have to infer them
trieved questions by the traditional system
from assumptions made by the bloggers and their
(TQA) and by the opinion one (OQA). We take
comments. Moreover, most of the opinion ques-
into account the first 1, 5, 10 and 50 answers.
tions have longer answers, not just a phrase snip-
pet, but up to 2 or 3 sentences. As we can ob-
Number of found answers
Q
T
A
serve in Table 2, the questions for which the
@1
@5
@10
@ 50
TQA system performed better were the pure fac-
TQA
OQA
TQA
OQA
TQA
OQA
TQA
OQA
tual ones (1, 3, 4, 8, 10 and 14), although in some
1
F
5
0
0
0
2
0
3
4
4
cases (question number 14) the OQA system re-
2
O
5
0
0
0
1
0
1
0
3
trieved more correct answers. At the same time,
3
F
2
1
1
2
1
2
1
2
1
4
F
10
1
1
2
1
6
2
10
4
opinion queries, although revolving around NEs,
5
O
11
0
0
0
0
0
0
0
0
were not answered by the traditional QA system,
6
O
2
0
0
0
0
0
1
0
2
but were satisfactorily answered by the opinion
7
O
5
0
0
0
0
0
1
0
3
8
F
5
1
0
3
1
3
1
5
1
QA system (2, 5, 6, 7, 11, 12). Questions 18 and
9
F
5
0
1
0
2
0
2
1
3
20 were not correctly answered by any of the two
10
F
2
1
0
1
0
1
1
2
1
systems. We believe the reason is that question
11
O
2
0
1
0
1
0
1
0
1
12
O
3
0
0
0
1
0
1
0
1
18 was ambiguous as far as polarity of the opin-
13
F
1
0
0
0
0
0
0
0
1
ions expressed in the answer snippets (“im-
14
F
7
1
0
1
1
1
2
1
2
provement” does not translate to either “positive”
15
F/O
1
0
0
0
0
0
1
0
1
or “negative”) and question 20 referred to the
16
F/O
6
0
1
0
4
0
4
0
4
17
F
10
0
1
0
1
4
1
0
2
title of a project proposal that was not annotated
18
F/O
1
0
0
0
0
0
0
0
0
by any of the tools used. Thus, as part of the fu-
19
F/O
27
0
1
0
5
0
6
0
18
ture work in our OQA system, we must add a
20
F/O
4
0
0
0
0
0
0
0
0
component for the identification of quotes and
Table 2: Results for English
titles, as well as explore a wider range of polar-
ity/opinion scales. Furthermore, questions 15, 16,
Q
T A