5.2 Construction of Training and Testing Sets
‘Test c’, we used the words in the question title part
We made use of the questions crawled from Yahoo!
as the main search query and the other words in the
Answers for the estimating models and evaluation.
information need part as search query expansion to
More specifically, we obtained 2 million questions
retrieve candidate recommended questions from Ya-
under two categories at Yahoo! Answers: ‘travel’
hoo! Answers website. We obtained an average of
154 resolved questions under ‘travel’ or ‘computer-
5
https://traloihay.net
s&internet’ category, and three assessors were in-
6
https://traloihay.net
volved in the manual judgments.
logId=LDC2006T13
7
https://traloihay.net jimmylin/resources.html
Given a question returned by a recommendation
method, two assessors are asked to label it with
‘good’ or ‘bad’. The third assessor will judge the
conflicts. The assessors are also asked to read the in-
formation need and answer parts. If a recommended
Q1: If I want a faster computer should I buy
question is considered to express the same or similar
more memory or storage space?
information need, the assessor will label it ‘good’;
InfoN If I want a faster computer should I buy
otherwise, the assessor will label it as ‘bad’.
more memory or storage space? What-
Three measures for evaluating the recommenda-
s the difference? I edit pictures and
tion performance are utilized. They are Mean Re-
videos so I need them to work quickly....
ciprocal Rank (MRR), top five prediction accura-
RQ1 Would buying 1gb memory upgrade
cy (precision@5) and top ten prediction accuracies
make my computer faster?
(precision@10) (Voorhees and Tice, 2004; Cao et
InfoN I have an inspiron B130. It has 512mb
al., 2008). In MRR the reciprocal rank of a query
memory now. I would add another 1gb
question is the multiplicative inverse of the rank of
into 2nd slot ...
the first ‘good’ recommended question. The top five
RQ2 whats the difference between memory
prediction accuracy for a query question is the num-
and hard drive space on a computer and
ber of ‘good’ recommended questions out of the top
why is...?InfoN see I am starting edit videos on my com-
five ranked questions and the top ten accuracy is cal-
puter but i am running out of space. why
culated out of the top ten ranked questions.
is so expensive to buy memory but notexternal drives? ...
