SECTION 4 HIGHLIGHTS THE MANAGEMENT OF THE INTER-FORMANCE OF A SYSTEM...

2002).

off only, we used a question type similarity

based on a matrix akin to the one reported in

Similarity Metric 1 is based on two process-

(Lytinen and Tomuro, 2002)

ing steps:

Similarity Metric 5 is based on question con-

(a) the content words of the questions are

cepts rather than question terms. In order to

-

measure used in In-

weighted using the

translate question terms into concepts, we re-

formation Retrieval

1

placed (a) question stems (i.e. a WH-word +

, where

is the number of

Z9

&

NP construction) with expected answer types

-

is the num-

questions in the QUAB,

?

(taken from the answer type hierarchy em-

ber of questions containing

and

is

ployed by F

ERRET

’s Q/A system) and (b)

the number of times

appears in the ques-

named entities with corresponding their corre-

tion. This allows the user question and any

sponding classes. Remaining nouns and verbs

QUAB question to be transformed into two

were also replaced with their WordNet seman-

vectors,

and

tic classes, as well. Each concept was then as-

;

"!

sociated with a weight: concepts derived from

(b) the term vector similarity is used to compute

named entities classes were weighted heavier

the similarity between the user question and

than concepts from answer types, which were

any question from the QUAB:

#

%$

J

in turn weighted heavier than concepts taken

%

?

%

?

?

9

%

?

from WordNet clases. Similarity was then com-

('

puted across “matching” concepts.

5

The resul-

Similarity Metric 2 is based on the percent of

tant similarity score was based on three vari-

user question terms that appear in the QUAB

ables:

question. It is obtained by finding the intersec-

S

= sum of the weights of all concepts matched

tion of the terms in the term vectors of the two

between a user query (

T

) and a QUAB query

questions.

(

TVU

);

W

= sum of the weights of all unmatched con-

Similarity Metric 3 is based on semantic in-

cepts in

T

formation available from WordNet. It involves:

X

= sum of the weights of all unmatched con-

(a) finding the minimum path between Word-

cepts in

TVU

;

and

,

Net concepts. Given two terms

and

TYU

was calcu-

The similarity between

T

<

each with

T

and

)

WordNet senses

S

3

!

'

W

3

!

U

'

X

, where

!

lated as

and

*

<

*

<

<

-

. The se-

,+.-

and

/

!

U

were used as coefficients to penalize the con-

<

is

mantic distance between the terms

0

tribution of unmatched concepts in

T

and

TVU

defined by the minimum of all the possible pair-

respectively.

6

<

:

wise semantic distances between

B

, where

Similarity Metric 6 is based on the fact that the

0

<

13254

B

is the path length between

and

B

.

=

?

(

%76

G

(

5

In the case of ambiguous nouns and verbs associated with

(b) the semantic similarity between the user

multiple WordNet classes, all possible classes for a term were

<

+

question

:

and the QUAB

considered in matching.

8

to be defined

6

We set

Z

@

= 0.4 and

Z[

= 0.1 in our experiments.

question

:9

;:

:</

<

:

QUABs:

Q1: Does Iran have an indigenous CW program?

(1a) How did Iran start its CW program?

(1b) Has the plant at Qazvin been linked to CW production?

Answer (A1):

(1c) What CW does Iran produce?

Although Iran is making a concerted effort to attain an independent production capability for all aspects of chemical

weapons program, it remains dependent on foreign sources for chemical warfare−related technologies.

Q2: Where are Iran’s CW facilities located?

(2a) What factories in Iran could produce CW?

(2b) Where are Iran’s stockpiles of CW?

(2c) Where has Iran bought equipment to produce CW?

Answer(A2):

According to several sources, Iran’s primary suspected chemical weapons production facility is located in the city of Damghan.

Q3: What is Iran’s goal for its CW program?

(3a) What motivated Iran to expand its chemical weapons program?

(3b) How do CW figure into Iran’s long−term strategic plan?

(3c) What are Iran’s future CW plans?

Answer(A3):

In their pursuit of regional hegemony, Iran and Iraq probably regard CW weapons and missiles as necessary to support their

political and military objectives. Possession of chemical weapons would likely lead to increased intimidation of their Gulf,

neighbors, as well as increased willingness to confront the United States.

Figure 6: A sample interactive Q/A dialogue.

QUAB questions are clustered based on their

this section to manually assess the impact of simi-

mapping to a vector of important concepts in

larity on a Q/A dialogue.

the QUAB.The clustering was done using the

5 Experiments with Interactive Q/A

K-Nearest Neighbor (KNN) method (Dudani,

Dialogues