. IN THE TECHNIQUE, TRANSFORMATION RULES AREMETHOD NATIVE CORPUS...

1994) - TÀI LIỆU BÁO CÁO KHOA HỌC CREATING A MANUALLY ERROR TAGGED AND SHALLOW PARSED LEARNER CORPUS PPTX 1994) - TÀI LIỆU BÁO CÁO KHOA HỌC CREATING A MANUALLY ERROR TAGGED AND SHALLOW PARSED LEARNER CORPUS PPTX
Khoa học TÀI LIỆU BÁO CÁO KHOA HỌC CREATING A MANUALLY ERROR TAGGED AND SHALLOW PARSED LEARNER CORPUS PPTX

1994). In the technique, transformation rules are

Method Native Corpus Learner Corpus

obtained by comparing the output of a POS tagger

CRF 0.970 0.932

and the human annotation so that the differences be-

HMM 0.887 0.926

tween the two are reduced. We used the shallow-

Table 4: POS-tagging accuracy.

7

BOS denotes a beginning of a sentence.

Method Original Improved

chunking. To see this, let us define the following

CRF 0.932 0.934

symbols:

: Recall of head noun identification, :

HMM 0.926 0.933

recall of error detection without chunking error,

recall of error detection with chunking error. and

Table 6: Improvement obtained by transformation.

are interpreted as the true recall of error detection

and its observed value when chunking error exists,

respectively. Here, note that

can be expressed

parsed corpus as a test corpus and the other man-

as

. For instance, according to Han et al.

ually POS-tagged corpus created in the pilot study

(2006), their method achieves a recall of 0.40 (i.e.,

described in Subsect. 3.2.1 as a training corpus. We

assuming that chunk-

), and thus

used POS-based and word-based transformations as

ing errors exist and recall of head noun identification

Brill (1994) described.

is

Table 6 shows the improvements together with the

just as in this evaluation. Improving

to

without any mod-

would achieve

original accuracies. Table 6 reveals that even the

ification to the error detection method. Precision can

simple application of Brill’s technique achieves a

also be estimated in a similar manner although it re-

slight improvement in both taggers. Designing the

quires a more complicated calculation.

templates of the transformation for learner corpora

may achieve further improvement.

6 Conclusions