. THE QUALITY FEATURES AND THEIR RELATION TO AN ACCURACY STUDY DESI...

10). The quality features and their relation to an accuracy study design are shown in

Figure 3. An accuracy study is designed to generate a comparison between

measurements obtained by a test and those obtained by a gold standard. As shown

in Figure 3. Design of a test accuracy study and features of its methodological quality,

one needs to independently measure the same clinical attribute on two occasions,

once by a test and second by a gold standard, and then to discern the relationship

between these measurements. In such studies, one possible source of bias is the use

a sample which is not representative of the whole spectrum of the clinically relevant

population. Accuracy studies may appear to be more optimistic if researchers have

deliberately discarded difficult cases from the study. Such omissions are more likely

to occur with convenience or arbitrary methods of sampling the study population.

Selection bias is less likely to be operative with the use consecutive or random

sampling.

The researchers of primary studies on test accuracy should provide sufficient

information on the manner in which the test was conducted. For example description

of preparation of the patients, measurements of biophysical recordings, details of

laboratory assays, computation of results and cut-off levels for defining abnormality

should all be provided. Similarly, the gold standard should be an appropriate one,

usually a test that is generally acknowledged to be the best available for use as the

reference test. In addition, accuracy studies require that observers assessing gold

standards verifying the diagnosis be blinded to measurements obtained from the test

and vice versa. Blinding avoids bias, as recordings made by one observer are not

influenced by the knowledge of the measurements obtained by other observers.

Moreover, during the verification process bias may arise if the result of the test

under evaluation influences whether study subjects undergo confirmation by the gold

standard. This may be the case in some studies where most of the test positive

cases but only a minority of the test negative cases are subjected to verification by

gold standard.

The purpose of quality assessment is to extract essential information on elements of

the study design. In particular, the recruitment, the spectrum and the flow of

subjects through the study should be assessed along with the execution of test and

blinding of its results to the gold standard. Table 2.

A hierarchy of evidence for primary test accuracy studies

Grade

Level of

evidence

Study design

A

1

An independent, blind comparison with reference standard

among an appropriate population of consecutive patients.

B

2

An independent blinds comparison with reference standard

among an appropriate population of non-consecutive patients or

confined to a narrow population of study patients.

B

3

An independent, non-blind comparison with reference standard

confined to a narrow population of study patients

C

4

An independent, blind comparison among an appropriate

population of patients, but reference standard not applied to all

study patients.

D

5

Reference standard not applied independently or expert opinion

without explicit critical appraisal, based on physiology, bench

research or first principles.

Modified from Clark et al, (31) Divakaran et al, (32) and Sackett et al (33)

See Figure 2 for relationship to test accuracy study design.

shows a hierarchy of accuracy evidence based on these features. Empirical evidence

of bias is emerging for many of the quality elements (11). It is, therefore, crucial

that any test accuracy review should include a comprehensive analysis of the

methodological quality of primary studies. These factors, together with

characteristics and results of the studies, should be displayed in tabular form, from

which, it should be possible to infer whether the test appears accurate when drawing

conclusion from a review.