. THE QUALITY FEATURES AND THEIR RELATION TO AN ACCURACY STUDY DESI...
10). The quality features and their relation to an accuracy study design are shown in
Figure 3. An accuracy study is designed to generate a comparison between
measurements obtained by a test and those obtained by a gold standard. As shown
in Figure 3. Design of a test accuracy study and features of its methodological quality,
one needs to independently measure the same clinical attribute on two occasions,
once by a test and second by a gold standard, and then to discern the relationship
between these measurements. In such studies, one possible source of bias is the use
a sample which is not representative of the whole spectrum of the clinically relevant
population. Accuracy studies may appear to be more optimistic if researchers have
deliberately discarded difficult cases from the study. Such omissions are more likely
to occur with convenience or arbitrary methods of sampling the study population.
Selection bias is less likely to be operative with the use consecutive or random
sampling.
The researchers of primary studies on test accuracy should provide sufficient
information on the manner in which the test was conducted. For example description
of preparation of the patients, measurements of biophysical recordings, details of
laboratory assays, computation of results and cut-off levels for defining abnormality
should all be provided. Similarly, the gold standard should be an appropriate one,
usually a test that is generally acknowledged to be the best available for use as the
reference test. In addition, accuracy studies require that observers assessing gold
standards verifying the diagnosis be blinded to measurements obtained from the test
and vice versa. Blinding avoids bias, as recordings made by one observer are not
influenced by the knowledge of the measurements obtained by other observers.
Moreover, during the verification process bias may arise if the result of the test
under evaluation influences whether study subjects undergo confirmation by the gold
standard. This may be the case in some studies where most of the test positive
cases but only a minority of the test negative cases are subjected to verification by
gold standard.
The purpose of quality assessment is to extract essential information on elements of
the study design. In particular, the recruitment, the spectrum and the flow of
subjects through the study should be assessed along with the execution of test and
blinding of its results to the gold standard. Table 2.
A hierarchy of evidence for primary test accuracy studies
Grade
Level of
evidence
Study design
A
1
An independent, blind comparison with reference standard
among an appropriate population of consecutive patients.
B
2
An independent blinds comparison with reference standard
among an appropriate population of non-consecutive patients or
confined to a narrow population of study patients.
B
3
An independent, non-blind comparison with reference standard
confined to a narrow population of study patients
C
4
An independent, blind comparison among an appropriate
population of patients, but reference standard not applied to all
study patients.
D
5
Reference standard not applied independently or expert opinion
without explicit critical appraisal, based on physiology, bench
research or first principles.
Modified from Clark et al, (31) Divakaran et al, (32) and Sackett et al (33)
See Figure 2 for relationship to test accuracy study design.
shows a hierarchy of accuracy evidence based on these features. Empirical evidence
of bias is emerging for many of the quality elements (11). It is, therefore, crucial
that any test accuracy review should include a comprehensive analysis of the
methodological quality of primary studies. These factors, together with
characteristics and results of the studies, should be displayed in tabular form, from
which, it should be possible to infer whether the test appears accurate when drawing
conclusion from a review.