ITEM 10.    Describe the number, training and expertise of the persons executing and reading the index tests and the reference standard.

 


Example
Subjects were classified as to whether or not they were heavy drinkers based on their response to the Self Administered Alcohol Screening Test (SAAST)[ref] and the Khavari questionnaire on the amount of alcohol consumed during the past year.[ref] Both questionnaires were administered by a research associate. The research associate was trained by a Ph.D. psychologist who specializes in alcohol treatment. He discussed with her how to score questions and how to follow up ambiguous information, and he observed her administering more than 10 questionnaires[1]

 

 

Variability in the manipulation, processing, or reading of the index test or reference standard will affect measures of diagnostic accuracy.[2,3] Many studies have shown reader variability, especially in the field of imaging.[4,5] The amount of the readers training can help readers to judge whether similar results are attainable in their own settings, with possibly less experienced readers.

 

Professional background, expertise, and prior training to improve interpretation and to reduce inter-observer variation all affect the quality of reading.[6,7] Readers are more likely to interpret results from (subjective) tests as abnormal in settings with higher prevalences of the target condition, a tendency known as context bias.[8]

 

The example describes the reference standard in a study of a model that uses results of commonly performed laboratory tests to identify men who are heavy drinkers.

 

 

References

1.

Hartz AJ, Guse C, Kajdacsy-Balla A. Identification of heavy drinkers using a combination of laboratory tests. J Clin Epidemiol 1997; 50:1357-68.
2.

Elmore JG, Feinstein AR. A bibliography of publications on observer variability (final installment). J Clin Epidemiol 1992; 45:567-80.
3.

Brealey S, Scally AJ, Thomas NB. Review article: methodological standards in radiographer plain film reading performance studies. Br J Radiol 2002; 75:107-13.
4.

Elmore JG, Wells CK, Lee CH, Howard DH, Feinstein AR. Variability in radiologists' interpretations of mammograms. N Engl J Med 1994;331:1493-9.
5. Brealey S, Scally AJ. Bias in plain film reading performance studies. Br J Radiol 2001;74:307-16.
6.

Ronco G, Montanari G, Aimone V, et al. Estimating the sensitivity of cervical cytology: errors of interpretation and test limitations. Cytopathology 1996; 7:151-8.
7.

Cohen MB, Rodgers RP, Hales MS, et al. Influence of training and experience in fine-needle aspiration biopsy of breast. Receiver operating characteristics curve analysis. Arch Pathol Lab Med 1987;111:518-20.
8. Egglin TK, Feinstein AR. Context bias. A problem in diagnostic radiology. JAMA 1996; 276:1752-5.