Observer variability when evaluating patient movement from electronic portal images of pelvic radiotherapy fields.
BACKGROUND AND PURPOSE: A study has been performed to evaluate inter-observer variability when assessing pelvic patient movement using an electronic portal imaging device (EPID). MATERIALS AND METHODS: Four patient image sets were used with 3-6 portal images per set. The observer group consisted of nine radiographers with 3-18 months clinical EPID experience. The observers outlined bony landmarks on a digital simulator image and used matching software to evaluate field placement errors (FPEs) on each portal image relative to the reference simulator image. Data were evaluated statistically, using a two-component analysis of variance technique, to quantify both the inter-observer variability in evaluating FPEs and inter-fraction variability in patient position relative to the residuals of the analysis. Intra-observer variability was also estimated using four of the observers carrying out three sets of repeat readings. RESULTS: Eight sets of variance data were analysed, based on FPEs in two orthogonal directions for each of the four patient image sets studied. Initial analysis showed that both inter-observer variation and inter-fraction-patient position variation were statistically significant (P<0.05) in seven of the eight cases evaluated. The averaged root-mean-square (RMS) deviation of the observers from the group mean was 1.1 mm, with a maximum deviation of 5.0 mm recorded for an individual observer. After additional training and re-testing of two of the observers who recorded the largest deviations from the group mean, a subsequent analysis showed the inter-observer variability for the group to be significant in only three of the eight cases, with averaged RMS deviation reduced to 0.5 mm, with a maximum deviation of 2.7 mm. The intra-observer variability was 0.5 mm, averaged over the four observers tested. CONCLUSIONS: We have developed a quantitative approach to evaluate inter-observer variability in terms of its statistical significance compared to inter-fraction patient movement. This will assist us in training and assessing observers required to perform this task on a routine basis.