Abstract
-
Background: Assessment presents one of the greatest challenges to evaluating
health professional trainee performance, as a result of the subjectivity of
judgements and variability in assessor standards. The present study aimed
to test a moderation procedure for assessment across four independent universities
and explore approaches to assessment and the factors that influence
assessment decisions.
Methods: Assessment tasks designed independently by each of the four universities
to assess student readiness for placement were chosen for the present
study. Each university provided four student performance recordings
for moderation. Eight different academic assessors viewed the student performances
and assessed them using the corresponding university assessment
instrument. Assessment results were collated and presented back to the
assessors, together with the original university assessment results. Results
were discussed with assessors to explore variations. The discussion was
recorded, transcribed, thematically analysed and presented back to all assessors
to achieve consensus on the emerging major learnings.
Results: Although there were differences in absolute scores, there was consistency
(12 out of 16 performances) in overall judgement decisions regarding
placement readiness. Proficient communication skills were considered a
key factor when determining placement readiness. The discussion revealed:
(i) assessment instruments; (ii) assessor factors; and (iii) the subjectivity of
judgement as the major factors influencing assessment.
Conclusions: Assessment moderation is a useful method for improving the
quality of assessment decisions by sharing understanding and aligning standards
of performance.