Skip to main content
placeholder image

Enhancing the defensibility of examiners’ marks in high stake OSCEs

Journal Article


Download full-text (Open Access)

Abstract


  • Background: Most assessments in health professions education consist of knowledge-based examinations as well

    as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making

    related to borderline grades assigned by examiners. Borderline grades are commonly used by examiners when they do

    not have sufficient information to make clear pass/fail decisions. The interpretation of these borderline grades is rarely

    discussed in the literature. This study reports the application of the Objective Borderline Method (version 2, henceforth:

    OBM2) to a high stakes Objective Structured Clinical Examination undertaken at the end of the final year of a Medicine

    program in Australia.

    Methods: The OBM2 uses all examination data to reclassify borderline grades as either pass or fail. Factor analysis was

    used to estimate the suitability of data for application of OBM2. Student’s t-tests, utilising bootstrapping, were used to

    compare the OBM2 with ‘traditional’ results. Interclass correlations were used to estimate the association between the

    grade reclassification and all other grades in this examination.

    Results: The correlations between scores for each station and pass/fail outcomes increased significantly after the mark

    reclassification, yet the reclassification did not significantly impact on students’ total scores. Examiners, students and

    program leaders expressed high levels of satisfaction and the Faculty’s Curriculum Development Committee has decided

    that the OBM2 will be used for all future clinical examinations. Implications of the OBM2 are discussed.

    Conclusions: The OBM2 provides a feasible, defensible and acceptable solution for classification of borderline grades as

    either pass or fail.

UOW Authors


  •   Shulruf, Boaz (external author)
  •   Damodaran, Arvin (external author)
  •   Jones, Philip (external author)
  •   Kennedy, Sean (external author)
  •   Mangos, George (external author)
  •   O'Sullivan, Anthony J. (external author)
  •   Rhee, Joel
  •   Taylor, Silas (external author)
  •   Velan, Gary (external author)
  •   Harris, Peter (external author)

Publication Date


  • 2018

Citation


  • Shulruf, B., Damodaran, A., Jones, P., Kennedy, S., Mangos, G., O'Sullivan, A. J., Rhee, J., Taylor, S., Velan, G. & Harris, P. (2018). Enhancing the defensibility of examiners’ marks in high stake OSCEs. BMC Medical Education, 18 (1), 1-9.

Scopus Eid


  • 2-s2.0-85051052142

Ro Full-text Url


  • http://ro.uow.edu.au/cgi/viewcontent.cgi?article=6527&context=smhpapers

Ro Metadata Url


  • http://ro.uow.edu.au/smhpapers/5462

Number Of Pages


  • 8

Start Page


  • 1

End Page


  • 9

Volume


  • 18

Issue


  • 1

Place Of Publication


  • United Kingdom

Abstract


  • Background: Most assessments in health professions education consist of knowledge-based examinations as well

    as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making

    related to borderline grades assigned by examiners. Borderline grades are commonly used by examiners when they do

    not have sufficient information to make clear pass/fail decisions. The interpretation of these borderline grades is rarely

    discussed in the literature. This study reports the application of the Objective Borderline Method (version 2, henceforth:

    OBM2) to a high stakes Objective Structured Clinical Examination undertaken at the end of the final year of a Medicine

    program in Australia.

    Methods: The OBM2 uses all examination data to reclassify borderline grades as either pass or fail. Factor analysis was

    used to estimate the suitability of data for application of OBM2. Student’s t-tests, utilising bootstrapping, were used to

    compare the OBM2 with ‘traditional’ results. Interclass correlations were used to estimate the association between the

    grade reclassification and all other grades in this examination.

    Results: The correlations between scores for each station and pass/fail outcomes increased significantly after the mark

    reclassification, yet the reclassification did not significantly impact on students’ total scores. Examiners, students and

    program leaders expressed high levels of satisfaction and the Faculty’s Curriculum Development Committee has decided

    that the OBM2 will be used for all future clinical examinations. Implications of the OBM2 are discussed.

    Conclusions: The OBM2 provides a feasible, defensible and acceptable solution for classification of borderline grades as

    either pass or fail.

UOW Authors


  •   Shulruf, Boaz (external author)
  •   Damodaran, Arvin (external author)
  •   Jones, Philip (external author)
  •   Kennedy, Sean (external author)
  •   Mangos, George (external author)
  •   O'Sullivan, Anthony J. (external author)
  •   Rhee, Joel
  •   Taylor, Silas (external author)
  •   Velan, Gary (external author)
  •   Harris, Peter (external author)

Publication Date


  • 2018

Citation


  • Shulruf, B., Damodaran, A., Jones, P., Kennedy, S., Mangos, G., O'Sullivan, A. J., Rhee, J., Taylor, S., Velan, G. & Harris, P. (2018). Enhancing the defensibility of examiners’ marks in high stake OSCEs. BMC Medical Education, 18 (1), 1-9.

Scopus Eid


  • 2-s2.0-85051052142

Ro Full-text Url


  • http://ro.uow.edu.au/cgi/viewcontent.cgi?article=6527&context=smhpapers

Ro Metadata Url


  • http://ro.uow.edu.au/smhpapers/5462

Number Of Pages


  • 8

Start Page


  • 1

End Page


  • 9

Volume


  • 18

Issue


  • 1

Place Of Publication


  • United Kingdom