References

Ilgen JS, Ma IW, Hatala R, Cook DA. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015; 49:(2)161-173 https://doi.org/10.1111/medu.12621

Joyce CM, Wainer J, Piterman L, Wyatt A, Archer F. Trends in the paramedic workforce: a profession in transition. Aust Health Rev. 2009; 33:(4)533-440 https://doi.org/10.1071/AH090533

Maurin Söderholm H, Andersson H, Andersson Hagiwara M Research challenges in prehospital care: the need for a simulation-based prehospital research laboratory. Adv Simul (Lond). 2019; 4 https://doi.org/10.1186/s41077-019-0090-0

McKenna KD, Carhart E, Bercher D, Spain A, Todaro J, Freel J. Simulation use in paramedic education research (SUPER): a descriptive study. Prehosp Emerg Care. 2015; 19:(3)432-440 https://doi.org/10.3109/10903127.2014.995845

McLaughlin K, Ainslie M, Coderre S, Wright B, Violato C. The effect of differential rater function over time (DRIFT) on objective structured clinical examination ratings. Med Educ. 2009; 43:(10)989-992 https://doi.org/10.1111/j.1365-2923.2009.03438.x

Tavares W, Boet S, Theriault R, Mallette T, Eva KW. Global rating scale for the assessment of paramedic clinical competence. Prehosp Emerg Care. 2013; 17:(1)57-67 https://doi.org/10.3109/10903127.2012.702194

Tavares W, LeBlanc VR, Mausz J, Sun V, Eva KW. Simulation-based assessment of paramedics and performance in real clinical contexts. Prehosp Emerg Care. 2014; 18:(1)116-122 https://doi.org/10.3109/10903127.2013.818178

Improving simulation through student assessment partnerships. https//tinyurl.com/yck88dcs (accessed 12 June 2022)

Tsai BM, Sun JT, Hsieh MJ Optimal paramedic numbers in resuscitation of patients with out-of-hospital cardiac arrest: a randomized controlled study in a simulation setting. PLoS One. 2020; 15:(7) https://doi.org/10.1371/journal.pone.0235315

Van Dillen CM, Tice MR, Patel AD Trauma simulation training increases confidence levels in prehospital personnel performing life-saving interventions in trauma patients. Emerg Med Int. 2016; 2016 https://doi.org/10.1155/2016/5437490

Williams B, Abel C, Khasawneh E, Ross L, Levett-Jones T. Simulation experiences of paramedic students: a cross-cultural examination. Adv Med Educ Pract. 2016; 7:181-186 https://doi.org/10.2147/AMEP.S98462

Yeates P, Cope N, Hawarden A, Bradshaw H, McCray G, Homer M. Developing a video-based method to compare and adjust examiner effects in fully nested OSCEs. Med Educ. 2019; 53:(3)250-263 https://doi.org/10.1111/medu.13783

Differential rater function over time (DRIFT) during student simulations

02 July 2022
Volume 12 · Issue 2

Abstract

Background:

The field of paramedicine continues to advance in scope. Simulation training is frequently used to teach and evaluate students. Simulation examinations are often evaluated using a standardised global rating scale (GRS) that is reliable and valid. However, differential rater function over time (DRIFT) has not been evaluated when using the GRS during simulations.

Aims:

This study aimed to assess if DRIFT arises when applying the GRS.

Methods:

Data were collected at six simulation evaluations. Raters were randomly assigned to evaluate several students at the same station. Each station lasted 12 minutes and there was a total of 11 stations. A model to test DRIFT scores was created and was tested against both a leniency and perceptual model.

Findings:

Of the models explored, one that included students, the rater, and the dimensions had the greatest evidence (−3151 Bayes factors). This model was then tested against leniency (K=−9.1 dHart) and perceptual models (K=−7.1 dHart). This suggests a substantial finding against DRIFT; however, the tested models used a wide parameter so the possibility of a minor effect is not fully excluded.

Conclusion:

DRIFT was not found; however, further studies with multiple centres and longer evaluations should be conducted.

The field of paramedicine has evolved considerably over the last three decades. Originally, the role of a paramedic was restricted to patient transportation from the field to a nearby hospital with little medical intervention along the way. Today, paramedics are skilled clinicians who are capable of performing advanced medical and procedural interventions in a prehospital setting (Joyce et al, 2009; Williams et al, 2016).

To ensure that paramedic students are ready to enter the workforce, academic institutions have been adopting novel techniques (Thompson and Houston, 2021).

Simulation training is a hands-on, practical teaching method that has become increasingly used within teaching institutions (McKenna et al, 2015; Van Dillen et al, 2016; Tsai et al, 2020; Thompson and Houston, 2021). It has been established that simulation training is a critical means of improving medical education. Students exposed to simulation acquire more competencies and expertise in practical skills than those whose curriculum does not require simulation training (Van Dillen et al, 2016).

Subscribe to get full access to International Paramedic Practice

Thank you for visiting International Paramedic Practice and reading our archive of expert clinical content. If you would like to read more from the only journal dedicated to those working in emergency care, you can start your subscription today for just £48.

What's included

  • CPD Focus

  • Develop your career

  • Stay informed