Paramedics providing pre-hospital emergency response and rescue services could be called on at any time to perform advanced cardiac life support (ACLS) in the field. The assumption is that they are well versed in ACLS knowledge and skills. American Medical Response, a US-based pre-hospital services provider network, reports that 17% of daily calls require paramedics to evaluate undifferentiated cardiac symptoms on a regular basis, with 1% of those calls involving an actual cardiac arrest. With the relative low frequency of ACLS skill use, paramedics may experience the same problems with retention of ACLS knowledge and skill as other healthcare providers: both knowledge and skill deteriorate rapidly if not used (Hammond et al, 2000; Wayne et al, 2006; Jensen et al, 2009; Hein et al, 2010); Spanos and Patterson, 2010; Lo et al, 2011; Settles, 2011). However, paramedics’ retention of ACLS knowledge and skill is not well documented in the literature (Studneck et al, 2011).
Some studies suggest that ACLS knowledge may be retained longer than skills (Smith et al, 2008; Jensen et al, 2009). Research suggests that frequent refresher training of knowledge and skills is successful in the maintenance or improvement of many skills, including ACLS (Hammond et al, 2000; Wayne et al, 2006; Woollard et al, 2006; Smith, et al, 2008; Oermann et al, 2011). Galvagno and Segal (2009) used 5-minute closed book tests consisting of short answer questions over nine months to promote retention of key ACLS and airway protocols among anaesthesia residents. They found that participants’ scores significantly improved over the nine months. Niles et al (2009) provided a sensorised CPR torso on a moveable cart for short computer guided CPR refreshers, over a 3-month period, for all healthcare providers in an intensive care unit. Those providers completing two or more short refreshers per month performed significantly better than those who practiced less than twice a month.
Current methods of teaching ACLS use a high dose, low frequency methodology, defined for purposes of this study as: traditionally learners completing one to two days of intense training (low dose) every two years (low frequency) without additional required refresher training. Schmidt and Lee (2005) suggested that these current methods may assist learners to achieve only the cognitive phase of learning, in which a learner can successfully pass the initial assessment but quickly forget the skill, without further use. Initial retention of knowledge and skills is improved by practice (Arthur et al, 1998). With overlearning, defined as additional training after achieving proficiency, providers do not need to consciously think about the process or steps to follow when performing a skill. Regular practice leads to skill development and refinement, the associative phase of learning, and eventually to the autonomous level, in which skills are performed automatically (Schmidt and Lee, 2005). Low dose, high frequency instruction, in this study is defined as a minimum of 10 minutes of practice per month (low dose) using an ACLS computer gaming program every month for 10 months (high frequency), for an extended period of time.
Registered nurses (RNs) who had maintained ACLS skill and currency for longer than 5 years scored higher on written tests, suggesting overlearning had occurred (Smith et al, 2009). While studies have examined ACLS education for paramedics, there is limited research on their continued competence. Among emergency medical technicians (EMTs), Studnek et al (2009) found that hours of continuing education and practice frequency were not significantly associated with success on a cognitive test— the National Registry of Emergency Medical Technicians Basic Certification examination. Initial higher educational levels, such as completion of a bachelor's or higher degree, and EMTs who were employed professionally rather than as volunteers, correlated with passing the examination, during a study of current EMTs.
Frequent refresher practice is absent from most current ACLS training programs and could provide one method to ameliorate the well documented lack of retention of knowledge and skill. Kerfoot et al (2007) used monthly web-based refresher questions to enhance cognitive memory in urology residents. These researchers found that spaced learning events over time resulted in more retention of knowledge than massed learning at a single point in time. The purpose of this study was to examine if there was a difference in ACLS cognitive performance, measured by a modified Megacode, between two groups of paramedics: those who practiced for 10 minutes monthly (low dose) for a high volume time period (10-month period) using brief computer-based ACLS scenarios, and those who did not refresh.
Materials and methods
This pre post-test quasi-experimental study was conducted over one year (2009–2010) to examine the effects of low dose, high frequency instruction (10 minute monthly refreshers) on the retention of ACLS knowledge among practicing paramedics. Institutional review board approval was received for the study. Participants were randomly assigned to a refresher (monthly practice of ACLS) or no refresher group. Their ACLS skills were assessed at baseline and again at one year.
Sample and survey
Paramedics employed by a pre-hospital services provider at three counties in California were eligible to participate in the study. Informational posters and face-to-face solicitation were used to recruit participants. A convenience sample of 128 paramedics were enrolled in the study; 53 of those participants completed the full study and were assessed on their retention of ACLS knowledge 12 months later. At the time of enrollment, participants were asked to complete an optional demographic survey to collect information about gender, education, and experience. Of the 53 paramedics who completed the yearlong study, only 23 chose to complete the survey. Participants were predominantly male (n=19, 83%), with some college education (n=22, 96%). They were all certified in ACLS and 22 reported providing ACLS more than 12 times in the last year. Details of the demographics, education and training for participants can be seen in Table 1.
| Responses for 23 (43%) of participants | ||||||
|---|---|---|---|---|---|---|
| Attribute | Monthly refresher (n=17) | No refresher (n=6) | Total (n=23) | |||
| Gender: | ||||||
| Male | 13 (76%) | 6 (100%) | 19 (83%) | |||
| Female | 4 (24%) | 0 (0%) | 4 (17%) | |||
| Not answereda | 0 | 0 | 0 | |||
| Highest education: | ||||||
| High school | 1 (6%) | 0 (0%) | 1 (4%) | |||
| Some college | 14 (82%) | 5 (83%) | 19 (83%) | |||
| 4 year | 2 (12%) | 1 (17%) | 3 (13%) | |||
| Paramedic education: | ||||||
| Other | 9 (56%) | 3 (50%) | 12 (55%) | |||
| Hospital | 2 (13%) | 0 (0%) | 2 (9%) | |||
| AA | 3 (19%) | 2 (33%) | 5 (23%) | |||
| BA | 2 (12%) | 1 (17%) | 3 (13%) | |||
| Not answered | 1 | 0 | 1 | |||
| ACLS certified? | ||||||
| Yes | 17 (100%) | 6 (100%) | 23 (100%) | |||
| Not answered | 0 | 0 | 0 | |||
| Used ACLS before? | ||||||
| Yes | 16 (100%) | 6 (100%) | 22 (100%) | |||
| Not answered | 1 | 0 | 1 | |||
| Number of times ACLS used in last 12 months: | ||||||
| 1 | 0 (0%) | 0 (0%) | 0 (0%) | |||
| 2–5 | 4 (25%) | 2 (33%) | 6 (27%) | |||
| 6–12 | 4 (25%) | 1 (17%) | 5 (23%) | |||
| >12 | 8 (50%) | 3 (50%) | 11 (50%) | |||
| Not answered | 1 | 0 | 1 | |||
Not answered responses are not counted in percentages
Megacode validation
To assess mastery of ACLS skills, paramedics are routinely required to perform a Megacode at the end of an ACLS course. A Megacode is a scenario for which there are a known series of actions based on assessment data that should be completed and for which the paramedic can be scored objectively. For this study, a standardised pre-test Megacode on ventricular fibrillation (VF) and a post-test Megacode on symptomatic bradycardia were created. Bradycardia was chosen because it requires higher level cognitive skills for appropriate assessment (Wayne et al, 2006). These two standardised study Megacodes were shorter (5–7 minutes) than traditional end-of-course Megacodes and focused on specific actions for these two specific heart conditions. Scenarios were validated for accuracy and key points by four expert faculty selected by the American Heart Association, including an emergency room physician and three paramedics.
Megacode scoring
A Laerdal Advanced Life Support manikin (model 205-05050) was used for the pre- and post-test Megacode scenarios. The performance of all participants was observed and scored by a trained rater at the study site. Scoring was based on the key points required to demonstrate the participant's ability to successfully perform in the scenario (defined as a ‘win’) or inability to perform in the scenario (defined as a ‘failure’). Key points for scoring of the scenarios were confirmed by an expert emergency room physician, four paramedics, and faculty trainers from the National College of Technical Instruction (NCTI).
Megacode confederates
A trained study staff member, who had current certification in Basic Life Support, assisted study participants with both the pre- and post-test Megacodes as a second set of hands or team member. The addition of this ‘confederate’ role allowed the study participants to focus on the cognitive aspects of ACLS rather than skills and actions. Confederates could perform all functions of the Megacode except for the physical assessment portion, which the participant was required to do. Additionally, confederates were allowed to perform multiple actions at the same time, simulating a team of assistants. Study participants could run the entire Megacode by assessing the patient and directing the confederate. Confederates responded to orders received from the study participant, who functioned as the team leader, and were specifically coached to not move or initiate any actions until orders had been received, avoiding cueing of study participants. Confederates were skilled in performing CPR, using a bag-mask ventilator, applying chest leads for potential equipment that might be used in the Megacode, and using the defibrillation paddles. They also were prepared to perform actual defibrillation of the manikin, using live electricity, should it be ordered. All confederates were taught to use closed loop communication, repeating back orders to indicate they had been heard and acted upon for the team leader, for example, ‘18 gauge IV started.’ Confederates also were required to be familiar with the medication boxes used in the study.
Megacode raters
All Megacode raters who scored the study pre- and post-tests were professional faculty of NCTI, specifically trained to use the Laerdal ALS scoring software, and practiced to proficiency prior to starting the scoring the study Megacodes. Raters were certified ACLS instructors, but were not involved with teaching the ACLS courses to study participants.
Study procedure
All participants completed a one-day (approximately 8 hour) face-to-face American Heart Association (AHA) ACLS refresher course during month one of the study. The course was based on AHA learning objectives and performance criteria and used traditional AHA training materials (videos and textbooks) to teach the didactic component and psychomotor skills. It included a standardised written test to assess cognitive knowledge and a manikin interfaced with a computer running ACLS software for the end-of-course hands-on psychomotor skills assessment Megacode. Following completion of the refresher course, participants completed the study pre-test Megacode on VF.
All participants were then randomised with equal probability using blocked randomisation to the monthly refresher group (experimental) or no refresher group (control). The control group was instructed to conduct ‘business as usual.’ Participants in the experimental group were oriented by site coordinators to the MicroSim™ education program (Laerdal Medical, Stavangar, Norway) to enable them to complete the monthly refresher scenarios. MicroSim is a computer-based gaming system designed to challenge a learner's medical knowledge and problem solving ability while strengthening critical thinking skills. Ten realistic emergency patient scenarios, each based on specific ACLS learning objectives, were used in this study, one for each month of practice. Each case was configured to test the decision-making process and competency of the individual learner. The program could be set for experience levels from a new student to the experienced professional. For this study, the experience level was set at physician provider level, as paramedics basically function independently in the field. The MicroSim inpatient emergency patient scenarios were based on the 2005 ACLS algorithms.
Over the course of ten months, to complete the assigned monthly refresher scenario, participants in the experimental group checked out a laptop computer from the station study site coordinator before beginning a shift, once a month. The monthly refresher sessions were completed on laptop computers during shift ‘downtime’ in the field, i.e. time in which participants were in an emergency medical system vehicle waiting for a call. Monthly reminders about the refresher training scenarios were sent via email or work pager text messages, with one reminder sent if computer check out and scenario completion were not done by a certain date.
In month 12, both the experimental and control groups completed the standardised post-test Megacode (symptomatic bradycardia). A confederate was again included in the scenario to allow the participant to function as the team leader and focus on the cognitive aspects of running a code. At the completion of the study, all participants received a $5 Starbucks gift card, were entered in a drawing for $250, and received a letter of thanks for their study participation from the AHA.
Data analysis
Descriptive data were summarised as frequencies and percentages. Survey responses regarding demographic information, education, and training were summarised for the experimental and control groups separately. The effect of completing monthly refresher scenarios on ACLS performance was tested using a binary logistic model that included a main effect for group (monthly refresher or no refresher) and the pre-test Megacode result as covariate to control for baseline differences in ACLS skills between groups.
This study was statistically powered around detection of an absolute difference of 20% between ACLS cognitive performance rates for the experimental and control group (80% power, alpha=0.05) yielding a required sample size of 128 (64 per group). Seventy five (59%) of the 128 participants enrolled and randomised did not complete the monthly training or post-test Megacode assessment. The largest of the study sites underwent a change in provider management during the study, contributing to higher than expected attrition of study participants. Data reported in the tables are based on the subset of 53 participants at two sites who completed both the pre-test and post-test Megacodes and monthly practice.
To attain adequate power for the statistical analyses, multiple imputation was used to handle incomplete post-test Megacode assessments using the monotone logistic method (default options) available in SAS 9.3 (MI, MIANALYZE and LOGISTIC procedures) such that statistical tests performed were adequately powered (at 80%). Imputation allows one to fill in missing data by using the observed data to predict the missing values (Schafer and Graham, 2002). The imputation process is repeated multiple times creating multiple datasets which are analysed separately; results from the statistical analyses are then summarised across the multiple datasets. Twenty imputed datasets were used to obtain the odds ratio (95% confidence interval) and p-value reported. For comparison, data were analysed using the listwise deletion method, where only the 53 participants completing both the pre-test and post-test Megacode were included in the analysis, and similar results (not reported here) were obtained.
Results
Of the 53 participants (41% of 128), 34 (64%) were assigned to the monthly refresher group and 19 (36%) were assigned to the no refresher group. 27 (79%) of the experimental monthly refresher group and 18 (95%) of the control group successfully completed the pre-test Megacode (referred to as a ‘win’). No difference in post-test skills was found between groups after controlling for differences in pre-test skills (participants in monthly refresher group with wins: 28 (82%) versus 15 (79%) participants in the control group; odds ratio (95% CI)=0.61 (0.12, 2.76); p-value=0.52). Table 2 summarises the agreement between the pre-test and post-test Megacode assessments for the two groups. Among the 53 participants, 38 (72%) passed both assessments, and 3 (6%) failed both assessments. Five (9%) participants, all part of the experimental monthly refresher group, failed the pre-test Megacode yet successfully passed the post-test Megacode. Seven (13%) participants passed the pre-test Megacode, but at 12 months failed to pass the post-test scenario, including four (12%) out of 34 participants in the experimental group and three (16%) out of 19 in the control group.
| Baseline score/Testout score | Monthly refresher (n=34) | No refresher (n=19) | Overall (n=53) | |||
|---|---|---|---|---|---|---|
| N (%) | N (%) | N (%) | ||||
| Failed/Failed | 2 (6%) | 1 (5%) | 3 (6%) | |||
| Failed/Win | 5 (14%) | 0 (0%) | 5 (9%) | |||
| Win/Failed | 4 (12%) | 3 (16%) | 7 (13%) | |||
| Win/Win | 23 (67%) | 15 (79%) | 38 (72%) |
Discussion
This study used a novel monthly ACLS refresher methodology, a computer-based gaming system, to maintain the cognitive knowledge of practicing paramedics in the field during their downtime in their vehicle waiting for a call. The paramedics in this study had practiced for an average of seven years. Based on training requirements, each participant had completed an ACLS course every two years, a minimum of three prior times, suggesting overlearning (described by Smith et al (2008) might have occurred. Although the psychomotor skills of ACLS may be used infrequently, the ACLS algorithms may be used in some form during many calls as part of an initial differential diagnosis. This frequent recall alone may serve to refresh the algorithms for the paramedics, allowing them to retain their knowledge.
Alternatively, the control group, knowing that they would be re-evaluated and compared to the monthly practice group, at the end of the study, might have studied on their own or been more motivated than usual to remember the algorithms. While tempting to suggest that refresher training might not be necessary after a certain number of years, there are algorithms that are rarely used, yet still required. We suggest that after a certain number of consecutive biennial recertifications, in health provider positions that use ACLS algorithms regularly, a planned refresher schedule using computer-based scenarios or gaming for some of those less frequently used ACLS algorithms might be a more cost-effective way of approaching mandatory ACLS training than the current refresher courses. This refresher training schedule should be developed based on a low dose, high frequency model. This would encourage overlearning (Schmidt and Lee, 2005).
Anecdotally, many participants who completed the 10 minute monthly refreshers reported practicing much longer than the required 10 minutes. They wanted to beat the machine. Some of those who dropped out of the study reported that they were not familiar with computer gaming and found it frustrating to know the answers but not be able to work the controls well enough to complete the game in a satisfactory time.
Although cognitive skills are important, the outcome that counts is skill at the bedside when it is needed. This study needs to be repeated in a more controlled environment where outcome tracking is possible. It is worth noting that a small cohort (n=5) of paramedics in the experimental group who refreshed did not pass the pre-test Megacode but were successful on the post-test Megacode. If only a small number of paramedics could improve their ACLS knowledge during their downtime, this could influence patient outcomes.
Limitations
The difficulty of conducting field research in busy and unpredictable work environments cannot be overstated. Although the study was originally planned for implementation at 10 sites, multiple problems occurred that limited the number of sites available to complete the study and resulted in only 53 (41%) of the participants being able to complete the study. The small sample size and use of paramedics from one state limits generalisability. Multiple imputation techniques for missing data were used to statistically address this limitation.
Conclusions
Paramedics can use downtime between calls in their vehicles to refresh their ACLS knowledge. Additionally, this study provides support for a new model of ACLS certification renewal using a combination of low dose, high frequency education and periodic practice of infrequently used skills. Challenges notwithstanding, the surprising lack of cognitive ACLS knowledge degradation among paramedics over time suggests that paramedics may be able to maintain strong ACLS skills simply through regular emergency patient contacts. In systems with a low call volume, integrating a system of low dose, high frequency instruction for refreshers may be particularly valuable because of the lack of practical experience opportunities.