Retrieval practice, retrieval __________, _________ practice, testing effect……wuh?!?!
Retrieval practice simply means to actively recall information following exposure (e.g., studying). Because tests are a particularly common and effective means by which to prompt the retrieval of specific pieces of information, the learning benefits of retrieval practice are also known as the testing effect. That is, effective tests can do more than simply assess learning; they can strengthen learning by prompting retrieval. It is important to clarify that the key to the testing effect is the retrieval and not the test per se. Therefore, the testing effect pertains to not only traditional assessments like tests and quizzes, but also to free recall. So, silently answering questions in your mind (e.g., self-testing) is an example of testing that promotes learning.
Landmark study by Roediger and Karpicke in 2006a
Although the testing effect has been described by studies that date back more than a century, researchers and articles often cite a 2006a study by Roediger and Karpicke as the source of renewed interest in the strategy and effect. In that study, the investigators asked three groups of undergraduates to read passages that were about 250 words long. One group of students learned the passages by studying (i.e., reading) them four times (SSSS group). A second group learned the passages by studying them three times and then completing a test in which they were prompted to retrieve information from the passages (SSST group). The last group studied the passages just one time and then performed the retrieval test three times (STTT group). All three groups were given a total of 20 minutes to learn each passage, following which their retention was assessed via free recall either 5 minutes or 1 week later. As you can see in Figure 1, there was a modest advantage with the SSSS strategy, as well as a modest disadvantage with the STTT strategy, immediately after learning the passages. However, the exact opposite pattern was observed one week later, as the STTT group’s recall scores were about 5% higher and 21% higher than those of the SSST and SSSS groups, respectively. The results of this study demonstrated that testing/retrieval practice can be a powerful means of improving long-term memory. These advantages to long-term recall have subsequently been confirmed by many different researchers and investigations (see Roediger and Butler 2011; Roediger and Karpicke, 2006b for review).
Retrieval practice and the ability to make inferences; it isn’t just about simple recall
One might be concerned that retrieval practice is just a form of drill and practice that merely teaches people to produce a fixed response to a specific cue. Karpicke and Blunt (2011) addressed this concern by comparing the effects of retrieval practice and concept mapping on meaningful learning, which includes the ability to draw conclusions and create new ideas. The investigators chose concept mapping for this comparison because it known to promote elaborative (i.e., complex) learning. In one experiment, one group of students learned a science text by repeatedly reading (i.e., studying) it, another group studied the text and then used it create a concept map, and a third group studied and then recalled the text two times. The total amount of time the concept mapping and retrieval practice groups were given to learn the text was standardized. The students returned the following week and completed a short-answer test that included both questions that could be answered verbatim from the text and questions that required inferences. As is displayed in Figure 2, the retrieval practice strategy resulted in superior scores on not just the verbatim questions, but also on the inference questions. That is, the advantages of retrieval practice extended beyond simple recall and to meaningful learning. These findings are supported by numerous other investigations (see Karpicke and Aue, 2015 for review), including a subsequent study by the same authors (Blunt and Karpicke, 2014).
Okay, so retrieval practice has been shown to enhance recall and meaningful learning, but does it work with the types of information that are relevant to APS members?
Yes………numerous studies support this claim. One notable example was a study by Larsen, Butler and Roediger (2009) in which two groups of medical residents first attended lectures on the treatments of both status epilepticus and myasthenia gravis. Immediately after the lectures, and then again about two and four weeks later, the residents studied (i.e., read) a review sheet pertaining to the treatment of one of those diseases and they completed a retrieval test that included feedback on the other treatment. Roughly six months after the lectures, the residents completed a final assessment that covered the treatment of both diseases. As you can see in Figure 3, the testing strategy resulted in scores that were about 11% and 17% higher than those associated with the studying strategy on the status epilepticus and myasthenia gravis sections, respectively. It is also worth noting that the overall effect size pertaining to those differences was large (Cohen’s d = 0.91). The same group of researchers went on publish similar findings with groups of first-year medical students (Larsen et al, 2013). In that follow-up study, a testing-based strategy produced superior recall and greater transfer of learning of four clinical neurology topics six months after the students first encountered them.
Our lab has also recently published numerous studies with relevant materials, and we observed several advantages with retrieval practice compared to more commonly-used reading and note-taking learning strategies. For example, we found that retrieval-based strategies resulted in superior recall of exercise physiology (Linderholm, Dobson and Yarbrough, 2016) and anatomy and physiology course information (Dobson and Linderholm, 2015a; Dobson and Linderholm, 2015b), including information that consisted of concepts and terminology that were previously unfamiliar to the students (Dobson, Linderholm and Yarbrough, 2015). We have also observed advantages to independent student learning that resulted in higher scores on course exams (Dobson and Linderholm, 2015a), as well as to the ability to synthesize themes from multiple sources (Linderholm, Dobson and Yarbrough, 2016), which is a skill that requires higher orders of cognition.
Just give me the take home messages.
- Dozens of studies have demonstrated that retrieval practice can promote superior recall and meaningful learning when compared to more commonly-used strategies like reading. (Karpicke and Aue, 2015; Roediger and Butler, 2011; Roediger and Karpicke, 2006b).
- Although some studies have provided evidence that essay and short answer (SA) questions can lead to a greater testing effect than multiple choice (MC) questions (Roediger and Karpicke, 2006b; Butler and Roediger, 2007), a recent study by Smith and Karpicke (2014) indicated that MC and SA questions are equally effective.
- Multiple repetitions of retrieval practice promote more learning than a single retrieval event (Roediger and Butler, 2011; Roediger and Karpicke, 2006b)
- The benefits of retrieval practice are enhanced if learners receive feedback after they retrieve (Roediger and Butler, 2011; Roediger and Karpicke, 2006b).
Great, but how do you apply retrieval practice in the classroom?
- Summative assessments. Tests prompt retrieval, so one way to incorporate more retrieval practice into your classes is to have your students complete both more exams and more cumulative exams.
- Formative assessments. There are numerous reasons to use low-stakes assessments like quizzes instead of tests. Quizzes may be just as effective at prompting retrieval, and they provide valuable feedback about performance to both instructors and students, but they typically elicit less anxiety and encourage less cheating. Suggested applications include starting class meetings with a short quiz that prompts students to retrieve information that will be developed during the lecture and/or end class meetings with a short quiz to get students to retrieve the important take home messages of the lecture.
- In-class retrieval assignments. A great way to break up the monotony of lectures is to have students complete retrieval assignments during class meetings. For example, have individuals or groups of students retrieve information and then present it to the rest of the class.
- Encourage students to use retrieval practice outside of class. One of the greatest benefits of retrieval practice is that it easy to use; all one needs to do is to recall information from memory. I encourage my students to use retrieval practice by first presenting to them some of the evidence of its effectiveness (described above), and then by suggesting some methods they may use to employ the strategy that (e.g., take turns quizzing or teaching fellow students, quiz one-self, or simply freely recall portions of the information). Again, it is important to emphasize that multiple retrieval events are more beneficial, and that each or most of those should include feedback. For example, have students study then retrieve then study again to receive feedback, etc.
- Dobson JL, Linderholm T, Yarbrough MB. Self-testing produces superior recall of both familiar and unfamiliar muscle information. Advances in Physiology Education 39: 309-314, 2015
- Dobson JL and Linderholm T, The effect of selected “desirable difficulties” on the ability to recall anatomy information. Anatomical Sciences Education 8: 395-403, 2015.
- Dobson JL, Linderholm T. Self-testing promotes superior retention of anatomy and physiology information. Advances in Health Sciences Education 20: 149-161, 2015.
- Butler AC, Roediger HL. Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology 19: 514-527, 2007.
- Blunt JR, Karpicke JD. Learning with retrieval-based concept mapping. Journal of Educational Psychology 106: 849, 2014.
- Dobson JL, Perez J, Linderholm T. Distributed retrieval practice promotes superior recall of anatomy information. Anatomical Sciences Education DOI: 10.1002/ase.1668, 2016.
- Karpicke JD, Aue, WR. The testing effect is alive and well with complex materials. Educational Psychology Review 27: 317-326, 2015.
- Karpicke JD, Blunt JR. Retrieval practice produces more learning than elaborative studying with concept mapping. Science 331: 772-775, 2011.
- Larsen DP, Butler AC, Roediger HL. Repeated testing improves long-term retention relative to repeated study: A randomized controlled trial. Medical Education 43: 1174-1181, 2009.
- Larsen DP, Butler AC, Lawson AL, Roediger HL. The importance of seeing the patient: Test-enhanced learning with standardized patients and written tests improves clinical application of knowledge. Advances in Health Sciences Education 18: 409-25, 2013.
- Linderholm T, Dobson JL, Yarbrough MB. The benefit of self-testing and interleaving for synthesizing concepts across multiple physiology Advances in Physiology Education 40: 329-34, 2016.
- Roediger HL, Butler AC. The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences 15: 20-27, 2011.
- Roediger HL, Karpicke JD. Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science 17: 249-255, 2006.
- Roediger HL, Karpicke JD. The power of testing memory: Basic research and implications for educational practice. Perspectives in Psychological Science 1: 181-210, 2006.
- Smith MA, Karpicke JD. Retrieval practice with short-answer, multiple-choice, and hybrid tests. Memory 22: 784-802, 2014.