Monthly Archives: March 2017

The Surprising Advantages Retrieval Practice

Retrieval practice,  retrieval __________,    _________ practice,  testing effect……wuh?!?!

Retrieval practice simply means to actively recall information following exposure (e.g., studying). Because tests are a particularly common and effective means by which to prompt the retrieval of specific pieces of information, the learning benefits of retrieval practice are also known as the testing effect. That is, effective tests can do more than simply assess learning; they can strengthen learning by prompting retrieval. It is important to clarify that the key to the testing effect is the retrieval and not the test per se. Therefore, the testing effect pertains to not only traditional assessments like tests and quizzes, but also to free recall. So, silently answering questions in your mind (e.g., self-testing) is an example of testing that promotes learning.

Landmark study by Roediger and Karpicke in 2006a

Figure 1. Repeated testing lead to better long-term recall when compared to repeated studying. Roediger and Karpicke, 2006a.

Although the testing effect has been described by studies that date back more than a century, researchers and articles often cite a 2006a study by Roediger and Karpicke as the source of renewed interest in the strategy and effect. In that study, the investigators asked three groups of undergraduates to read passages that were about 250 words long. One group of students learned the passages by studying (i.e., reading) them four times (SSSS group). A second group learned the passages by studying them three times and then completing a test in which they were prompted to retrieve information from the passages (SSST group). The last group studied the passages just one time and then performed the retrieval test three times (STTT group). All three groups were given a total of 20 minutes to learn each passage, following which their retention was assessed via free recall either 5 minutes or 1 week later. As you can see in Figure 1, there was a modest advantage with the SSSS strategy, as well as a modest disadvantage with the STTT strategy, immediately after learning the passages. However, the exact opposite pattern was observed one week later, as the STTT group’s recall scores were about 5% higher and 21% higher than those of the SSST and SSSS groups, respectively. The results of this study demonstrated that testing/retrieval practice can be a powerful means of improving long-term memory. These advantages to long-term recall have subsequently been confirmed by many different researchers and investigations (see Roediger and Butler 2011; Roediger and Karpicke, 2006b for review).

Retrieval practice and the ability to make inferences; it isn’t just about simple recall

Figure 2. Retrieval practice resulted in higher scores on verbatim and inferential questions. Derived from Karpicke and Blunt, 2011.

One might be concerned that retrieval practice is just a form of drill and practice that merely teaches people to produce a fixed response to a specific cue. Karpicke and Blunt (2011) addressed this concern by comparing the effects of retrieval practice and concept mapping on meaningful learning, which includes the ability to draw conclusions and create new ideas. The investigators chose concept mapping for this comparison because it known to promote elaborative (i.e., complex) learning. In one experiment, one group of students learned a science text by repeatedly reading (i.e., studying) it, another group studied the text and then used it create a concept map, and a third group studied and then recalled the text two times. The total amount of time the concept mapping and retrieval practice groups were given to learn the text was standardized. The students returned the following week and completed a short-answer test that included both questions that could be answered verbatim from the text and questions that required inferences. As is displayed in Figure 2, the retrieval practice strategy resulted in superior scores on not just the verbatim questions, but also on the inference questions. That is, the advantages of retrieval practice extended beyond simple recall and to meaningful learning. These findings are supported by numerous other investigations (see Karpicke and Aue, 2015 for review), including a subsequent study by the same authors (Blunt and Karpicke, 2014).

Okay, so retrieval practice has been shown to enhance recall and meaningful learning, but does it work with the types of information that are relevant to APS members?

Figure 3. The testing strategy resulted in superior performance on both sections of the six month assessment. Derived from Larsen, Butler and Roediger, 2009.

Yes………numerous studies support this claim. One notable example was a study by Larsen, Butler and Roediger (2009) in which two groups of medical residents first attended lectures on the treatments of both status epilepticus and myasthenia gravis. Immediately after the lectures, and then again about two and four weeks later, the residents studied (i.e., read) a review sheet pertaining to the treatment of one of those diseases and they completed a retrieval test that included feedback on the other treatment. Roughly six months after the lectures, the residents completed a final assessment that covered the treatment of both diseases. As you can see in Figure 3, the testing strategy resulted in scores that were about 11% and 17% higher than those associated with the studying strategy on the status epilepticus and myasthenia gravis sections, respectively. It is also worth noting that the overall effect size pertaining to those differences was large (Cohen’s d = 0.91). The same group of researchers went on publish similar findings with groups of first-year medical students (Larsen et al, 2013). In that follow-up study, a testing-based strategy produced superior recall and greater transfer of learning of four clinical neurology topics six months after the students first encountered them.

Our lab has also recently published numerous studies with relevant materials, and we observed several advantages with retrieval practice compared to more commonly-used reading and note-taking learning strategies. For example, we found that retrieval-based strategies resulted in superior recall of exercise physiology (Linderholm, Dobson and Yarbrough, 2016) and anatomy and physiology course information (Dobson and Linderholm, 2015a; Dobson and Linderholm, 2015b), including information that consisted of concepts and terminology that were previously unfamiliar to the students (Dobson, Linderholm and Yarbrough, 2015). We have also observed advantages to independent student learning that resulted in higher scores on course exams (Dobson and Linderholm, 2015a), as well as to the ability to synthesize themes from multiple sources (Linderholm, Dobson and Yarbrough, 2016), which is a skill that requires higher orders of cognition.

Just give me the take home messages.

  • Dozens of studies have demonstrated that retrieval practice can promote superior recall and meaningful learning when compared to more commonly-used strategies like reading. (Karpicke and Aue, 2015; Roediger and Butler, 2011; Roediger and Karpicke, 2006b).
  • Although some studies have provided evidence that essay and short answer (SA) questions can lead to a greater testing effect than multiple choice (MC) questions (Roediger and Karpicke, 2006b; Butler and Roediger, 2007), a recent study by Smith and Karpicke (2014) indicated that MC and SA questions are equally effective.
  • Multiple repetitions of retrieval practice promote more learning than a single retrieval event (Roediger and Butler, 2011; Roediger and Karpicke, 2006b)
  • The benefits of retrieval practice are enhanced if learners receive feedback after they retrieve (Roediger and Butler, 2011; Roediger and Karpicke, 2006b).

Great, but how do you apply retrieval practice in the classroom?

  • Summative assessments. Tests prompt retrieval, so one way to incorporate more retrieval practice into your classes is to have your students complete both more exams and more cumulative exams.
  • Formative assessments. There are numerous reasons to use low-stakes assessments like quizzes instead of tests. Quizzes may be just as effective at prompting retrieval, and they provide valuable feedback about performance to both instructors and students, but they typically elicit less anxiety and encourage less cheating. Suggested applications include starting class meetings with a short quiz that prompts students to retrieve information that will be developed during the lecture and/or end class meetings with a short quiz to get students to retrieve the important take home messages of the lecture.
  • In-class retrieval assignments. A great way to break up the monotony of lectures is to have students complete retrieval assignments during class meetings. For example, have individuals or groups of students retrieve information and then present it to the rest of the class.
  • Encourage students to use retrieval practice outside of class. One of the greatest benefits of retrieval practice is that it easy to use; all one needs to do is to recall information from memory. I encourage my students to use retrieval practice by first presenting to them some of the evidence of its effectiveness (described above), and then by suggesting some methods they may use to employ the strategy that (e.g., take turns quizzing or teaching fellow students, quiz one-self, or simply freely recall portions of the information). Again, it is important to emphasize that multiple retrieval events are more beneficial, and that each or most of those should include feedback. For example, have students study then retrieve then study again to receive feedback, etc.

 References

  1. Dobson JL, Linderholm T, Yarbrough MB. Self-testing produces superior recall of both familiar and unfamiliar muscle information. Advances in Physiology Education 39: 309-314, 2015
  2. Dobson JL and Linderholm T, The effect of selected “desirable difficulties” on the ability to recall anatomy information. Anatomical Sciences Education 8: 395-403, 2015.
  3. Dobson JL, Linderholm T. Self-testing promotes superior retention of anatomy and physiology information. Advances in Health Sciences Education 20: 149-161, 2015.
  4. Butler AC, Roediger HL. Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology 19: 514-527, 2007.
  5. Blunt JR, Karpicke JD. Learning with retrieval-based concept mapping. Journal of Educational Psychology 106: 849, 2014.
  6. Dobson JL, Perez J, Linderholm T. Distributed retrieval practice promotes superior recall of anatomy information. Anatomical Sciences Education DOI: 10.1002/ase.1668, 2016.
  7. Karpicke JD, Aue, WR. The testing effect is alive and well with complex materials. Educational Psychology Review 27: 317-326, 2015.
  8. Karpicke JD, Blunt JR. Retrieval practice produces more learning than elaborative studying with concept mapping. Science 331: 772-775, 2011.
  9. Larsen DP, Butler AC, Roediger HL. Repeated testing improves long-term retention relative to repeated study: A randomized controlled trial. Medical Education 43: 1174-1181, 2009.
  10. Larsen DP, Butler AC, Lawson AL, Roediger HL. The importance of seeing the patient: Test-enhanced learning with standardized patients and written tests improves clinical application of knowledge. Advances in Health Sciences Education 18: 409-25, 2013.
  11. Linderholm T, Dobson JL, Yarbrough MB. The benefit of self-testing and interleaving for synthesizing concepts across multiple physiology Advances in Physiology Education 40: 329-34, 2016.
  12. Roediger HL, Butler AC. The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences 15: 20-27, 2011.
  13. Roediger HL, Karpicke JD. Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science 17: 249-255, 2006.
  14. Roediger HL, Karpicke JD. The power of testing memory: Basic research and implications for educational practice. Perspectives in Psychological Science 1: 181-210, 2006.
  15. Smith MA, Karpicke JD. Retrieval practice with short-answer, multiple-choice, and hybrid tests. Memory 22: 784-802, 2014.
 John Dobson is an Associate Professor in the School of Health and Kinesiology at Georgia Southern University. John received his M.S. and Ph.D. in Exercise Physiology at Auburn University. Although most of his research has focused on the application of learning strategies that were developed by cognitive scientists, he has also recently published articles on peripheral neuropathy and concussion-induced cardiovascular dysfunction. He teaches undergraduate and graduate Anatomy and Physiology, Structural Kinesiology, Exercise Physiology, Cardiovascular Pathophysiology courses. He has been an active member of the American Physiological Society since 2009, and he received the Teaching Section’s New Investigator Award in 2010 and Research Recognition Award in 2011.
Boredom, the Evil Destroyer of Motivation vs. Inquiry, the Motivation Maker

Students have an innate desire to learn and more learning takes place when doing rather than when listening. (4)  This begins in pre-school and kindergarten when children have fun while learning by playing with blocks, coloring, drawing, etc.  This is their first experience with active learning.  But then as education progresses through grade school, high school and college, something bad happens.  That is, fun learning activities are slowly replaced with often very boring listening activities filled with inane factoids, and consequently, students often become disinterested.  The disinterest is seen in the form of poor class attendance, and the lack of motivation is palpable through continual yawns, bobbing heads, and walking to the back of the classroom and looking at student laptops to see how many are streaming Netflix or shopping for shoes.  As educators that take part in this process, we actively destroy their innate desire to learn.  We do not do this intentionally, as all of us want our students to learn as much as possible.  However, with the ever increasing and endless mountain of information, we cannot teach them everything, and often feel that we should be actively teaching, rather than letting them actively learn. (3)  Thus, after hours, days and years of sitting in class “listening”, the traditional “sage on the stage” can slowly chip away at the inner desire to learn.  But, if this internal motivation can be decreased by boring activities, can it also be increased by fun or intriguing activities?

 

As educators, we hold an awesome power that has the potential to inspire and increase student motivation.  Student-centered learning activities that include but are not limited to collaborative group testing, inquiry-based learning, team-based learning and laboratory exercises (5) provide students with the opportunity to apply their minds, to have fruitful discussions with their peers (2) and to see and appreciate the complex beauty that science and medicine are.  If we can provide our students with learning activities that open their imaginations and make them feel excitement, we can actively increase their innate desire to learn, and improve their chances of success. (1)  In doing so, the awesome potential power that we hold can become fully realized in the form of life-long learners.

 

References

  1. Augustyniak RA, Ables AZ, Guilford P, Lujan HL, Cortright RN, and DiCarlo SE. Intrinsic motivation: an overlooked component for student success. Adv Physiol Educ 40: 465-466, 2016.
  2. Cortright RN, Collins HL, and DiCarlo SE. Peer instruction enhanced meaningful learning: ability to solve novel problems. Adv Physiol Educ 29: 107-111, 2005.
  3. DiCarlo SE. Too much content, not enough thinking, and too little fun! Adv Physiol Educ 33: 257-264, 2009.
  4. Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, and Wenderoth MP. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A 111: 8410-8415, 2014.
  5. Goodman BE. An evolution in student-centered teaching. Adv Physiol Educ 40: 278-282, 2016.

 

 

Robert A. Augustyniak is an Associate Professor and Physiology Discipline Chair at Edward Via college of Osteopathic Medicine- Carolinas Campus, Spartanburg, SC. Rob received his Ph.D. in Physiology at Wayne State University School of Medicine, Detroit, MI, and subsequently completed a post-doctoral fellowship at the University of Texas Southwestern Medical Center, Dallas, TX. A cardiovascular physiologist by training, his studies have focused on the blood pressure regulation during exercise and in heart failure and hypertensive states. In 2009, Rob became a founding faculty member at Oakland University William Beaumont School of Medicine where he began to focus on the scholarship of medical education. These research interests continued to grow when he moved to Spartanburg, SC in 2013. He is profoundly interested in how medical student motivation impacts learning and in finding best practices in teaching and assessment that can increase motivation. For the past several years, he has been and continues to be active within the leadership of the APS Teaching Section.