Author Archives: Allison Hood

Teaching High Level Learning Goals in Science Classes: A Lesson from Librarians

Bloom’s Taxonomy provides a way to classify learning outcomes into lower order and higher order goals. On the lower end of the spectrum we ask students to remember, understand, and apply by doing tasks such as define, list, explain, and interpret. On the higher end of the spectrum, we want students to analyze, evaluate, and ultimately create by doing tasks such as organize, compare, critique, and design (1). As educators, we all want to push our students toward the highest levels of Bloom’s Taxonomy, but how do you teach someone to create? It feels like a daunting task. I don’t think anyone ever attempted to directly teach me these higher-level skills, but instead I somehow learned them in graduate school by trial and error (in the form of a lot of red writing on drafts that I submitted to my thesis advisor). This is why I was so excited to discover the synthesis matrix. A synthesis matrix is a table that is set up to extract relevant information from sources, which can include non-scholarly, scholarly, and even student generated data. It provides a way to organize research that allows for easy comparison of the key information from many sources (3, 4, 5). I first learned about the synthesis matrix when I was teaching First Year Seminar (FYS) at Dickinson College. Learning outcomes for FYS include the ability to critically analyze information from multiple perspectives and use that information to create clear academic writing (2). Using class time to teach these skills was very different from what I typically do in my biology classes where it is a struggle just to keep up with the vast amount of content. Therefore, it is an understatement to say that I was out of my element teaching FYS. Fortunately, each FYS class at Dickinson is assigned to librarians. I was fortunate to have Dickinson librarians Nick Lonergan and Jessica Howard help design assignments and teach methods that help students achieve the FYS learning goals. Nick and Jessica designed a synthesis matrix assignment to help students find relevant non-scholarly and scholarly sources and extract information from the sources to help them compare viewpoints on different concepts found in each reference. In this case, the synthesis matrix was used as a homework assignment to prepare students to organize and synthesize information from multiple references in future writing assignments. The power of the synthesis matrix immediately hit me as I realized that this is what experts do in their heads. Many years of reading and analyzing both the work of others and our own research leads to the formation of a mental synthesis matrix that we can pull from as needed in our respective fields. I think my life would have been a lot easier if I knew about the concept of a paper synthesis matrix in graduate school. Since my discovery of the synthesis matrix in FYS, I have used it in different ways in all of the biology classes I teach at Dickinson College including Introductory Biology (Biology 132), Physiology (Biology 333), and Molecular Pathophysiology, which is a research and writing intensive class (Biology 433). Some ways I have used it include:

Homework assignment: On the simplest level, the synthesis matrix can be used to assess student ability to find appropriate references and extract relevant information from those references. An example of this is described above for FYS and I can easily see this working well in Introductory Biology classes. I have also done this in Molecular Pathophysiology (Biology 433) as a homework assignment prior to assigning a literature review writing assignment.

Classroom Activity: In Physiology (Biology 333), I have lab groups (6 groups of 4 students each) find a primary publication on a topic related to an upcoming lab project and analyze it for specific information related to research methods and results. In order to avoid overlap and make sure they found the right type of paper, I have the students email the paper they found for approval. If two groups found the same paper or if it is not the right type of source (for example, some students will try to use a review), I will ask them to find another paper. In lab I draw a synthesis matrix on the board and distribute blank handouts of the same synthesis matrix. We then go around the room and as students report their findings, I fill in the synthesis matrix. When it is done, I demonstrate how to use the matrix to synthesize the results of multiple references to come to overarching conclusions and design new experiments. We use this to guide the design of a class research project and in future writing assignments.

Model Creation: The most complex way I have used the synthesis matrix is in Molecular Pathophysiology (Biology 433). As a research based Writing in the Discipline (WID) class, we focus all of our attention on analyzing primary literature and doing novel experiments in lab. Throughout the semester, I encourage students to draw their own textbook style models of what data show. This can be done by synthesizing results from a single primary publication (if the authors did not already generate a model), multiple primary publications (as seen in review articles), and even by incorporating student lab results with published results. The synthesis matrix can be set up to accommodate all of these approaches. For example, instead of labeling columns by reference #, columns can be labeled by figure # for a single primary publication. Similarly, a column for class lab results can be added to incorporate class results (Figure 1). This is my favorite way of using the matrix. It is so powerful for students to see how one small experiment they did fits in with the big picture of what others have published.

Of all the teaching methods I have tried over the years, the synthesis matrix is the closest I have come to teaching students how to think like an expert. It has also allowed me to do a better job of breaking the research and writing process down into component parts. If you tell a college senior to write a one page introduction section of a scientific paper with 5 references, many of them think they can produce one page of writing in a couple of hours (I know I thought that as a senior in college!). However, if you force them to do a synthesis matrix that includes analysis of the relevant information in 5 primary papers prior to writing about them, they quickly realize how much work is involved.

Ultimately, the most important lesson I learned though all of this is that teaching in science classes can benefit greatly from methods used in classes outside of our discipline. If you Google “synthesis matrix”, it is a commonly used method promoted on Library, Academic Coaching, and Writing Center websites at many colleges and universities (3, 4, 5). However, I never heard of it until librarians introduced me to it while teaching FYS. Interacting with scholars outside of my discipline has helped me to integrate the teaching of higher level learning goals alongside lower level learning goals related to content in my classes.

References

Vanderbilt University, Center for Teaching, Bloom’s Taxonomy. https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/. 2018 Vanderbilt University, Accessed December 28, 2018.
Dickinson, First-Year Seminar. https://www.dickinson.edu/homepage/99/first_year_seminars . Accessed December 28, 2018.
Ashford University, Synthesis Matrix. https://writingcenter.ashford.edu/synthesis-matrix . 2017 Bridgepoint Education. Accessed December 28, 2018.
Johns Hopkins Sheridan Libraries, Write a Literature Review. http://guides.library.jhu.edu/lit-review/synthesize . 2017 Johns Hopkins Sheridan Libraries. Accessed December 28, 2018.
Academic Coaching and Writing, A Synthesis Matrix as a Tool for Analyzing and Synthesizing Prior Research. https://academiccoachingandwriting.org/dissertation-doctor/dissertation-doctor-blog/iii-a-synthesis-matrix-as-a-tool-for-analyzing-and-synthesizing-prior-resea . 2018 Academic Coaching and Writing LLC. Accessed December 28, 2018.

Tiffany Frey is an Assistant Professor of Biology at Dickinson College in Carlisle, PA. She received her Ph.D. in Molecular Medicine from Johns Hopkins University School of Medicine and also has a certificate in Adult Learning from Johns Hopkins University School of Education. She teaches Introductory Biology, Physiology, and Molecular Pathophysiology at Dickinson College. Her research interests are focused on understanding the cellular and molecular basis of autoinflammatory disease and incorporating and assessing the effects of scholarly teaching methods in her courses. Outside of work, she enjoys spending time with her family (husband, 2 children, and dog Charlie), reading, participating in exercise classes, and running in local races.

Affective Teaching and Motivational Instruction: Becoming More Effective Educators of Science

As educators, we’re intimately familiar with learning objectives such as, “Using Fick’s principle, calculate the diffusion of a substance across a membrane.” Also, as scientists, we are familiar with technical objectives such as, “Using a micropipette, transfer 5μL of Solution A into the chromatography chamber.” In terms of learning conditions, the first is an intellectual skill and the second is a motor skill.¹ One area in which we don’t often give much thought is the third type of skill that was identified by Gagné and Medsker — the affective skill. This is the area that is most often neglected by educators because it is the hardest to evaluate and quantify. We can’t explicitly say to a student, “By the end of the semester you will develop a love of physiology.” We can hope to achieve this through the semester, but as educators, the best that we can do is hope to instill these attitudes, choices, and values in our learners that persist beyond our brief time with them in the classroom.

Instilling attitudes in our learners is a complex goal. This is, in part, because stating an affective goal is at times counterproductive to the goal and interferes with learning. In the example above, it is clearly ridiculous to expect that all students will leave our classrooms with a true passion for our subject matter. Some clearly will, but others will not. That will be shaped by the attitudes with which students enter our classrooms. Those attitudes consist of the knowledge that a learner has about a subject – the cognitive aspect, how the person feels about the subject — the affective aspect, and how the person behaves in response to those influences — the behavioral aspect.² So despite our best interests to instill a care for the animal and human models we frequently use in experiments, it is completely beyond our ability to control the behavior of our learners outside of the classroom. That doesn’t mean that we shouldn’t still try because the majority of our students will come away with those lessons intact. Additionally, affective learning is difficult to assess. We can test the knowledge and skills necessary and ask about student feelings³, but at the end of the day, our students will make a choice on their behaviors on their own. For that reason, we should not make affective learning objectives part of our formal instruction plan. Because there are so many methods that depend on the affect you might want to influence, I’m going to focus on two areas that are most common: attitude and motivational instruction.

Katz and Stotland identified five types of attitudes.⁴ These types of attitudes vary with differing levels of affective and cognitive components, but the key takeaway is that individual experiences and the results and consequences of previous choices dramatically shape the attitudes with which our learners enter our classrooms. Reward for behavior not only reinforces the behavior, but also the cognitive and behavioral components that drive that behavior.¹ When we focus purely on the cognitive and the motor skill aspects of learning, we can often get away with a fair amount of do-as-I-say-not-as-I-do-style instruction. The problem with this is that students look to the faculty and other instructors for role model behavior. Thus, the more accurately that we reflect the attitudes that we want to instill in our learners, the more the students will reflect those ideals.³ One of the easiest ways to bring about these changes of attitudes are through in-class discussions.⁵ This positive benefit is most likely due to differences that are raised during discussion, sometimes prompting the discovery of a discrepancy between existing attitudes in a learner and new facts that are being presented. The learners then have a choice on how to adapt to the new desired attitudes. Most importantly, never underestimate group acceptance of attitudes, as immediate social reinforcement can be a powerful driver in solidifying attitudes.

Having discussed attitude, motivational instruction is another key area that is relevant to affective learning. No two students enter the classroom with the same motivation. One student may be enrolled in your class because of a deep passion for your subject matter while another is there simply to satisfy a requirement for their major. This mix of intrinsic and extrinsic motivations will drive the overall outcomes of affective learning. The student who is highly motivated by an intrinsic interest in your subject or the student who is extrinsically driven by the reward of a good grade (or fear of a bad grade) will generally excel in class, albeit for different reasons. The student who is there out of obligation to meet a requirement may have very little motivation to do anything beyond what is required of them to get by. To help with those students who are lacking in motivation, JM Keller broke motivational instruction into four components: attention, relevance, confidence and satisfaction.⁶ Gaining the attention of students through demonstrations, discussions, and other active learning techniques may help keep student motivation high. Practical application of concepts and ideas will generally inspire higher motivation than abstract or arbitrary examples.⁷ Keeping the material relevant will generate motivation for intrinsic learners by providing self-improvement and for the extrinsic learners by providing a reward, such as doing well on the exam. Confidence is a harder area to approach, as students must first believe they are capable of meeting the stated objectives. Making the material too easy will not lead to feelings of accomplishment, while making the material too challenging will undermine confidence in all learners.¹ Finally, satisfaction can be achieved by learners of all types, regardless of motivation type when outcomes match objectives. Keeping motivation high by providing opportunities to apply learning will drive further motivation to continue learning.

Last week I completed a comprehensive review of our capstone thesis writing course, which has changed dramatically over the past year and a half while I have been the course director. Initially, the goal of the course was to have students write a literature research paper on a physiological topic of their choosing where their grade was entirely dependent upon the finished paper. The students were frequently frustrated with a lack of guidance in the course and the faculty regularly complained about the burden of reading papers of sometimes-questionable quality. Clearly there were issues with the affective components of this course from both the student and faculty side. I’ve de-emphasized the actual paper and refocused the course on the process of writing with stated learning outcomes such as: 1) Develop the language that helps us talk about science; 2) Strengthen research skills to become educated consumers of science; and 3) Gain specialized knowledge in a selected area of physiological research. Focusing the course in this way has yielded measurable results in course evaluations and faculty perceptions of paper quality from the students. By focusing on the affective components of writing and giving students more opportunities to apply their new skills, overall satisfaction has improved. Like all works of science, though, this course continues to evolve and improve. In short, to be effective teachers, we need to go beyond the intellectual and motor skills and make sure we address the affective learning of our students as well.

¹ Gagné RM and Medsker LK. (1996). The Conditions of Learning. Training Applications. Fort Worth: Harcourt Brace College Publishers.

² Baron RA and Byrne D. (1987). Social Psychology: Understanding Human interaction. 5^th ed. Boston: Allyn and Bacon.

³ Dick W and Carey L. (1996). The Systematic Design of Instruction. 4^th ed. New York: HarperCollins Publishers.

⁴ Katz D and Stotland E. (1959). A preliminary statement to a theory of attitude structure and change. In Psychology: A Study of Science. vol 3. New York: McGraw-Hill.

⁵ Conrad CF. (1982). Undergraduate Instruction. In Encyclopedia of Educational Research. 5^th ed. New York: The Free Press.

⁶ Keller JM. (1987). Development and use of the ARCS model of instructional design. Journal of Instructional Development. 10;3. 2-10.

⁷ Martin BL and Briggs LJ. (1986). The Affective and Cognitive Domains: Integration for Instruction and Research. Englewood Cliffs, New Jersey: Educational Technology Publications.

Ryan Downey is an Assistant Professor in the Department of Pharmacology & Physiology at Georgetown University. As part of those duties, he is the Co-Director for the Master of Science in Physiology and a Team Leader for the Special Master’s Program in Physiology. He teaches cardiovascular and neuroscience in the graduate physiology courses. He received his Ph.D. in Integrative Biology from UT Southwestern Medical Center. His research interests are in the sympathetic control of cardiovascular function during exercise and in improving science pedagogy. When he’s not working, he is a certified scuba instructor and participates in triathlons.

Likely or unlikely to be true? I like to have students hypothesize

Throughout my science education career, if I was asked what I do, I responded “I write standardized tests.” Let me assure you, this doesn’t win you too many fans outside of science education assessment circles. But in my opinion, there is nothing better to help one develop an understanding and intuition about how students learn than interviewing hundreds of students, listening to their thinking as they reason through questions.

When I listen to students think aloud as they answer questions, I learn a lot about what they know and about their exam-taking processes too. For example, while interviewing a student on a multiple true-false format physiology question, the student answered all the true-false statements then said “Wait, let me go back. There’s always some exception I might be missing.” For this student, physiology always broke the rules and the exams they typically took tried to test whether they knew the exceptions. Although my intention for the question was to have the students apply general conceptual knowledge, the student, like most others I interviewed, was instead spending a lot of time making sure they had recalled all the right information. Eventually, moments like this led to a simple change in question format that created an interesting shift in the way questions elicited thinking from faculty and students alike.

The interview mentioned above occurred during the process of writing a programmatic physiology assessment, Phys-MAPS.² The goal of this assessment and the others in a suite of Bio-MAPS assessments was to build tools that could measure student learning across biology majors. Our working team³ and I chose to build all the assessments using a multiple true-false format, where for each question, a short scenario is described, followed by several (often 4-6) statements about the scenario that students identify as either true or false. We chose this format for its high utility assessing how students can hold both correct and incorrect ideas about a topic simultaneously,⁴ highly pertinent to learning across a major. In addition, the multiple true-false format has the benefit of facilitating easy and quick grading for a large number of students while still allowing for a rich understanding of student thinking comparable to essay assessments.⁵

Example of Modified Multiple True-False Design (from a question similar to but not on the Phys-MAPS)

However, as I was creating the physiology-specific assessment and Dr. Mindi Summers was creating the ecology-evolution-specific assessment, we ran into challenges when writing statements that needed to be absolutely “true” or “false.” Sometimes we had to write overly complex scenarios for the questions because too many constraints were needed for a “true” or “false” answer. In addition, discipline experts were refusing to ever say something was “true” or “false” (especially, but not solely, the evolutionary biologists). Thus, many of our statements had to be re-written as something that was “likely to be true” or “unlikely to be true”, making the statements bulky and long.

Dr. Summers was the first to bring up in our working group meeting the idea of modifying the true-false format. She suggested changing the prompt. What initially read “Based on this information and your knowledge of biology, evaluate each statement as true or false,” became “Based on this information and your knowledge of biology, evaluate each statement as likely or unlikely to be true.” I was instantly sold. I thought back to the student who had spent so much extra time trying to search her brain for the exceptions to the general rules. Surely, this was going to help!

It did. For starters, the discipline experts we were consulting were much more inclined to agree the answers were scientifically accurate. And for good reason! We science experts do not often work in the absolutes of “true” and “false”. In fact, I’m pretty sure a whole field of math was created for exactly this reason. I also saw a difference in how students responded to the new language. In my interviews, I noticed students took considerably less time on the assessment and I never again heard a student stop to try to remember all the exceptions they might know. Better yet, I started hearing language that reflected students were applying knowledge rather than trying to remember facts. For example, in the previous true-false format, I often heard “Oh, I just learned this,” and then I would watch the student close their eyes and agonize trying to remember a piece of information, when all the information they needed to answer the question was right in front of them. With the new “likely or unlikely to be true” format, I was hearing more “well that’s generally true, so I think it would work here too.” It appeared that students had shifted to a more conceptual rather than factual mindset.

But what really convinced me that we were on to something worthwhile was the awareness of some students of what they were truly being asked to do. “Wait, so basically what you want me to do is hypothesize whether this would be true [in this new scenario] based on what I already know?” YES!!! (I do my inner happy dance every time.)

We educators hear the message from a million places that we should teach science as we do science. I maintain that this should count towards how we assess science knowledge and skills too, asking students to apply their knowledge in new contexts where there is no known answer. But when science explores the unknown, how do you ask about the unknown and still have a right answer to grade? (Easily, on a scantron, that is.) As scientists, we use our knowledge to make predictions all the time, not thinking that our hypotheses will absolutely be true, but that they are the mostly likely outcome given what we already know. Why not show our students how much we value that skill by asking them to do the same?

¹ Answer: Likely to be true.

²More information about the Phys-MAPS and all of the Bio-MAPS programmatic assessments can be found on: http://cperl.lassp.cornell.edu/bio-maps

³ The Bio-MAPS working group includes: Drs. Michelle Smith, Jennifer Knight, Alison Crowe, Sara Brownell, Brian Couch, Mindi Summers, Scott Freeman, Christian Wright and myself.

⁴ Couch, B. A., Hubbard, J. K., and Brassil, C. E. (2018). Multiple–true–false questions reveal the limits of the multiple–choice format for detecting students with incomplete understandings. BioScience 68, 455–463.

⁵ Hubbard, J. K., Potts, M. A., and Couch, B. A. (2017). How question types reveal student thinking: An experimental comparison of multiple-true-false and free-response formats. CBE Life Sci. Educ.

Dr. Katharine (Kate) Semsar finally found a job that uses all her diverse training across ecology, physiology, genetics, behavioral biology, neuroscience, science education, and community building. Kate is the Assistant Director of STEM Programming for the Miramontes Arts & Sciences Program (MASP), an academic community for underrepresented students in the College of Arts & Sciences at the University of Colorado Boulder.

She received her PhD from North Carolina State University and continued her training at University of Pennsylvania. She then became a science education specialist with the Science Education Initiative in the Integrative Physiology department at the University of Colorado Boulder, studying how students learn and collaborating with faculty to incorporate fundamental principles of learning in their courses. She continued her science education research with the Bio-MAPS team before finally landing in her dream career, teaching and mentoring students in MASP. Despite the career shift, she still loves watching people’s reactions when she tells them she used to write standardized assessments.

Teaching for Learning: The Evolution of a Teaching Assistant

An average medical student, like myself, would agree that our first year in medical school is fundamentally different from our last, but not in the ways most of us would expect. Most of us find out that medical school not only teaches us about medicine but it also indirectly teaches us how to learn. But what did it take? What is different now that we didn’t do back in the first year? If it comes to choosing one step of the road, being a teaching assistant could be a turning point for the perception of medical education in the long run, as it offers a glimpse into teaching for someone who is still a student.

At first, tutoring a group of students might seem like a simple task if it is only understood as a role for giving advice about how to get good grades or how to not fail. However, having the opportunity to grade students’ activities and even listen to their questions provides a second chance at trying to solve one’s own obstacles as a medical student. A very interesting element is that most students refuse to utilize innovative ways of teaching or any method that doesn’t involve the passive transmission of content from speaker to audience. There could be many reasons, including insecurity, for this feeling of superficial review of content or laziness, as it happened for me.

There are, in fact, many educational models that attempt to objectively describe the effects of educating and being educated as active processes. Kirkpatrick’s model is a four-stage approach which proposes the evaluation of specific aspects in the general learning outcome instead of the process as a whole (1). It was initially developed for business training and each level addresses elements of the educational outcome, as follows:

Level 1- Reaction: How did learners feel about the learning experience? Did they enjoy it?
Level 2- Learning: Did learners improve their knowledge and skills?
Level 3- Behavior: Are learners doing anything different as a result of training?
Level 4- Results: What was the result of training on the business as a whole?

Later, subtypes for level 2 and 4 were added for inter-professional use, allowing its application in broader contexts like medicine, and different versions of it have been endorsed by the Best Evidence in Medical Education Group and the Royal College of Physicians and Surgeons of Canada (1) (2). A modified model for medical students who have become teachers has also been adapted (3), grading outcomes in phases that very closely reflect the experience of being a teaching assistant. The main difference is the inclusion of attitude changes towards the learning process and the effect on patients as a final outcome for medical education. The need for integration, association and good problem-solving skills are more likely to correspond to levels 3 and 4 of Kirkpatrick’s model because they overcome traditional study methods and call for better ways of approaching and organizing knowledge.

Diagram 1- Modified Kirkpatrick’s model for grading educational outcomes of medical student teachers, adapted from (3)

These modifications at multiple levels allow for personal learning to become a tool for supporting another student’s process. By working as a teaching assistant, I have learned to use other ways of studying and understanding complex topics, as well as strategies to deal with a great amount of information. These methods include active and regular training in memorization, deep analysis of performance in exams and schematization for subjects like Pharmacology, for which I have received some training, too.

I am now aware of the complexity of education based on the little but valuable experience I have acquired until now as a teacher in progress. I have had the privilege to help teach other students based on my own experiences. Therefore, the role of a teaching assistant should be understood as a feedback process for both students and student-teachers with a high impact on educational outcomes, providing a new approach for training with student-teaching as a mainstay in medical curricula.

References

Roland D. Proposal of a linear rather than hierarchical evaluation of educational initiatives: the 7Is framework. Journal of Educational Evaluation for Health Professions. 2015;12:35.
Steinert Y, Mann K, Anderson B, Barnett B, Centeno A, Naismith L et al. A systematic review of faculty development initiatives designed to enhance teaching effectiveness: A 10-year update: BEME Guide No. 40. Medical Teacher. 2016;38(8):769-786.
Hill A, Yu, Wilson, Hawken, Singh, Lemanu. Medical students-as-teachers: a systematic review of peer-assisted teaching during medical school. Advances in Medical Education and Practice. 2011;:157.

The idea for this blog was suggested by Ricardo A. Pena Silva M.D., Ph.D. who provided guidance to Maria Alejandra on the writing of this entry.

María Alejandra is a last year medical student at the Universidad de Los Andes, School of Medicine in Bogota, Colombia, where she is has been a teaching assistant for the physiology and pharmacology courses for second-year medical students. Her academic interests are in medical education, particularly in biomedical sciences. She is interested in pursuing a medical residency in Anesthesiology. Outside medical school, she likes running and enjoys literature as well as writing on multiple topics of personal interest.