Monthly Archives: May 2016

Teaching Toolbox: Tips and Techniques for Assessing What Students Know

What has to shift to change your perspective? Thomas Kuhn coined the term paradigm shift and argued that science doesn’t progress by a linear method of gathering new knowledge, rather, a shift takes place when an anomaly subverts the normal practice, ideas and theories of science. Students learn through interaction with the surrounding environment mediated by prior knowledge from new and previous interactions with family, friends, teachers, and other sociocultural experiences (Falk & Adelman, 2003). Deep understanding of concepts depend on the interaction of prior experience with new information. As Kuhn stated in his 1962 book The Structure of Scientific Revolutions, “The challenge is not to uncover the unknown, but to obtain the known.”

In order to assess what students know, you need to find out what they already knew. An assessment can only provide useful information if it is measuring what it is intended to. In the medical field, assessments are used all the time, for example, an MRI is a useful diagnostic tool to determine the extent of tissue damage but it is not necessarily useful for establishing overall health status of an individual. Assessing what a student knows with a multiple choice test may also not be useful in establishing an overall picture of what knowledge a student possesses or how that knowledge is applied, especially if the items are not measuring what they are supposed to. Construct validity provides evidence that a test is measuring the construct it is intended to. How to measure construct validity is beyond the scope of this article, for information, see the classic work by Messick (1995). Outside of the psychometrics involved in item or assessment construction, I’ll provide some quick tips and techniques I have found useful in my teaching practice. What can you do to separate real learning with deep understanding from good test taking skills or reading ability? How can you assess what students know simply and effectively?

Instruction in a classroom environment needs to be connected with assessment rather than viewing instruction and assessment as separate activities. Understanding student thinking can be done with formative assessment which benefits students by identifying strengths and weaknesses and gives instructors immediate feedback regarding where students are struggling so that issues can be addressed immediately. By providing students with context in the form of a learning goal at the start of a class, the clear objective of the lesson allows them to begin making connections between what they already know and new information. When designing or preparing for a class, ask yourself:

What do I assume they already know?
What questions can I ask that will help me confirm my assumptions?
What are the most common misconceptions related to the topic?

Tips for checking students background knowledge

On a whiteboard or in a presentation, begin with one to three open ended questions and/or multiple choice questions. Ask students to respond in two to three sentences, or circle a response. It’s important to let them know that the question(s) are not being graded, rather, you are looking for thoughtful answers that will help guide instructional decisions. Share the results at the start of the next class or with a free tool like Plickers for instant feedback.
Short quizzes or a survey with Qualtrics, Google Forms, or Doodle Poll can be used via Black Board prior to class. Explain that you will track who responded but not what the individual student responded at this point. Share the results and impact on course design with students.
Group work. Using an image, graph, or some type of problem regarding upcoming course content, have students come up with a list of observations or questions regarding the material. Use large sheet paper or sticky notes for them to synthesize comments then review the themes with the class.

Formative assessment is used to measure and provide feedback on a daily or weekly basis. In addition to learning goals communicated to students at the beginning of each class and warm up activities to stimulate thinking about a concept, formative assessment can include comments on assignments, projects or problem sets, asking questions that are intentional towards essential understanding rather than a general, “Are there any questions?” at the end of a lesson. To add closure and summarize the class with the learning goal in mind, provide index cards or ask students to take out a piece of paper and write in a couple of sentences what the most important points of the lesson were and/or ask them to write what they found most confusing so that it can be addressed in the next class. Formative assessments provide tangible evidence for you to see what your students know and how they are thinking and they provide insight and feedback to students in improving their own learning.

Summative assessment includes quizzes, tests and projects that are graded and used to measure student performance. Creating a well-designed summative assessment involves asking good questions and using rubrics. In designing an assessment that will accurately measure what students know, consider:

What do you want your students to know or be able to do? This can also be used in each lesson as a guiding objective.
Identify where you will address the outcomes in the curriculum.
Measure what they know with your summative assessment.
Based on the measurement, what changes can be made in the course to improve student performance?

Good questions

Measure what you intend for them to measure.
Allow students to demonstrate what they know.
Discriminate between students who learned what you intended versus those that did not.
Examine what a student can do with what they learned versus what they simply remember.
Revisit learning goals articulated at the beginning of a topic, unit or course.
Use a variety of questions such as multiple choice, short answer and essay questions.

Rubrics

Used for oral presentations, projects, or papers.
Evaluate team work.
Facilitate peer review.
Provide self-assessment to improve learning and performance.
Motivate students to improve their work.

Online rubric resources for educators include, Rubistar, Online Instruction Rubric, and Value Rubrics.

Students do not enter your classroom as a blank slate. Assessing and determining what students know targets gaps in knowledge. By incorporating an activity or a question in a small amount of time at the start and end of a class, you can check on potential and actual misconceptions so that you may target instruction for deep understanding. Background checks of prior knowledge provide awareness of the diversity of your students and their experiences further designing and improving instruction for active, meaningful learning. Creating a bridge between prior knowledge and new material provides a framework for students for a paradigm shift in learning and makes it very clear for them and for you to see what they learned by the end of a lesson or the end of a course.

References

Falk JH, Adelman, L.M. Investigating the Impact of Prior Knowledge and Interest on Aquarium Visitor Learning. Journal of Research in Science Teaching. 2003;40(2):163-176.

Kuhn TS. The Structure of Scientific Revolutions. 4th ed. Chicago: The University of Chicago Press; 1962.

Messick, S. (1995). Standards of validity and the validity of standards in performance assessment. Educational measurement: Issues and practice,14(4), 5-8.

Jennifer (Jen) Gatz graduated from Ithaca College in 1993 with a BSc in Exercise Science and began working as a clinical exercise physiologist in cardiac and pulmonary rehabilitation. Jen received her MS in Exercise Physiology from Adelphi University in 1999, founded the multisport endurance training company, Jayasports, in 2000, and expanded her practice to include corporate health and wellness for Brookhaven National Laboratory, through 2012. Along the way, Jen took her clinical teaching practice and coaching experience and returned to school to complete a Master of Arts in Teaching Biology with NYS teaching certification from Stony Brook University in 2004. A veteran science teacher for 12 years now at Patchogue-Medford High School in Medford, NY, Jen is currently teaching AP Biology and Independent Science Research. A lifelong learner, Jen returned to Stony Brook University in 2011 and is an advanced PhD candidate in Science Education anticipating the defense of her dissertation in the fall of 2016. Her dissertation research is a melding of a love of physiology and science education focused on understanding connections among cognitive processes, executive functioning, and the relationship to physical fitness, informal science education, and environmental factors that determine attitudes towards and performance in science. In 2015, Jen was a recipient of a Howard Hughes Medical Institute Graduate Research Fellowship.

Summative Assessment – Does the End Justify the Means?

I recently heard two students in academic difficulty recount painfully similar stories about how their own studies had come off the rails following the attempted suicide of younger siblings, who were themselves college undergraduates. What are the chances of hearing two such stories in one day? Well, according to Emory University’s statistics, there are 1000 suicides per year among college students and as many as 1 in 10 students have made a plan for suicide at some point¹.

I do not pretend to understand such shocking statistics. Known stressors for college students include interpersonal factors such as new social environments and relationships, personal factors like poor sleeping and eating habits and financial problems, academic workload and poor grades². There are many things here I cannot help with directly, but on the academic side it does make me reflect on what can I do as a professor to help?

I am a physiologist, not a counselor or a psychiatrist. However, I can begin by learning what counselling services my university has (they are excellent as it turns out – and I bet yours are too), and I can do a better job of guiding distressed students to seek their help; if the need arises I can ask students straight out if they have suicidal thoughts and I can dial 911 if necessary. But another thought occurs to me….at certain points I become the focal point for student stress and that happens each time I choose to set a high stakes exam.

It is an old axiom that assessment drives student learning but with such power comes great responsibility! The stress incurred by students through testing (especially when graded) must come with some tangible educational benefit. In other words, I must weigh the costs and benefits of deciding to set up a particular assessment and especially how much summative testing to include in the block. After all, we know that the rate of forgetting is significant, even after the mega high stakes United States Medical Licensing Exams³.

One strange observation I have made over time is that students and faculty often align with wanting more testing; students want to lessen the burden of information per test and faculty want more complete sampling of the material. I have struggled in three different institutions to reduce summative testing load and to replace some tests with formative testing instead. Each time, student score distributions at the end of a course were not affected, whereas student stress levels seemed lower and the classroom was a more relaxed and enjoyable place.

Is all testing bad or can assessment be a win-win where positive educational impacts outweigh the negatives? Progressive testing methods such as project-based assessment and collaborative assessment align with 21^st century goals of graduating students with competencies in critical thinking, communication skills, technology literacy etc., perhaps without the same level of stress that cramming for knowledge-based tests produces. Recent studies have convincingly shown that frequent zero-stakes testing used as a means to rehearse content produces major learning gains in what has been coined the “testing effect⁴”. Commercially available adaptive learning platforms are also available in which the technology helps students to continually self-assess towards achievement of mastery⁵.

As a faculty member I can help to address student burnout and stress by carefully considering my choices of summative assessment and maximizing testing for learning. I believe we need to be intentional about teaching students how to learn by addressing learning preferences, motivation and self-regulated learning habits. The dismaying statistics I started with suggest universities should also provide more learning opportunities on wellness, nutrition, resiliency, lifestyle management, financial planning, etc., as part of all our programs. I realize there are many other factors to think about and hope some discussion will follow to explore these gaps.

Resources

Emory Cares 4 U. Suicide Statistics http://www.emorycaresforyou.emory.edu/resources/suicidestatistics.html Accessed 4/22/16
Ross SE, Niebling BC, Heckert TM. Sources of stress among college students. College Student Journal 33 p312-318, 1999
Ling Y, Swanson DB, Holtzman K, Deniz Bucak S. Retention of basic science information by senior medical students Academic Medicine 83(10 Suppl):S82–S85, 2008
Karpicke JD, Roediger HL 3rd. The critical importance of retrieval for learning. Science 319:966–968, 2008
Flashcards. Memory Aids. An automatic study plan for every lecture. https://www.osmosis.org/ Accessed 4/22/16

J.D. (Jon) Kibble graduated from the University of Manchester in 1994 with his BSc and PhD in physiology. In his first faculty position at the University of Sheffield Medical School, Dr. Kibble started a research laboratory to investigate the molecular physiology of renal tubular ion transport. His passion for teaching was ignited at this time as he began to teach medical physiology and anatomy. Next he became a Course Director for Medical Physiology at St. George’s University in the West Indies and later at The Memorial University of Newfoundland in Canada. The experience of teaching over 4,000 medical students in different parts of the world established his academic base as a medical physiology teacher.
Jon moved to the United States in 2008 to join the founding faculty of the University of Central Florida, College of Medicine. In 2010 he was appointed as Assistant Dean for Medical Education and is responsible for overseeing the development of basic science content throughout the curriculum. His scholarly work includes publication of learning resources in the form of a textbook on medical physiology, flashcards and electronic resources for adaptive learning. His primary research interest relates to the efficacy of formative assessment and understanding student engagement in self-assessment.
Jon became a Fellow of UK Higher Education Academy in 2007, is deputy editor of the journal Advances in Physiology Education, currently chairs the American Physiological Society’s Teaching Section and is a member of the International Union of Physiological Society’s Education Committee. He was the recipient of the Alpha Omega Alpha Robert J Glaser Distinguished Educator Award, 2015.

Guided Inquiry: A Flexible Technique for Engaging Students

Like many instructors, I am continually looking for ways to better engage students and, hopefully, benefit their learning. To this end I have incorporated a variety of techniques into my courses; including inquiry labs, case-based studies, clicker questions (without actually using clickers), and various electronic supplements. These have been met with varying degrees of success, as assessed solely by student feedback. And although I continue to use all of these to some extent, my new “favorite” active learning activity is guided inquiry. I find it a flexible method that can used in class, as homework, or in labs. It can also be used to focus on graphical interpretation, which is a skill I have a particular interest in.

What is Guided Inquiry?

Guided inquiry is a form of inquiry-based learning, the latter of which is a broad category of learning where students focus on solving problems, scenarios or posed questions in a manner that aids them in constructing knowledge. The phrase “guided inquiry” is not always defined consistently, with some experts only using this term when the solution to the presented scenario is not already known, and others using it to describe any process that “guides” students through the learning cycle. I personally apply the phrase in the same manner in which is it presented by the POGIL (Process Oriented Guided Inquiry Learning) Project, where it is used to describe the following process:

Model Exploration. Models are often figures or graphs, but could include objects, videos, etc. Exploration commonly involves direct questions, which can be answered by appropriately examining the model and interpreting the information correctly. In a presentation of figures of a homeostatic feedback loop, model exploration might involve listing the components of the loop.

Concept invention. Additional questions require students to identify patterns in the model. In the example of the homeostatic feedback loop, such questions might focus on the interactions between components and their reliance on one another.

Students are asked additional questions that require them to apply the concept to a different or new scenario. Such questions may be convergent (students should have similar answers) or divergent (where there may be multiple reasonable answers).

As in many types of active learning, this process is best carried out in small groups of students. When process skills are added into the overall activity, such as teamwork or oral communication, the activity is often described with the trademark POGIL acronym, although this label should only be used after the activities have been reviewed and approved by the POGIL office.

How I Use Guided Inquiry in my Physiology Courses

When I started teaching 15 years (or so) ago, I was the “typical” lecturer, presenting information on slides (overheads to start), hoping that the students would passively absorb the information. In my teaching there have always been certain figures, such as the oxyhemoglobin dissociation curve, that I would spend significant time presenting information about. Today, as a reformed educational facilitator, I have a goal of using guided inquiry (or another active learning technique) to have students investigate such figures. To this end, I use guided inquiry in lectures, in some labs and occasionally as homework. The benefits of using these “in class” include the ability to roam the room and eavesdrop on student conversations and to include a “report out”, which can stimulate discussion even amongst the groups.

I am Interested, How do I Get Started?

Guided inquiry activities can take significant effort to put together, especially without good examples to work from. If you are a member of the Human Anatomy and Physiology Society (HAPS), you can access five activities free from their site. There are also currently two books that contain POGIL activities for physiology, and they would be a great way to get started. The collection of guided inquiry activities for physiology is growing, which should provide additional options for those of us looking for added options to use in our classes.

Ron Gerrits is a Professor of biomedical engineering at Milwaukee School of Engineering (MSOE). He earned his BS degree in biomedical engineering from MSOE in 1994 and his Ph.D. in Physiology from the Medical College of Wisconsin (MCW) in 1999. That same year he returned to MSOE to become the coordinator of the health science courses. Since that time he has taught a variety of courses, including cell biology, microbiology, nutrition, physiology, pathophysiology and pharmacology, to nursing, biomedical engineering and perfusion students. His main professional interest is science education. To this end he has been active with the Biology Scholars program, the Human Anatomy and Physiology Society, Project Lead the Way, and various summer programs for high school students. He has also been the program director of the Masters of Science in Perfusion program since 2002.