As of March 2020, when the SARS-COV-2 pandemic sent teachers and students home to figure out online instruction and learning, I had been teaching high school biology/AP biology for 27 years and anatomy & physiology at the two local community colleges for 7 years. Since I had been practicing flipped coursework for years, I knew that my biggest challenge would be how to fairly assess my students and their learning. This challenge would be compounded by an at-home virtual testing environment without any proctoring.
As I pondered the best approach to my assessment challenge, I was naturally drawn to the College Board’s 2012-13 redesign of the AP (Advanced Placement) Biology curriculum and examination. In the redesign, the AP curriculum focuses on four “Big Ideas” or broad themes covering a number of subtopics/concepts that are further broken down into learning objectives for students. The examination focuses on measuring student learning and skills using what the College Board (AP Higher Education, 2012-2013) calls an “evidence-centered-design approach that parallels the curriculum’s understanding-by-design approach.” The examination consists of a mix of multiple-choice and short-answer/free-response questions. I know from my many years of grading student AP essays/short answers that, when students turn to Google for their answers, they often fail. Students will frequently regurgitate the rubrics for grading the prompts rather than dissecting and answering the question. Subsequently, the students fail to demonstrate their own learning or understanding of the material. This is unfortunate as it is also a missed opportunity for feedback, correction and/or remediation.
In designing a new accelerated online physiology course, I really wanted the course assessments to mimic the AP Biology style of assessments. I wanted them not only to be aligned with course objectives, but to require students to think about and demonstrate the skills and concepts they were learning. I was skeptical, but hopeful I could also find an approach in which I would not have to rewrite the entire examination from scratch each term. In my search for related pedagogies, I ran across an article in the May 2020 HAPS Educator, “Testing in the Age of Active Learning: Test Question Templates Help to Align Activities and Assessments,” and recognized the name of one of the authors, Dr. Greg Crowther (Everett Community College, Everett, WA) from a previous association. I reached out to Greg and requested some more details about Test Question Templates (TQTs). What I found was a pedagogical gold mine!
The TQTs are based on somewhat general learning objectives, much like the four Big Ideas of the AP Biology exam. Students often ignore these learning objectives because they don’t know what they mean or how they will be assessed, but TQTs are formatted as input-output statements that tell the student exactly what they will be assessed on. Two examples (“Example A” and “Example B”) are provided for the students, followed by a prompt encouraging students to create their own test question following the template format.
The timing of my find was perfect for incorporating TQTs into the design of the new course. Since I am totally online, I took the time to video each TQT. On video, I present the input-output statement for each TQT and present Example A, along with approaches to answering the question or solving the problem. My TQT videos are attached to a weekly discussion board in the course management system, where students are then encouraged to work on solving Example B and creating a third example. I frequently visit the discussion board and provide feedback and guidance as needed throughout the week.
Below is an example of a TQT input-output statement and examples given to students ahead of the examination in the discussion board and used to model the examination question:
TQT 3.1. Given the chemical structure or chemical formula of an ion or molecule (chemical structure or text description), list the most likely mechanism(s) by which it crosses cell membranes.
- Example A: See structure below left. By which process(es) is this molecule most likely to cross cell membranes? Explain your reasoning. [add chemical structure of a molecule like urea]
- Example B: See structure below right. By which process(es) is this molecule most likely to cross cell membranes? Explain your reasoning. [add image of a peptide like insulin]
- Example C: Make up an example (think of an ion or molecule that you’ve heard of) and ask your classmates!
In the previous unit, students had been instructed on chemical structures/formulas and bonding properties. In this unit, students are asked to extend and apply their understanding of chemical structures, bonding properties (polar, nonpolar, ionic) with their new knowledge of cell membrane structure (phospholipid) and cell transport mechanisms (passive or active).
Examinations are carefully aligned with the objectives, formative assessments and exact input-output statements given to students in the TQTs. The examination contains 10-11 short answer questions and approximately 25-30 multiple choice questions. I have added a statement on the examination for students to sign, reminding them not to use any outside resources (people, notes, internet….) along with the consequences for doing so. Students are reminded to use what they are learning in the course to answer and solve exam problems/questions. I explain to students how I will know if they don’t follow the rules.
I will admit that the new course has gotten off to a rough start. For reasons I can only guess at, more than half my students are procrastinating until the last minute to start assignments (lecture, reading, lab, formative assessments, TQTs…). This approach is not consistent with my suggestions to space out their learning, practice, or repetition of concepts that we know is so important to learning and applying the information to new situations.
Not surprisingly, students who participated during the week and spaced-out lecture segments, formative assessments and TQTs did much better on the examination than those who did not. Those who chose alternative approaches to the course material often googled their way through the examination and failed miserably. Using Google, they could identify a molecule, how it is made, and where it is found, but they couldn’t answer the questions asked.
It has taken several examinations to convince many of the students that physiology is not simply about googling or memorizing facts, but about developing critical thinking skills and a higher-order understanding of the material that will persist beyond the course. More students are now actively preparing, studying and asking more complex questions throughout the week than previously (as evidenced by the course management system analytics and student contact). Many have shown improvement not only on their overall exam scores, but in their demonstrations of reasoning on assignments and exams.
After the initial rough weeks of getting students on board, students are now reaching out via email to report progress in their learning, growth, and ability to connect the material to their work as CNAs and Medical Assistants. For example, one young man in the course writes, “As we’ve progressed onward to future chapters I feel like my knowledge is increasing gradually and I personally feel that like I CAN do this, it has been a struggle I’m not going to lie and say it was a breeze but, I feel like I’m truly getting a ton of knowledge from these chapters, I’ve found much interest on the systems we’ve been studying especially with the TQT examples and formative questions that you help me with your feedback.” Another young lady states, “I am sorry I am not doing well. I have never been forced to study before and though the TQTs are hard I am finding that I am learning a lot and am really interested in learning more. I am glad I didn’t give up.”
In summary, both the AP Biology redesign assessment questions and the TQTs have allowed me to better assess my students’ knowledge and skills. These approaches have also given me insight into student misconceptions and helped me provide feedback, remediation, and other support as needed. I can easily write (or rewrite) questions based on the TQT input/output statements without having to rewrite entire examinations each term. Students are learning that simply googling will not let them ace the exams; instead, they are learning to more carefully read the questions and answer the questions based on their own understanding.
“ACKNOWLEDGMENTS: The author thanks Greg Crowther for help implementing TQTs and for feedback on this blog post.”
References:
- AP Higher Education (2012-2013). AP Course and Exam Redesign. https://aphighered.collegeboard.org/courses-exams/course-exam-redesign
- Crowther, G., Wiggins, B., Jenkins, L. HAPS Educator (May 2020). “Testing in the Age of Active Learning: Test Question Templates Help to Align Activities and Assessments.”