Thomas M. Nosek, Ph.D.
Professor Emeritus, Department of Physiology and Biophysics
Case Western Reserve University
In this final article of the series, the discussion comes to a completion with the remaining aspects that show the results of the tools that were presented in part I and the implementation techniques in pat II.
17. Administer weekly quizzes.
We administer a 10 multiple choice question, computer-based quiz to both resident and Internet students (using the testing function in the CMS) every Monday before class that covers the material from the previous week’s lectures. Grades on these quizzes constitute 15% of the final grade in the course. The intent of the quizzes and their 15% contribution to the final course grade is to encourage students to keep up with the course material.
Advantages: The courses move along very quickly and if a student gets behind studying the material, they are at a severe disadvantage – there typically is not enough time for the students to catch up. Once a student’s answers are submitted, their grade is immediately reported so the students have immediate feedback on their performance.
Disadvantages: Faculty must write these quiz questions and time must be taken to administer the quizzes. Staff must be available to help students who have problems with the software on the day of the quiz. It is inevitable that a few students will have problems with their computers. Thus, the department has purchased 10 computers dedicated for use by students having difficulty during the quiz/exam administration. A room or group of rooms with Internet service and power sources for computer chargers must be available for the administration of quizzes and exams.
18. Provide weekly Study (Homework) Questions.
At the beginning of each week, we provide approximately 10 multiple choice study questions for each chapter in the assigned textbook that is covered that week. Students are encouraged to work together on these questions; they are free to use the textbook, class notes, and any other learning resource. However, the students are on their honor to enter their own answers to the question in the testing function of the CMS by noon each Saturday. Immediately after the due date for the questions, the answers with detailed explanations are provided to the students. Grades on these Study Questions constitute 15% of the final grade in the course.
Advantages: We have found that students who read the textbook do better than students who do not. These study questions encourage students to read the textbook. They key the students in on the most important points in each chapter helping them prepare for quizzes and exams. This learning resource has been rated very highly by the students.
Disadvantages: These study questions are created each year by the TA’s. Unfortunately, the TA’s are not always experienced in writing questions and many questions must be corrected after they are posted. It is best if these questions are written by faculty or at least reviewed by them before the questions are released to the students.
19. Administer computer-based Block examinations.
The textbook we use is organized by organ systems. The course is divided into Blocks by these organ systems. A faculty member is assigned to coordinate each Block. This faculty member is responsible for soliciting quiz and Block exam questions from each of the faculty members lecturing during the Block. At the end of each Block, a computer-based multiple-choice examination is administered through “Exemplify/ExamSoft” which we purchase for each of the students. The Block exam average constitutes 70% of the final course grade. These exams are secure so that they can be used from year to year with only slight modifications to better word questions that are found to be confusing and to modify them to accommodate for different faculty teaching the material.
Advantages: Blocks are typically 4 weeks long and there are 4 Blocks/semester. This breaks up the semester into 4 parts and gives the students 4 opportunities/semester to demonstrate their knowledge of the material. The computer-based exam system gives them immediate feedback on their performance.
Disadvantages: As noted above for quizzes, staff must be available to help students who have problems with the exam software on the day of the exam. As noted above, it is inevitable that a few students will have problems with their computers. Thus, the department has purchased 10 computers dedicated for use by students having difficulty during the exam administration. A room or group of rooms with Internet service and power sources for computer chargers must be available for the administration of Block exams. For the Internet students who cannot come to Cleveland to take the Block exams, we administer the exams through ProctorU, a service paid for by the department. This service uses the CMS to administer the exam through their proprietary software where a proctor observes the students through their computer camera and reports any inappropriate activity.
20. Use a “Difficulty Factor” (DF) to adjust quiz and Block exam grades for difficulty.
It is very difficult to predict how well students will perform on multiple choice quizzes and Block exams. Our expectation is that the median score on each Block exam will be 85% with 100% – 85% = A, 84% – 70% = B, and anything less than 70=C (with a C considered a failing performance in graduate school). Until a track record can be established for an exam, performance on each exam will be used to calculate the DF = the difference between the median on an exam and 85%. For example, if the median on an exam is 80%, 5% points are added to each student’s grade to bring the class median to 85%. If the median on an exam is greater than 85%, the DF = 0. After 3 years of administering a secure exam, the DF is calculated from the average of the DF’s for the previous 3 years.
Advantages: This system of adjusting the Block exam grades has been effective in making the final class average in the course close to 85% and a distribution of grades approximately 50% A’s and 50% B’s with only a few C’s. The students see the grading system as non-competitive; i.e., they do not see themselves as being in competition with other students for a limited number of A’s. Although it has never happened, it is possible for all students to earn an A in the course.
Disadvantages: None
21. As soon as possible after quizzes and Block exams are completed, hold a review session to discuss the correct questions.
Review sessions are conducted by the TA’s immediately after the class that follows the administration of the Monday quizzes and immediately after the Block exams. In these review sessions, the right answers to each question are provided. Students are allowed to contest the answer to a question. All contested questions of substance are taken to the faculty member responsible for the quiz or Block exam for evaluation and possibly changing the right answer or accepting multiple right answers. When this happens, the question is immediately corrected for use on the next year’s quiz/exam.
Advantages: It is very useful for students to have immediate (or near immediate) feedback on their performance on quizzes and Block exams. In this way they can identify concepts that they have not mastered and correct their thinking before moving on to the next topic.
Disadvantages: Steps must be taken to keep quiz and Block exam questions secure during these review sessions.
22. Provide faculty advisors to the MSMP students.
Each of the MSMP students is assigned a faculty advisor from the primary and secondary departmental faculty. We limit the number of students assigned to each faculty member to no more than 10/year. The advisors are responsible for writing letters of recommendation for their advisees. Students are encouraged to meet personally with their advisor at the beginning of the program and then at least after each Block exam in the first year core courses. During the second year, they meet with their advisors as needed. Internet students are encouraged to contact their advisor via phone or Skype on the same schedule.
Advantages: Students have access to a faculty member who can advise them on how to best navigate the courses and the MSMP program. Based on the student’s unique situation, they can also advise them on which electives will be most helpful to them when they apply to a medical professional program. Personal meetings are important so that the advisor can write a personalized letter of recommendation.
Disadvantages: Advisors are not equally knowledgeable about the intricacies of getting into professional medical programs and don’t provide the same quality of advice. Students tend to seek advice from the faculty members who have the reputation of being the best advisors, even when they are not assigned to that advisor. It is difficult to recruit enough faculty to be advisors to keep the student/advisor ratio at 10/incoming class.
23. Provide peer advisors.
At the beginning of the academic year, each matriculating MSMP student is assigned a peer advisor from the second year MSMP students. They are encouraged to meet and have social activities together. Each peer advisor is given $10/advisee to defray the cost of a social activity; ex. pizza party, bowling party, etc.
Advantages: Students appreciate meeting second year MSMP students who have successfully completed the first year core courses. Especially at the beginning of the program, this contact helps the incoming students become acclimatized to CWRU, the MSMP program, and Cleveland.
Disadvantages: It is not always easy to recruit enough 2nd year MSMP students to keep the student/advisor ratio at approximately 5.
24. Provide Teaching Assistants who have already had the core courses.
At the end of the academic year, we interview students who have successfully completed their first year in the program and who want to be Teaching Assistants (TA’s). Typically, these students have a perfect 4.0 GPA and have had some TA experience during their undergraduate experiences. The TA’s conduct weekly review sessions for the core courses, hold office hours for personalized Question and Answer sessions, administer the quizzes and Block exams, and post the materials provided by faculty in the CMS. Three of the nine TA’s have been assigned to work exclusively with the Internet students, being sure to contact them at least once/week.
Advantages: The TA’s provide an invaluable service to the MSMP program, relieving the AA and faculty from having to deal with many routine administrative duties. This experience is very valuable to the TA’s, greatly enhancing their resumes. This position pays $15/hour for up to 20 hours/week of work thus giving the second year students some income. These TA positions have become very prestigious, especially when considering that every TA has gained admittance to the medical professional program of their choice after graduating with the MSMP degree.
Disadvantages: We hire 9 TA’s/year and this comes with a price tag that must be met by the income generated by the program.
25. Require students to complete a faculty/course/program evaluation at the end of each Block of the course.
At the end of each Block exam in the core courses and at the end of each elective, each student is required to complete a Block/faculty/program evaluation administered through SurveyMonkey. Student grades are not released until they complete all surveys. The Director of the MSMP program creates the template for this survey and the TA’s appropriately modify it and post it online for each Block of the courses.
Advantages: This feedback is provided to the Block coordinators, the MSMP Administration Committee members, the faculty teaching in the Block, and the TA’s. This feedback is coordinated by the AA. The Block coordinator and the members of the MSMP Administration Committee see the full evaluations. The faculty and TA’s only see their own evaluations. This information is very important for faculty and TA’s to know how their interactions are valued by the students and by the Block coordinators and MSMP Administration Committee members to evaluate the courses and program and take steps to continuously improve it.
Disadvantages: The AA must take time to keep track of students who have completed the evaluations and explain to those students who have not completed them why their course grade has not been posted.
Dr. Nosek earned his B.S. in Physics from the University of Notre Dame in 1969 and his Ph.D. in Biophysics from The Ohio State University in 1973. After post-doctoral research in the Cardiovascular Physiology Training Program in the Department of Physiology and Pharmacology at the Bowman Gray School of Medicine of Wake Forest University, he went to the Department of Physiology at the Medical College of Georgia (1976-1997) where he was the Coordinator of the Muscle Cell Biology Research Group (conducting research on the cellular basis of muscle fatigue) and the Coordinator of the Computer Aided Instruction Research Group (editing and being a section author of “Essentials of Human Physiology: A Multimedia Resource” published by the DxR Group). He served as Director of the medical physiology course taught to first year medical students and was the Director of the Departments Ph.D. program. In 1997, he moved to Case Western Reserve University School of Medicine where he was Associate Dean of Biomedical Information Technologies (creating the Computer-Based Integrated Curriculum through 2006) and Professor of Physiology and Biophysics until he retired in 2014 becoming Professor Emeritus. He served as the department’s Director of Medical Education. He was founding Director of the MS in Medical Physiology Program at CWRU from 2010 – 2019 when he became Director Emeritus.