Confessions of a converted lecturer

I watched "Confessions of a converted lecturer", a special talk on science education by Harvard physics professor Eric Mazur. The talk started a little slow, but became very interesting at the half hour mark. Well-worth watching. Here are my notes from the talk.

Prof. Mazur taught the introductory physics course at Harvard. It was mostly aimed at pre-med students, but unlike most pre-med physics classes, it was calculus based. The first year he taught the course, he went to a colleague who had taught it previously and asked him what textbook to use. The colleague said, "Halliday and Resnick", so Mazur assigned that as the text. But now that there was a textbook, what would he say in his lectures? The colleague had mentioned some other books and one of them was out-of-print. So Mazur thought, "Perfect! That's the book I'll use." Mazur would then spend 10 hours preparing each lecture out of this out-of-print book. When he first stated teaching, he thought he was a great teacher because the students did well on the tests and because he got good evaluations. Mazur provided photocopies of his lecture notes for the students and consequently, a few students complained that "Prof. Mazur lectures out of his lecture notes!"

This made Mazur realize that lectures focus on delivery of information. This was certainly important before Gutenberg invented the printing press, because lecturing was the only way to transmit information to the next generation. But now with books and internet, teachers should focus on assimilating information. Education is not just about information transfer. Students need to build mental models. After all, in a Shakespeare class, you don't have the instructor reading Twelth Night to the class. The students are expected to read the play before coming to class.

Meanwhile, Mazur came across a study on the effective of physics teaching. The study gave a so-called "force concept inventory" (FCI) test to the students before and after the course. An example question is: "A light truck and a heavy truck collide. How do the forces they exert on each other compare?" Possible answers are "A) The light truck exerts more force than the heavy truck, B) The heavy truck exerts more force than the light truck, C) The forces are the same, D) There are no forces. The trucks are just in each other's way." The correct answer is (C), using Newton's Third Law.

The student data was divided into four groups: students who had award-winning teachers, students in small instruction groups, students who had experienced a lot of hands-on demonstration in their class, and students who had teachers with the worst evaluations. There was little difference between the FCI pre-test and post-test scores. In fact, it made no difference what group the students had been in! All the students failed to learn basic physics concepts.

Mazur thought, "Well, I teach at Harvard" and wanted to prove the study's results wrong. He gave the FCI to his students at the beginning of his course and found that most of his students scored below 23/29. A score below 23 indicates Aristolian thinking and a failure to grasp Newtonian concepts. Then he gave the test at the end of the course. There was a little improvement but still over half the class scored below 23. Many of these students had scored 5's on the AP Physics exam.

You can define a statistical "gain" to characterize the student's improvement due to the course. If you plot Harvard's results along with data from other colleges, you find a gain of 23% which is much lower than the maximum possible gain of 100%.


Mazur considered various possibilities. Bad teacher? NO. Dumb students? NO. Blame the test! So he wrote his own test. He designed his exam so that each "conventional" question would be paired with a "conceptual" question. The conventional question was one you would find in a traditional textbook. The conceptual question was word based with no numbers or algebra. Moreover, he chose to do this on a topic that students had very little pre-existing intuition: DC circuits. For Newtonian mechanics, students might have real world experience that interfered with their reasoning. This could be avoided by testing them on DC circuits. An example paired conventional and conceptual question are shown below.

Conventional question


Conceptual question


Mazur found that the conventional questions gave misleading impressions of the students's performance. He had expected that conceptual question to be really easy, compared to the conventional question which requires cranking out a page of algebra. Any physicist can solve the conceptual problem in 30 seconds, with 5 seconds on parts (a)-(d) and 25 seconds on part (e). But the students freaked out on the conceptual question. The average score on the conventional question was 6.9/10. The average score on the conceptual question was 4.9/10, with a huge peak at 2/10. The students had been tripped up by the following fallacy. They thought that when the switch was closed, the current would be divided equally between the two paths (the wire with the short circuit and the wire with the light bulb). Therefore, they had all gotten part (b) correct, which is what resulted in the huge peak at 2/10. Mazur examined the correlation between the conventional question score and the conceptual question score. Students who did well on the conceptual question tended to do well on the conventional question. However, the converse was not true. Students who did well on conventional question did not necessarily do well on the conceptual question.


He realized that students were simply learning by "plug and chug", recipes, and memorization. No wonder people often complain that physics is boring! Physics is boring because students apply recipes they don't understand to solve problems. The way students try to solve problems is to look at the "problem solving strategies" box in the textbook. If that doesn't work, they find a worked problem and try to substitute the numbers. There is no real understanding of the concepts. Unfortunately, in physics, recipes only work sometimes. If a recipe only works for 75% of the textbook problems and not for the other 25%, then students get frustrated.

Imagine that you are in a room of 150 students taking an exam. As you turn the page to the first problem, you see the conventional question with the circuit diagram we just discussed. what is your first thought? Kirchoff's Law problem! So, in a split second, you have already determined the appropriate recipe to use and there is no physics left. It's become an exercise in algebra. But do students actually understand Kirchoff's Law? Clearly not, because they did poorly on the conceptual question. [My comment: Physics has turned into zoology! We just classify problems by which recipe applies to them!]

Mazur was at a loss. But then he remembered what happened after he gave the class the FCI. The students were appalled by their poor performance and wanted Mazur to go through the test question by question. He didn't have time to do that in class, so he reserved a lecture hall at night for discussion. Let's go back to the question about the light truck and the heavy truck. Mazur's first explanation was "use Newton's Third Law, done." The students looked utterly confused. So he tried again. The second explanation was "a = F/m, so even though the forces are the same, the heavy truck experiences less acceleration and feels less from the impact." Still, blank looks. Then Mazur gave up and told the class to discuss the problem with their neighbors. Chaos ensued.

He realized that students have a much better idea of what difficulties their classmates are having. The instructors learned the material so long ago, that they have forgotten these difficulties. Education is a two step process: 1) information transfer, 2) assimilation of the information. Therefore, we should give students more responsibility for gathering information.

Mazur now uses a peer instruction style teaching method. Peer instruction includes 1) pre-class reading, 2) in-class instruction focusing on depth, not coverage, and 3) ConcepTests. Each ConcepTest follows the sequence: 1) Question, 2) Thinking, 3) Individual answer, 4) Peer discussion, 5) Revised/group answer, 6) Explanation. In practice, this is how it works. The lecturer will put the question on the projector and ask the class to think about it individually. Then each student has a "clicker" and electronically selects an answer from multiple choice. After that, the students pair up and try to convince their partner that their answer is correct. Finally, the students answer the question, possibly changing their answer after the discussion. Since the instructor can see the distribution of the answers, he/she gets feedback about the class's understanding and can adjust the instruction accordingly. Thus, instead of a traditional lecture, the class becomes ConcepTests interspersed with snippets of demonstrations and the lecturer talking.

The benefits of peer instruction are that 1) there is little time to goof off, 2) there is two-way information transfer. The instructor can assess student understanding and the students can assess their own understanding without any impact on their grade.

Other colleges besides Harvard have implemented peer instruction and the results are astounding. The overall gain is 0.48 for peer instruction compared to 0.23 for traditional lecturing. Moreover, the gains are not instructor dependent.


What about problem solving? Mazur decided to stop doing example problems in class. He felt like he was simply writing the textbook on the board. Think of the following analogy. You don't learn the piano by listening to CDs.

Previously, Mazur had said that better conceptual understanding leads to better problem solving, but the converse is not true. He showed data proving this point. Since he never repeated any questions on exams, he decided it would be safe to give the exam from his traditional 1985 class to his peer-instructed 1991 class. The 1991 students did statistically better on the test than the 1985 class.

In retrospect, Mazur thinks that courses should be defined not by content, but by learning outcomes. For example, for a course on mechanics, a learning outcome could be "understanding Newton's three laws." Unfortunately, Mazur has surveyed his colleagues, and there is no consensus on learning outcome.

The audience had many questions about the practical implementation of peer instruction. Mazur said that preparing the course still takes the time, but instead of making lectures, all the time is put into writing good ConcepTests. You can develop good questions by 1) looking at mistakes on exams and 2) asking for feedback about what was confusing on the reading. For each reading assignment, Mazur has students answer two questions on the material and give feedback on what was confusing. This accounts for 10% of their overall grade.

Also, Mazur has found that it is better to reduce coverage of material. The traditional lecture course covered way too much. The peer instruction course covers less material, but the gain in understanding far outweighs the small loss of coverage.

He also tries to motivate the students by writing exams with conceptual questions. In fact, the first exam is completely conceptual. As he puts it, "it's amazing how students are driven by the exams." He also makes all the exams open-book. After all, in reality, people don't memorize. They look up information while they are solving problems.

Someone asked about labs. Mazur said that because of overcrowding, his students took lab biweekly. One half would do it one week and then the other half would do it the next week. He once gave an exam involving circuits and it turned out that because scheduling, half the students had done the circuits lab before the exam, and the other half were going to do the lab afterwards. The lab made no difference on the exam performance. In fact, the students who had done the lab did a little worse than the ones who hadn't. This just goes to show how cookie-cutter and badly-written the labs are. Mazur has just now started on the huge project of revising the labs.

The audience pointed out that students and colleagues will resist peer instruction. Students have two main complaints. First, there are the students who did well on conventional problems (5's on AP Physics exam), but do poorly on the ConcepTests. They try to blame the instructor and ask "when will we do real physics?" Second, there are students who write on the course evaluation, "Prof. Mazur doesn't teach us anything! We have to learn everything ourselves!" They are mad because they are paying $47,000 a year to be taught by their peers. Mazur tries to convince them by showing them the data and telling them anecdotes. As for colleagues, Mazur said that schools outside Harvard have implemented peer instruction and when his colleagues go to visit those places, they encounter peer instruction and warm up to it. As Mazur put it, "change comes from outside." Mazur had very little influence on "evangelizing" his colleagues about peer instruction. The point is that students and instructors have very deep rooted conceptions about how they learn.

To summarize, traditional indicators of success (class evaluations, exam scores) are very misleading. Education is no longer about information transfer; it's about how to use information.

My comments: I think the success of peer instruction is simply an illustration of ratios. Physics is a very difficult subject. In an ideal world, everyone would have an amazing tutor who would teach them one-on-one. But that is far too expensive, so we have these huge lectures in college. Peer instruction leverages students as teachers.

If I were the "smart" student in the class, I would object to peer instruction. Why should I come to class and have to teach my classmates while not learning anything from them?

The truth is that college is about learning how to teach themselves. What an instructor can do is show you how to approach the problem, what are the right questions to ask. In learning, just as in research, asking the right questions is 75% of the battle. In a sense, peer instruction helps students learn how to teach themselves, because they are forced to teach someone else and explain their reasoning. It's just like what all teachers say; they understood the material much better after teaching it.

I think one thing that Mazur fails to mention is that we still need teachers to inspire students. Difficult abstract subjects like math and physics benefit the most from an inspiring instructor.