Wednesday, April 3, 2019

The Curse of Knowledge vs The Dunning--Kruger Effect: an instructor's dilemma

updated 3/15/20
Damned if you do, damned if you don't. -- George of Trebizond (c. 1433)

Laurence J. Peter: “Look around you where you work, and pick out the people who have reached their level of incompetence. You will see that in every hierarchy the cream rises until it sours.” -- Laurence J. Peter The Peter Principle 1970

Too often, our confidence that we know what is going on is greater at the beginning of an episode, than it is at the end. -- Sloman S & Fernbach P (2017) The Knowledge Illusion p.19

Introduction. In 1960, in the early years of the Cold War, my freshman year, I was placed in ("invited to") a course called "Honors Physics." Three days a week at 8 AM our professor, who, we were told was a renowned researcher, would come in and fill six boards with notes. Except for the two class "geniuses" who, having built a cyclotron in high school, just sat there occasionally nodding their heads, we spent all our time furiously copying. The prof, an apparently shy person, entertained no questions and was not accessible outside of class.


Help understanding the prof's notes was to be provided in small-group seminars by graduate physics assistants. We, the freshmen -- geniuses excepted and usually absented -- often couldn't understand what the prof was getting at. More than occasionally, a graduate assistant would confess the same problem. The seminars soon evolved into deciphering sessions.

Of course, we worried about our grades, since nobody -- geniuses excepted -- ever made above a 50 on the quizzes. (The geniuses regularly got 100's.) On the final, the geniuses were excused by the prof from taking the exam. The rest of us were graded, we were told, by the prof. I got a thirty-six, which seemed to be about average, so far as an informal consensus determined. No matter, the class average was a 34. So I was awarded a B. After all, we surmised, how could a class of "specially invited honors students" be flunked en masse?


Caught Between The Curse of Knowledge (COK) and The Dunning-Kruger Effect (DKE). Many of us have had a teacher or professor who, we felt, consistently "talked over our heads." This was mostly, I would hope, not done intentionally. What usually happens is that the teacher overestimates how much their students know. This happens often, if not more, as you go up the educational ladder (where it is more embarrassing to admit to ignorance). The professor exhibits this psychological bias, the Curse of Knowledge, by not taking into consideration the possibility of class disparities in prior knowledge. (College entrance exams are presumed, I suspect, to have precluded such ignorance down, even, to course-level specifics.)

Students often exhibit a bias complementary to the COK, the DKE. This is a tendency to overestimate the knowledge one has acquired from minor acquaintance with a topic or area of concern. It seems that some people, even in high places, believe that, from TV, or conversation, or reading fiction, or newspaper reports, they know as well, if not better, as anybody, -- excepting, "nerds" and "wonks" and "so-called" experts -- about, say, law, espionage, economics, politics, and diplomacy, etc.

Countering the Biases. The instructor can counter both biases to some extent by giving pretests that don't affect the students' grades. The procedure is
a. create (an often difficult procedure) and administer the pretest (See, e.g., Your Image of University Life: a course prequiz );
b. have students indicate what they think their grade will be (they can write it on the pretest;
c. grade the test and let the students see their graded papers.
d. tell them about the Dunning-Kruger Effect.

I have used this technique for years for students through high school up through doctoral education. For upper-level research writing, make the students' grades partially dependent upon their writing in critical "editors-subgroups" reviews of each other's papers.(See Examples of Papers and Critiques.)

Institutional Resistances to Trying to Counter COK and DKE biases. Sometimes students react to pre-testing by dropping out of the course. I had one such student - in a course focussed on policy evaluation -- tell me that he thought his experience as a highly trained laboratory technician would get him through "the BS of the soft-science stuff."

I had a dean worry to me that letting students on to how much they would have to learn might reduce the number of applicants in the long run.

I talked with a professor -- who really worked at accommodating his students' personal weaknesses -- who told me how his college was going to relax admissions requirements so as to increase tuition revenues. He didn't think he could deal with both increased numbers of students whose literacy and math levels were much below the already feeble standards that were traditional.

At another university of my long acquaintance, professorial folklore has it that new faculty who win recognition for excellent teaching take it to be an omen of future failure to get tenure. Trustee opinion: "If a professor spends so much time trying to engage students well, how can he or she be participating in committees and doing good research besides?"

An interesting situation is when a governor of a university exhibits DKE with respect to how educational processes proceed. At a meeting of faculty and university trustees, I heard the following comment from a trustee, "I hear that there are members of the faculty who have published books this last year. That's really good! Very productive! What I want now is to see you all publish two books next year."

To pursue these issues, see What is Worth Knowing? A Philosophical Distraction from a Problem in Leadership


Cordially, EGR