Monday, October 24, 2011

Mistyping People: placement test error and worse

No placement test -- a test which places subjects in a category -- is 100% accurate. It will generate a number of "false positives," persons who don’t meet the specs, but nonetheless test positive. False positives will be practically indistinguishable from true positives.

Can we trust such tests? For example, suppose a student has been test-identified as a drug user, how likely is it that that student is truly a user? Or, if a placement test indicates that a student is ready for instruction at a third grade, is that student really ready to begin at that starting point? Do tests purporting to show that students have learned beginning calculus actually let some through who really need to learn more?

Our normal admission processes to public school are haphazard, usually nothing more than checking the child’s age. This process will allow in quite a few "false positives;" that is, students who at the point of admission appear no less capable -- they walk, they talk, they can fog a mirror -- than the students who possess the skills to succeed at that level. And a flood of incompetence commences.

Progressing upward from grade to grade in this manner generates the flood of inept "false positives" which now fills our colleges -- as suggested by the epidemic dimensions of cheating and plagiarism throughout high school and well beyond.

To examine these issues further, see Classification Error in Evaluation Practice: 
the impact of the "false positive" on educational practice and policy

Cordially,
-- EGR