Monday, October 24, 2011

Mistyping People: placement test error and worse

revised 1/17/19

No placement test -- a test which places subjects in a category -- is 100% accurate. It will generate a number of "false positives," persons who don’t meet the specs, but nonetheless test positive. False positives will be practically indistinguishable from true positives.


Can we trust such tests? For example, suppose a student has been test-identified as a drug user, how likely is it that that student is truly a user? Or, if a placement test indicates that a student is ready for instruction at a third grade, is that student really ready to begin at that starting point? Do tests purporting to show that students have learned beginning calculus actually let some through who really need to learn more? (See Identifying the "At Risk" Student. What is the Concern? )

Our normal admission processes to many schools are "standardized" and haphazard -- particularly in public schools. Entrance criteria are usually nothing more than checking the child’s age and appearance of "normality" -- and, in many a private schools, parent ability to pay the tuition.

This process will allow in quite a few "false positives;" that is, students who at the point of admission appear no less capable -- they walk, they talk, they can fog a mirror -- than the students who possess the skills to succeed at that level. And, semester after semester, a cascade of incompetence commences.

Progressing upward from grade to grade in this manner likely generates the flood of inept "false positives" which now fills our colleges -- as suggested by the epidemic dimensions of cheating and plagiarism throughout high school and well beyond.

To examine these issues further, see Classification Error in Evaluation Practice: 
the impact of the "false positive" on educational practice and policy

Cordially,
-- EGR