Thursday, November 7, 2013

Is It Really a Test? Or Just Another Task?

revised 1/25/19
Per aspera ad astra. Through difficulties, to the stars.

From a test-taker's point of view every test is a task. But not every task is actually a test, even if it looks like one. What is it that makes a task a test? This is a question of great practicality. State governments allocate funds to school districts on the basis of efficiency. This efficiency is determined by tasks-activities provided by state departments of education and imposed on local school districts. But what is necessary if this procedure, this task-activity, is to be anything more than a charade?

To avoid overlooking assumptions built into a widely accepted notion of testing, let's use a substitute concept: rank-task. We begin our investigation by talking about "rank-tasks" rather than about "tests." A rank-task is a type of activity some outcomes of which can be ranked, as, for example, better, the same, or worse. Think of a rank-task as any procedure which assigns someone a number. This can be interpreted as a rank to compare that person to others involved with the procedure.

Cinderella's Prince, looking to fit the glass slipper, would be undertaking such a rank-task. Some feet are too small; others, too large; only Cinderella's, just right. But trying to sort football players by the numbers on their jerseys is not a rank-task because there is generally no significance to the comparison of any two numbers other than that they indicate a different wearer.

Tests are, at the minimum, rank-tasks. They can be performed with more or less skill. But the skill demonstrated may not be what it is we wish to measure. For instance, Students take SAT preparation courses to learn test-taking skills, not the information the tests are designed to measure. Often these test-taking skills can be as crucial to a good score as actually knowing the material covered in the test.

For example, The Princeton Review has for many years provided materials and training in simple test taking procedures which seem to be able to raise SAT scores significantly. The SAT's are intended to measure scholastic aptitude. But the effectiveness of the Princeton Review's materials suggests that the SAT's are also measuring something else, namely the ability to take standardized tests of this type. This observation illustrates the very practical nature of our seemingly theoretical observations about testing. (See "Rationales for Intervention: From Test to Treatment to Policy: a forensic theory of warrants & rebuttals")

Among the readers of this article are certain to be individuals who did not get a scholarship, or who failed to get in to the college or university of their choice because of the score they received on the SAT. And there is a fair chance that the reason they did not get a higher score was not because they lacked scholastic aptitude, but because they lacked certain test-taking skills.

To consider other things that make a rank-task a test, see Justice Through Testing

--- EGR