Is It Really a Test? Or Just Another Task?

revised 1/25/19

Per aspera ad astra. Through difficulties, to the stars.

From a test-taker's point of view every test is a task. But not every task is actually a test, even if it looks like one. What is it that makes a task a test? This is a question of great practicality. State governments allocate funds to school districts on the basis of efficiency. This efficiency is determined by tasks-activities provided by state departments of education and imposed on local school districts. But what is necessary if this procedure, this task-activity, is to be anything more than a charade?


To avoid overlooking assumptions built into a widely accepted notion of testing, let's use a substitute concept: rank-task. We begin our investigation by talking about "rank-tasks" rather than about "tests." A rank-task is a type of activity some outcomes of which can be ranked, as, for example, better, the same, or worse. Think of a rank-task as any procedure which assigns someone a number. This can be interpreted as a rank to compare that person to others involved with the procedure.

Cinderella's Prince, looking to fit the glass slipper, would be undertaking such a rank-task. Some feet are too small; others, too large; only Cinderella's, just right. But trying to sort football players by the numbers on their jerseys is not a rank-task because there is generally no significance to the comparison of any two numbers other than that they indicate a different wearer.

Tests are, at the minimum, rank-tasks. They can be performed with more or less skill. But the skill demonstrated may not be what it is we wish to measure. For instance, Students take SAT preparation courses to learn test-taking skills, not the information the tests are designed to measure. Often these test-taking skills can be as crucial to a good score as actually knowing the material covered in the test.

For example, The Princeton Review has for many years provided materials and training in simple test taking procedures which seem to be able to raise SAT scores significantly. The SAT's are intended to measure scholastic aptitude. But the effectiveness of the Princeton Review's materials suggests that the SAT's are also measuring something else, namely the ability to take standardized tests of this type. This observation illustrates the very practical nature of our seemingly theoretical observations about testing. (See "Rationales for Intervention: From Test to Treatment to Policy: a forensic theory of warrants & rebuttals")

Among the readers of this article are certain to be individuals who did not get a scholarship, or who failed to get in to the college or university of their choice because of the score they received on the SAT. And there is a fair chance that the reason they did not get a higher score was not because they lacked scholastic aptitude, but because they lacked certain test-taking skills.


To consider other things that make a rank-task a test, see Justice Through Testing


Cordially
--- EGR

The University as Rumpelstiltskin

And when the girl was brought to the King he took her into a room which was quite full of straw, gave her a spinning-wheel and a reel, and said, "Now set to work, and if by to-morrow morning early you have not spun this straw into gold during the night, you must die." -- Rumplestiltskin, Brothers Grimm.

David Leonhardt in “Colleges Are Failing in Graduation Rates” (New York Times of September 9, 2009 , B1) tells us that universities are failing in their “core mission”: to turn teenagers into college graduates, (straw into gold?) The evidence? Highly selective, usually private, universities have a higher graduation rate than open-admissions or non-competitive, usually public universities. As a result, Leonhardt claims, social inequality has “soared” and productivity growth has slowed down.

But is “turning teenagers into college graduates” really “the core mission" of universities? And has social inequality increased since the beginning of the 1900’s when very, very few of middle and working class students were able to get into college?

A third question: has economic productivity over these last 110 years fallen also?

Answers: no, no and no.

Leonhardt based his column on an article by Mark Schneider, a political scientist who is a tad more circumspect about making broad claims. (See Schneider,M. The costs of failure factories in American higher education. Educational Outlook. No. 6. October 2008. American Enterprise Institute for Public Policy Research)

Although Schneider’s title contains the rock’em-sock’em hype, “failure factories” to characterize public universities, he hedges his bets: what he actually says is, if we worry about students not graduating from high school and attempt to coerce high schools with low graduation rates into doing something about it, then why ignore the public university “failure factories” whose graduation rates are significantly lower than our public high schools?

Why, indeed? But why hold high schools, with their compulsory attendees, more responsible for outcomes than are public universities? Those that attend public universities are not forced there by law.

To examine these issues further, see Moral Responsibility in the Education Industry: 
how much can school reform enhance a student's occupational fitness?

Cordially
EGR

Measuring Educational Outcomes: truth, tricks and hype

It sometimes seems that the only real result of math education is to convince its worst students – and a substantial number of us mediocre ones – that numbers are magic. “Anything can be measured,” goes the myth; and, once a number is associated with something, it becomes much more real, much more defined. All problems are soluble once they have become mathematized – even TV pushes this philosophy on the dramatic series, Numbers.

This hocus-pocus has some startingly impressive roots. Psychologist Edward L. Thorndike – unskilled in basic algebra -- wrote: "Whatever exists at all exists in some amount. To know it thoroughly involves knowing its quantity as well as its quality." Heavy philosophy!

But what is the quantity of a chess game? Or of the relation "<>"? Do chess games or "<>" not exist? Do chess masters wait with bated breath for someone to mathematize a chess game so that they can “know it thoroughly?”

The harm done with this exaggeration is that it creates the expectation that when and only when a technical, measurable solution to a problem can be devised, can that problem be solved.

So it is that problems must be left to mathematical geniuses, since their solution is beyond the ken of most people who can only stumble through math counting on their fingers.

To examine these issues further, see Measurability and Educational Concerns

Cordially

---EGR