Fragmented Testing

A couple things make standardized testing difficult in any given educational setting:

  • pressure to pass / excel
  • expensive exam-taking and scoring process
  • valid test construction
  • contested utility

If the pressure leads to undue stress among students or teacher-aided cheating, or if the time lost to test-taking takes too much from the school schedule, or if test writers cannot crank out valid tests fast enough, or if one day of testing simply does not lead to gains, then standardized testing fails to achieve its desired effect on student learning.

An alternative test environment may remove these obstacles. Imagine a fragmented exam schedule instead of the massive 4 hour exams currently administered to students. Periodically throughout the academic year, students would respond to targeted questions benchmarking progress in well-defined fields of study. These questions can be administered via computer on both a scheduled and a random basis; depending on the student’s individual achievement and time passed between questions, the administrator can choose more conceptually complex questions. Following either a minimum number of correct questions or a maximum amount of time spent producing an answer, the administrator can decide whether to advance a student or to begin remedial intervention.

A fragmented standardized exam of this sort may be incentivized through small rewards for each correct answer, say, $1. If a student answers one correct question every school day, they could walk out the last day with $180 in their pocket and a good cache of knowledge – plus the prospect of earning / learning more. Allowing up to ten questions per day, kids could walk away with a small scholarship.

Adding Incentives to Test Scores

From Education Week:

To explore what might happen if students had a little incentive to try harder, a trio of researchers focused on a sample of 2,600 students from 59 schools in seven states who were taking NAEP tests in reading. Within each school, the students were randomly assigned to one of three test-taking conditions. Under the first condition, the seniors were paid $20 at the start of the test-taking session. Another group was offered $5 in advance and $30 at the end of the session if they correctly answered two randomly chosen questions on the test. The control group received no special incentives.

The results of the experiment were posted today in the online version of Teachers College Record. The authors are Boston College’s Henry Braun and Irwin Kirsch and Kentaro Yamamoto of the Educational Testing Service.

Students who were assigned to the second group, earning the $30 bonus at the successful completion of the test, did somewhat better than the first group. Both paid groups outperformed the control group, who received no incentives. These results should surprise nobody. People perform tasks better given tangible rewards. The prospect of $35 outweighs the prospect of “opportunity” simply because money is tangible compared to the more abstract concept of opportunity. But here’s the question: ought school districts pay students to take standardized exams?

To answer this, let’s think back to the discussion over the cost of education. Public school students experience around $7 in subsidized instruction per hour in the classroom, whereas high-end private school students experience around $24 in purchased instruction per hour in the classroom. If public school students want to compete for spots in the Ivies with prep-school students, they need to make their education about three times more valuable (in crudely quantified terms). Students from low-income families have few choices – they either need to be very bright and very determined, or they need to sacrifice a large amount of resources to invest in extra educational opportunities. More often than not, we only see the very bright and very determined students gain the most from the public education system.

But what if state and local governments were to reward high performing students instead of subsidizing their education? Let’s say a standardized exam has 150 questions. At $30 per correct question, it could cost only $4500 to pay a kid to ace the test. This is less than half the $10000 it costs to subsidize that same kid’s education over one school year – an approach which we know leads many to failure.

You might say that replacing school funding with test rewards amounts to sabotage. How can the kids pass the test if nobody teaches them? My answer to that is: those who truly want to earn that $4500 will seek out an education. When that happens, you’ve created an open market, lowering costs, boosting innovation, increasing productivity, and posting gains in educating youth.

Unfortunately, I don’t see this transformation happening soon if at all.

Definitions: “Incentives” v “Accountability”

Incentives give individuals reasons to act; usually, people act on certain information that signals a positive contribution to one’s own wellness. An incentive might look like this: If I do X, I will gain Y and Y makes me happy.

Accountability enables the collective to sanction individuals for undesired action. In a way, accountability serves as an inverted incentive: you know a failure to perform certain actions will result in a negative contribution to your wellness, so you try harder to perform accordingly. An accountability measure may look like this: If I fail to do X, I will lose Y and Y makes me happy.

Both incentives and accountability can influence the same set of actions. For instance:

X  ->  Y     Johnny works for a living, Johnny gets paid well.

-X -> -Y    Johnny does not work for a living, Johnny does not get paid well.

Notice the subtle difference. Incentives presuppose that individuals do not already have Y and accountability presupposes that individuals do already have Y. (By “presuppose,” I mean that the conditional holds by strength of the consequent term. We don’t care about the cases in which someone doesn’t act on the incentive or doesn’t care about being held accountable.) Y makes the individual happy; but, insofar as the individual has differing interests in increasing her happiness as opposed to avoiding a decrease in her happiness, she will respond differently to incentives than to accountability measures.

For instance: you might act on my offer to give you a free ounce of milk for each cookie you buy from me, but you might not respond if I threaten to take from you an ounce milk every time you don’t buy a cookie.

So be careful when – in arguing questions of policy – you bring up a point about providing incentives when the issue is over accountability, and visa versa. You may have confused your terms.