Fragmented Testing

A couple things make standardized testing difficult in any given educational setting:

  • pressure to pass / excel
  • expensive exam-taking and scoring process
  • valid test construction
  • contested utility

If the pressure leads to undue stress among students or teacher-aided cheating, or if the time lost to test-taking takes too much from the school schedule, or if test writers cannot crank out valid tests fast enough, or if one day of testing simply does not lead to gains, then standardized testing fails to achieve its desired effect on student learning.

An alternative test environment may remove these obstacles. Imagine a fragmented exam schedule instead of the massive 4 hour exams currently administered to students. Periodically throughout the academic year, students would respond to targeted questions benchmarking progress in well-defined fields of study. These questions can be administered via computer on both a scheduled and a random basis; depending on the student’s individual achievement and time passed between questions, the administrator can choose more conceptually complex questions. Following either a minimum number of correct questions or a maximum amount of time spent producing an answer, the administrator can decide whether to advance a student or to begin remedial intervention.

A fragmented standardized exam of this sort may be incentivized through small rewards for each correct answer, say, $1. If a student answers one correct question every school day, they could walk out the last day with $180 in their pocket and a good cache of knowledge – plus the prospect of earning / learning more. Allowing up to ten questions per day, kids could walk away with a small scholarship.

Adding Incentives to Test Scores

From Education Week:

To explore what might happen if students had a little incentive to try harder, a trio of researchers focused on a sample of 2,600 students from 59 schools in seven states who were taking NAEP tests in reading. Within each school, the students were randomly assigned to one of three test-taking conditions. Under the first condition, the seniors were paid $20 at the start of the test-taking session. Another group was offered $5 in advance and $30 at the end of the session if they correctly answered two randomly chosen questions on the test. The control group received no special incentives.

The results of the experiment were posted today in the online version of Teachers College Record. The authors are Boston College’s Henry Braun and Irwin Kirsch and Kentaro Yamamoto of the Educational Testing Service.

Students who were assigned to the second group, earning the $30 bonus at the successful completion of the test, did somewhat better than the first group. Both paid groups outperformed the control group, who received no incentives. These results should surprise nobody. People perform tasks better given tangible rewards. The prospect of $35 outweighs the prospect of “opportunity” simply because money is tangible compared to the more abstract concept of opportunity. But here’s the question: ought school districts pay students to take standardized exams?

To answer this, let’s think back to the discussion over the cost of education. Public school students experience around $7 in subsidized instruction per hour in the classroom, whereas high-end private school students experience around $24 in purchased instruction per hour in the classroom. If public school students want to compete for spots in the Ivies with prep-school students, they need to make their education about three times more valuable (in crudely quantified terms). Students from low-income families have few choices – they either need to be very bright and very determined, or they need to sacrifice a large amount of resources to invest in extra educational opportunities. More often than not, we only see the very bright and very determined students gain the most from the public education system.

But what if state and local governments were to reward high performing students instead of subsidizing their education? Let’s say a standardized exam has 150 questions. At $30 per correct question, it could cost only $4500 to pay a kid to ace the test. This is less than half the $10000 it costs to subsidize that same kid’s education over one school year – an approach which we know leads many to failure.

You might say that replacing school funding with test rewards amounts to sabotage. How can the kids pass the test if nobody teaches them? My answer to that is: those who truly want to earn that $4500 will seek out an education. When that happens, you’ve created an open market, lowering costs, boosting innovation, increasing productivity, and posting gains in educating youth.

Unfortunately, I don’t see this transformation happening soon if at all.

Test Less?

When it comes to assessment, the United States is an international outlier. As Stanford University’s Linda Darling-Hammond has shown, many nations with better and more equitable educational outcomes test far less than we do. They typically test just one to three times before high school graduation, and use multiple-choice questions sparingly, if at all. Excessive testing wastes educational resources and fosters the use of cheap, low-level tests, while adding high stakes narrows and dumbs down the curriculum. The results provide little instructional value to students, teachers, schools, or districts.

via Education Week: A Better Way to Assess Students and Evaluate Schools.

A great point, however, be careful when comparing across borders: a country’s institutional structures and civic attitudes can and do influence political outcomes. Compared to alternatives, testing may be the best option for American schools … but we have to look at alternatives first.

Standards

Sometimes some people confuse themselves over the terms “education standards” and “standardized testing.” (That’s to say, not everyone confuses themselves over the terms or everyone does not confuse themselves all the time — a logic pun which points out that the political situation in education is fraught with confusion from whichever angle you look at it.)

A set of statements and/or committee reports detailing a school district’s pedagogy constitute “education standards.” These statements can range from loose principles such as, “all students will demonstrate proficiency in reading, writing, and arithmetic,” to strict mandates like, “all students must score at least 1900 on the SAT.” State and district standards fall somewhere in between the vague and the draconian, usually, mixing the two with extraordinary bureaucratic flair.

A test scored on a normalized scale distributing test-takers into statistical groups comprises a “standardized test.” The “standard” here refers to the fact that, when designing and scoring the exam, the testing company tries to approximate the likelihood that any given student will achieve any given test score. That is to say, if you are no less and no more intelligent than other test-takers, and if you have received no test-specific preparation, you will most likely receive a normalized score that falls in the middle range of all test scores.

The standardized test is a complicated beast; good (also read: expensive) standardized tests (like the LSAT) carry higher statistical validity than less good (also read: cheap) exams (like a pop quiz). The stronger tests are actually a good indication of what a student know relative to other students.

This last part is important. If a district sets its education standard to standardized testing – and nothing else – then the only indication of learning progress will be progress relative to other students. Therein lies the source of much confusion; so long as good tests, tests which are statistically evaluated and updated following each issuing, provide the benchmark for success, there can be little indication of what students have learned other than what they have learned in relation to other students.

The bottom line? Standards-based reform, a kind of pedagogy that ties classroom learning to standardized testing, does not provide for a progressive learning experience. It does provide competitive learning. But in order for that to be any good, the competition needs to be just as good or better.