Fragmented Testing

A couple things make standardized testing difficult in any given educational setting:

  • pressure to pass / excel
  • expensive exam-taking and scoring process
  • valid test construction
  • contested utility

If the pressure leads to undue stress among students or teacher-aided cheating, or if the time lost to test-taking takes too much from the school schedule, or if test writers cannot crank out valid tests fast enough, or if one day of testing simply does not lead to gains, then standardized testing fails to achieve its desired effect on student learning.

An alternative test environment may remove these obstacles. Imagine a fragmented exam schedule instead of the massive 4 hour exams currently administered to students. Periodically throughout the academic year, students would respond to targeted questions benchmarking progress in well-defined fields of study. These questions can be administered via computer on both a scheduled and a random basis; depending on the student’s individual achievement and time passed between questions, the administrator can choose more conceptually complex questions. Following either a minimum number of correct questions or a maximum amount of time spent producing an answer, the administrator can decide whether to advance a student or to begin remedial intervention.

A fragmented standardized exam of this sort may be incentivized through small rewards for each correct answer, say, $1. If a student answers one correct question every school day, they could walk out the last day with $180 in their pocket and a good cache of knowledge – plus the prospect of earning / learning more. Allowing up to ten questions per day, kids could walk away with a small scholarship.

Selective and Magnet School Enrollment in Chicago Public Schools

The new enrollment process for coveted CPS classrooms.

Adding Incentives to Test Scores

From Education Week:

To explore what might happen if students had a little incentive to try harder, a trio of researchers focused on a sample of 2,600 students from 59 schools in seven states who were taking NAEP tests in reading. Within each school, the students were randomly assigned to one of three test-taking conditions. Under the first condition, the seniors were paid $20 at the start of the test-taking session. Another group was offered $5 in advance and $30 at the end of the session if they correctly answered two randomly chosen questions on the test. The control group received no special incentives.

The results of the experiment were posted today in the online version of Teachers College Record. The authors are Boston College’s Henry Braun and Irwin Kirsch and Kentaro Yamamoto of the Educational Testing Service.

Students who were assigned to the second group, earning the $30 bonus at the successful completion of the test, did somewhat better than the first group. Both paid groups outperformed the control group, who received no incentives. These results should surprise nobody. People perform tasks better given tangible rewards. The prospect of $35 outweighs the prospect of “opportunity” simply because money is tangible compared to the more abstract concept of opportunity. But here’s the question: ought school districts pay students to take standardized exams?

To answer this, let’s think back to the discussion over the cost of education. Public school students experience around $7 in subsidized instruction per hour in the classroom, whereas high-end private school students experience around $24 in purchased instruction per hour in the classroom. If public school students want to compete for spots in the Ivies with prep-school students, they need to make their education about three times more valuable (in crudely quantified terms). Students from low-income families have few choices – they either need to be very bright and very determined, or they need to sacrifice a large amount of resources to invest in extra educational opportunities. More often than not, we only see the very bright and very determined students gain the most from the public education system.

But what if state and local governments were to reward high performing students instead of subsidizing their education? Let’s say a standardized exam has 150 questions. At $30 per correct question, it could cost only $4500 to pay a kid to ace the test. This is less than half the $10000 it costs to subsidize that same kid’s education over one school year – an approach which we know leads many to failure.

You might say that replacing school funding with test rewards amounts to sabotage. How can the kids pass the test if nobody teaches them? My answer to that is: those who truly want to earn that $4500 will seek out an education. When that happens, you’ve created an open market, lowering costs, boosting innovation, increasing productivity, and posting gains in educating youth.

Unfortunately, I don’t see this transformation happening soon if at all.

University Tenure, RIP

From The Chronicle of Higher Education:

What does vanishing tenure mean for higher education? For starters, some observers say that college faculties are being filled with people who may be less willing to speak their minds: contingent instructors, usually working on short-term contracts. Indeed, the American Association of University Professors says instructors need tenure to guarantee that they can say controversial things inside and outside the classroom without being fired.

But others argue that the disappearance of tenure is actually not the worst thing that could happen in academe. The competition to secure a tenure-track job and then earn tenure has become so fierce in some disciplines that academe may actually be turning away highly qualified people who don’t want the hassle. A system without tenure, but one that still gave professors reasonable pay and job security, might draw that talent back.

Ultimately, though, the future of tenure may hinge on a different calculation: Does its absence hurt students enough in the classroom—something research has shown—that the cost savings to institutions are no longer worthwhile?

via Tenure, RIP – Labor & Work-Life Issues – The Chronicle of Higher Education.

(1) I’m not certain if there is any broad danger to free speech in academia. A few high profile cases exist, but they seem much more rare than we might speculate.

(2) I do believe competition will improve all levels of education in the United States. However, other institutional changes must accompany any major change in faculty hiring. Administrators must implement a stable, non-controversial method of evaluating teachers before attempting to reduce tenured faculty. Administrators must bite the bullet too; there ought to be an equivalent evaluation standard for non-instructional staff. Furthermore, schools need to implement broad incentives, making sure successes come with immediate and commensurate rewards.

(3) I don’t believe, prima facie, that reducing tenured faculty harms student achievement. In my personal experience, some of the best teachers I’ve had were not tenured and some of the worse still are. It should be easy to find many instances where a stable tenured faculty harmed student progress – in fact, we know many instances within urban school districts and less-than-Ivy-calibre colleges.

So should tenure die? I don’t think all tenured positions should vanish overnight, and I don’t think any tenured position in a college, primary, or secondary school should get the ax without a careful consideration of other institutional changes (a la (2) above). But neither do I consider tenure indispensable to the educational system.

How (not) to Assess Teacher Performance

Michele Kerr suggests (1) teachers should be able to remove disruptive students more easily from the classroom; (2) teachers should only be assessed on the results of those students with 90 percent or higher attendance; (3) students who don’t achieve “basic” proficiency in a subject should be prohibited from advancing to the next grade level; and (4) teachers should be assessed on student improvement, not an absolute standard.

Would you agree to tying your performance reviews to student test scores if these four elements were in place?

via Education Week.

I do not agree, since there doesn’t seem to be anything left to review once you take away classroom management, attendance, and – well – teaching. The fourth point looks like a plateau with even less substance.

Test Less?

When it comes to assessment, the United States is an international outlier. As Stanford University’s Linda Darling-Hammond has shown, many nations with better and more equitable educational outcomes test far less than we do. They typically test just one to three times before high school graduation, and use multiple-choice questions sparingly, if at all. Excessive testing wastes educational resources and fosters the use of cheap, low-level tests, while adding high stakes narrows and dumbs down the curriculum. The results provide little instructional value to students, teachers, schools, or districts.

via Education Week: A Better Way to Assess Students and Evaluate Schools.

A great point, however, be careful when comparing across borders: a country’s institutional structures and civic attitudes can and do influence political outcomes. Compared to alternatives, testing may be the best option for American schools … but we have to look at alternatives first.

End Teacher Tenure, says Tim Knowles

Tim Knowles, director of the Urban Education Institute at The University of Chicago, points out in the Wall Street Journal that tenure in K12 institutions fails to improve the public school system.

An award-winning study of Illinois school districts over an 18-year period found an average of two tenured teachers out of 95,000 were dismissed for underperformance each year. Nationally, between 0.1% and 1% of tenured teachers are dismissed annually, according to the Center for American Progress.

It’s not news that students suffer when very low-performing teachers are allowed to remain in the classroom. But teachers suffer too. In a forthcoming article, my colleague Sara Ray Stoelinga of the University of Chicago Urban Education Institute illustrates how teacher tenure creates perverse practices in schools across Chicago. In interviews with 40 principals, 37 admitted to using some type of harassing supervision—cajoling, pressuring or threatening—to get teachers to leave in order to circumvent the byzantine removal process mandated by the union contract. One principal plotted to remove a teacher who had trouble climbing stairs by assigning her to a fourth-floor classroom. Another reassigned a teacher who had been teaching eighth-graders for 14 years to a first-grade classroom.

Our public school system relies on a perverse collection of counter-incentives and unaccountability to improve their school, their students, and their community.