Fragmented Testing

A couple things make standardized testing difficult in any given educational setting:

  • pressure to pass / excel
  • expensive exam-taking and scoring process
  • valid test construction
  • contested utility

If the pressure leads to undue stress among students or teacher-aided cheating, or if the time lost to test-taking takes too much from the school schedule, or if test writers cannot crank out valid tests fast enough, or if one day of testing simply does not lead to gains, then standardized testing fails to achieve its desired effect on student learning.

An alternative test environment may remove these obstacles. Imagine a fragmented exam schedule instead of the massive 4 hour exams currently administered to students. Periodically throughout the academic year, students would respond to targeted questions benchmarking progress in well-defined fields of study. These questions can be administered via computer on both a scheduled and a random basis; depending on the student’s individual achievement and time passed between questions, the administrator can choose more conceptually complex questions. Following either a minimum number of correct questions or a maximum amount of time spent producing an answer, the administrator can decide whether to advance a student or to begin remedial intervention.

A fragmented standardized exam of this sort may be incentivized through small rewards for each correct answer, say, $1. If a student answers one correct question every school day, they could walk out the last day with $180 in their pocket and a good cache of knowledge – plus the prospect of earning / learning more. Allowing up to ten questions per day, kids could walk away with a small scholarship.



(Part I. Part II in progress…)

Agora won several Goya Awards for production (set, costume, cinematography, special effects, etc) and its screenplay. The film has received mixed reviews in the United States. Roger Ebert wrote in the Sun-Times,

This is a movie about ideas, a drama based on the ancient war between science and superstition. At its center is a woman who in the fourth century A.D. was a scientist, mathematician, philosopher, astronomer and teacher, respected in Egypt, although women were not expected to be any of those things.

while V.A. Musetto complained in the NY Post,

The story revolves around Hypatia (Rachel Weisz), a real-life philosopher whose proclamations about the sun being the center of the universe ran counter to religious beliefs of the time.

There are a few exciting battle sequences and the sets are lavish, but mostly the film meanders aimlessly for more than two hours.

Then Patrick Goldstein interviewed the director for the Los Angeles Times during Cannes,

At several points during the film, he takes us swooping up and away from Alexandria, allowing the audience to see the world from high above, as if watching from the cockpit of a satellite orbiting the Earth. I asked him why he chose that perspective. “For me, this wasn’t just the story of a woman, but the story of a city — and a civilization, and a planet — so I wanted to find a way visually to capture that. When you see things from a distance, you can see how relative things are. The ideas that so inflame people up close, that feel so scary and menacing, they look very different when you see them from a different perspective.”

The words “center,” “revolve,” “meander aimlessly,” “perspective,” “relative,” as well as other astronomical vocabulary pepper the reviews with various denigrating puns or praises, depending on the critic’s general assumptions concerning film aesthetics. I find it amusing that the reviewers – much like the besieged students in my favorite scene – come close to discovering a correct interpretation of the film’s message, but ultimately fall back on preconceived, fallacious notions.

Read more of this post


I often have difficulties reading the New York Times, mainly because it is so poorly written these days. A recent online opinion piece – Equalizing Mediocrity by Sandra Stotsky – shows why. Here’s the opening paragraph:

The case for national standards rests in part on the need to remedy the inconsistent aims and inferior quality of many state standards and tests in order to equalize academic expectations for all students. The argument also addresses the urgent need to increase academic achievement for all students. In mathematics and science in particular, the United States needs much higher levels of achievement than its students currently demonstrate for it to remain competitive in a global economy.

At this point, I would ask my students: what does the author argue for or against? I would expect my students to respond, “Not sure…”

(1) Look at the initial sentence and whittle it down to its most essential parts. It basically reads, “the case for P rests (partly) on a need for a remedy to X.” You should ask what else the case rests on if it only rests “in part” on a need for X. You should ask whether policies Q, R, S… also rest on this need. And you should also ask just what the author means in her use of the verb “to rest.” Does she mean X entails P (or Q, R, S…)? Or does she mean P (Q, R, S…) is only valid when X obtains, and would otherwise be invalid? (another way to phrase this is, if there is no context X, is there a basis for P?). Since none of this is clear, we can expect the argument to come out muddled.

(2) Check the evidence the author presents. She has none; the entire article relies on general assumptions. She shares no facts concerning educational inequalities, no clear indication as to which tests or measures may gauge academic success, and no indication as to why US students “need” to post higher achievement in math and science – nor any mention as to which nation posts highest math and science scores. How can she argue either way without these basic facts on hand?

(3) The author does not come down hard on either the pro or the con. She just kind of wavers. The assumptions about US students math and science achievements might strengthen the case for national education standards, but her prejudices against such standards dissuades her from taking that stance. To put it lightly, the author hasn’t thought out the issue for herself.

These problems occur very often in the New York Times columns and blogs. (I haven’t counted, but I stopped reading NYT on a regular basis once I felt nearly all their opinion pieces advocated various fallacies.) True, they’re problems that crop up in many places online – occasionally here, too. But when I see blatant horse**** in a piece lamenting the mediocrity, my blood begins to boil.

PS: What is the phrase “more uniformly mediocre” supposed to mean?? If you have a collection that, on the whole, appears neutral in some respect, how exactly can it become more neutral? Probably, she means to say, “Common Core will make each student mediocre, whereas before only the aggregate was mediocre,” which in and of itself does not preclude some very exceptional students… *sigh* By the way, my education followed a “common core” and it turn out fine…

Selective and Magnet School Enrollment in Chicago Public Schools

The new enrollment process for coveted CPS classrooms.

Adding Incentives to Test Scores

From Education Week:

To explore what might happen if students had a little incentive to try harder, a trio of researchers focused on a sample of 2,600 students from 59 schools in seven states who were taking NAEP tests in reading. Within each school, the students were randomly assigned to one of three test-taking conditions. Under the first condition, the seniors were paid $20 at the start of the test-taking session. Another group was offered $5 in advance and $30 at the end of the session if they correctly answered two randomly chosen questions on the test. The control group received no special incentives.

The results of the experiment were posted today in the online version of Teachers College Record. The authors are Boston College’s Henry Braun and Irwin Kirsch and Kentaro Yamamoto of the Educational Testing Service.

Students who were assigned to the second group, earning the $30 bonus at the successful completion of the test, did somewhat better than the first group. Both paid groups outperformed the control group, who received no incentives. These results should surprise nobody. People perform tasks better given tangible rewards. The prospect of $35 outweighs the prospect of “opportunity” simply because money is tangible compared to the more abstract concept of opportunity. But here’s the question: ought school districts pay students to take standardized exams?

To answer this, let’s think back to the discussion over the cost of education. Public school students experience around $7 in subsidized instruction per hour in the classroom, whereas high-end private school students experience around $24 in purchased instruction per hour in the classroom. If public school students want to compete for spots in the Ivies with prep-school students, they need to make their education about three times more valuable (in crudely quantified terms). Students from low-income families have few choices – they either need to be very bright and very determined, or they need to sacrifice a large amount of resources to invest in extra educational opportunities. More often than not, we only see the very bright and very determined students gain the most from the public education system.

But what if state and local governments were to reward high performing students instead of subsidizing their education? Let’s say a standardized exam has 150 questions. At $30 per correct question, it could cost only $4500 to pay a kid to ace the test. This is less than half the $10000 it costs to subsidize that same kid’s education over one school year – an approach which we know leads many to failure.

You might say that replacing school funding with test rewards amounts to sabotage. How can the kids pass the test if nobody teaches them? My answer to that is: those who truly want to earn that $4500 will seek out an education. When that happens, you’ve created an open market, lowering costs, boosting innovation, increasing productivity, and posting gains in educating youth.

Unfortunately, I don’t see this transformation happening soon if at all.

University Tenure, RIP

From The Chronicle of Higher Education:

What does vanishing tenure mean for higher education? For starters, some observers say that college faculties are being filled with people who may be less willing to speak their minds: contingent instructors, usually working on short-term contracts. Indeed, the American Association of University Professors says instructors need tenure to guarantee that they can say controversial things inside and outside the classroom without being fired.

But others argue that the disappearance of tenure is actually not the worst thing that could happen in academe. The competition to secure a tenure-track job and then earn tenure has become so fierce in some disciplines that academe may actually be turning away highly qualified people who don’t want the hassle. A system without tenure, but one that still gave professors reasonable pay and job security, might draw that talent back.

Ultimately, though, the future of tenure may hinge on a different calculation: Does its absence hurt students enough in the classroom—something research has shown—that the cost savings to institutions are no longer worthwhile?

via Tenure, RIP – Labor & Work-Life Issues – The Chronicle of Higher Education.

(1) I’m not certain if there is any broad danger to free speech in academia. A few high profile cases exist, but they seem much more rare than we might speculate.

(2) I do believe competition will improve all levels of education in the United States. However, other institutional changes must accompany any major change in faculty hiring. Administrators must implement a stable, non-controversial method of evaluating teachers before attempting to reduce tenured faculty. Administrators must bite the bullet too; there ought to be an equivalent evaluation standard for non-instructional staff. Furthermore, schools need to implement broad incentives, making sure successes come with immediate and commensurate rewards.

(3) I don’t believe, prima facie, that reducing tenured faculty harms student achievement. In my personal experience, some of the best teachers I’ve had were not tenured and some of the worse still are. It should be easy to find many instances where a stable tenured faculty harmed student progress – in fact, we know many instances within urban school districts and less-than-Ivy-calibre colleges.

So should tenure die? I don’t think all tenured positions should vanish overnight, and I don’t think any tenured position in a college, primary, or secondary school should get the ax without a careful consideration of other institutional changes (a la (2) above). But neither do I consider tenure indispensable to the educational system.

Salmon on Expensive Home Foreclosures

Exerpt from Felix Salmon (HT: Around the Sphere)

Streitfeld’s piece is bylined Los Altos, California, a town where the median home is $1.5 million. In such towns, you don’t need to be a millionaire to find yourself in a multi-million-dollar home. Let’s say you’re a tech geek who found yourself with $200,000 for a downpayment on a house over the course of the dot-com bubble. So you buy a million-dollar home, and then start up a series of companies. You need to live, of course, and you can’t afford to pay yourself a salary, so you do two or three cash-out refinancings on a home which by 2007 was worth $2.5 million. Before you know it, you’ve got a $2 million mortgage, no way of paying it, and a home which is worth significantly less than the mortgage. Realistically, you have no choice but to default.

Even after accounting for your initial $200,000 downpayment and a series of mortgage-interest payments along the way, you still took out of the house much more money than you put in: the cost of living there over the past 10 years has probably been negative to the tune of well over half a million dollars. Essentially, the house has paid you $50,000 a year — money which is easy to spend, and is now long gone.

In any event, these were jumbo mortgages when they were taken out, and they’re jumbo mortgages now — none of this has anything to do with Fannie or Freddie, except insofar as the homeowning majority of the population might yet wake up and, emulating the rich, default on their underwater homes. And so the GSEs are desperately, and unconvincingly, trying to persuade them not to do so:

Knowing the costs and factoring in the time horizon, some borrowers have made the calculation that it is better to purposely default on the mortgage. While I understand how that might well be a good decision for certain borrowers, that doesn’t make it good social policy. That’s because strategic defaults affect many other families and communities. And these costs – or as they are known in economic jargon, externalities – are not factored into the individual borrower’s calculations.

Well, sure, it’s not good social policy to strategically default. Fine. That doesn’t stop the rich, and it shouldn’t stop the rest of us either. I think it’s pretty clear which direction we’re headed in, and moralistic exhortations aren’t going to turn the tide.