In case you missed it, reporters for the New York Times asserted that "economic integration" has been enormously successful in raising the academic achievement of black students in Wake County, NC. The article requires a subscription, but here is the relevant part:
Economic integration initiatives differ from each other, and from many traditional integration efforts that relied on mandatory transfer of students among schools. Some of the new initiatives involve busing but some do not; some rely on student choice, while some also use a lottery. And so it is difficult to measure how far students travel or how many students switch schools.
The most ambitious effort and the example most often cited as a success is in the city of Raleigh, N.C., and its suburbs.
For seven years the district has sought to cap the proportion of low-income students in each of the county's 143 schools at 40 percent.
To achieve a balance of low- and middle-income children, the district encourages and sometimes requires students to attend schools far from home. Suburban students are attracted to magnet schools in the city; children from the inner city are sometimes bused to middle-class schools at the outer edges of Raleigh and in the suburbs.
The achievement gains have been sharp, and school officials said economic integration was largely responsible. Only 40 percent of black students in grades three through eight in Wake County, where Raleigh is located, scored at grade level on state reading tests in 1995. By the spring of 2006, 82 percent did.
It's a wonderful story. There's just five problems with it.
1. An earlier New York Times article making the same claim (and also requiring a subscription) noted that economic integration began in 2000. It would be surprising, to say the least, for almost any intervention to raise academic achievement that much in merely six years.
2. The earlier article stated that about "about 2.5 percent--or about 3,000 children--are assigned to schools for economic balance". How does a program that directly affects only 2.5% of the students achieve such large gains?
3. On The Daily Howler blog, Bob Somerby points out that black students all throughout North Carolina achieved almost exactly the same gain. In ten years black students in Wake County achieving "proficiency" increased by 28 percentage points. But in North Carolina as a whole--note that the vast majority of the state does not have the benefit of economic integraton--the increase was 29 percentage points. (The Web page Mr. Somerby linked to for the data has, unfortunately, been changed, but I'm willing--for now--to take his word for it.)
4. North Carolina's "proficiency" increase doesn't seem to show up in the standardized NAEP test scores, raising a serious question about whether North Carolina's proficiency test was dumbed down. (True, the NAEP scores quoted are for students of all races, while the proficiency gains were stated for just blacks. But if black students really did increase proficiency that much, even if non-black students didn't improve at all, the state average should have increased noticeably.)
5. Finally, even if the gain is real, how do we know that other factors weren't much more important? For example, as noted by the link in point 4, NCLB has put serious pressure on poorly-performing schools. My wife, who taught in the regular Wake schools, saw the results: after NCLB, all of a sudden, resources were redirected, redirected especially toward getting poor and minority kids to learn how to read better.
I tell my students that establishing causality using social science data is difficult. Always consider multiple possible causes. Find a control sample. Try to understand how the data were collected: question whether what's actually being measured is what is supposedly being measured.