RSS Feeds Feeds: Articles | Issues
Articles About TAP Subscribe Donate
TAPPED  |  Beat the Press

Remember Me
Forgot your password?

The symbol identifies content for paid subscribers only.


 


Momma said wonk you out

KLEIN ACCOUNTABILITY WATCH: TESTING EDITION.

Yesterday, I joined Seyward Darby in lauding the impressive improvement in test scores during Arne Duncan's tenure as CEO of Chicago's public schools. Dana Goldstein, however, has a useful corrective on this sort of thinking:

A major problem of the testing apparatus under No Child Left Behind is that states can make up their own standards. A report from the Center for American Progress and the U.S. Chamber of Commerce found that Illinois is in the middle of the pack when it comes to the rigor of its standards.

So blogger-sociologist Eduwonkette, who works handily with statistics, looked at Chicago's performance not according to Illinois tests, but according to the National Assessment of Educational Progress. Also known as "The Nation's Report Card," the NAEP is administered by the Department of Education to students across the country and, in typical American fashion, counts for nothing, despite experts' recognition of its findings as the best benchmark we've got. Eduwonkette found that under Duncan's tenure, gaps between black and white students actually grew...This doesn't meant Duncan is a bad superintendent, or that we can't learn anything from him, or that he shouldn't be secretary of education. His leadership on early childhood education, polytechnic secondary schools, and careful growth of the charter sector is a model. But we have to be very careful when we talk about student achievement and the achievement gap, because we just don't have agreed-upon ways of measuring success and failure. Indeed, that's a major problem with NCLB that I hope Duncan will address as secretary.

So what we can actually say is that during Duncan's tenure in Chicago, student achievement on an Illinois-designed test improved and the black-white achievement gap shrank. On the national test, however, those improvements were not in evidence, and the achievement gap actually grew. Which does go to show the difficulty of accurately measuring educational achievement. Teachers aren't just bullshitting when they say that the problem with merit pay and rigorous "accountability" is that there's no agreement on how we examine this stuff in a reliably useful way. That doesn't mean we shouldn't try, but it does imply a certain humility as to the implications of the results, at least until we figure out the methodological problems.

Update: Via Dana comes a very cool site -- the product of a joint venture between CAP and the Chamber of Commerce -- that lets you compare the results, testing scores, and data collection of various states.



COMMENTS

Perhaps under Obama we'll stop measuring student ability through standardized testing period. Okay, that'll never happen, but every year I get students coming in from high school who say they've always tested well in English and are confused, then, that they're getting C's and D's on their essays. I started in Illinois and now I'm in Wyoming - the students are equally bad at writing.

Here's a small and maybe meaningless anecdote - I subbed for a year in a school district in southern Illinois, and one day taught an AP English course. These were advanced placement juniors and seniors reading To Kill a Mockingbird. This should not be a difficult book for eighth graders - which isn't to say it's not an excellent book - yet these highly placed students were reading it in their second semester.

There was an interesting point brought up in the comments of the Eduwonkette post--should we be more concerned about closing the gap, or about seeing all scores rise at the same rate? It's a fair enough point to argue that without closing the gap, there can be no equality in access to powerful/well paying positions, but surely there is a point at which it would be preferable to have all students reach a certain skill level than to avoid having some groups of students outperform others? If for no reason than academic skill level is only going to be one component in making a candidate an attractive hire--in a genuinely meritocratic system, I would think adequate (adequate meaning adequate for performing a high level job, not basic) skills plus other attractive work qualities would at least balance out against higher than necessary skills but a lack of other desirable traits.

Its also worth pointing out that the difficulty of tests changes from year to year which often results in the appearance of improvement without actual improvement.

But sure, lets keep taking fantastical claims of improvement at face value. Everyone benefits when test scores go up!

In NYC I know that some teachers "cook" their tests, and gives students the answers during the state, city, and predictive states. How does this prove tests are effective educationally and statistics show the achievement gap disappearing?

OK - after 6 years teaching 1st grade in an "inner city" (I use the term loosely because of Los Angeles' strange geography) school, this is what I found.

First, year by year, we got better at teaching little 6 year olds to fill in bubbles correctly and to fill in only one bubble per questions.

The tests had many of the same questions each year, so we got better at making sure the kids knew particular facts.

Our test giving skills improved (even when we didn't do things like emphasize particular words in the questions, or read the questions again).

So surprise surprise, the kids' scores went up.

Also, the school encouraged us to have the parents of particularly slow students sign waivers exempting their kids from the tests. (We had to have the waivers because otherwise, the absent kids' scores would be counted as 0.)

One year, the school counselor had us all meet in the auditorium where we were encouraged to "review" the tests a week before they were given. Very illegal. But I'm sure our school was not the only one.

Then there were stories about teachers who erased and corrected their students' answers.

The school administration turned a blind eye, and indeed, seemed to encourage "anything that will get our test scores up."

Some years, the school would be given extra money from the state for "improving" its scores -- we teachers never found out where that money went.

Even without the money, the administrators were judged by their schools' scores, so the incentive was to get them up by hook or by crook.

Meantime, we had to jettison our most stimulating lessons and methods, because they took time from test prep activities which went on all year, and were monitored by surprise classroom visits.

Yes, I am bitter. We were doing better teaching before "accountability" hit.

There has to be a better way.

signed, ex-teacher

Post a comment



Type the characters you see in the picture above.

Search for:

About Ezra Klein

Ezra Klein is an associate editor at The American Prospect. An archive of his articles for The American Prospect can be found here.

Email | RSS | Twitter

Link Blog:


Renew your print subscription or e-subscription.
Get an e-subscription for $14.95.
Give the gift of political insight. Send The American Prospect to a friend.
Change your email address or street address.
YES! I want to receive The American Prospect
— the essential source for progressive ideas.
Explore The American Prospect's award-winning investigative journalism and provocative essays in a free trial issue. Continue receiving The American Prospect at only $19.95 for a one-year subscription - a savings of 60% off the newsstand price!
First Name
Last Name
Address 1
Address 2
City
State
ZIP     
Email

Should you decide not to continue receiving the magazine after the initial free issue, simply write "cancel" on the invoice and you will not be billed.

© 2009 by The American Prospect, Inc.  |  Privacy Policy  |  Permissions and Reprints