Translate

Monday, January 3, 2022

Data is bullshit. Part One

Yes. Yes it is. 

Of course, it doesn't have to be. There is lots of useful data in the universe. The Covid infection rate and the number of available hospital beds in your town are important pieces of information. Cholesterol level. Blood pressure. Miles to the gallon. Yards per carry. Odds of getting a heart on the river. Each of these data points has an outcome associated with it that is meaningful and actionable. 

There are data that are related to schooling that are also meaningful. Graduation rates. Attendance. Student-teacher ratio. Number of students qualifying for subsidized lunches. The trouble is that when school reformers talk about data, they are talking about test scores. 

So, as I wrote last time, all of these terrible ideas that incoming superintendent Alberto Carvalho is likely to bring to LAUSD--school choice, performance pay for teachers, the firings--all of them depend entirely on "data," by which he means "testing," by which he means state-sponsored, corporate-controlled assessments. If you are wondering what the hell I'm talking about: 1) You need to pay closer attention, and 2) You obviously don't teach math or English, or now science, or... 

In any case, this approach is unsound because the data are unsound. 

I threw that "are" in there for those of you who insist that data must be plural. However, I'll be using the word as a singular mass noun to represent the scores on these tests, and using it specifically to attack the notion that these scores--this "data"-- is the supreme (or any) expression of what students know, how effective their teachers are, or whether their schools have "failed." This data is a political tool and not meaningful in any educational sense. Teachers should resist the entire enterprise. Some ideas on how to do that later. 

For those of you thinking "Oh my test scores! I could lose my job!" I'm sorry, but you are right to be concerned. For any of you thinking, "My scores! I must be a shitty teacher!" You might be a shitty teacher, but it has nothing to do with your test scores, which are not your test scores. They are your students' scores. 

If you've been thinking that the testing is crazy and pointless and you've been thinking the same thing since No Child Left Behind, take comfort in the fact that you have been right for twenty years. Read on for corroboration and please comment and add your own thoughts. For those of you newer to the madness and thinking "Am I crazy? Or is this insane?" Take comfort in the fact that you are not crazy. It is insane, and you need to give yourself permission to acknowledge that. It's not you, it's the testing. You want proof?

Let's begin with the emphasis on testing and the elevation of test data generation over other elements of schooling. In case you haven't heard, we're in the middle (if we're lucky) of a fucking pandemic. Schools are under assault while struggling to keep students and staff safe, and people increasingly don't want to work there or go there. Teachers are desperate to reconnect with their students, to support their mental health while helping them engage and focus on their studies. So what do you think would be the most important concern for educators high enough on the food chain to actually decide things? 

If you guessed standardized testing, you win! Libraries are battlefields and there's no staff and schools can't even follow their own safety rules, but the testing abides. The testing obsession among educators who aren't teachers is as stifling as the gun obsession among 2nd Amendment fetishists. The destruction just happens more slowly.

In her excellent The Answer Sheet blog (no relation) at the The Washington Post, education writer Valerie Strauss posted on the subject and includes some very good questions from Bob Schaeffer from Fairtest. It's a good read.

Right now you should be on break, but in a week the lunacy will resume and many of you out there will be trying to open a testing session, trying to get students to finish a session, or wondering why there are sessions at all while we're still in a fucking pandemic. You are asking the right question and very probably using the right words--at least with your friends--to describe the ridiculousness. 

Some of you, however, might actually be listening to the testing fanatics who are leading your so-called Professional Developments (really just test training and ass covering) or pushing you to be sure you catch the "make-up" testers, and you might actually be thinking, "Well, at least it's for a good cause." 

Well just forget it. You (may) have to give the tests in order to keep your job, but unless your idea of a good cause is enriching the testing-industrial complex and their shareholders while destroying the public in public education, there is no reason for any teacher to feel good about giving these tests. Period. 

If you have any doubt about this, consider the following (if you haven't already--like a bazillion times): First, teachers are prevented from seeing the questions from the tests--including the ones their students got right and wrong--and are prevented from seeing the scores in time to shape instruction. And, get this, teachers are definitely prohibited from discussing the assessments with even their own colleagues (sign the affidavit!). So ask yourself: Why would such a discussion--one that might actually do some good if by good you mean helping students and, incidentally, raising test scores--be outlawed

Second, imagine a world so gripped by a pandemic that whole school systems had to close down for months. Then they open again and improvise their way through a year of masks/no masks, vax/no vax, and "What happened to all the Covid tests?" Under these circumstances, what could the value of administering these assessments possibly be? Except, of course, to support the narrative of failing schools. In that case, what better time to test? 

(By the way, check the "Nation's Report Card"  if you're up for a little mind tease. It takes some doing, but if I'm reading the long-term trends correctly--and I like to think that I am--it turns out the actual scores on perhaps the least egregious of the Big Tests, the NAEP, have been pretty stable recently and have gone up significantly over time. Hardly a picture of crisis. There is a genuine and serious issue regarding score gaps and equity, but even those gaps have diminished. However, you probably haven't heard how great a job we're doing in schools. That's not the carefully constructed and immensely profitable narrative.)

The truth is the testing exists to enrich the industry and generate data that will elevate certain schools and punish others along with their teachers (aka "hold them accountable"). Which might make some kind of sense if you believe or pretend to believe that teachers are slackers who can and should be shamed and hectored into magically levitating scores, which are in fact controlled by a million variables only one of which is a teacher--or six or seven teachers if you're in high school. And it might make very good sense to you if you're the type of person who cocks their head and looks up at the stars and muses: If only we could get rid of all the bad ones... And oh yes, fuck the teacher unions! 

Or maybe you really believe, or pretend to believe, that scores on a glitchy exam given once a year tell us what we need to know about students and their learning. And if that doesn't work, then we have lots more exams for all the other times of the year! Yes, you might be thinking, "That's totally worth it! Beating up teachers and hijacking instruction is a small price to pay for the higher test scores that will then prove that beating up teachers and hijacking instruction really works! To produce higher test scores!

Except for one thing: Data is bullshit

Next up: What's the plan? Testing is easy; fixing things is hard.




No comments:

Post a Comment