Translate

Tuesday, October 25, 2022

NAEP? NOPE. - Seemed like a good day to replay this one. From Sept. 19 this year.

 The Test Score Tango.

Preface: You may already know how I feel about Big Testing and test scores. If not and you're interested, I've written about it here, and here, and here, and here. That said, the least terrible of the Big Tests is probably the National Assessment of Educational Progress, or NAEP, if only because you don't have to shut down an entire school every year to give it, and because it doesn't sort the high-score schools from "failing" ones. 

Yet, when the latest NAEP scores dropped a couple weeks ago the usual suspects were predictably aghast. Hilariously, some of the same dopes that declared the teacher shortage a fake now credulously announced the loss of twenty years of progress based on one set of scores on one test given to nine-year-olds in the third year of a fucking pandemic. 

Kids, there are demons out there bent on the destruction of public schools, and they'll do whatever it takes to make it happen. When I was in the classroom, it felt like I was in the middle of one big chaotic shitstorm. It may feel that way to you, too. But it is not. It is strategy. It's a program. Think of it that way and it's easier to recognize. You just have to know what to look for. Then maybe we can fight back. 

And now, the Test Score Tango.

A lot of the early NAEP score headlines were overwrought and eerily similar. The usual suspects, of course. The New York Times had "The Pandemic Erased Two Decades of Progress in Math and Reading." But also from outlets that should know better. From Chalkbeat: "Math and reading scores plummet on national test, erasing 20 years of progress." And of course piling on from The74: "‘Nation’s Report Card’: Two Decades of Growth Wiped Out by Two Years of Pandemic" (all emphasis mine). A little later The New Yorker had this terrible "interview" full of leading questions and reformster* rhetoric.

Getting out early with sensational claims helps shape the narrative. Fortunately, several observers responded quickly with a more measured, nuanced analysis than the doomsayers. This article from Education Week does a good job explaining what the scores actually mean. Jan Resseger    situates the test results within the broader context of the trauma and disruption caused by the pandemic, and suggests the limits on conclusions that may be drawn from them. And this thoughtful article from Jill Barshay titled, "6 Questions to Better Understand Math and Reading Scores" asks really good questions and doesn't pretend to have all the answers. In advance. 

Leonie Haimson of classsizematters.org stresses the lack of clarity in the data and pushes back on the popular assertion that closing schools and the consequent remote learning were a key driver in the score decline. 

Haimson also challenges the conventional proposals for what administrators and education Big Thinkers call "learning recovery." Addressing "negative impacts on learning and test scores," Haimson writes, will require not simply "longer days and years." Instead, she advocates for smaller class sizes rather than what she calls "false and damaging policy prescriptions."

It's difficult to know whether all the hyperventilation comes from genuine misunderstanding, or underlying partisanship, or if it's just for clicks, but it is surely... overblown.

Many of the agitated commenters were no doubt expecting--and knew their readers (and funders) were expecting--the pandemic to have a severe impact on student learning. And many remain particularly focused on the closing of school buildings and the shift to remote learning. Many expected those effects to have been disastrous, and they are heavily invested in blaming teachers and their unions for the damage. When the scores came out, it was easy to read them as confirmation of their expectations. 

But the truth is, in the best of times the NAEP scores don't mean what a lot of people think they mean or would like them to mean. And in pandemic times, it's even harder to draw conclusions.  

One of the things about NAEP that doesn't mean what people think it means is "grade level." In 2016's "The NAEP proficiency myth" from Brookings, Tom Loveless demolishes the notion that NAEP proficiency is "synonymous with grade level" and urges states not to use NAEP’s proficient level as a basis for education policy

Some interested parties--including organizations promoting a "failing schools" narrative to advocate for school choice--continue to deliberately misinterpret the NAEP levels and conflate "proficient" with grade level. By this definition, then, in a given year two-thirds! of test takers might fall short of "grade level." 

The truth is, however, that what most of us think of as "grade level" more closely aligns to NAEP's Basic level. By simply being honest about the meaning of the NAEP scores, instead of two-thirds below grade level we in fact have two-thirds at or above. The sky may be a bit cloudy, but it's not falling.

The Covid pandemic has obviously had a profound impact on students, and the new NAEP scores clearly reflect that.    However, in an example of the scores not meaning what some people would like them to mean, several partisan observers have used the scores to criticize the ways schooling was conducted during the pandemic. In doing so, they either misconstrue or ignore the data completely.

At the center of the project to blame teachers and unions for the drop in scores is the issue of school closings and the reliance on remote learning. Lots of interested parties have passed around the same study to show a link to learning loss. The NAEP scores themselves reveal a more complicated picture.  

For example, as Haimson and others have noted, the evidence for a correlation between school closings/remote learning and the drop in test scores is decidedly inconclusive. And even the NCES itself warns us that "Users are cautioned against interpreting NAEP results as implying causal relations." As Peter Greene put it in 2019: "Everyone ignores that advice, but NAEP clearly acknowledges that there are too many factors at play here to focus on any single one."

What can we know based on this limited set of data? As author and former writing teacher John Warner describes it in a NAEP discussion with Nick Covington of the Human Restoration Project: We know "a big thing happened." And "not only was school disrupted, the entire lives of everybody in the country and the world was disrupted" and "the notion that we could have avoided this with some kind of different school or different choices, going back in person sooner... it's not reflected in any of the data." (7:50)

Although the impact of the pandemic and the disruption and trauma that have accompanied it are clear and universal, the impact on student learning of going remote is, as yet, unclear. Facts are stubborn things.

Another thing we can learn from NAEP concerns the testing regime spawned by No Child Left Behind. If you assume that the promoters of NCLB were acting in good faith and not simply trying to inject an air bubble into the bloodstream of public schooling, the testing regime it initiated (not NAEP, but the hundreds of other tests kids have to take every year) hasn't succeeded on its own terms. Rather than more and more testing leading to a rise in scores, with the exception of a limited burst in the early 2000s, the NAEP scores have flattened. Big Testing hasn't worked


Those are things we can legitimately infer from the limited NAEP data we have right now. But what about "Two Decades of Growth Wiped Out!"? It would be easy to dismiss it as professional alarmism, but in this case the stakes are too high to simply chalk it up to selling papers. 

There are demons out there, and they are perched, waiting to jump to conclusions and manufacture analyses that serve the project to dismantle authentic** public schooling. They can't wait even a few weeks until more complete data is available. 

As Jill Barshay writes"A more detailed report from the Department of Education on student achievement during the pandemic is expected in October. It will list state achievement scores for fourth and eighth graders on another NAEP test. Hopefully, we can unravel more of these knots together."    Maybe that's what the vultures are afraid of.

In this environment, information like the NAEP scores is routinely politicized. The scores are used to sell programs and products from organizations that profit from brokenness. And of course, the scores are cynically weaponized by the school choicers and their enablers in the press in order to promote a failing schools narrative and advance the choice and anti-union agenda.  

But the scores do not mean what the screamers say they mean. In order to turn the volume up to 11, they compare the raw scores to previous scores, find a match twenty years ago, and announce twenty years of progress loss. The thing is, it's twenty years of test score progress, not twenty years of student progress. 

In an interview conducted by Liz Mineo for the Harvard Gazette, Dr. Andrew Ho, professor at the Harvard Graduate School of Education and former member of the NAEP governing board, explains:

[E]ach point on the NAEP scale is roughly three weeks of an academic year, and the overall decline was seven points, which is roughly five months in terms of an academic school year of learning that this cohort of students is relatively behind.

Five months. Not twenty years. But the average overall decline doesn't tell the whole story, of course. Predictably, students who were most challenged before the pandemic were most challenged by the pandemic. For students who were already struggling, who were already coping with insufficient resources--including educational resources--and whose families were likely to be most severely impacted by the pandemic, results were more alarming. From Dr. Ho:

But this average betrays the inequality where the decline for the higher-scoring students was only three points, or nine weeks, whereas the decline for the lowest-scoring students was 12 points, which is 36 weeks, which is almost approaching an entire academic year. 

Terrible. Unjust. Unacceptable. We need to address this, but this is not just a test score. This is a reflection of the racism and resource inequality that permeates our system. And still, the most impacted students lost "almost approaching an entire academic year." NOT twenty. 

The kids being told right now that the scores are down and the pandemic has cost them twenty years are being crushed for the objectives of the crushers.

Dr. Ho goes on to discuss the broader impact of the pandemic on students' lives apart from one set of test scores and reminds us what returning to "normal" means for students getting back to school. He also suggests the need for a "learning infrastructure," perhaps in the form of a "National Assessment of Educational Equity." In his words:

I’m not just worried about academic learning. I’m worried about the structures of educational opportunity and inequality that have increased over the pandemic; this is just the tip of the iceberg. I hope we can pay attention to the entire structure of educational opportunity. Academic learning outcomes are easy to measure, but also easy to overemphasize, as if students are only trying to get better at math and reading and not also trying to reconnect with their classmates and their teachers and remember how to sit still, listen, learn, and enjoy learning and playing with others.

Again, even among the most severely impacted, most vulnerable students, the setback was 12 points, 36 weeks, "approaching an entire school year." Which, after what they've gone through, seems predictable.

Not good, but not twenty years. All the screaming is premature. Before the additional scores (8th grade) and details in October. Before students have a chance to fully readjust to in-person schooling and reinvigorate their intellectual lives. Before students have had the chance to right themselves before we've given them a chance to catch up. For many of them, before they've had a chance to fully mourn their personal losses. 

Yet the drumbeat goes on. In the newspapers. Online. On the TV news. Among administrators. Sometimes friends and family shaking their heads and tsk tsking.

What can we do? I'm no longer on the job but when I was, I tried to follow this first rule of survival:

Do not fall for their bullshit. Because they tell you something is bad doesn't mean it is. Because they tell you scores are everything doesn't mean they are. Because they say your scores are down and you failed doesn't mean it's true. 

First of all, they are not your scores. They are your students' scores. And it's only a test. You know your students, and you know how little of their intelligence and talent and humor is reflected in any single test score. 

Nod your head, pretend to care, ignore the bs. Remember that most of the world doesn't understand your world, and 99% of the press reports and all the talk on the morning news is from people who either don't know anything or are deliberately lying for their own purposes. 

That goes for the bosses, too. Don't expect your admin to honestly tell you why you're doing this test or that test or why you're spending so much time prepping for them. Don't break your head trying to figure them out. Remember that they have a different agenda. They are speaking a language different from yours. Your concerns are not their concerns. Their only honest answer to "why" is the same as yours, the same as your students': Because we're told to. 

If you're in a position to resist, do so. Ask other,  "non-why" questions--they hate that. Challenge their analysis of the scores. Their interpretation. With the rest of the world who are not your bosses, push back. At parties, with family, on social media. And write letters to editors; that's still a thing. Write a blog. You're an educator, let's educate the ones who don't know better and confront the ones who do.

You are right and they are wrong. Don't let the bastards grind you down.


*h/t Peter Greene

**h/t Steven Singer



 



Wednesday, October 19, 2022

One-Room Schoolhouse


One-Room Schoolhouse


“Fernando.”
    Fernando looks at his friend Dahlia, who nods. He then rises grinning from his seat and bounces toward the teacher table at the front of the room. I hand him his book and, after giving it a cursory glance, Fernando twirls and retraces his steps. His friends giggle, and so does he.
    Between handing out books 17 and 18, I take a look around the room. Fernando is talking with his friends Irma and Jasmine. He has already put the book away in his backpack, not moved by the fact that there are still eighty-five minutes left in our block period. His friends don’t stop him and don’t ask to see the book. Unlike Fernando, they can read, but not in English. The same is true of Song, who arrived from Korea four months ago and seems most interested in the size of the book. He pretends to weigh it in his hands as he says something to Kim, a friend he has made in the class. They both laugh broadly.
    That’s the predictable, even easy part of the story. Now I look to my right and see Tina, focused and already reading ardently. Behind her, Juan is also eagerly chewing through hunks of text, only he’s reading out loud to his friends Milton and Rudy. His mouth seems to taste the words as it forms them, measured, deliberate. In the front corner of the room, Jose sits and stares, first at the cover of the book as he traces the drawing with his finger, then out the window.
    The thirty-two students gathered together in this ninth grade English class are so diverse an observer might fairly ask if there’s been a mistake. I have students from fourteen years old to eighteen, some brand new to the country, some born within blocks of the school. They have English reading levels from below the first grade to above the eleventh. To friends and family I’ve described it as anything from a balancing act to a grand pageant, from a juggling act to just plain impossible. A hundred years ago, students of all ages and skill levels in a community might be taught in one room by a single teacher. They called it a one-room schoolhouse.

I wrote that ten years ago. 

I had just moved from the school where I had taught for fifteen years to a brand new "pilot school" campus and we were creating a school where none had been. When we arrived we had to invent systems and structures that would serve our students and remain true to our mission of social justice and environmental responsibility. And because we were a k-12 span school, we had to do it for elementary, middle, and high school kids from 5 to 19 years old.

For those of you who don't know, pilot schools, established in 2007 through a side agreement between the Los Angeles Unified School District and United Teachers Los Angeles, are essentially the district's answer to the charter schools that have been siphoning off students and resources from authentic public schools for years

There is a much more fulsome interrogation of charter schools in a near-future post, but briefly: From the CDE website, "In 1992, California became the second state in the nation to adopt public charter school legislation." That measure was followed with additional legislation dissolving limits on charter schools and forcing districts and even existing neighborhood schools to divert more and more resources--up to and including giving away the public school's actual facilities--to support the charters with which they were now forced to compete for students. 

The district's responses to this existential threat have been reliably hapless and politically self-serving. Of course there was the capitulation to the testing-industrial complex that grew out of No Child Left Behind and its evil progeny. Once fully addicted to the poison pill of competition and synthetic accountability, the district resorted to mimicking its tormentors, adopting the assumptions and solutions of the school choicers and internalizing them.

So, rather than focus on the contributions of authentic neighborhood schools, their value to their communites, and the importance of strengthening them, the district instead opted for sham self-determination (school-based management, expanded school-based management, something called an iDesign Schools Division), or offered its own recipe for school choice (Public School School Choice and Open Enrollment).

In fact, LAUSD itself  got on the charter train almost immediately, with one superintendent even floating the idea of a charter district of LAUSD's very own. 

The pilot school model is simply one more answer to the wrong question. Not "How do we best serve our students, our communities, and our world?" But rather, "How do we  compete?" 

Because they are advertised as having several "autonomies" that allow them to operate independent of the school district, today's pilot schools are peddled as alternatives both to charter schools and to traditional schools starved of resources and trapped in a bureaucracy more concerned with scores than students. 

These autonomies include decisions with respect to testing and curriculum, budget, governance, and the school calendar. Most critically, the school--and by school, I mean principal--maintains control over staffing. The stated objective is to bring together like-minded individuals committed to a shared mission. One Elect-to-Work agreement reads: "The Principal may release staff members that are unwilling to support and implement the vision and mission" of the school. 

Theoretically, this independence facilitates the innovation promised by charter schools concerned with being more efficient--and more segregated--versions of public schools.

We're a pilot school! We're not like those other public schools. We have autonomies! We have a theme! We're small for personalized instruction! We can get rid of those troublesome teachers who won't get with the program or who--gasp!--don't believe in your kids.  

Sadly, in our school we quickly found limits we didn't anticipate. In spite of our outwardly posh surroundings, we were basically a bare-bones operation. We shared the campus with five other schools and both our staffing and our physical space were severely limited. Our middle/high school counselor had to keep track of about six hundred students. There were classes we couldn't offer because there were not enough classrooms. We were denied access to certain spaces--the freakin brand new  theatre, for example--and even our lunch times were subject to complicated negotiations. 

We had no teachers or classrooms to create an infrastructure for language learners, so even newcomers were programmed into traditional English classes, often with no additional support. And on the first day of school we found out we were a "full inclusion" school and students with special needs were immersed in the standard program, even if their previous experience was in a special day class. We were told this was in keeping with our "social justice" mission. 

In both these cases, it was clear to me that our model was not a sufficient response to the needs of our students. I didn't really understand the scope of the war being waged against public schools. As I've written, I was siloed in the day to day struggle of trying to serve my students and stay healthy. 

Even then, though, I could see that what we were doing--the posing, the authoritarian professional developments and prescriptive procedures, the minimal resources and skeleton crew, the disappearing of teachers who spoke out, and the compulsive testing--was not school. It was, rather, a reaction to the Walmartization* of schooling driven by the privatizers and school choicers in general and the charter school movement in particular. 
*Found here, and later here, and by Diane Ravitch here.

Charter schools have long benefited from a gigantic public relations/advocacy machine that has been very successful at shaping public opinion, intimidating dissenters, and influencing public officials--including district officials--to support them. Unfortunately, this pilot school pretend-charter model was a symptom, not a solution. 

However, saying so, and speaking up about the many ways we were falling short, did not make me a favorite with administration.

Pro tip: When your principal is assigned and paid by the district and exercises total authority over staffing (there's autonomy, for you), there's no such thing as independence from the district. 

The district's response to the enemies of public schooling was anemic and wrong-headed, but it was not entirely cynical. We wanted to be better. But we should not have wanted to be like them. 

So it was in this context that I was sitting at a table in my one-room schoolhouse, handing out books to the most eclectic group of students I had faced in my career. I was also trying to understand my school and district and the many ways forces outside my classroom were becoming more and more powerfully influential in the way I conducted my interactions with kids. I was very frustrated.

More of that frustration in the next post when I'll share what I thought about it all then, what I think about it now, and some of what I've been able to see from outside the silo. 

See you next time.



Jack Schneider

In place of a post, I'm sharing these two fantastic twitter threads, one on the "unstated theory of change" underlying recent "dominant approaches to education reform" and one pulling back the curtain on vouchers.

If you are not familiar with the work of Jack Schneider    @Edu_Historian, do yourself a favor. This is his website at the University of Massachusetts Lowell which has links to the awesome podcast he does with Jennifer Berkshire @BisforBerkshire

Together they co-authored A Wolf at the Schoolhouse Door: The Dismantling of Public Education and the Future of School

You can also get to the podcast here.



And because this one is great, too:

 A thought about vouchers and what they're really about...



Sunday, October 9, 2022

Listen to "LAUSD Up -- Periodic Reporting on the Los Angeles Unified School District"

Saturday, October 8, 2022

LAUSD Up -- Periodic Reporting on the Los Angeles Unified School District

Hello from Los Angeles.

It's been a pretty big few weeks at LAUSD. In April, the LAUSD School board voted to add seven "optional" days to the school year, ostensibly to address learning "lost" during the pandemic. UTLA then filed an unfair practice charge citing the district's failure to negotiate before imposing an illegal alteration to the school year.

 

The bulldozer approach from the Superintendent's office failed as, after the complaint and a vote to boycott the extra days, UTLA got exactly what it had been asking for all along: a negotiated proposal for extended learning time.

Of course you wouldn't know that from the L.A. Times. Predictably, the Times characterized the agreement as a response to "pressure from the teachers union," rather than an admission that contractual matters must be negotiated. Quelle surprise. (paywall)

In a statement, politician and world-class suit wearer Alberto Carvalho said the district was "pleased that UTLA has accepted our proposal for the Student Acceleration Days, and we look forward to continued discussions with other labor partners." I'll bet he does.

Side note: Throughout the process there was a bunch of screeching from the reformster Los Angeles Parents "Union." The organization has turned down the volume since the agreement but it remains virulently anti-UTLA and has returned to old stand-bys school closures and masking, among other complaints and accusations. 

According to US News & World Report, LA Parents Union was co-founded by Alma Marquez, a professional organizer and advocate who also worked for Green Dot and who went on to found the National Parents Union, an anti-union organization cobbled together to oppose the NEA and AFT.

The rhetoric used by the group and its supporters is massively over the top, hostile and aggressive. They consistently characterize teachers--and particularly teachers unions--as acting counter to the interests of students.

Be aware of their true nature if they happen to pop up at your school sometime.

As I wrote here, Carvalho brings his "my way or the highway" management approach with him from Miami, and when he swaggered into town he made a lot of promises in the "This is how it's going to be" category. 

Bullies gonna bully, I suppose, but having spent a few years under the heel myself, I can say with confidence that when a knee-breaker is brought in to browbeat the teachers into submission, it's usually the bully who walks away limping. 

Perhaps Carvalho just forgot that he needs to work with partners to achieve results, and that some of his most important partners are teachers. Maybe he knows now, maybe not. Stay tuned. We are likely to see this again.


Another big thing that gripped the district--at least according to all the breathless coverage in the local media--was the cyber ransoming of data. Over Labor Day weekend, LAUSD was cyber attacked and gave up hundreds of gigabytes of data (it's not even clear yet whose or what kind). Weeks later the hackers set a deadline for ransom and, when the deadline came and went, they released 500GB of data.  

It's impossible for us on the outside to know at this point how much damage has been done. We got this from the Super:

I don't need to tell you how uncomfortable it is counting on the district to act quickly and effectively. Many of you may remember the payroll debacle, and just the letters ssis isis misis might give you a headache. 

But even if we close our eyes and whisper to ourselves over and over again they-got-this-they-got-this, the district and its administrators have rarely given us a reason to believe they're being truthful with us. Rather, as usual, their interests lie in minimizing the damage and pretending there's nothing to see here.

UTLA had this tweet:

Nice, but it's difficult to see how UTLA can do much. One thing they can do, however, is follow member and NBC teacher       @phylis_hoffman's advice and push for an independent auditor to assess the harm and risks associated with the hack. 

UTLA may not be able to do much, but it can do this. 

If you all have any additional info on the breach, info that can flesh out these stories, please share and we'll get the news out there. You can comment (anonomously, if you like) here, or you can email me at nowwaid@gmail.com. 


And finally, it was hot!

I saw lots of anti-asphaltism and chatter about demands for green space and I know that UTLA is committed to fighting for green schools and spaces:


And I must say I'm all for it. But a word of caution: Have you ever seen the district take care of anything they created? 

It reminds me of a Seinfeld episode: You know how to plant the trees; you just don't know how to grow the trees. Or feed them or water them or protect them in any way. Or even remember where you planted them. 

The most likely scenario: The district makes a big deal out of it and holds a ceremony. They plant a bunch of trees at a high-profile school, and it'll be on the news with the Super getting his picture taken, and that's it. The custodians will try for a while to keep the trees alive, until they're three people short because of transfers and illness. The drip system will fail and the earliest maintenance ticket is next summer. Some teacher will step up and try but soon the other teachers who offered to help have meetings and papers to grade, and the student volunteers have homework or soccer practice or little sisters and brothers to take care of.

And then the trees die. And the students who walk by the dead trees every day wonder why nobody gives a crap about them.

We used to say that every good program--drama, marching band, chess club, debate, robotics--is just one dedicated teacher from extinction. The system has no institutional memory and feels no responsibility for continuity. 

Please don't do this to the trees.


Finally, just a reminder to join your union. If you're already a UTLA member, here's a list of endorsements you should pay attention to. As always, your mind and your vote belong to you--please use them. 

  • Dr. Rocio Rivas - LAUSD Board District 2
  • Kelly Gonez - LAUSD Board District 6
  • Erin Darling -LA City Council District 11
  • Hugo Soto-Martinez - LA City Council District 13
  • Danielle Sandoval - LA City Council District 15
  • Lindsey Horvath - LA County Board of Supervisors District 3
  • Karen Bass - LA Mayor
  • Yes on Measure ULA


If you're not a member of your union, why not? Below is an Economic Policy Institute chart reminding us that, in addition to standing between us and fifty kids in a class and zero paid sick days, unions are crucial in mitigating income disparity. If you don't have access to a union or are prohibited from collective bargaining, vote and organize. Power concedes nothing without a demand.




Monday, September 19, 2022

Listen to "NAEP? Nope." The Test Score Tango.

NAEP? NOPE.

The Test Score Tango.

Preface: You may already know how I feel about Big Testing and test scores. If not and you're interested, I've written about it here, and here, and here, and here. That said, the least terrible of the Big Tests is probably the National Assessment of Educational Progress, or NAEP, if only because you don't have to shut down an entire school every year to give it, and because it doesn't sort the high-score schools from "failing" ones. 

Yet, when the latest NAEP scores dropped a couple weeks ago the usual suspects were predictably aghast. Hilariously, some of the same dopes that declared the teacher shortage a fake now credulously announced the loss of twenty years of progress based on one set of scores on one test given to nine-year-olds in the third year of a fucking pandemic. 

Kids, there are demons out there bent on the destruction of public schools, and they'll do whatever it takes to make it happen. When I was in the classroom, it felt like I was in the middle of one big chaotic shitstorm. It may feel that way to you, too. But it is not. It is strategy. It's a program. Think of it that way and it's easier to recognize. You just have to know what to look for. Then maybe we can fight back. 

And now, the Test Score Tango.

A lot of the early NAEP score headlines were overwrought and eerily similar. The usual suspects, of course. The New York Times had "The Pandemic Erased Two Decades of Progress in Math and Reading." But also from outlets that should know better. From Chalkbeat: "Math and reading scores plummet on national test, erasing 20 years of progress." And of course piling on from The74: "‘Nation’s Report Card’: Two Decades of Growth Wiped Out by Two Years of Pandemic" (all emphasis mine). A little later The New Yorker had this terrible "interview" full of leading questions and reformster* rhetoric.

Getting out early with sensational claims helps shape the narrative. Fortunately, several observers responded quickly with a more measured, nuanced analysis than the doomsayers. This article from Education Week does a good job explaining what the scores actually mean. Jan Resseger    situates the test results within the broader context of the trauma and disruption caused by the pandemic, and suggests the limits on conclusions that may be drawn from them. And this thoughtful article from Jill Barshay titled, "6 Questions to Better Understand Math and Reading Scores" asks really good questions and doesn't pretend to have all the answers. In advance. 

Leonie Haimson of classsizematters.org stresses the lack of clarity in the data and pushes back on the popular assertion that closing schools and the consequent remote learning were a key driver in the score decline. 

Haimson also challenges the conventional proposals for what administrators and education Big Thinkers call "learning recovery." Addressing "negative impacts on learning and test scores," Haimson writes, will require not simply "longer days and years." Instead, she advocates for smaller class sizes rather than what she calls "false and damaging policy prescriptions."

It's difficult to know whether all the hyperventilation comes from genuine misunderstanding, or underlying partisanship, or if it's just for clicks, but it is surely... overblown.

Many of the agitated commenters were no doubt expecting--and knew their readers (and funders) were expecting--the pandemic to have a severe impact on student learning. And many remain particularly focused on the closing of school buildings and the shift to remote learning. Many expected those effects to have been disastrous, and they are heavily invested in blaming teachers and their unions for the damage. When the scores came out, it was easy to read them as confirmation of their expectations. 

But the truth is, in the best of times the NAEP scores don't mean what a lot of people think they mean or would like them to mean. And in pandemic times, it's even harder to draw conclusions.  

One of the things about NAEP that doesn't mean what people think it means is "grade level." In 2016's "The NAEP proficiency myth" from Brookings, Tom Loveless demolishes the notion that NAEP proficiency is "synonymous with grade level" and urges states not to use NAEP’s proficient level as a basis for education policy

Some interested parties--including organizations promoting a "failing schools" narrative to advocate for school choice--continue to deliberately misinterpret the NAEP levels and conflate "proficient" with grade level. By this definition, then, in a given year two-thirds! of test takers might fall short of "grade level." 

The truth is, however, that what most of us think of as "grade level" more closely aligns to NAEP's Basic level. By simply being honest about the meaning of the NAEP scores, instead of two-thirds below grade level we in fact have two-thirds at or above. The sky may be a bit cloudy, but it's not falling.

The Covid pandemic has obviously had a profound impact on students, and the new NAEP scores clearly reflect that.    However, in an example of the scores not meaning what some people would like them to mean, several partisan observers have used the scores to criticize the ways schooling was conducted during the pandemic. In doing so, they either misconstrue or ignore the data completely.

At the center of the project to blame teachers and unions for the drop in scores is the issue of school closings and the reliance on remote learning. Lots of interested parties have passed around the same study to show a link to learning loss. The NAEP scores themselves reveal a more complicated picture.  

For example, as Haimson and others have noted, the evidence for a correlation between school closings/remote learning and the drop in test scores is decidedly inconclusive. And even the NCES itself warns us that "Users are cautioned against interpreting NAEP results as implying causal relations." As Peter Greene put it in 2019: "Everyone ignores that advice, but NAEP clearly acknowledges that there are too many factors at play here to focus on any single one."

What can we know based on this limited set of data? As author and former writing teacher John Warner describes it in a NAEP discussion with Nick Covington of the Human Restoration Project: We know "a big thing happened." And "not only was school disrupted, the entire lives of everybody in the country and the world was disrupted" and "the notion that we could have avoided this with some kind of different school or different choices, going back in person sooner... it's not reflected in any of the data." (7:50)

Although the impact of the pandemic and the disruption and trauma that have accompanied it are clear and universal, the impact on student learning of going remote is, as yet, unclear. Facts are stubborn things.

Another thing we can learn from NAEP concerns the testing regime spawned by No Child Left Behind. If you assume that the promoters of NCLB were acting in good faith and not simply trying to inject an air bubble into the bloodstream of public schooling, the testing regime it initiated (not NAEP, but the hundreds of other tests kids have to take every year) hasn't succeeded on its own terms. Rather than more and more testing leading to a rise in scores, with the exception of a limited burst in the early 2000s, the NAEP scores have flattened. Big Testing hasn't worked


Those are things we can legitimately infer from the limited NAEP data we have right now. But what about "Two Decades of Growth Wiped Out!"? It would be easy to dismiss it as professional alarmism, but in this case the stakes are too high to simply chalk it up to selling papers. 

There are demons out there, and they are perched, waiting to jump to conclusions and manufacture analyses that serve the project to dismantle authentic** public schooling. They can't wait even a few weeks until more complete data is available. 

As Jill Barshay writes"A more detailed report from the Department of Education on student achievement during the pandemic is expected in October. It will list state achievement scores for fourth and eighth graders on another NAEP test. Hopefully, we can unravel more of these knots together."    Maybe that's what the vultures are afraid of.

In this environment, information like the NAEP scores is routinely politicized. The scores are used to sell programs and products from organizations that profit from brokenness. And of course, the scores are cynically weaponized by the school choicers and their enablers in the press in order to promote a failing schools narrative and advance the choice and anti-union agenda.  

But the scores do not mean what the screamers say they mean. In order to turn the volume up to 11, they compare the raw scores to previous scores, find a match twenty years ago, and announce twenty years of progress loss. The thing is, it's twenty years of test score progress, not twenty years of student progress. 

In an interview conducted by Liz Mineo for the Harvard Gazette, Dr. Andrew Ho, professor at the Harvard Graduate School of Education and former member of the NAEP governing board, explains:

[E]ach point on the NAEP scale is roughly three weeks of an academic year, and the overall decline was seven points, which is roughly five months in terms of an academic school year of learning that this cohort of students is relatively behind.

Five months. Not twenty years. But the average overall decline doesn't tell the whole story, of course. Predictably, students who were most challenged before the pandemic were most challenged by the pandemic. For students who were already struggling, who were already coping with insufficient resources--including educational resources--and whose families were likely to be most severely impacted by the pandemic, results were more alarming. From Dr. Ho:

But this average betrays the inequality where the decline for the higher-scoring students was only three points, or nine weeks, whereas the decline for the lowest-scoring students was 12 points, which is 36 weeks, which is almost approaching an entire academic year. 

Terrible. Unjust. Unacceptable. We need to address this, but this is not just a test score. This is a reflection of the racism and resource inequality that permeates our system. And still, the most impacted students lost "almost approaching an entire academic year." NOT twenty. 

The kids being told right now that the scores are down and the pandemic has cost them twenty years are being crushed for the objectives of the crushers.

Dr. Ho goes on to discuss the broader impact of the pandemic on students' lives apart from one set of test scores and reminds us what returning to "normal" means for students getting back to school. He also suggests the need for a "learning infrastructure," perhaps in the form of a "National Assessment of Educational Equity." In his words:

I’m not just worried about academic learning. I’m worried about the structures of educational opportunity and inequality that have increased over the pandemic; this is just the tip of the iceberg. I hope we can pay attention to the entire structure of educational opportunity. Academic learning outcomes are easy to measure, but also easy to overemphasize, as if students are only trying to get better at math and reading and not also trying to reconnect with their classmates and their teachers and remember how to sit still, listen, learn, and enjoy learning and playing with others.

Again, even among the most severely impacted, most vulnerable students, the setback was 12 points, 36 weeks, "approaching an entire school year." Which, after what they've gone through, seems predictable.

Not good, but not twenty years. All the screaming is premature. Before the additional scores (8th grade) and details in October. Before students have a chance to fully readjust to in-person schooling and reinvigorate their intellectual lives. Before students have had the chance to right themselves before we've given them a chance to catch up. For many of them, before they've had a chance to fully mourn their personal losses. 

Yet the drumbeat goes on. In the newspapers. Online. On the TV news. Among administrators. Sometimes friends and family shaking their heads and tsk tsking.

What can we do? I'm no longer on the job but when I was, I tried to follow this first rule of survival:

Do not fall for their bullshit. Because they tell you something is bad doesn't mean it is. Because they tell you scores are everything doesn't mean they are. Because they say your scores are down and you failed doesn't mean it's true. 

First of all, they are not your scores. They are your students' scores. And it's only a test. You know your students, and you know how little of their intelligence and talent and humor is reflected in any single test score. 

Nod your head, pretend to care, ignore the bs. Remember that most of the world doesn't understand your world, and 99% of the press reports and all the talk on the morning news is from people who either don't know anything or are deliberately lying for their own purposes. 

That goes for the bosses, too. Don't expect your admin to honestly tell you why you're doing this test or that test or why you're spending so much time prepping for them. Don't break your head trying to figure them out. Remember that they have a different agenda. They are speaking a language different from yours. Your concerns are not their concerns. Their only honest answer to "why" is the same as yours, the same as your students': Because we're told to. 

If you're in a position to resist, do so. Ask other,  "non-why" questions--they hate that. Challenge their analysis of the scores. Their interpretation. With the rest of the world who are not your bosses, push back. At parties, with family, on social media. And write letters to editors; that's still a thing. Write a blog. You're an educator, let's educate the ones who don't know better and confront the ones who do.

You are right and they are wrong. Don't let the bastards grind you down.


*h/t Peter Greene

**h/t Steven Singer