I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Monday, November 23, 2015

Blogging Update: Happy Thanksgiving!

Dear Readers,

Apologies, but work, school (NOT where I work), personal, and family obligations have kept me from blogging over the past week.

I've decided to avoid lame, over-quoting blogging (you know who you are) and take off the rest of the week. Pro-public education snark recommences after the holiday.

Have a happy and safe Thanksgiving!

Jersey Jazzman

Actual exponential growth...

Thursday, November 12, 2015

Proud Teachers Are Banned In New Jersey #ProudToBeAParsippanyTeacher

How dare these unionized teachers express pride in their profession!
Teachers Ordered to Remove “Proud to be a Teacher” Signs
“Pride in Parsippany”, boasts the great township sign at the intersection of Rt. 46 and Vail Road. The teachers of Parsippany’s public school children however, are not to be afforded the opportunity to exhibit that pride according to an edict issued today by the school district’s chief administrator. Teachers have been ordered to remove a simple black and white sign that says only “I am Proud to be a Teacher” in plain block letters, followed by a smaller tag line, “Parsippany-Troy Hills Education Association”.
The message, uniformly printed on standard white printer paper, had been posted on classroom doors in all district school buildings for several weeks before acting superintendent Nancy Gigante issued a directive that they be taken down. Gigante cited Board of Education policy forbidding political actions or statements on building premises as the reason for ordering the removal of the signs.
“We view this statement as a positive message to both students and parents”, said Joe Kyle, PTHEA president. “ We have always been proud of our role as teacher, coach, counselor and guide to the children of Parsippany, and we never intended for the sign to be anything other than a declaration of that pride.”
Kyle went on to say that the PTHEA would be challenging the validity of Gigante’s interpretation of the Board policy in question. “Our members were proud to go out of our way to willingly bail out the Board after the recent Middle School disaster, and so we are astounded that a printed declaration of that pride is deemed a political action statement”. PTHEA members worked diligently to restore a workable schedule in both Central and Brooklawn Middle Schools, going above and beyond their contracted duties. [emphasis mine]
A little history is in order here:

Way back in 2010, at the start of NJ Governor Chris Christie's War on Teachers, Christie decided that the biggest problem New Jersey faced was overpaid school superintendents. It was all a distraction from his chronic underfunding of New Jersey's schools, fueled by his tax giveaways to the wealthiest citizens of the state. Christie needed a villain to blame for his failure to fund schools according to the state's own law; he found one in Parsippany superintendent Lee Sietz.

Seitz, you see, had the temerity to expect that someone who was doing a decent job ought not to take a huge pay cut just because Chris Christie says he should. So Christie launched a disturbingly personal attack against Seitz for daring to making a low six-figure salary while running a large, successful suburban school district. Christie used Seitz to justify imposing a superintendents pay cap that has since chased talented school leaders out of the state.

Unfortunately, Parsippany has suffered from Christie's nakedly political ambitions. Seitz's replacement came into the district and immediately decided his top priority was to make sure his teachers were wearing the right sort of pants while at work.

No, I am not making this up. Click through -- I dare you.

When the Parsippany Board of Education announced Seitz's replacement would be resigning this year, there was actually applause. That's how bad things had become in Parsippany: this past year, the district's middle school was in complete chaos because the administration couldn't even generate basic school schedules.

So now Parsippany is currently operating under the "leadership" of an acting superintendent who appears to spend most of her time correcting the mistakes of her predecessor. You would think this acting superintendent, Nancy Gigante, would have larger concerns than worrying whether her teachers were proud of their chosen profession.

You'd be wrong:

Here are Passippany's teachers, following orders and throwing simple signs declaring their pride in their profession into the trash.

This is what you've come to, America: your teachers aren't even allowed to express pride in their profession in their own classrooms.

This is exactly the sort of school "leader" Chris Christie wanted to replace the experienced, professional superintendents he chased out of this state: petty bureaucrats who share his bullying tendencies. This is where public education is headed...

Unless we all stand up and say: "Enough." Stay strong Parsippany. On behalf of teachers across New Jersey and the nation: thanks to Joe Kyle and our colleagues in Parsippany-Troy Hills for taking on this fight.

Send this around and let everyone know that you are:


ADDING: It's worth pointing out that Scott Rixford, the superintendent who replaced Seitz and spent his days checking teachers' pants before resigning this past fall, was one of the top lieutenants in Christie's Department of Education when it was being run by Chris Cerf. Rixford was the executive director for one of the Regional Achievement Centers (RACs), Cerf's brainchild. The RACs, fueled by reformy billionaire Eli Broad's money, were supposed go into "failing" schools and turn them around with superior managerial knowhow.

Think about it: a guy who couldn't even produce a schedule for a middle school was the guy Chris Cerf and Chris Christie put in charge of telling schools how to improve.

Heckuva job, fellas...

Wednesday, November 11, 2015

Charter Schools, An Exchange: Part VI (Final)

This is the last installment of an exchange about charter schools between myself and Dmitri Mehlhorn. 

Part I: Mehlhorn's opener.

Part II: My reply.

Part III: Mehlhorn's response.

Part IV: My second reply.

Part V: Mehlhorn's second response.

Part VI (this post): My final reply.

I hope everyone has found this enlightening. My sincere thanks to Dmitri Mehlhorn for engaging me in this debate.

* * *

In his final installment in this debate, Dmitri Mehlhorn posed five questions to me. Here are my responses.

Q: "Jazzman, why don’t TPS (traditional public schools) test results go down as charters expand?"

A: We don't really know that they don't. But even if they don't, there are other pernicious effects for district schools when charters expand. 

As I explained in Part II, the evidence we have about the effects of charterization on neighboring public district schools is quite limited. The Texas study Mehlhorn cited earlier (Booker et. al, 2008) shows a small positive effect for hosting districts; however, the proliferation rate of charters was quite small. In that study, most districts had well under 5 percent of their students in charters, so it's reasonable to think any "cream skimming" -- the process where charters take only those students who are more likely to thrive under a "no excuses" model -- had only a small influence on the district schools.

What happens when the charter enrollments get up to 40 percent, as they are in Washington, D.C., or soon will be in Newark? We really don't know, and it's going to be very difficult to figure out the answer when it happens. The plain fact is that there are many confounding variables when trying to figure out the effects of charter school growth on sending districts' outcomes. Test scores may go up or down, but pinning the cause on charter schools is going to be a formidable task.

In Washington, D.C., for example, the past decade has seen large changes in the city's demographics. Can we really attribute any changes, positive or negative, to charter school proliferation when the city is undergoing such large shifts in its student population characteristics? 

In addition: how sure are we that the students enrolling in charter schools would have otherwise gone to public schools? There is good reason to believe charters are attracting at least some of the population of students who would have gone to private or parochial schools. That could explain, in part, why public district schools'  test scores remain static when charter enrollment grows.

All of this, however, is speculation. I and others will continue to dig into the issue... but let's be clear about one thing: test scores are not the only measure of the potentially pernicious effects of charter schools.

As I said previously in this debate: high quality research shows that charter schools have a negative effect on the budgets of their hosting schools. This makes sense, as charters are redundant systems of school administration, and do not allow districts to fully leverage economies of scale. In other words: when every charter school has its own high-paid superintendent and administrative staff, that's inefficient. And we have more and more evidence that is the price to be paid for "choice."

I'd rather see money go into the classroom. I'd rather we cut back administration as much as possible, and put our funds into hiring the people who actually educate our children. Having multiple systems within a single community runs counter to that goal.

I am also very concerned "no excuses" charters are reinstalling a hidden curriculum of obedience within our schools that is only being foisted on students of particular races and socioeconomic classes. I've cited Pedro Noguera several times during this debate, and I'll do so again right now: we don't teach kids in the suburbs the way "no excuses" charters teach urban students of color. Chanting and marching and teacher tracking and rote repetition is not the type of education suburban parents seek for their children. Why, then, is it acceptable for children coming from segregated, economically disadvantaged communities?

This has consequences outside of the charter schools. When the public schools feel the pressure of charter "accountability" breathing down their necks, are they more likely to open up their curricula to exploratory and experiential learning, or narrow them with a focus on test prep? Are they more likely to have their youngest students marching through the halls, or will they instead give them extra time for play? Will they allow their teachers and principals to have the autonomy with accountability of professionals, or force them towards pre-scripted instructional methods that are "proven" to be "successful"?

Those who define "success" through test scores are not likely to ask such questions.

Q: "Jazzman, why aren’t race and history relevant to the burden of proof here?"

A: They are -- and history teaches us school "choice" is often a front for school segregation. Further, the "choice" offered urban parents is not suburban "choice."

Here's an ugly truth: school "choice" has a segregationist history. Here's another ugly truth: today's "choice" plans appear to be leading to more segregation, not less. The Civil Rights Project at UCLA found that "that charter schools, in many ways, have more extensive segregation than other public schools."

I approach the idea of reintegrating schools with some caution as I think there can easily be a deficit perspective inherent in the idea: that somehow we need to "save" students of color by mixing them in with more white students whose families have the "right" values. That said, we also have to acknowledge that this country deliberately segregated itself, and that segregation extends to its schools.

I don't see that the charter movement has done anything to address this reality. The "high performing" charters we hear so much about in the press (thanks to those charters' extensive public relations machines) seem to be very happy to embrace a school system where urban "choice" means choosing between a racially segregated, underfunded public school and a racially segregated, better-funded charter school.

As I've asked throughout this series: is this really the best that we can do?

And let me state once again: suburban "choice" is not urban "choice." Contrary to the claims of those who don't seem to know much about how suburban schools work (more on that soon), suburban "strivers" do not go into separately governed school systems that abrogate the rights of students and parents. Disruptive children in suburban schools -- and yes, they are there, believe me -- do not get booted out of their schools except under the most extreme circumstances.

Those who point to the suburbs to justify the proliferation of charters are making a false analogy. As they do, poverty and institutional racism calcify, resources are distributed inequitably, and a pedagogy of compliance is foisted on the most underserved and deserving of our children.

Again: is this really the best we can do?

Q: "Jazzman, why don’t you push for good charters?"

A: I'd rather push for equity: equity of resources, equity of access, equity of lives.

There is a place for school choice if we adequately fund schools and equalize the lives of children. But choice is not a substitute for equity.

That's worth repeating, loudly:

Choice is not a substitute for equity.

Why would anyone put promoting school choice at the top of their social justice agenda when our schools are being massively underfunded? Why would anyone think expanding charters is a top priority when 22 percent of our children live in poverty, and 45 percent live in low-income households?

I am a capitalist. I believe a well-regulated market generally is the most efficient way to move capital where it is needed, and I believe ingenuity, talent, and hard work should be rewarded. I have no problem with people who create useful goods and services making lots and lots of money as their reward.

But America's economic system is leaving far too many of our families behind, and our children are paying the price. It is utterly shameful that the most powerful nation in the history of this planet allows so many of its children and their families to live in squalor while a select few gather up all of the income gains we have made over the past few decades.

The idea that a few more schools operating outside of democratic control can make up for this is, frankly, ridiculous. Who could possibly believe this? Who could possibly think setting up more of these schools is a serious response to our current situation? Who could possibly imagine that New Orleans or Newark will all of a sudden become an economic paradise where poverty and inequality are magically swept away by a few bumps in test scores thanks to charter schools?

It's long past time for the charter cheerleading sector to stop pretending they have found a large-scale solution to poverty. Too many in the charter industry pat each other on the rump because their schools put the pennants of elite colleges on their walls, never stopping to think for a moment that millions of people in this country are doing necessary work that does not require a college degree, yet are struggling to survive. Should they all go to Yale? Should all the truck drivers and brick layers and home health care aides get Ivy League degrees? Would that make things better?

Social mobility is not social equity. The minute you guys realize this is the minute I'll consider pushing for more "choice." Until then, I've got bigger concerns -- and so should you.

Q: "Jazzman, how much improvement would be enough?"

A: Hey, I'm not the one who said charters are closing the "achievement gap"; I'm only pointing out they aren't closing it.

Dmitri, you missed the point of my argument on effect size, so I'll try one more time:

The spokespeople for the charter sector -- not me, but them -- are the ones saying their schools are somehow proving that poverty isn't destiny. They are setting up the comparison between affluent and economically disadvantaged schools and students, not me. They are the ones saying their schools are closing the "achievement gap."

They are wrong:

Yes, higher test scores are better than lower scores -- duh. But tiny, incremental gains are not enough to close the "gap" that the charter cheerleaders say they are closing.

Again I ask: is this really the best we can do? A few more answers correct on a bubble test? For a select few students who manage to make it through the gauntlet that is a "no excuses" school?

A fully functioning, well-funded, and equitable education system is a necessary precondition for a just society. We need good schools if we are going to lift people out of poverty; nobody disputes this. But you can't expect schools to solve the problems of chronic poverty on their own.

This, however, appears to be exactly the argument the charter sector keeps making for itself: expand charters and the effects of poverty on school outcomes will simply fade away. Except the evidence shows it won't. So let's stop pretending that school "choice" a some sort of substitute for a meaningful, wide-scale program of social and economic justice. Clearly, it isn't.

Q: "Jazzman, do we already have some areas of agreement? If so, what do they mean?"

A: We'd be more likely to find agreement if your side would stop bashing teachers unions and public schools.

Here's the reformy LA Times, claiming charters are "...free of sometimes stultifying union rules." Here's a charter staffer saying her school is "Free from union contracts..." Here's a pro-charter film excoriating the "...sorry statistics of union-fueled schools..." Here's the NY Post saying that the UFT, NYC's teachers union, is "...associated with yet another education failure" because its charter school "...can’t compete with the non-union charters." Here's a report from the "liberal" MSNBC that claims charters are "...unencumbered by teachers unions or antiquated rules and regulations that can choke innovation." Here's a pro-charter think tank claiming charters are better off when they "... reject the fixed salary levels that have been comfortably adhered to and influenced by teachers unions."

Time and again, the charter industry has stated, both implicitly and explicitly, that their "freedom" from teachers unions is a key part of their success. I've been looking at this a good long time, and I can only find one theory that explains a charter advantage that comes from having a non-unionized staff:

That fraction of the charter sector that is "high-performing" has schools that churn their staff and keep experience levels low. They are, consequently, able to keep staffing costs low compared to the public schools in their host communities. Experienced teachers on traditional union-negotiated step guides make considerably more than their charter counterparts, but that doesn't matter much for the charters, as they have only a few teachers with experience into their second decade and beyond.

These charters are able to offer higher starting salaries than the neighboring district schools. For that higher salary, teachers work longer hours and more days, allowing the school to spend more time drilling its students and, in some cases, offering extracurriculars. Consequently, the children score higher on test scores.

That's it. That's the "non-union charter advantage": a less-experienced staff working longer hours, incentivized by higher starting salaries but ultimately having shorter teaching careers.

This is a smart strategy from a business sense, but I think it's ultimately bad for the teaching profession to move more and more educators into what is essentially temp work. Further, I don't think this is scaleable. "High-performing" charters, in my opinion, are going to exhaust their human resource pool soon; then what?

In any case, there is no reason to believe charters have found some magical efficiencies because they don't have to worry about unions stifling them. Those who make that case are engaging in unwarranted union bashing instead of honestly and carefully analyzing how charters really get their test score gains.

The charter industry has also built the justification for its existence on claiming our public schools have "failed." Not only is this extremely unhelpful in garnering the necessary financial support our schools have been lacking; it's also unjustifiably sanctimonious considering that charter schools do not do he same job as public schools.

Even charter operators like KIPP, who pride themselves on attempting to serve the most disadvantaged students, do not follow the same enrollment procedures as public schools. In Camden, KIPP decided it would only enroll kindergarten students in its first year and grow each year after that. But the Camden Public Schools don't have that luxury. Neither can they, like Mastery's and Uncommon's schools, pick and choose which grades they would like to serve and how many students they would like to enroll.

Success Academy's attrition practices are now well known; however, even if we put them aside, Eva Moskowitz herself says she won't take new students into her schools after Grade 4. She says it's "not really fair" to the higher-performing students to be slowed down by the lower-performing ones who might come in. Again, that's fine for her schools, but the public schools she is muscling out through colocation don't have that option.

As I've said before, I am sympathetic to the point of view of urban parents who want the positive peer effects they see suburban families enjoying for their own children. I've never criticized a parent for enrolling their child in a charter school, and I never will. But it's unfair to seek out that peer effect while simultaneously claiming public schools are "failing," even as they serve a different population of students.

The charter industry has made a lot of noise lately about putting aside the district vs. charter battle and moving on to areas of agreement. If their spokespeople are sincere, however, they ought to stop beating down teachers unions and public schools. I'm not saying unions are always right, and I'm certainly not saying urban public schools don't need a lot of improvement, starting with -- but certainly not limited to -- more resources.

But justifying your existence by continually wagging your finger at the teachers and the schools that are doing a different job than the charters is, frankly, obnoxious. My charter school friends: a little humility would go a long, long way.

Thank you, Dmitiri, for this exchange.

Wednesday, November 4, 2015

Charter Schools, An Exchange: Part V

Dmitri Mehlhorn and I continue our exchange about charter schools.

Part I: Mehlhorn's opener.

Part II: My reply.

Part III: Mehlhorn's response.

Part IV: My second reply.

[ADDING: Here is the rest of the exchange:

Part V: Mehlhorn's second response.

Part VI (this post): My final reply.]

Leave your thoughts in the comments below. My final reply will be up shortly.

* * *

Five Questions for Charter Critics
Jazzman-Mehlhorn Episode V

Reform skeptic Jersey Jazzman is hosting a debate about charter schools. He and I have spent 4 posts and about 10,000 words discussing the past few years’ worth of research by charter advocates, detractors, and academics. We will probably have a “long tail” of comments in social media, but this is the final formal installment of my appeals to Jazzman and the other skeptics who have engaged over the past year or so, including Sue Alexander, Dr. John Thompson, Deb Stahl, Jack Covey, Ajay Srikanth, Daniel Katz, Greg Clark, Mindy Rosier, Ben Spielberg, Peter Greene, Ben Faber, Randi Weingarten, Dr. Bruce Baker, Gail Richmond, Ed Harris, Jennifer Berkshire, and others.

If you’re up for it, I have five questions for all of you. Of course, this is not an exhaustive list – we could talk for years about test scores, poverty, fixed costs, and other issues. But in the spirit of trying to make progress in at least some areas, I’d be grateful for your honest take in these five areas, not conflated with other issues.  

(1) Jazzman, why don’t TPS test results go down as charters expand? 

Mark, the skeptics’ most common and most powerful arguments involve “skimming.” Compared to TPS [traditional public schools], these arguments claim that charter populations qualify more for reduced-price rather than free lunch (i.e., mild poverty rather than extreme poverty); charters push kids out more often; charters don’t always backfill open seats; charters deliver peer groups rather than better education; charters don’t serve as many SPED [special education] or severe SPED students; charters only serve students whose parents are motivated enough to apply; etc. These claims have elements of truth, but the key question is their aggregate impact including offsetting issues.

Mark, do you acknowledge the role this claim plays in the architecture of charter skepticism? After all, if skeptics are right on this issue, charter test-score gains are illusory, and thus charter expansion will hurt TPS by leaving them with a hard-to-teach cohort. If skeptics are wrong, then the charters’ test-score gains result from something the charters are doing, and charter expansion does not damage TPS by leaving them with a hard-to-teach cohort.[i]

Mark, as you know, both sides have invested heavily in studying TPS student achievement as choice expands. I won’t revisit all prior posts, but you know I’m looking at Washington DC; New Orleans; Florida; some boroughs in New York City; and some districts in Texas.

Mark, where is the evidence backing up the idea that hard-to-teach cohorts will be increasingly concentrated in TPS, driving down results? Is it possible for you to acknowledge the weight of the current evidence on this subject?  Will you have the courage to articulate a testable explanation for why the facts have repeatedly been the opposite of skeptics’ predicted results? 

(2) Jazzman, why aren’t race and history relevant to the burden of proof here?

In your most recent piece, Mark, you characterized my burden of proof argument as follows: “if someone can gather up a few parents who want something, they should get it, absent a compelling reason that they shouldn’t.” I would like to better understand why you and other skeptics are so quick to dismiss the racial and historical issues suggest a different presumption.

Let’s start with race. On average, graduation rates, test scores, and other metrics suggest that African American students and Hispanic students are poorly served in existing TPS, especially by comparison with white, Jewish, and Asian students. Parents in African American and Hispanic communities often speak out on the topic of school choice, in interviews, focus groups, polls, and charter applications. I have cited some of the results in previous posts, but the short version is a lot of these parents really want more charter schools.  Tens of thousands of parents (or more), who bear the legacy effects of centuries of state-sponsored racism, should not be dismissed as “a few parents” who’ve been “gathered up” by “someone.” Their agency and their numbers suggest a different presumption.

Turning from race to history, your post refers to Russell’s Teapot. That is a perfect frame for what many parents of color say. For white, Jewish, and Asian parents in TPS, choice and the threat of choice long ago enabled meaningful control over their children’s schools. Thus, current anti-charter laws have unique and disparate force among low-income black and Hispanic parents, who become compelled to send their children to struggling TPS.  The anti-charter policy therefore amounts to the following: “I, Jersey Jazzman, offer you the Russell’s Teapot monopoly public school system. This public school system is a century-old experiment that has not yet delivered results for your community. We cannot give up, however, until we try to supplement those schools with more social services, more resources, and other innovations. Until every possible test has been tried (other than charter schools of course), you will be required to send your community’s children to these TPS into the infinite future.”

Mark, this is not a choice between two impossible-to-assess claims.[ii]  As you know, we are not talking about authorizing a specific charter, but rather laws that limit the expansion of *any* charters. As you note, our nation has gathered considerable evidence about this question.  We know that nonprofit and private actors compete with public actors for public dollars in other sectors, including healthcare, energy, and military procurement. We know that wealthy parents already have extensive school choice.  In light of that, why isn’t the burden of proof on the monopoly public school system, which overwhelmingly employs relatively prosperous white people, and which was constructed without meaningful input from (and often deliberately in opposition to) communities of color? 

(3) Jazzman, why don’t you push for good charters?  

You seem to agree that the best charter school chains can deliver results.  Why don’t you push to replace charter caps and limits, like we have here in Virginia, with enlightened charter authorization policies like those of Boston? 

(4) Jazzman, how much improvement would be enough?   

If you are open to facts, I am trying to understand what evidence would be sufficient for you to prove the theory that charters actually work better than TPS for low-income students of color.   
You dismiss the CREDO 2015 results in a few ways, but I think you’re missing the forest for the trees. To try to clarify the issue here, let me try to stipulate as many of your objections as possible.

First, let’s talk about whether a fraction of a standard deviation is meaningful.  You don’t like my healthcare example, and we can’t seem to agree on human height,[iii] so let’s use the market returns example that you approved. You said that a difference of 0.1 standard deviations was not a big deal, because “if my investment goal is to make a million dollars, getting $275 over $232 is not of any practical significance for me” (emphasis yours). Jazzman, that’s more than an 18% difference in end-of-period wealth, and I used a starting number of $100 just to make the math obvious. The median net worth of a household in America today is about $81,000; using that as a starting investment delivers a 10-year difference of $35,000 in extra wealth. And remember, 10 years still a shorter period of time than a child spends in K-12 schools. 

That said, you suggest that 0.1 or .11 standard deviations is not the right benchmark, because those are the aggregate average gains Hispanic ELL students, who seem to benefit especially from charter schools.[iv] So, let’s broaden to include all black and Hispanic students in poverty, who benefit by less in math (.08 and .07) and reading (.06 and .04).[v]  If you apply those numbers to the financial market calculations from the prior paragraph, you get gains of ~ $27,000; $24,000; $20,000; and $13,000.  To me, none of those are small potatoes, but the real point is that it depends on your perspective. Parents and policy-makers would certainly be on safe ground in concluding that gains of that magnitude are worth embracing. Any instrument that reliably delivered those kinds of gains would instantly become the industry norm on Wall Street.

You argue that the effect sizes of charter schools are merely “a few more bubbles filled in correctly.” Yet Mark, a few bubbles per year on average, across a large population, represents marginal additional mastery every single year. It’s not a perfect or complete measure, but it may just be the difference for a kid on the edge. Maybe the best way to frame it is not that students advance faster; rather, maybe it’s that they don’t fall behind. Either way, even if you and many others do not think that those bubbles matter, millions of educators and parents care about them.

All of which leads back to the original question: how much would be enough?  What would charter schools have to show you for you to accept the theory that these schools, because they do not have to deal with the union contracts and other bureaucratic hurdles, actually deliver better results for low-income children of color?   

(5) Jazzman, do we already have some areas of agreement? If so, what do they mean?

For policy-makers interested in choice policy, it seems as though we might agree on a few things.  Having read your work, and that of many other conscientious charter skeptics, I thought I’d take a stab at areas of agreement, and see if you could agree to any of them, or articulate an alternative package of agreements, and then suggest what policy-makers might learn from those areas of choice when deciding whether or not to maintain charter caps. 

For my part, I agree that for suburban areas, and for white, English speaking, prosperous children, traditional public schools currently perform as well as charter schools, or better. I agree further that, even for poor students facing residential segregation, charters alone will not eliminate the achievement gap between the children of wealthy families and the children of poor families (although charters will at least start to equalize the narrow issue of school choice, which wealthy families have and these families do not). I agree that some charter schools have been much more effective than others, and that charter authorizers should therefore be careful in expanding charters and aggressive in closing them. I agree that numeracy and literacy, as measured by test scores, are not a complete measure of student achievement or school performance, and that therefore over time we need to monitor lifetime student outcomes.   I certainly agree that schools alone won’t solve all of humanity’s problems, from wealth inequality to violence to sickness.  I also agree that more money, if spent properly on students, could help students obtain better outcomes. 

I have not been able to identify your explicit concessions, as they often appear as caveats or rebuttals.  I believe, however, that you acknowledge the shameful history of American traditional public schools, including the consequences of residential segregation, and the racial motives of public actors who built the modern K-12 system from the late 1800s to the early 1900s. I am confident you condemn situations where TPS student discipline is turned over to physically violent police, or where teachers are abusive, just as I would condemn those situations in charter or private schools. I bet you also would acknowledge that wealthier parents use both political power and the threat of choice to navigate traditional systems for the benefit of their kids.  I suspect you agree that not all TPS expenditures have been wise, or for the benefit of students – indeed, you may even agree that public sector traditional public schools have an extensive history of fiscal waste, fraud, and abuse.  I suspect you also agree that public funding, multi-sector provision, and private citizen choice works in some areas, such as public services in Denmark, or healthcare provision (as opposed to healthcare finance) in the United States. 

Is that a fair summary, Jazzman?  If so, can you think of any compromises to recommend to policy-makers who might be reading this exchange? 

* * * *

Thank you again for your platform and the patience of your audience.  I hope this has moved a few minds, or at least opened up a few lines of communication. 

[i] I agree there remains the fiscal challenge of spreading fixed costs over fewer students. As I’ve noted, this is a real issue, but it’s a very different and more focused conversation than the skimming allegation.  Moreover, if we dive into the fiscal data, I believe we’ll find the same result: that the aggregate impact of charter expansion on TPS fiscal management is net positive, not net negative.  We’ll save that for another conversation, however.
[ii] When you ask the question: “is this really the best we can do?”, you are setting up an absurd standard.  By that standard, **no action** would ever be taken, ever, because no single action is ever by itself the best we can do.  The question at hand is whether we should expand charters or not. 
[iii] On height, the question is not whether the gain would make you an NBA basketball player, but whether most people would care. In the world of cosmetics, a product that delivered an average height gain of one-third of an inch would become a multibillion-dollar product. Also, when you double down on the Loveless / Spielberg examples, you use final adult height, rather than what clinicians and biostatisticians refer to as “height velocity” – i.e., how fast you grow. Height velocity is the apt comparison for growth over a long period, and standard deviations of height gain are pretty wide. 
[iv] Your protests about my focus on Hispanic ELL are a little bit overwrought, as I suspect you realize on reflection. As I have written in the past, this is the national cohort relevant to my daughter’s friend from soccer, whose personal story you entirely ignore in your rebuttal. It is also the most clear case for charters, because these parents are least likely to have either voice or choice to navigate TPS, so you’d expect charter gains to be especially stark.  To be clear, the Hispanic ELL population sub-group is an aggregate result, as it includes the results from those cities whose charters perform very poorly, as well as for those cities whose charters perform well.  If I had isolated only the better-performing cities or better-performing charters, the result would be much greater.
[v] The cohorts of students of color in poverty are the benchmark relevant to our debate, because those are the students who are least well served in the status quo. Of course I care about the other students, but those generally are not as poorly served in the status quo.

Sunday, November 1, 2015

Moskowitz, Petrilli, and the Hard Truth About America's Schools

I'd be lying if I denied I haven't taken some measure of satisfaction in watching the perpetually self-impressed Eva Moskowitz implode this week. Now that we know for certain that her charter schools do, in fact, counsel out "disruptive" students, we can drop any pretense that she is some sort of a pedagogical genius.

The truth is her descriptions of how she has changed water into cartons of milk never held up to even the mildest of scrutinies. Earlier this year, she gave an interview on local public television where she bragged on how much "fun" learning is at Success Academies:
(1:48) But imagine if schools acted as if children had the freedom to choose whether or not to go to school. In fact, they had the freedom to walk out of the classroom at any moment -- now, they don't at our schools (interviewer chuckles), but imagine if they did. I would venture to argue that at most schools in America, they wouldn't come and if they came they would leave, because boredom, unfortunately, is one of the most common experiences of children. 
And what we do is, we engage children with rich, wonderful literature, with discovery-oriented science five days a week, with chess and debate and coding and art and music and sports, so that kids love being at school.
Uh-huh. Let's go back to this week's NY Times article, the one that caused all the fuss:
Some of the parents whose children were on the “Got to Go” list acknowledged that they did not agree with how the school managed behavior. But several also said that both before and after the list was created, they thought school and network employees were trying to push them out. 
Folake Wimbish said her son, who has attention deficit hyperactivity disorder, was suspended 19 times last year, in first grade, and missed 26 days. Success said her son was intellectually gifted but struggled with behavior, “often hitting, kicking, biting and spitting at other children and adults.”

In early December, while Ms. Wimbish was pushing the school to evaluate her son for special education services, she was called to a meeting in Lower Manhattan with the network’s assistant general counsel and its associate special education manager, Julie Freese. She said Ms. Freese told her that, because of his suspensions, her son was missing out on his education, and she needed to think about his well-being.
“She said, ‘Why don’t you just put him in another school, because he’s suffering,’ ” Ms. Wimbish said. 
Ms. Wimbish withdrew her son at the end of the year, because with the suspensions and calls to pick him up, she said, “I started feeling like I was going to have a breakdown.” He now attends Public School 119 in Brooklyn, where Ms. Wimbish said he was very happy and had not been suspended once.
Sounds like Eva's magical book choices didn't work out so well that time, huh?

There have been uncorroborated reports about drilling-and-killing in Success Academy schools. Those reports seem to line up, however, with what we know about SA from the dog-and-pony shows they put on for the press:
Touring a first grade classroom on the immaculate third floor of a co-located building on 140th Street, visitors were, sure enough, shown displays of rigor intermingled with joy.
In one first grade classroom the children, who were looking at a passage from "Charlie and the Chocolate Factory" and trying to create mental images of characters, sat with their hands folded in their laps, eyes tracking the teacher, and backs straight. 
Students who are not sitting correctly or who fidget are asked to change their posture in front of the class, and at least one student got a “check mark” for bad behavior for not keeping his hands folded in another classroom. (Three check marks result in a time-out, five in a written letter of apology, and so on.)
Students were often instructed to “sit like a professional.” A sign on one wall read “$cholar dollar fines” and noted the monetary charges for various misbehaviors: $1 for an untucked shirt, $5 for “not loud and proud,” and up to $10 for talking during a “zero” noise period. (Success students are called scholars.) 
The message for principals: Getting students under control creates the conditions for academic achievement. 
Students’ grammar was corrected on the spot, as were teachers’ missteps. Harlem Success 5 principal Khari Shabazz, a fixture at Success’ events, frequently gave feedback to teachers as they were teaching, asking the teacher to phrase a question in a different way, or to let a certain student talk more. 
Leaders at Success and other charters occasionally use walkie-talkies to instruct young or struggling teachers from the back of a classroom. In a third grade math class, students stood to read aloud word problems, or “number stories," a Success-specific phrase.
After a student correctly read a problem aloud, his classmates shouted, in unison, “His brain is on fire!” 
Students were particularly engaged in math classes, straining to be called on and enthusiastically creating models and pictures to complete division and multiplication word problems. 
“I think this problem is way past what you can do,” Shabazz said to one class of third graders, apparently challenging them to prove him wrong. In the back of the class, a fidgeting student raised his hand for help or attention. His teacher took him aside and said, “Look at me and get it done.” When the student raised his hand again, Shabazz simply pointed to his watch, indicating that there were only a few minutes left to solve the problem, as displayed on a timer projected onto the smartboard. The student looked distraught. [emphasis mine]
This is what Eva Moskowitz thinks children will choose if you let them. This is what she thinks is the antithesis of boredom. This is what she describes as "joyful."

As well as the reporter who wrote this, Eliza Shapiro. I suppose it's possible the "fidgeting" and "distraught" student was an anomaly... but then that's sort of the point, isn't it? If your child likes this sort of learning, terrific; if not...
Monique Jeffrey said her son, who was in kindergarten last year, was suspended so many times she “stopped counting.” In the middle of the year, Ms. Jeffrey said, the school’s education manager, Rebecca Fleischman, told her that her son had emotional and behavioral issues the school could not handle and that she should look for another school. Ms. Jeffrey withdrew him at the end of the year. 
Nicey Givens, the mother of another student on the list, said her son, also a kindergartner last year, was suspended many times, in some cases, the school told her, for fighting. Ms. Fleischman said in an email that a special education committee of the school district recommended that the boy be placed in a type of special education class the school did not offer in his grade. Ms. Givens recalled that Ms. Fleischman told her the school did not have the resources to serve her son and offered to help find him a placement in a regular public school. Her son now attends P.S. 287.
Eva Moskowitz has taken plenty of gratuitous swipes at America's schools, its teachers, and its colleges of education. She has pretended to have the answer: More chess! More feedback from principals! Rigor!

It all turns out to be a steaming pile of dung. But if you had been actually listening to the tripe she was spewing, and you were intellectually honest, you would have already known that. For example -- and to his great credit -- here's Michael Petrilli, one of the country's top reformsters:
American Federation of Teachers president Randi Weingarten is crowing about the allegations that Eva Moskowitz’ Success Academies charter schools regularly suspend or counsel out disruptive students. “Eva touts Success Academy’s ‘equity’ and record,” Weingarten tweeted, “while blasting public education. Now we learn her secret — pushing kids out.” 
What makes this sort of demagoguery more disappointing than usual is the nature of the issue at hand. As Weingarten’s own members know all too well, classroom disruption is a major problem. In a Public Agenda survey, 85% of public school teachers said that the experience of most students suffers because of a few chronic offenders. 
Addressing this challenge — enabling serious learners to learn, without denying serial disrupters opportunities to resume learning — takes smarts (finding better approaches to school discipline) and guts (making some hard choices). Unfortunately, today’s shrill debates are encouraging little of either. 
Everyone agrees that suspensions and expulsions should be rare. Kids who are repeatedly suspended or expelled are more likely to lag academically, drop out of school and end up in the criminal justice system. The direction of causation is hard to untangle, but, still, overly harsh discipline is worth curbing. 
But what the school-discipline reformers never talk about is the impact that disruptive behavior has on classmates. As common sense — and solid research — tells us, that impact is real and harsh. A study published in the journal Education Next found that “troubled students have a statistically significant negative effect on their peers’ reading and math test scores” and that “the addition of a troubled peer also significantly increases misbehavior of other students in the classroom.”
Again, to his credit, Petrilli has been remarkably consistent about this, even if he is reluctant to admit the corollary to his theory: the "high-performing" charters don't actually have a pedagogical secret sauce. Peer effects matter, and they explain at least part of "success" of these charters. Add in more resources, longer days/years, staffing structures that maximize efficiencies (but largely aren't replicable), and, in some cases attrition and selective enrollment policies... it would be remarkable if these schools didn't get better test-based outcomes.

The issue, however, is the price to be paid for what are largely marginal gains in test scores (especially when compared to the effects of socio-economic status on outcomes). Peter Greene, as usual, is quite perceptive in his answer:
Petrilli's point is not completely without merit, and as teachers often lack sufficient time and resources, many do perform a certain amount of educational triage by considering which students need us most. And every teacher knows the frustration of having a classroom tyrannized by one serial disrupter. But "prioritize" students? That sounds like a level of judgmental school administration that I'm not comfy with, and I suspect would provide an avenue for biases and concerns for compliance to run roughshod over actual care and concerns for the well-being of students. 
Look-- Success Academy is not nobly rescuing the top strivers from difficult situations. They are picking winners and losers based on the school's preferences and the school's convenience, based on Moskowitz's two guiding values-- compliance and test scores. When a six year old cracks under that sort of misguided pressure, that's not revealing some sort of character deficiency or lack of striverness. It's revealing an institutional incompetence in dealing with six year olds. 
But I appreciate Petrilli's willingness to just say it-- charters are only for the chosen few, those that the school finds deserving. What I'd really like to know next is how a system in which a school is the final arbiter of what level of education a child deserves fits together with the reformy ideals of school choice?
Amen -- but let me extend the thought a little further:

I've made clear in the past that I teach in an affluent, well-resourced school district. My own children attended public schools in a similar district. And I can tell you disruptive students are not an issue exclusive to urban schools serving large numbers of children in economic disadvantage.

Every teacher deals with children who don't listen to and follow directions, who are noisy, who distract their fellow students, who show disrespect (intentionally or otherwise), and who have a variety of personal characteristics that impede the smooth functioning of a classroom and a school. Sometimes these children have profound learning disabilities that impact not only their own learning but that of their classmates. Sometimes they clash with teachers or fellow students who may not work well with a particular sort of person. Sometimes their home lives are less than ideal, even if they are not disadvantaged.

Suburban schools, however, only counsel out these children as an absolute last resort. Only children with the most severe emotional or intellectual disabilities are placed out of district in the leafy 'burbs, and almost always after protracted attempts at accommodating them within the schools in their districts. And the kids who call out without raising their hands or who don't "track" their teachers are not considered threats to the learning of others; they are treated as children who teachers must patiently and persistently educate.

This is yet another example of why suburban "choice" looks nothing like charter school "choice." The expectation in affluent, highly resourced districts is that all children, regardless of their special education status or behavior in class, will be educated by the district. Yes, you will find tracking and gifted programs, but the children will all be in the same building with the same access to facilities and resources and extracurriculars. And if you ever tried to change that -- to put the "disruptors" into a separate building, away from their peers -- there would be hell to pay.

Petrilli believes that the disruption in urban schools has gotten so bad that we have no choice but to isolate the "strivers" from everyone else. Why they need to be in schools that are not state actors and therefore do not extend the same rights to students and parents, Petrilli doesn't say. All he knows is that it's not fair to keep those kids who can sit quietly with their hands in their laps in the same class with kids who call out.

What he never stops to ask, however, is:

1) Are urban kids, particularly students of color, really acting out that much more than kids in suburban schools?

2) If so, why?

First of all, we know for a fact that school punishments are distributed with a clear racial bias; on this there is no question. So before we start suggesting that urban schools are out of control, maybe we ought to question the prejudices too many of us bring to the discussion.

Second, as Jose Vilson has written about extensively, our schools still suffer from cultural incompetency and -- let's not mince words -- straight up racism. To be clear: this is not at all a problem confined to charter schools. But it is clear that we are nowhere close to where we should be when it comes to how we treat students of color. 

Which brings us to a third point: why would we ever be surprised that there is friction in our urban schools given the way we ignore the needs of their students? This nation purposefully segregated its citizens. It then refused to adequately fund its urban schools, even as it ignored the needs of children outside of their schools. It then installed into those schools a hidden curriculum of obedience, even as affluent suburban children benefitted from schools that served as engines of social replication.

All this, and then we're shocked -- shocked, I say! -- to find that students in urban schools think they're getting a raw deal. Most channel their frustration in positive ways; are we surprised that some do not?

Again: there's plenty of reason to believe the "problem" of disruptive children in urban schools is overstated. But even if we think it's only marginally worse than in suburban schools, don't we have an obligation to take a step back and examine what might be creating these problems? And if some charter schools are having a problem with discipline...

... maybe we ought to ask: is separating the "strivers" from the "disrupters" and putting them into schools that suspend them for the tiniest infraction really the best we can do for these beautiful and deserving kids?

These are hard questions. Asking them unmasks hard truths. Those truths leads to difficult conversations. But if we really care for the education of children in urban schools, we'd better start having these conversations, and stop swallowing the facile nonsense of Eva Moskowitz and others who have been shown, time and again, to be little more than self-promoting hucksters.

ADDING: More from Daniel Katz:
Moskowitz is losing total control of her situation.
Success Academy is run in a very particular way.  It has a dynamic, forceful, and very visible personality at the top of the organization.  The policies, tone, and demeanor of the organization flow entirely from that person who exerts an extraordinary level of control of the operation right down to the classroom.  There is a very narrow band of acceptable behaviors and attitudes.  Teachers who embody those behaviors and attitudes can rise very quickly with some becoming school principals in their mid-20s, and students who do similarly well are rewarded with toys and other goodies. Those who do not thrive are subjected to rigorous and frequent “corrections” that either mold them into proper form or convince them to leave. The network has an arguably paranoid attitude towards “outsiders,” frequently declaring to themselves that figures in the press and public are out to get them because they have cracked the code and are disruptors of the status quo.  Those who leave and speak out about the network’s inside information are viciously attacked.
But Success Academy has grown far too large to keep the lid on everything now.  Moskowitz enrolls 11,000 students in 34 schools.  She has around 1000 teachers and staff.  With such numbers and given their policies, there will likely be 1000s of former “scholars” and 100s of former teachers in short order, and all of them are not going to be intimidated into silence about what they saw while there.  The simple fact is that Moskowitz absolutely cannot keep total control over what people say and know anymore, and it is her own policies of driving away students she does not want and burning out teachers that has put her in this position.  So even if she fully recovers from this month, I think it is likely we will see many more months like this.
The next couple of years will be interesting.

ADDING MORE: If a guy from the reformy Fordham Institute can admit this, why can't everyone else?
“I see nothing wrong with charters functioning as a poor man’s private school. I see much that’s right about that,” said Robert Pondiscio, a senior fellow at the Thomas B. Fordham Institute. But he maintains that the schools should be more transparent about their competitive advantage. “What’s not right is to say, ‘We are just the same as any other public school,’ when you are not behaving the same way as a public school.” [emphasis mine]
Is this really so hard to admit? By the way:
Lindsay Kelly, a spokeswoman for KIPP DC, said that KIPP schools backfill when seats become available in all grades as a matter of principle. But they do it “in the least-disruptive way possible,” she said — enrolling them only at the beginning of the school year, a common practice in charter schools.
So you're not doing the same job as the public district schools, by your own admission. Good to know.

Thursday, October 29, 2015

Charter Schools, An Exchange: Part IV

Dmitri Mehlhorn and I continue our exchange about charter schools.

Part I: Mehlhorn's opener.

Part II: My reply.

Part III: Mehlhorn's response.

 Leave your thoughts in the comments below. We'll go one more round before we finish this series. Again, my thanks to Dmitri for participating in this debate.

ADDING: Here is an index for the entire exchange:

Part I: Mehlhorn's opener.

Part II: My reply.

Part III: Mehlhorn's response.

Part IV: My second reply.

Part V: Mehlhorn's second response.

Part VI (this post): My final reply.

* * *

Dmitri Mehlhorn starts off his latest chapter in this debate by quoting me:
“As Jazzman wrote to me: ‘Almost everything you say is factually correct; however … you arrive at conclusions that simply are not warranted.’”
Notice the ellipsis? Here’s what I said in its entirety:
Your essay is, in many ways, a perfect example of "reform" thinking. Almost everything you say is factually correct; however, you are so selective in your citations, and so lacking in context when discussing them, that you arrive at conclusions that simply are not warranted.”
Reads a little bit different when you get the whole quote, doesn’t it?

In a way, I’m glad Mehlhorn did this: unwittingly, he has given a perfect demonstration of why the charter sector's arguments in favor of the rampant expansion of charter schools simply do not hold up. He cherry-picks his data and places it completely out of appropriate context. Then he mischaracterizes my arguments against his position by setting up straw men, stuffed with poor paraphrases of my words. In addition, he misinterprets the research he himself cites, creating the impression that charters in the aggregate are getting practically significant gains.

To top it all off, he brushes aside my concerns about the pernicious effects of charters, as if the real fiscal problems that come from having redundant systems of school management can be ignored if there are meager gains in test scores, which are crappy, noisy measures of learning to begin with.

I’m going to dismantle Mehlhorn's points one at a time; afterward, I’m going to summarize why I see the argument for charter expansion as little more than a distraction from doing what really needs to be done to address the growing inequality, chronic poverty, and endemic institutional racism from which our nation currently suffers.

Fair warning: I'm going to be blunt. That doesn't mean I don't appreciate Dmitiri's time and effort in having this exchange.

The Burden of Proof

As Ajay Srikanth points out in the comments below the last post, Mehlhorn has misinterpreted the use of “strict scrutiny” in his defense of charters. I do, however, agree with one point he makes: anyone who advocates for a particular policy should be able to give a “compelling public rationale” for it. What Mehlhorn fails to apprehend, however, is that the burden of proof is not on those who express skepticism about a policy; it’s on those who take an affirmative stance for it.

Let’s suppose I’m a state official, and Dmitiri comes to me with an application to start the Russell’s Teapot Charter School of Excellence. I look at his application and say: “Sorry, you haven’t shown me a ‘compelling public rationale’ for approving this program.”

It isn't sufficient for Dmitri to say simply say: “Well, you can’t prove my charter school WON’T be successful! And I’ve got parents lined up to enroll their kids! You MUST give me the charter if you can’t prove I WON’T have a great school!”

Would anyone think this was reasonable? Yet that’s exactly the position Mehlhorn takes: if someone can gather up a few parents who want something, they should get it, absent a compelling reason that they shouldn’t.

This is a terrible way to guide any public policy, let alone education. Charter school advocates are the ones who need to demonstrate why they should be allowed to run schools that are not state actors yet continue to receive public funding. And we certainly have enough evidence available at this point to make reasonably informed judgments about the role charter schools should play going forward; we should examine it and see if the arguments of the charter sector for widespread expansion hold up.

Of course, we have to interpret that evidence correctly. Unfortunately, Mehlhorn, like so many “reformers,” demonstrates he is either incapable or unwilling to do just that.

The CREDO Studies, Again

Mehlhorn once again cites the CREDO studies of charter school effects as evidence in favor of charter expansion. Here’s his argument (without any ellipses):
Is it just some cities, or is it a national story? Let’s go back to Stanford University CREDO’s 2015 study of 41 urban regions. Jazzman is right that results vary by city. For instance, charters in Newark, Detroit, and DC do better than traditional schools; in Las Vegas, Fort Worth, and West Palm Beach, charters do worse. But what is the aggregate result for students who otherwise don’t have choices? What about for students like Sally, who is in the cohort that CREDO refers to as Hispanic ELL?  Nationally, across both well-run and poorly run city charters, charters improve literacy and numeracy for Sally’s cohort by 0.1 standard deviations over their matched peers in traditional schools.” [emphasis mine]
Notice that Mehlhorn asks what the “aggregate result” is for students; he then cherry-picks the highest performing subgroup’s results, and not the aggregate he himself called for.

The subgroup effect sizes in the 2015 CREDO report are found in Table 5, pages 17-18. Here, in its entirety, is the table:

I’m sorry to be blunt, but this is really shameless. Mehlhorn isolates the subgroup with the largest effect size from all the others, and doesn’t even acknowledge that for some other subgroups – whites and Native Americans* – the effects are negative. Are we supposed to just wish those away?

The aggregate effects of charter schools according to the CREDO report is 0.055 SDs in math and 0.039 SDs in reading -- that's what we should be talking about. But let’s, solely for the sake of argument, ignore Mehlhorn’s trickery. Is a 0.1 effect size practically significant?

Effect Sizes

Mehlhorn says 0.1 SDs is significant. He cites statistics on cancer deaths as evidence; unfortunately, he completely misunderstands their implications when he makes his analogy. The average age of cancer victims’ deaths and the standard deviations do not in any way describe the effects of a treatment on cancer: they are purely descriptive. An effect size, in contrast, looks at the effect of some intervention, like enrolling in a charter school.

Mehlhorn is closer to the mark in suggesting that an investment strategy that “beats the market” by 0.1 SDs has significance; here, the implied treatment would be investing in some way that maximizes gains. However, as I explained in my last post, statistical significance is not the same as practical significance. In any scenario, more returns on investment are preferable to fewer. But if my investment goal is to make a million dollars, getting $275 over $232 is not of any practical significance for me.

Let me suggest another analogy that might make things more clear for Mehlhorn: let’s suppose I really want to play in the NBA, but I’m too short. I’m only 5’10”**, the average height for males in the United States. But I hear about a pill I can take that will increase my size “significantly.” I buy it and take it, and then measure myself after the treatment period is over. It turns out I grew by 0.1 SDs.

The standard deviation for height is 3 inches; I grew less than one-third of an inch. Are the Knicks going to be calling?

When it comes to schooling, I constantly hear from “reformers” that charter schools are proving: "Poverty is not a final destination for these kids. Charter schools have shown that kids from impoverished backgrounds cannot be written off."

Mehlhorn himself engages in similar rhetoric: “Charter critics can decry residential segregation and ask for all schools to become good, but until that happens, it is simply immoral to cap charter schools and thus deny poor parents the same choice that wealthy parents already have.[emphasis mine]

This was the entire point of the chart I showed in my first post, and that I present once again here:

The comparison you have made, Dmitiri, is between the test score gap created by inequitable distribution of wealth and the gains of charter schools. But even if we accede to your brazen cherry-picking of results in the CREDO study, it’s clear the effects of charter schools do not come anywhere close to providing urban parents with an educational experience that is equivalent to those in the suburbs.

Yes, the charters, according to CREDO, do marginally better in the aggregate. The questions we ought to ask, however, are:
  •        What is the price being paid for this marginal increase?
  •         Is this really the best we can do?

Days of Learning

Mehlhorn, despite my cautions in my last post, seems to be hung up on this idea that the correct way to express charter school gains is in added “days of learning.” As I said previously, smart researchers I respect think this is fine. I don’t, and the next time I argue with them about this, I’m going to pull out Mehlhorn’s post. It is a nearly perfect illustration of how badly a layperson can misinterpret the conversion from effect sizes in standard deviations to “days of learning”; I hold it up as proof that the conversion is so conceptually flawed that it should never be used in studies of educational interventions again.

Here, in full, is Mehlhorn’s argument based on “days of learning”:
In education, the most commonly used benchmark converts standard deviations into days of learning. By this method, CREDO shows charters providing Hispanic urban ELL students with effectively a 40% longer school year (72 days). Over the course of a K-12 career, that’s the equivalent of adding more than 5 extra years of schooling. That’s enough extra schooling for a four-year college plus a one-year masters program. Jazzman protests that this translation is imprecise, and he is correct. We don’t know for sure exactly 72 extra days of learning. It might only be 50 (just free junior college for Sally), or it might be 100 (throw in a free Ph.D). Either way, these are not small potatoes.” [emphasis mine]
No. No, no, no.

The implication Mehlhorn makes here – one that, admittedly, is logical for a layperson – is that the “days of learning” gains translate into additional content. But that can’t possibly be right, because the tests used to judge the “days of learning” do not assess students’ abilities in any content beyond their current grade level.

To illustrate, let’s use the Common Core standards in math for grade 6. If a student was “a year ahead,” we’d expect that student would be learning content for grade 7. One of the standards in grade 7 math is for students to calculate the circumference of a circle given its diameter. It’s possible a sixth grader knows how to do this… but we’ll never know from the test, because the grade 6 test, if it’s aligned to the standards, doesn’t test grade 7 material. So the grade 6 test won’t ask a student to calculate the circumference, and we won’t know if she can.

It would be better if instead of using the phrase “days of learning” the CREDO authors used the phrase “days of drilling.” Because that’s all the test really can measure; again, if it’s aligned to the standards, it won’t measure a student’s knowledge of content taught and tested in the next year.

This was the point of the Matt DiCarlo post I linked to in my previous response: yes, high-performing charters do have more actual school days. The higher effect sizes are a product, in part, of that extra drilling. But it is completely unwarranted to extrapolate that these extra days are days spent learning new content; the tests can’t tell us whether that occurred or not. Consequently, the notion a K-12 charter student has the equivalent of a year’s worth of graduate education is without any merit, even if said in jest.

To further complicate matters: Mehlhorn links to a paper by Hill & Booth on the translation of effect sizes into “years of learning.” He claims: "In education, the most commonly used benchmark converts standard deviations into days of learning." The problem is that at no point in his source do the authors sanction what the CREDO studies do: breaking down that hypothetical year into days. One reason is that they show the different transitions from grade to grade (K-1, 1-2, 2-3, etc.) have different effect sizes (p. 3). In general, the transitions in earlier grades have much greater effect sizes than the later grade transitions; this makes equating “days of learning” in each grade indefensible.***

Look, I can understand the desire of researchers to translate their findings into terms comprehensible to laypeople. But “days of learning” just doesn’t work. What does work is what’s actually being measured: the proportion of questions answered correctly on a test. I can state with confidence that in a multi-day test with many items, the effect sizes found in the CREDO studies amount to, at most, a few more bubbles filled in correctly.

Is upending our entire education system worth that?

The Econometric Studies

I must admit, this statement of Mehlhorn’s was a big surprise:
Jazzman concedes the research from Mathematica and others that students who attend high-performing charter schools – such as KIPP and Achievement First – have better outcomes than students who apply to those schools but don’t win admissions in their lotteries. Jazzman dismisses these results by saying that those are just the good charter schools.

If he’s right, charter restrictions should stay in place for all charters that have not been validated by robust research. Validated charters should expand. Here in Virginia, my daughter’s friend Sally will have the option to attend KIPP schools. That is a huge relief for me, and I look forward to Jazzman’s support.”
If what Melhorn is saying is that only those schools, such as KIPP, that have been shown in studies to get marginal gains in tests scores should be allowed to expand, his position is likely going to send shock waves throughout the charter sector. Once again, here’s a graph from Bruce Baker:

KIPP is around 5 percent of the entire charter sector. The plurality of charters are small-time operations, with one or two schools in their network. Is Mehlhorn really saying none should be allowed to expand?

Of course, KIPP has plenty of money to throw around, so they can afford to commission Mathematica to extensively study their schools. Too bad for the smaller operators, I guess…

Let’s once again go over the limitations of these studies:
  • They are restricted to a few operators like KIPP who represent only a fraction of the charter sector.
  • The “lottery” studies are only generalizable to the pool of students who enter the lottery, and only the schools that are oversubscribed.
  • Most importantly: they measure their effects through noisy, crappy standardized tests in only two subjects.
What do we really know about the total educational experience of children who attend “high-performing,” oversubscribed charters? Do they get the full range and breadth of educational experiences that suburban children in well-resourced schools receive? Do they have opportunities for extra-curricular sports – I mean, aside from the truly awful charters that have become a de facto recruiting system for the NCAA? What about the arts? Lab sciences and history?

This is an area that has been grossly underreported in the literature, yet it’s probably more important than math and reading outcomes on standardized tests. As Jonathan Kozol has noted, these tests are largely seen as annoyances in affluent suburban districts. Yet it seems to me that KIPP and other CMOs base most, if not all, of their claims of success on these tests.

Wayne Au and others have documented extensively that the accountability measures imposed since No Child Left Behind – measures that, I would argue, have largely driven the expansion of charters – are narrowing the curriculum, particularly for children in schools deemed to be “failing.” Charter expansion, it seems to me, is playing right into this dynamic. Charters are fighting on the field of the standardized test, even though the test is, even by the current charter-lovin’ administration’s own admission, a pernicious influence.

The questions stand: what is the price being paid for these small gains in measures that are largely correlated to income? And will moving school governance and public funds away from state actors improve urban school outcomes enough to make a real difference in social mobility and income inequity?

I say no, and I turn to a study Mehlhorn has now cited twice for a little evidence: Angrist and Dynarski(2013).

Oversubscribed Boston Charter Schools

Again: one study in one city of only oversubscribed charters is not enough evidence to justify the radical policy shifts Mehlhorn seems to favor. This is such an obvious point I feel silly having to make it, but if Mehlhorn insists on generalizing Angrist & Dynarski to the entire charter sector, I have little choice but to restate what should be plain to him.

This said, let me restate that Josh Angrist and Susan Dynarski are two highly respected researchers and their work should be taken seriously. So let’s jump in and see what they and their team found:

There is no doubt that this limited group of students in this limited group of charter schools in this one city did better on their one state graduation requirement test. I don’t discount this at all, even as I question its generalizability. I do, however, think it is far too simplistic to simply say that the secret of the charter schools is their “no excuses” pedagogy and structure.

The authors, in a footnote on p.5, admit that per pupil expenses are larger at Boston’s oversubscribed charters than at the district schools. There is a problem in making comparisons because of grade level differences; however, that problem is further compounded by the fact that district schools usually have spending obligations that charter schools do not. As I said in my earlier post, high-performing charters have been shown to have a spending advantage in previous studies. Might that be the case in Boston?

The authors point out that the charters they are studying have a longer school day and a longer school year than the counterfactual public schools (p. 4). If that’s the cause of the improvement in test scores, fine… why, then, do we need charter schools? Why not just spend more on personnel to extend the school day and year? I’ve seen no evidence that unions would have an objection to this, so long as their members are paid for their extra work.

The authors also admit that peer effects could, in fact, account for some of the differences between charter and district school outcomes (p. 20). How large is the effect of charter enrollment on peer achievement? 0.13 SD – the same effect size Mehlhorn crows about from his cherry-picked subgroup in the CREDO report.****

I actually have an issue with how the authors define “peer effect”: I don’t think it should be limited merely to one’s cohort peers’ test scores, as cohorts tend to mix in high school courses and test score histories likely don’t fully describe which peers influence classroom interactions. But even with this issue aside, the authors do acknowledge a substantial body of research that shows peer effects are real (footnote 27, p. 21). Why, then, is Mehlhorn so anxious to downplay those effects, yet oversells charter effects of the same or even lesser magnitude?

Again, Mehlhorn himself has suggested an equivalency between charter school “choice” and elite suburban schools. So what does Angrist & Dynarski’s paper say about this? Do Boston’s charters create an experience for their students that rivals that of suburban schools?

This study shows they most definitely do not. Yes, the charters raise SAT scores, likely the result of all that extra time in school drilling. But there is no evidence these charters are able to propel their students into the top quartile of SAT scores (Table 5). There is a very small effect size when measuring these students’ ability to get a 3 on AP exams, and no effect on getting a 4 or 5 (Table 6), the usual standard for getting transcript credits at elite colleges. Charter attendance does not increase the likelihood of graduating from high school; in fact, it decreases the chance of on-time graduation (p. 15). And while charters seem to have a small, positive effect on college enrollment and persistence, the study shows that charter students have no advantage in enrolling in the most elite colleges (Table 8).

Let me restate this so as to be absolutely clear: yes, Boston’s oversubscribed charters do show a substantial gain on test-based outcomes, and that shouldn’t be discounted. But it is a massive, unwarranted leap to go from that conclusion to suggesting that urban charter schools are closing the income gap in education; they most certainly are not.

And we still come back to the most important question: are these gains, which are largely marginal, worth the radical policy of shifting America’s schools away from democratic control and toward a market-like system?

Other Stuff Mehlhorn Gets Wrong

I’ve already gone on way too long; however, there’s still much left in Mehlhorn’s post to debunk. Quickly:

- Yes, charters were the first choice in Newark for many families. But here’s what’s telling: as I showed in this policy brief, the “popular” schools in Newark, both charter and public, are some of the most segregated schools in the city. I can’t think of better evidence to show that what parents are really choosing in a “choice” system are their children's peers.

- If there is any evidence that Los Angeles has a better system for authorizing charters than other cities, I haven’t seen it. The CREDO report for LA puts charter gains at 0.07 SDs for reading and 0.11 SDs for math. Again: one-third of an inch.

- The notion that the nation should pattern its education policies on DC and Tennessee is utterly laughable.

- As to New Orleans: again, Doug Harris himself urged great caution in generalizing his preliminary results. Why Mehlhorn doesn’t listen to him, I have no idea. It's also worth noting that New Orleans students who were displaced by Katrina into public schools outside of the city showed substantial gains after their first year.

- When the United States installs social programs equivalent to Sweden's, I’ll entertain a conversation about whether anything they’ve learned about vouchers can be applied to our county.


So what’s the bottom line here? Dmitiri and I agree that charters, on average, show a gain in test scores; however (please don't put an ellipsis in place of this phrase, Dmitiri), while he thinks that gain matters, I’ve shown he’s just plain wrong. The gain is, in the aggregate, very small. In no way can it be compared to the advantage children in affluent school districts enjoy over their less advantaged peers. In no way can charters be said to be providing anything close to an equivalent educational experience as can be found in the leafy ‘burbs. In no way can anyone credibly make the case that charters are substantially closing the so-called “achievement gap.” Anyone who even infers that urban charter “choice” is equivalent to suburban choice is selling a lie.

Furthermore, the gains that charters show are all based on noisy, crappy, unvalidated standardized tests whose outcomes can largely be explained by economic status. Even the most ardent adherents of the value of these tests have begun to back away from embracing them as the public comes to realize they are poor measures of the complete education of a child.

So what price is being paid for these marginal gains on these poor measures? What costs does charterization exact from all of the stakeholders in American education so that students can make small, inconsequential gains on a series of bubble tests? Mehlhorn seems to think that so long as students who don’t enroll in charters maintain their scores on these same questionable measures, no one should worry about any other pernicious effects from expanding charters.

But life is more than a test, Dmitiri, and the downsides of charterization are too large to ignore:

- Because charters are not subject to the same standards of oversight and accountability as public district schools, there is a large and growing trail of fraud, waste, and abuse within the charter sector.

- Because charters set up redundant systems of school governance, they negatively impact public district budgets (see all my links in Part II). The impact may or may not show up in test scores; however, as I have shown in my previous work, funding inadequacy can negatively impact programs in the arts, physical education, and student support services. I have also presented evidence that charters themselves do not provide many of the support services at-risk and special education students may need to be successful in school. These negative effects likely don't show up in test scores; that doesn't mean they aren't a serious problem.

- Because charters have been found not to be state actors, they limit the rights of parents, students, and teachers (again, see Part II).

- Because many “high-performing” charters hire teaching staff from outside the communities where they are situated, teachers of color may suffer an inordinate consequence of charterization.

- Because many high-performing charters engage in “no excuses” pedagogy, they may be teaching their students to be compliant in a way that would not be tolerated in affluent suburban districts.

Can anyone seriously make the case that answering a few more questions correctly on a standardized test is worth all this?

I’ll say it yet again: I am not against charter schools per se. There may well be a place for “choice” within our school systems. I am certainly sympathetic to the notion that urban parents want a better education for their children. I do not think parents should be forced to send their children to schools that are unsafe or have disruptive environments.

But the evidence is increasingly clear: those charters that show gains appear to rely on a combination of increased funding, longer schools days and years, a test-focused curriculum, peer effects, and, in some cases, student selection, attrition effects, and decreased class sizes. There is no “secret sauce”: no special magic that can only be concocted by charter schools.

Many times, charter advocates make the case that they are free from “bureaucracy,” which allows them to “innovate” and “do more with less.” I see no evidence that this is true. We could put more money into all schools to lengthen the school day and year and reduce class sizes. We could desegregate our schools and our communities to spread peer effects more evenly. We could further narrow the curriculum in all schools and focus only on tested subjects (this would be a very bad idea). “Charteriness” is not a necessary precondition for any of these changes.

So why does the “reform” movement focus so much of its attention of charter school expansion? I contend it’s not so much that the “reformers” want to make money off of charters or bust unions (although, for some of them, these are nice sidebenefits). Rather, the “reformers,” no matter how well-intentioned some may be, are looking for a quick fix to problems that must be addressed within a context that is much larger than education policy.

Yes, we need good schools to have a just society. But social mobility is not social equity. Moving a few more students of color up the socio-economic ladder (and, again: we don't have much evidence charters are really doing this) does not address a core problem America faces: millions of people are doing necessary work yet falling further and further behind. They are not seeing any of the productivity gains this country has made over the last several decades; instead, they continue to do work that everyone admits must be done while making less and less.

Shuffling students around within their district borders into schools that get only slightly better test scores does nothing to address this core problem in American society. Creating a false narrative that school outcomes are the cause of inequity, and not the symptom, doesn’t help the problem; it perpetuates it.

If the best we can do for America’s most disadvantaged and deserving children is to build more schools under private control that largely teach a hidden pedagogy of compliance and get small gains on bubble tests, we have failed. We have failed because we are having the wrong conversation, and we are deluding ourselves into thinking we have equalized the lives of advantaged and disadvantaged children merely through changing how schools are governed.

It’s long past time to start being honest with ourselves about this. Mehlhorn ended with a quote by Upton Sinclair; let me end with it as well:

"It is difficult to get a man to understand something, when his salary depends upon his not understanding it."

* A technical note: I’m not even sure the category “Hispanic Students w/ELL Status” is consistently reported throughout the sample CREDO used. In New Jersey, school-level test scores are not reported for this group. Does CREDO’s sample disaggregate the student level data consistently across all states? The technical appendix doesn’t seem to say.

** I’m actually 6’ 1”. Because you really needed to know…

*** There’s also the question of whether Hill & Booth’s work, based on “vertically scaled” tests, can be applied to state tests that are not scaled this way. I’ll save that discussion, which I think is important, for another time.

**** To clarify: "Not surprisingly, given the positive selection of charter applicants, charter enrollment is associated with increases in peer achievement in the first post-lottery year: the (marginally significant) effect here is roughly 0.13 in each subject." (p. 20)