No child left behind?

(Crossposted to Jewschool.)

“I have never let my schooling interfere with my education.” –Mark Twain

***

Mechon Hadar has placed all the audio from their recent Independent Minyan Conference online, so those of us who weren’t there can listen and respond from afar. So we’ll be writing a series of posts here reacting to various sessions from the conference. And I’m starting with the keynote address by Professor Steven M. Cohen (not to be confused with Rep. Steve Cohen) on “The Groomed and the Bloomed: Varied Paths to Engagement in Independent Minyanim”. You can listen to the MP3 and view the PowerPoint presentation.

Cohen’s address is based on the data from the 2007 National Spiritual Communities Study, about which we have already blogged at length.

The dichotomy in Cohen’s title is between the “groomed” — people who have a Jewish educational background (defined, as we will see, in very specific ways) — and the “bloomed” — people who do not.

I find myself in agreement with the overarching ideas with which Cohen bookends his presentation, but I find the methodology and classifications used to support this conclusion to be horrifying.

Cohen says that this study arises from conversations with Rabbi Elie Kaunfer of Mechon Hadar about the difficulties of obtaining philanthropic funding for the independent minyan world. According to Kaunfer, funders are reluctant to support independent minyanim because they perceive them to be “an elite phenomenon” that is only for “the groomed”, while the funding environment is more favorable to projects directed at “the unaffiliated”. Cohen argues that “there has to be a place for elevating and mobilizing the large number of people in whom we have invested enormous sums and many hopes for the future of American Jewry.” If all of this human capital is not acted upon, then, he asks, “what was that all about?” He says that the expansion of Jewish education in the last several decades has led to an educated population who is seeking Jewish activity and high-quality Jewish community, and is now unable to find that in the established synagogues, so they have created new communities that meet their needs. I made a similar argument a few years ago in a widely distributed article, “Profile of an ‘Unaffiliated’ Jew.” Cohen concludes by saying that when historians write the story of early 21st-century American Jewish life, they’ll write about “the efflorescence of Jewish communities qua communities”, and the creation of these communities can only be accomplished by people who have some preparation. He says that the product is not the growth of the individual (which, he argues, is the goal of those funders who focus on “saving Jews from falling off the Jewish cliff”), but the production and activities of individuals when they get together.

I would go further and suggest that it’s not a zero-sum game. A rising tide lifts all ships. The creation of meaningful Jewish communities ultimately has the potential to benefit everyone (as long as the doors of these communities are open to everyone), whereas (e.g.) sending thousands of people to Israel on Birthright is insufficient if they have no community to be a part of when they return to North America.

So we definitely seem to be on the same page when it comes to the overall goals. But then we get into the methodology.

Cohen categorizes the 730 independent minyan participants who responded to the survey into four groups, based on what he describes as “when they could have turned on to Jewish life,” or as he labels his pie chart, “Earliest Sign of Intensive Jewish Involvement.” The first category is anyone who went to day school, which comprises 39% of respondents. The second category, “high school”, is people who didn’t go to day school, but went to Jewish camp AND participated in a Jewish youth group. This includes 25%. The third category, “college”, is people who aren’t in the first two categories, and were involved in a Jewish organization during college, totaling 23%. The last category, “bloomed”, is for people who are in none of the other three categories, and is 13% of the respondents. The rest of the presentation breaks down the responses to many of the other questions in the survey into how each of these categories answered them.

(By the way, the Jewish Week completely misinterprets the plain-sense meaning of these categories: “Cohen found that 39 percent of minyan-goers had attended Jewish day schools, 25 percent had experienced a Jewish connection in high school, 23 percent had been active in Hillel and other Jewish activities in college…”. If I had read only that article, I would have thought that more respondents had gone to day school than had been active in Jewish activities during college.)

Cohen uses these categories to construct his next chart: “Mean Levels of Jewish Educational Background for Selected Minyanim”. For all of the minyanim with significant samples in the survey, their participants were averaged together based on this formula: the “none of the above” category was assigned a score of 0, the “college” category was assigned 33, the “high school” category was assigned 67, and the “day school” category was assigned 100.

Just to make this scale perfectly clear: Someone who went to day school all the way through high school and has been Jewishly active her entire life would be assigned 100. Someone else who went to day school through 5th grade and has had zero Jewish involvement since then would also be assigned a “Jewish educational background” of 100. I come from a Jewishly active family, went to public schools from kindergarten through master’s degree (except for 4 years undergrad), went to Jewish camp, was active in NFTY, and was active in Hillel during college. I’m a 67. A friend of mine is going to be a rabbi in a few months, studied Talmud full-time for a year before entering rabbinical school, was active in Hillel, went to public school, and didn’t go to Jewish camp. She’s a 33. Another friend of mine comes from a family with strong Jewish traditions, grew up at a lay-led minyan within a synagogue, went to public school, didn’t go to Hillel during college because he preferred to go to the local synagogue instead, and is now active in the independent minyan scene. According to Steven Cohen, he’s a 0. Cohen talks about him as if he is a tabula rasa, with no previous Jewish involvement before he showed up at Kol Zimrah.

Based on this scale, Kol Zimrah (the community that I co-founded) is ranked lowest in “level of Jewish educational background”. Cohen specifically calls Kol Zimrah out during the presentation, saying “I was really interested to know why you guys are so low on this,” then qualifying this with “It’s not bad. You’re reaching out.” The reason Kol Zimrah is low on this scale is simple: most of us (including the real people described above) didn’t go to day school. (According to the survey, most independent minyan participants in general didn’t go to day school. But for Kol Zimrah, it’s a larger majority.) Does this mean that we’re less educated, or that we “turned on to Jewish life” later in life? Not necessarily.

Cohen’s categories suffer from three faulty assumptions: 1) Earlier education equals more or better education. 2) “Intensive Jewish involvement” is limited to a small defined set of experiences. 3) Education is equivalent to schooling. Let’s examine these assumptions one at a time.

1) Earlier education equals more or better education. I was talking about this with someone else, who noted that outside the Jewish world, the standard way to ask about education level in surveys (e.g. exit polls) is to ask about the highest level of education attained: high school, college, graduate school, etc. In that context, this is easier — someone who graduated from college can be assumed to have graduated from high school, and if for some reason they didn’t, it doesn’t matter. No such assumptions can be made with the categories of Jewish education that Cohen uses: someone who went to day school wasn’t necessarily involved in anything Jewish during college, and vice versa. But Cohen’s algorithm seems to make this assumption anyway: that people “turn on” to Judaism at a specific point in time, and their Jewish education continues at a uniform rate from that time forward, thus someone who starts later can never catch up. How else to explain the idea that someone who went to day school is always considered to be at a higher level of education than someone who didn’t, regardless of what they did afterwards? Or that in determining a community’s overall education level, one day school graduate (plus two people with no Jewish background) is equivalent to three people involved in Hillel? If you’re going to make the implicit claim that day school is that much more important than anything subsequent, you’ll have to back it up with evidence. And good luck sorting out correlation from causation.

2) “Intensive Jewish involvement” is limited to a small defined set of experiences. When the idea of “earliest” only includes items from a specific list, anything not on the list is discounted. Day school is one way to have “intensive Jewish involvement” before high school, but not the only one. I’ll join Steven Cohen in ignoring Hebrew school and bar/bat mitzvah. But other significant Jewish involvement before high school may include being part of a Jewishly active family (is Cohen really ignoring the value of the Jewish home and looking only at institutional experiences?), being part of an active Jewish community, and Jewish camp (which doesn’t only begin in high school). Cohen’s algorithm treats people with these experiences as equivalent to people with no Jewish involvement at that time. The same problems arise with the other life stages. Furthermore, as people get older, they can pursue Jewish education independently, in ways that wouldn’t show up on a survey.

3) Education is equivalent to schooling. All of the parameters used in determining level of education are about inputs, not outcomes. Back in the world of secular surveys, if you say that you’re a college graduate, that means that you had to fulfill some sort of requirements to graduate from college, and if you answer “some college”, you had to be accepted to college. But the categories in this survey indicate only that you had the opportunity to obtain Jewish education, and provide no assessment of whether you in fact did so. There is a whole field of educational assessment out there, and perhaps its methods would be helpful if one is really attempting to assess education. As anyone following the policy debates knows, standardized testing is highly imperfect; still, it would be a vast improvement over asking people whether they went to day school and what activities they participated in. (The 8th-grade standardized tests don’t ask “Did you go to preschool?” and then assign bonus points if the answer is yes. It may be the case that people who went to preschool perform better on the tests, but that should be tested, not assumed.) The only question on the survey that attempts to assess education (rather than schooling) is the one about whether you can understand a simple sentence in Hebrew. A single question, based on self-reporting, is grossly insufficient for any real kind of educational assessment.

So the methodology for this study has serious flaws. It may be that these assumptions are based on patterns from previous Jewish population studies, and Cohen left out the rigorous support because he was constrained by time and speaking mostly to non-academics. But if these minyanim and their populations are a new phenomenon, then we can’t assume that the old patterns still hold up; if we want to study these populations, we need to put our assumptions aside and test them objectively.

(Is Nate Silver Jewish? Can we bring him in to referee these studies?)

***

The presentation also included some discussion of denominational identification, much of which was already covered in the preliminary results of the survey (which we’ve already discussed), so I’m not going to write about it here. But Cohen also broke down the four new categories by denomination of origin and current identification. Unsurprisingly, a large number of independent minyan participants (in all categories) now consider themselves non-denominational. (The preliminary report didn’t list this separately, but included it within Other Jewish.) Also unsurprisingly, most people in the day school category grew up Orthodox or Conservative, with very few Reform. Some would see this as evidence that Reform Jews don’t take Jewish education seriously, but I see integration into the broader society as a positive value that Reform Judaism has long stood for. (The problem is the poor quality of supplementary Jewish education, not its existence.) This is a simple demographic explanation for why Kol Zimrah has fewer people who went to day school: Kol Zimrah has more people who grew up Reform (including its founders), and most Reform Jews don’t go to day school. While Kol Zimrah aims to be open to all who want to be there, going to the minyan that we founded does not constitute “reaching out”.

Cohen talked about the fact that very few independent minyan participants currently identify as Reform (while Conservative identification has a much smaller dropoff), and this time around, he said things that I agree with: The image of Reform has come to imply lack of Jewish activity, and therefore people who are Jewishly active are reluctant to identify as Reform. He quoted someone on an earlier panel who said “I grew up Reform, BUT [Jewishly committed]“, and marveled at the use of “but”. In my role as frame police, I crack down hard on this sort of use of “but”, so it stings doubly when it comes from the inside, indicating that some Reform Jews take Reform Judaism no more seriously than other movements take it. Cohen says that if he were speaking to the Reform movement, he would suggest that they capitalize on the passion generated in NFTY, etc., to create an “intent” form of Reform Judaism. I couldn’t agree more. I’d like to see this.

But let’s backtrack for a minute. This part of the presentation was one of several instances when Cohen demonstrated that his propensity to make unwarranted overgeneralizations applies also to his audience. What he actually said, before amending his remarks, was “If I were speaking to a Reform audience, which none of you are, so you don’t care…”. WTF? How does he know who was in the room, whether they identify as Reform, and whether they care about the future of Reform Judaism? If I had been there, I would not have let this go unchallenged. In other sections, when he talked about “groomed” versus “bloomed”, he addressed the audience as if they were all in the “groomed” category. This is despite the fact that his own charts showed that some percentage of “bloomed” people (albeit smaller than the percentage of “groomed” people) were in leadership positions in independent minyanim, and therefore might well have been at the conference representing their communities.

***

If Steven Cohen or anyone involved in the survey is reading this, you are of course invited to respond. I look forward to continuing the conversation.