Analytical Techniques: The Power of Anecdotes

Summary: We look at the role of anecdotes in researching the cult.  They can be powerful tools to either validate or challenge your existing thinking.  Anecdotes don’t prove trends or general conclusions, but they are a great tool for alerting you to possible trends, changes in direction, or conclusions you’ve missed.  This article talks about how we use anecdotes on Wall Street. But the best part is a case study, with one of our commenters reporting on a great chance encounter who interviewed a Scientologist at length in an airport bar, as well as my quick take on what to do next with an anecdote that challenges some of my beliefs about the cult.

Anecdotes are powerful tools: Today, I want to look at the power of anecdotal evidence in analyzing Scientology.   Stories from current and former members can be a powerful tool to check your assumptions and your thoughts about what is going on inside the cult. These are particularly important to help you make sure that reality has not changed without your noticing.  In other words, anecdotes that don’t fit into your current hypothesis of what is going on are one of the most powerful tools in improving your analytical work.

In order to make anecdotes work, one has to have a foundation of intellectual honesty. In other words, you have to be open to the possibility that some new piece of anecdotal data will unravel a theory, potentially even one that you are inordinately fond of.  You can’t rush to defend a theory without thinking dispassionately about what the new data point means. Pride in doing good analysis comes not in being right about a particular theory, but in being able to adapt your thinking and to continue to hone in on useful and actionable conclusions, even if they are heading in a different direction in your prior work.

While anecdotes are powerful, “the plural of anecdotes is not data.”  What I’m saying here is not at all conflict with what I have said in multiple comments on Tony’s blog and elsewhere about anecdotes as inherently insufficient to prove general conclusions.  As you may recall, I have said on numerous occasions that clear and convincing anecdotal evidence that Scientology auditing has produced big “wins” for some people in some circumstances is not sufficient to “prove” that auditing works in a general case across a broad population of people. As scientists say, “the plural of ‘anecdote’ is not ‘data.'” That’s because anecdotes, no matter how credible the teller, aren’t structured rigorously the way that statistically valid data points in a clinical drug trial would be.  So you can’t get from “a big bag of positive auditing success stories” to the assertion that “auditing works and is an effective form of therapy.”  

In other words, anecdotes are great ways to get you to continually challenge your existing views and to guide your work by digging deeper into inconsistencies in your scenario of what is happening and your predictions about what will happen.  For that, one or two anecdotes can be sufficient to open up a whole new area of research.  However, those same anecdotes are not proof of your new theory or model.

Incidentally, I am working on a longer piece that looks at the apparent contradiction of how anecdotes can be valid individually, but any number of them cannot be combined together to establish a true statement.  It should be out in a week or two.

How anecdotes make you rich and famous on Wall Street:  In the late 1990s, Oxford Health was an HMO growing explosively, and the stock was on a rocket ride.  But one analyst, who checked in with doctors who were Oxford providers, started to hear that they were having trouble getting paid, though she knew that Oxford had always been very timely in physician payments to date.  She talked to more doctors, did some more research, and eventually made a gutsy call: Oxford would miss their profit forecast for the quarter for the first time ever, and by an immense margin. Her research helped get her clients out of the stock while it was still high and avoid catastrophic losses when the company reported several weeks later that they were hemorrhaging money and the stock collapsed.  An article from the New York Times talks about the Oxford case (I can’t remember the name of the brave analyst who went against the grain and was roundly criticized until she was proven magnificently right).  And this article from the Wall Street Journal at about the same time gives more depth on the thought process of using anecdotes in a very powerful way.

Case study:  Let’s consider a case of a really interesting anecdote which was sent in by “B. B. Broeker,” a longtime commenter on Tony’s blog.   He ran into a Scientologist at the airport in Tampa and had a long chat with a longtime supporter of the cult, which he relayed to me.  He said:

I was in Tampa for business not long ago. When my business meetings went more smoothly than I’d predicted, I saw my chance. I drove across the bay to Clearwater, parked near the Super Power building, and took a leisurely walk around the Scientology complex. It was a pretty unremarkable visit, but I was glad to have seen up close the buildings that have occupied so much of my mental real estate since becoming a Scientology watcher.

On my way home, I stopped at the airport bar, and sat next to a chatty woman in late middle age. She was, based on her interaction with the bartender, on what I figure was her fourth or fifth glass of chardonnay, and was engaging the guy on the other side of her in a trite conversation about the deleterious effect electronic gadgets are having on communication.  Needless to say, I stayed buried in my phone.

While I avoided a conversation for a while, I eventually gave in after she directly asked me how my (crappy) food was.   As it happens, I was reading Mike Rinder ‘s blog when I finally surrendered.  It turned out she lived in the greater Clearwater area, and I mentioned that I’d just been there. She named a couple restaurants and asked if I’d gone to them, and I said no, I’d just visited on a pilgrimage of sorts to the Scientology complex.

Her jaw sort of dropped, and I figured, “oh, shit, she thinks I’m a clam, and doesn’t know what to say.” So I hurried to add, “yeah, I find them fascinating.” She fumbled a bit, and eventually said, “you have no idea what’s about to happen there.   I’m a Scientologist.”

Now it was my turn to be taken aback, but I quickly recovered.  “Yeah! Super Power is finally opening! The IAS gala! Golden Age of Tech Phase II is debuting! And … you’re leaving town?”

She seemed suspicious, but answered. “Yeah, I’m headed out of town for a while. I’ve got lots of friends [at my destination], and I need to get away for a bit.”

(beat)

“How do you know all that, about all the events?”

“Oh, I read a lot. Like I said,  I find your religion fascinating.”

Well, after telling her what I do for a living (I was soooo tempted to say I was a psych, but I made a conscious decision to not antagonize her, both because I didn’t want to be mean and to see if I could get this tipsy woman to open up), she seemed to decide that I was good people, and she told me her life story.

She grew up in one of the richer suburbs of a large city, but her family wasn’t really wealthy, and she didn’t really fit in with the other kids.  Consequently, she had a hard time of it in school. “I didn’t need Scientology to teach me how to stop being effect and start becoming cause.  I had to learn that in high school.”

She got into Scientology in her 20’s.  Her boyfriend introduced her to the church.  They got married, and her new husband started a company which he ran on LRH “admin tech.”  It succeeded, and was later sold, and they moved to Clearwater.

It was at this point that she confessed that he wanted a divorce, and that she felt like she needed some time apart to figure things out.  That’s why she was headed out of town.  He wanted to stay for the events, and she decided to let him have them, while she got her head straight.

“I’m really sorry to be missing what’s happening – especially the developments in the tech and processing – but I can watch them all on DVD when I’m [at my destination].”  I guess she was planning to be gone quite a while.

We talked about the tech, and how much it helped her and her husband relate better (I courteously ignored their impending divorce), and how study tech is probably the greatest advance in human development in the past thousand years.  She even talked about the amazing efficacy of Narconon  – she had referred family members to the center and tried it for her own drinking problem.  She felt the tech and the counselors had saved their lives.  (I chose not to comment on how she was throwing back the vino – probably on glass five or six – at that very moment.) She volunteered that her husband was on a fairly high Bridge level, and had been for a number of years, but I didn’t know if it was a faux pas to ask about her own case, so I didn’t.

Anyway, I continued to demonstrate I was knowledgeable about the subject, so I wasn’t that surprised when she said, “you know so much about Scientology. Have you ever taken any courses?”

“No, I haven’t.”

“Why not?”

“Well, I’ve read a lot of LRH, and … well, I guess it’s just not for everyone.”

She sort of accepted that, but after a while eventually returned to the topic – not in a proselytizing way, but as if she were genuinely curious why someone who had familiarized himself with the Founder’s work wouldn’t want to rehabilitate his spirit.

“Is it it the press?  You know you can’t trust the papers.”

“Oh, I know.  But I agree with LRH – ‘look, don’t listen.'”  (She smiled wide at that.) “I just don’t feel like I need Scientology. “

Again, she seemed to accept my position, but then she asked me a question I never would have expected:

“Everyone thinks we’re crazy, or we’re weird. I mean, people seem to hate us. You don’t – don’t get me wrong.  But why do you think people hate Scientology?”

It was touching, and not a little bit sad. She really wanted to know, and really had absolutely no idea, why the vast majority of people outside her little bubble believe that something at the core of her life is ridiculous and/or contemptible.  In keeping with my approach of not antagonizing her, and because I thought it would lead to a more illuminating discussion, I played it soft:

“Well, there’s the money aspect–“

which prompted her to talk about how much training the auditors all had, especially with the GAT II release and with Super Power, and about how that costs lots of money, and there’s all sorts of self-study courses besides.

“Right, but I wasn’t talking about donations for coursework or auditing.  I mean, the fundraising.  The Ideal Orgs.  The IAS.  You’ve been in for eons – do you get the sense that they’re regging you harder?”

“Well, maybe. But they really don’t pressure you to give what you can’t afford.  I’ve never felt pushed to give more than makes sense. Sure, really wealthy people – and there are a lot of quietly wealthy people in the Church – give a lot, but it’s nothing to them.  Normal people aren’t forced to give that much.  It’s just not expected”

“Are you guys IAS patrons, or anything like that?  Did you get pushed to prepay for Super Power?”

“No, we’ve got two bridges to pay for, and college for the kids.  We give what we feel we can, but our bridges come first.  And no one makes us feel bad about that.”

I don’t know about you, but I found that fascinating.  Sure, it could be a PR line, but it was delivered pretty genuinely, by a woman who had heretofore demonstrated no ability to effectively shade the absurd disconnect between her idealized vision of the tech and the reality of her experience in the Church. (See: her impending divorce, her Narconon “success” story.)  Now, whether she actually isn’t being coerced into donating, or whether she no longer can discern coercion – whether she actually isn’t giving a lot to the IAS, or whether she no longer has a sense of what “a lot of money” actually represents – I don’t know.  But I believe that *she* believes that there truly isn’t a regging problem.  And that’s interesting in and of itself.

Anyway, we chatted for a little while longer, but I soon had to head to the gate. As we parted, I caught her name off her boarding pass.  I checked her on Kristi Wachter’s completions list, and she had indeed been in the Church for quite a long time.  And I suspect she’ll never leave.

Thanks to BBB for a well-written narrative, and for doing a great job helping the lady he was talking to to open up.  Great job on sucking up the snark and wit to ask bland questions to help her feel comfortable.

How to analyze this data point:  There are  a couple areas where the lady’s statements fall outside my beliefs about how the cult operates.  Here are what I noticed and how I’d react to them (not to refute her statements, but to dig deeper to see what’s really going on):

  1. Regging is at tolerable levels:  The lady says that she doesn’t feel overly hounded for money, even though she is reasonably well off in semi-retirement, which I would assume makes her a prime target for enthusiastic FSM’s.  Given the horrific stories that have emerged from so many quarters, I’m surprised to see someone who is relatively sanguine about the amount of fund-raising in the cult.  It’s not likely that all those stories of obscene fund-raising techniques are wrong, but this lady apparently spoke truthfully (“in vino, veritas”?) about how she doesn’t feel overly pressured to donate all the time.  There are several possible explanations, and we would need further follow up to determine which might be applicable:   a) her husband might be the target of all the regging, since he controls the money in the family; b) they’ve reached the status of a “sideliner,” having made clear to the cult that they’re not giving more money ever; c) the cult is toothless to follow up on e-mails sent out in order to get people to attend events; d) the cult is more sophisticated in fundraising approaches, spending less time on members who are assessed as less likely to give, or e) something else entirely.  A detailed follow-up interview, if it were possible, with suitably gentle and wide-ranging questions might be able to give some perspective.
  2. Focus on the “Bridge” instead of events and donation: the picture in publicly available testimony is that the cult is making it difficult for people to move up the Bridge because it’s forcing them to redo long-ago levels and courses.  The fact that so many recent escapees say that having to redo “Objectives” caused them to blow may be a function of a self-selected audience; we’re not interviewing people still in the cult (which is why this conversation is so interesting).  I am intrigued that this person’s story challenges what many of us take on faith about lack of progress on the “Bridge.”  I would want to ask a whole bunch of follow-up questions including understanding how much Bridge progress they’re making, and whether they have had the setbacks (kicked back to “Objectives”) that others complain about. In other words, are they just engaging in a little cognitive dissonance, like touting the benefits of Narconon while belting back the drinks?  Or is there some sophistication in how the cult is targeting its members to maximize the total revenue per customer (like a casino who knows which customers prefer blackjack to poker, so they don’t shoehorn a craps player into a roulette game that they don’t really want to play).  Or, again, is something else in play?
  3. Narconon:  the fact that this lady was quickly getting bombed while talking about drug and alcohol “tech” is amusing. But beyond this, it’s reasonable to guess that a possible reason she’d be doing something that most addiction and rehab experts would say belies any actual rehabilitation, is that the cult’s definition of “recovered alcoholic” differs from the one used in the rest of the world by a fair margin.  In other words, the cult may rely on definitions to get people to think Narconon works.  She may think that because she has completed Narconon that that is what determines whether she’s an alcoholic or not.  On Tony’s blog the other day, a commenter quoted a story of one cult member saying of a nearby OT VIII who smoked madly, “he could quit at any time, he just chooses not to.”  It would take a follow-up interview to see if the lady believes Narconon works because it teaches you that you have the power to stop drinking any time you want, but that doesn’t mean you necessarily have to stop today.  If that is indeed the definition culties use for “success” at Narconon, it’s no wonder it’s easy for them to repeat claims of an 85% success rate for the program with a straight face.

The more data points one collects on a regular basis, the better prepared you are to detect changes in the environment that would allow you to update your scenario. The faster that you detect and respond to change, the more effective you’ll be… in capitalism, if you figure out that a company’s business is deteriorating, you can sell the stock before others think there might be a shortfall, and can often avoid huge losses.