From the book “Black Box Thinking” of Matthew Syed.  

On a cool morning in the spring of 1978, seventeen teenagers from New Jersey and New York were driven to Rahway State Prison, one of the most notorious detention centers in North America. As they walked up the gravel path to the forbidding set of buildings, the youngsters joked and giggled.

The kids—fourteen boys, three girls, of different ethnic groups, aged between fifteen and seventeen—had one thing in common: all had been in trouble with the law. Terence, a seventeen-year-old African American, had stolen cars. Lori, a pretty white sixteen-year-old with a wide smile and large earrings, was a thief and a drug dealer. Angelo, a teenager with unkempt hair and a wispy mustache, had robbed shops in his neighborhood.

Nearly half of all serious crime in America was, at the time, committed by children between ten and seventeen. These seventeen kids, still joking as they reached the gates of the prison, were not just an isolated group of delinquents, they were symbolic of a wider social problem facing the United States.

Their visit to Rahway was part of a crime-reduction program called “Scared Straight.” The idea was that by giving these youngsters a glimpse of prison life—what it is really like inside a maximum security installation—they would be shocked, or at least nudged, into a change of behavior. The program, which had been conceived by the inmates, had been running for two years.

The kids didn’t buy the premise, of course. Nobody was going to frighten them out of stealing and mugging. They were too tough to be intimidated by anyone, least of all the jailbirds at Rahway.

“They don’t scare me,” one of the youngsters said with a shrug of the shoulders.

“I think it’s going to be great going in and seeing all them burnouts,” Lori said, laughing.

As they walked through the metal detector at the entrance of the prison, however, the youngsters experienced a first tremor of apprehension.

“Line up against the wall!” a sergeant shouted. “You may think this is a sightseeing trip. It isn’t. When you went through the door, the man who brought you lost jurisdiction over you. You’re in our hands. You’ll do as we say. The first thing is to stop smoking! And don’t chew gum! And take off those hats!”

This was not what they were expecting. They were ordered to walk in single file into the main prison area as an iron door slammed behind them. They were now in the bowels of a maximum security prison. Up on the balcony convicted prisoners looked down on them. “There’s a sweet mother****** right there, with the yellow shirt on!” a muscular black convict yelled. “When you are here, you’ll be my bitch” another said menacingly. The kids looked at the guards for a reaction, but there was no response. Their fear heightened.

They were then walked through a cell block called “the hole,” populated by prisoners in solitary confinement. The sexual jibes at this stage are too shocking to report. The kids became ever more uncertain. The swagger had vanished. You could see the confusion and fear on their faces. But they were not even thirty minutes into their initiation.

For the next two hours, they were locked in a small room with twenty lifers: prisoners who have been given minimum sentences of twenty-five years. Together, their terms added up to nearly a thousand years. This is where the intervention really began. One at a time, the lifers stood up and offered an insight into what the youngsters could expect if they ever came to Rahway.

“Two of you guys I don’t like,” a convict with a life sentence for murder screamed at the kids. “I don’t like you and I don’t like you. You got one time to smile at me and I am going to turn your teeth upside down. You understand? I have just got out of the hole today and I am going to turn your teeth upside down.”

The kids had arrived at Rahway with the vague idea that prison was an easy ride. They thought they could just breeze through. They thought they were tough. As they listened, they were systematically disabused of their naïveté. Another inmate asked:

“When we got sexual desires, who do you think we get? Take a wild guess . . . We get young, dumb mother*******, just like you. I am in here ten years and I am going to die in this stinking joint. And if they want to give me these three bitches right here I would leap over them like a kangaroo just to get to one young, pretty . . . One day you are lying on your blanket, and your mind is drifting over those thirty foot walls and you are thinking about who’s with your girl when three guys will slide into your cell, wrap you up in that blanket, and I don’t care how tough you think you are or how strong you might be, but they are going to kick you onto the side of that bed, and they are going to [rape you]”

None of the kids were talking now. One or two were crying. The lifers were not acting out of spite. They were, in effect, issuing warnings, admonishing the kids to change before it was too late. This was an attempt to deter the next generation of criminals. The lifers didn’t want the youngsters to make the same mistakes they had.

“We don’t get paid for doing this,” the kids were told. “We don’t get no extra reward, no extra benefits, no nothing. We do it because we want to do it. Because we might help you.”

Another convict said: “I have been here seven years. I regret every day I have been here . . . You have the best opportunity in the world [to avoid prison] . . . You would have to be a fucking fool not to take it.”

The kids were inside Rahway for three hours, but it seemed like three days. They had seen the reality of prison and were adamant they would never go back. Crime no longer seemed cool, but a game that led to hopelessness and desperation. On the way home they were silent. At one point the driver had to stop the car so that one of the boys could vomit.

“I was just so scared, I don’t want to go to one of them things,” Lori, the girl with the big earrings, said. “It scared the shit out of me, I didn’t like it at all.”

“I think it will change my life,” another said, wide-eyed. “I mean I have got to cut some of this [crime] out. All of it, if possible . . . I am going to try very hard.” Others talked about going to college: anything to avoid jail.

The prison visit was recorded by Arnold Shapiro, a documentary maker. His film of the visit was later broadcast by KTLA, Channel 5 in Los Angeles and fronted by Peter Falk of Columbo fame. Viewers were riveted by the grim reality of prison life and by the seemingly incredible results of the Scared Straight program. Falk revealed that of the seventeen youngsters, sixteen were still going straight three months later. He also reported that the wider program had had a dramatic impact on reoffending rates. Falk said:

” Over 8,000 juvenile delinquents have sat in fear on these hard wooden benches and for the first time they really heard the brutal reality of crime and prison. The results of this unique program are astounding. Participating communities report that 80 to 90 percent of the kids that they send to Rahway go straight after leaving this stage. That is an amazing success story. And it is unequalled by traditional rehabilitation methods” 

Politicians lined up to praise the program. Newspaper columns were penned. Social commentators praised the approach of Scared Straight. Feckless kids were pushed into line and brought face-to-face with the consequences of their actions. It was the kind of short, sharp shock treatment that pundits had been crying out for. It was razor-edged deterrence.

During the week of March 5, 1979, Shapiro’s documentary was shown in two hundred major cities. The following month it won the Oscar for best documentary feature at the Academy Awards. The Scared Straight program was rolled out across the United States, Canada, the UK, Australia, and Norway. Its effectiveness was attested to by judges, correction officers, and other experts.

The data seemed remarkable. As George Nicola, a juvenile judge who worked in New Brunswick, a few miles from Rahway, put it: “When you view the program and review the statistics that have been collected, there is no doubt in my mind . . . that the juvenile awareness project at Rahway State prison is perhaps today the most effective, inexpensive deterrent in the entire correctional process in America.”

But there turned out to be one rather large problem with Scared Straight. It didn’t work. Rigorous testing would later prove that the kids who were taken on prison visits were more likely to commit offenses in the future, not less—as we shall see. A more appropriate name for Scared Straight might have been Scared Crooked. It was an unequivocal failure. It damaged kids in a number of ways.

But first we will ask: How is this possible? How can something be a failure when the statistics seem to show that it is a success? How can it be failing when virtually every expert is lining up to endorse it? This is the power of closed loop. 

In 1999, Scared Straight! 20 Years Later was broadcast in the United States. The documentary was fronted this time by Danny Glover rather than Peter Falk, and revisited those seventeen, scrawny teenagers who had appeared in the original film. The results were as seemingly miraculous as the original program had led audiences to believe.

Many of the interviewees talked about their new lives. Almost all credited the three-hour visit to Rahway two decades earlier as having turned their lives around. Terence, the young black kid who had once stolen cars and broken into stores, was now a part-time preacher at his local Baptist church, with a wife and two sons. “Chances are, if I wouldn’t have gone to Rahway, I would probably be locked up and could be in my grave,” he said.

Lori, the sixteen-year-old with the wide smile and big earrings, who had been dealing drugs, was now a thirty-six-year-old bookkeeper and mother. “I just thought it was a day away from school,” she said. “I don’t think I have ever been as afraid in my whole life . . . It made me not want to be an idiot anymore . . . I started going to school more after that.”

Angelo, the kid with the unkempt hair and wispy mustache, was now thirty- seven years old, tiled floors for a living and had three kids. He said “If I didn’t go to Rahway, I think I would have done hard time,” he said. “If that one day didn’t happen, I might not have my family. And my family to me right now is everything; it is the most beautiful experience in the world.”

This, then, is how the phenomenon of Scared Straight looked to millions of TV viewers. The statistics look good, too. This was a scheme, unlike most social programs, that actually bothered to collect data. According to the evidence, around 80 to 90 percent of people who attended the program went straight. As stated in the documentary: “That is an amazing success story. And it is unequalled by traditional rehabilitation methods.”

But if we rewind to the late spring of 1977, a rather different picture was starting to emerge. In April of that year, James Finckenauer, a professor at the Rutgers School of Criminal Justice, decided to test Scared Straight. He wasn’t just interested in the observational statistics. As a scientist he knew that these could be misleading. He was not interested in hype or slickly presented documentaries either. He wanted to know if the scheme really worked. In short, he wanted to run a Randomized Clinical Trial (RCT). 

Finckenauer has silver-white hair and inquiring eyes. He has published dozens of papers and won multiple awards for his research, but his most striking quality is his conversational style. He is cautious, considered, and attentive. He also has a laserlike quality, as if he is trying to cut through the surface to find the truths lying beneath. These qualities would serve him well as he forensically unpicked the Scared Straight phenomenon.

Before starting the RCT, Finckenauer probed the existing evidence for Scared Straight. Where did the 80 to 90 percent figure for kids going straight come from? He found that it was based on a questionnaire sent to the parents or guardians of children who had visited Rahway. (Another source of the data was letters of commendation sent in by the sponsoring agencies which brought kids to Rahway. These were not terribly reliable. These agencies may have had all sorts of hidden incentives to believe in the program.) There were four yes-or-no questions:

  • Have you noticed a marked change in your child’s conduct since their visit to the prison?
  • Has there been a slight change in their behavior since their visit to prison?
  • Do you think another visit is necessary for your son/daughter?
  • Are there any specific areas you think we might be of some assistance to you, or your son or daughter?
  • There was also space to write comments

But what did a “marked” change actually mean? What did a “slight” change mean? The questions were open to all kinds of interpretation. Finckenauer also discovered that many of the kids who visited Rahway had not been delinquent or even pre-delinquent in the first place. It hardly counts as a success that they didn’t commit crime afterward if they were already on the straight and narrow. Furthermore, the letters to parents were often sent within weeks of the prison visit. That was scarcely enough time to judge a change in behavior.

And yet these were only minor quibbles. The deeper flaws go to the heart of what constitutes valid evidence. The first is that only those who responded to the questionnaire were included in the statistics. Those who didn’t respond were entirely absent from the data. Consider how that might have distorted the result. It is possible that only the parents of children whose behavior improved bothered to respond. Parents whose kids continued to behave badly might have thrown the questionnaire in the bin, or at least responded in fewer numbers. This could have skewed the stats beyond recognition. 

This is a type of so-called “selection bias” and it should sound familiar. It is pretty much the same problem that bedeviled medieval medicine when only those who recovered from bloodletting were able to testify to its effectiveness. The evidence sounded terrific but that is because it was dangerously incomplete. Those who did not recover from bloodletting were never given a chance to express an opinion. Why? Because they were already dead.

The deepest problem with the Scared Straight statistics, however, related to the counterfactual. Even if everyone had responded to the questionnaire (which they hadn’t), we still wouldn’t know whether the outcomes had been caused by the intervention or by something else. Perhaps behavior would have improved without the intervention. Perhaps it improved because the local economy was improving, or because of a new scheme at school, or some other factor. Perhaps the outcome would have been even better without the intervention.

In August 1978, Finckenauer divided a set of delinquent youths into two random groups. One group attended the Scared Straight program. The other group (the control group) did not. He then sat and waited to measure the results. Despite the hype, the stellar-looking stats, the slick PR, the Oscar-winning documentary, the commendations from politicians, the tributes from corrections officers, and the widespread adoption of the scheme around the world, this was the first time the project had been subjected to the most rigorous kind of failure test.

And the results, when they finally arrived, were dramatic. Scared Straight didn’t work. The children who attended Rahway were more likely to commit crimes than those who did not. “The evidence showed that the kids who went on the program were at greater risk of offending than those who didn’t,” Finckenauer said. “The data when you compared the treatment and control group was clear.”

This was, to many people, a surprise. The program looked good. The logic seemed compelling. It had parents lining up to say that it had “cured” their kids. The questionnaire data seemed solid, too. But all of these things were true of bloodletting. Only with an RCT could we cut through the ambiguity and see the real effect of the program. Finckenauer says:

” People were convinced of the success of Scared Straight because it seemed so intuitive. People loved the idea that kids could be turned around through a tough session with a group of lifers. But crime turns out to be more complex than that. Children commit offenses for many different, often subtle reasons. With hindsight, a three-hour visit to prison was unlikely to solve the problem. The intentions of the inmates were genuine: they really wanted the kids to go straight. But the program was having unintended consequences. The experience of being shouted at seemed to be brutalizing the youngsters. Many seemed to be going out and committing crime just to prove to themselves and their peers that they weren’t really scared “

Defenders of the scheme reacted angrily to Finckenauer’s report.

Judge Nicola, who had lavishly praised the program in the documentary, said: “. . . the [Scared Straight] program doesn’t need defending.”

Robert J. McAlesher, the staff adviser to Scared Straight, was even more blistering. “We question the motives of dilettantes [i.e., Finckenauer] who compromise their intellectual integrity by thrusting themselves into the national limelight with meaningless statistics deceptively presented as the result of scientific study.” 

When we are presented with evidence that challenges our deeply held beliefs, we tend to reject the evidence or shoot the messenger rather than amend our beliefs. Indeed, many of the defenders of Scared Straight responded to the results of Finckenauer’s RCT by saying that they had become more convinced of the efficacy of the program, not less. This is precisely what the theory of cognitive dissonance would predict. But even those with no prior commitment to Scared Straight continued to be attracted to the program, like moths to a flame. The hard data showed that it was counterproductive, but the narrative of kids being deterred from crime by mean-talking inmates was too seductive to ignore. By the 1980s, Scared Straight–style programs were in operation in Georgia, South Carolina, and Wisconsin. Further programs were set up in New York, Virginia, Alaska, Ohio, and Michigan.

It was as if the research conducted by Finckenauer had never happened.

By the 1990s similar programs were burgeoning. The Los Angeles Police Department ran a scheme where one of the components was kids visiting the city prison to be “shouted and screamed at” by convicts. At a program in Carson City, Nevada, a youngster was reported as saying that the part of the tour that made the greatest impact was “all the inmates calling us for sex and fighting for our belongings.” The idea was soon exported to the UK, Australia, and Norway.

Meanwhile, the hard evidence against the scheme was multiplying. RCTs were conducted on Scared Straight–style programs from the West to the East Coast of America. They found the same thing: Scared Straight doesn’t work. It often damages kids. One of the trials showed a 25 percent increase in delinquency in the treatment group compared with the control group.

But none of this seemed to matter. The glitzy narrative was far more seductive than the boring old data.

Even government officials eulogized the program. In 1994, a Scared Straight–style scheme in Ohio was commended in the official publication of the U.S. Office of Juvenile Justice and Delinquency Prevention. The experts had been bewitched by the narrative fallacy. In 1996, almost twenty years after Finckenauer’s RCT, the New York Times reported that the original program at Rahway was at the height of its popularity, hosting around ten groups per week or 12,500 kids per year.

But then in 2002 the Campbell Collaboration arrived on the scene. This is a global, nonprofit organization devoted to evidence-based policy. They conducted what is called a “systematic review.” This is where the data from all the randomized trials are collated into a single spreadsheet. By pooling the results from all the individual trials (seven were used in the so-called meta- analysis), a systematic review represents the gold standard when it comes to scientific evidence. It is the ultimate failure test.

Forgive me if you know what’s coming, but the results were emphatic. Scared Straight doesn’t work. It increases crime. Some research indicates that this increase can be as high as 28 percent.

In exquisitely understated language, the authors effectively damned its entire rationale: “We conclude that programs like Scared Straight are likely to have a harmful effect and increase delinquency . . . Doing nothing would have been better than exposing juveniles to the program“

Scared Straight was, in many ways, ahead of its time. Unlike most social programs, which collate no data whatsoever, it actually sent out questionnaires and gathered statistics. But, as with medieval bloodletting, observational stats do not always provide reliable data. Often, you need to test the counterfactual. Otherwise you may be harming people without even realizing it.

And this is really the point. It doesn’t require people to be actively deceitful or negligent for mistakes to be perpetuated. Sometimes it can happen in plain view of the evidence, because people either don’t know how to, or are subconsciously unwilling to, interrogate the data.

But how often do we actually test our policies and strategies? How often do we probe our assumptions, in life or at work? In medicine, as we have seen, there have been almost one million randomized trials. In criminal justice, they scarcely exist. Policy, almost across the board, is run on narrative, hunch, untested ideology, and observational data skewed to fit predetermined conclusions.

Closed loops are not just an intellectual curiosity, they accurately (and sometimes terrifyingly) describe the world in which we live.

On January 1, 1982, an intruder broke into the home of a nineteen-year-old called Michele Mika. After rummaging through several rooms, he took a knife from the kitchen, entered Ms. Mika’s bedroom, and murdered her. Michele’s mother later found her facedown in bed with an eight-inch carving knife in her back. After she was killed, Ms. Mika was sexually assaulted for several hours. The motive was pure sexual gratification.

More than twenty-five years later, on March 17, 2007, police arrested Angelo Speziale, a forty-five-year-old living in Hackensack, New Jersey. Speziale was one of the original seventeen youngsters profiled in Scared Straight! He was the kid with the unkempt hair and wispy mustache who had robbed shops in the neighborhood. He had also been interviewed in the follow- up feature twenty years later, by which time he had three kids and a job tiling floors.

Like most of the people interviewed for the follow-up program, Speziale claimed that the visit to Rahway had transformed his life. It sounded almost inspirational. “If I didn’t go to Rahway, I think I would have done hard time,” he said. Danny Glover, the narrator, said: “Angelo, thirty-seven, is now a law- abiding family man.”

But the reality was rather different. In 2005, Speziale was arrested for shoplifting and police obtained a DNA sample. During routine testing they discovered that it matched the DNA of the sperm found in the corpse of Michele Mika. Mika and Speziale, it turned out, had lived on opposite sides of the same duplex on Teaneck Avenue at the time the murder had taken place.

The makers of the documentary did not deliberately mislead audiences about Speziale. They couldn’t have known that he was deceiving them when he said he had “gone straight.” They couldn’t have realized that just three years after he had visited Rahway, he had raped and murdered an innocent nineteen-year-old. Only the test provided by DNA revealed the truth.

But the documentary makers did know by the early 1980s that Scared Straight was increasing crime. And yet they continued to make celebratory programs on the project. A&E, an American cable and satellite channel, introduced Beyond Scared Straight, a new series, in 2011. By 2014 it was in its eighth season. Arnold Shapiro, the producer (who also made the original 1978 documentary), continues to defend the scheme, despite the overwhelming evidence against it. He argues that Scared Straight today involves more counseling and less shouting. But the logic of conducting the interventions in prisons has always relied on a confrontational component.

As the Daily Beastput it:

“The episodes themselves do emphasize the horrors of prison life more than discussion. At the beginning of one filmed at Maryland’s Jessup prison, a 50-year-old man convicted of first-degree murder barks into a 17-year-old dropout’s face, “Don’t smile at another man in prison, ’cause if you smile at another man in prison, that makes them think that you like them, and for you to like another man in prison, something seriously is wrong with you.”

In his three-hour visit to Rahway in 1978, Speziale endured a number of degradations, but one event is particularly chilling in hindsight. The youngster was forced to stand in front of the group and read out a newspaper report of a knife attack that had taken place in prison. “Rahway inmate stabbed to death in cell block,” the sixteen-year-old read, voice trembling. “He was stabbed about a dozen times in the neck, chest, head and back. Robinson was pronounced dead on arrival at Rahway General Hospital.”

There is no evidence of any connection between the fact that Speziale was humiliated into reading out loud the details of a savage knife attack on his visit to Rahway in 1978 and the fact that he perpetrated a similar crime a few years later. This is almost certainly a coincidence. But what we do know is that these visits, on average, damage the kids who are taken on them. We have known that for more than three decades.

In 2010, Speziale pleaded guilty to sexual assault and stabbing and was sentenced to twenty-five years. He is now back in Rahway prison, where this story began. It is an endlessly disturbing and cautionary tale. But the deepest irony of all, and the one that takes us to the heart of the closed-loop phenomenon, is that Speziale might soon be delivering Scared Straight–style confrontations to the next generation of delinquents.