So I know, you thought I’d actually blog about a ride at some point. I promise I will, but this issue is directly relevant to what this blog discusses, and besides, it’s bugging me and needs to be said.
Over the last few days, I’ve encountered some rather pointed examples of journalism gone wrong. The first one, aired recently on NPR’s show Radiolab (the episode is entitled “The Fact of the Matter,” and you can listen to it here), was about a phenomenon called “Yellow Rain,” which may or may not have been a chemical weapon used as part of a genocide campaign against the Hmong people after the end of the Vietnam War. Full disclosure: I love Radiolab. It’s a show about the weird connections science can help us make between topics. Mostly, the show feels like someone has burrowed into my brain and transplanted the way I think about things directly onto the radio.
But this segment was different. It involved an interview with a Hmong elder, and ended with the translator in tears and the show’s hosts pretty callously dismissing and/or completely missing the point she was trying to make. As she was explaining about her uncle’s experiences with the mysterious substance called “Yellow Rain” and the deaths he believed he had seen it cause, the hosts were busily explaining away the chemical warfare story the Hmong have come to believe by using scientists who claimed that yellow rain was explosive diarrhea from bees. When you start dismissing someone’s personal genocide experience with “bee poop,” you better be sure you’ve got some great science behind you. Unfortunately, they not only skipped the whole “great science” part, but they handled the entire approach to the interview very poorly indeed. The show is now trying madly to backpedal and apologize its way out of the mess they created. You can read this rather sad and entertaining example of how not to admit you’re wrong, here.
The main point I came away with, however, wasn’t about whether or not the hosts were rude to the particular interview subjects (they were), or even if “Yellow Rain” is bee feces. As I said, the science wasn’t particularly compelling. What I walked away with was the distinct sense that the dismissal of the many, many Hmong refugee stories about “Yellow Rain” was based on the fact that this evidence was merely from their own experiences, and that anything less than a test conducted somewhere in a lab, any lab, was not really valid for making a claim. It didn’t matter that the scientific evidence in question made no logical sense. Science, even bad science, must trump grandpa’s stories about stuff falling from the sky causing instant, excruciating death, right? Even if there are hundreds of grandpas making that claim.
Ah, but what does this have to do with bikes, you say? Here’s where my brain starts making those darned connections again. Two days ago, The New York Times published an article entitled: “To Encourage Biking, Cities Lose the Helmets.” The story was mainly talking about the reporter’s experience of renting a bike share bike in Paris, and riding it there helmetless. She contrasted this with her feeling that she definitely needed a helmet while riding in New York, and wondered how successful a bike share program could be if folks felt they needed a helmet in order to participate in it.
If you read anything to do with cycling, you’ve heard all this stuff before. She used some scientific studies, though nothing particularly new or compelling. In fact, I would put what’s cited right up there with the bee poop. There’s the usual “helmet laws make folks without helmets think they can’t ride a bike” studies, which always seem self-evident to me, and the way they are used is often totally defeatist: people think this is hard, so let’s not do anything about it! There is a dodgy statistical analysis moment, where a supposedly intelligent scientist suggests that if we think we should wear helmets on bicycles, we should wear helmets all the time, because, apparently, this is an all-or-nothing argument: damnit, if we can’t prevent all head injuries, we’re not preventing any! And she doesn’t really address the fact that comparing cycling in Europe to cycling in America is like comparing eating an apple to eating a banana, yet we continue to present it as if the comparison means something. All of this is would be par-for-the-course, and not even pretending to be “scientific,” if it were not for this bit at the end:
Before you hit the comment button and tell me that you know someone whose life was probably saved by a bike helmet, I know someone, too. I also know someone who believes his life was saved by getting a blood test for prostate specific antigen, detecting prostate cancer. But is that sense of salvation actually justified, for the individual or society? Back in New York I strapped on my helmet for a weekend bike ride in Central Park. But I’m not sure I’ll do the same two years from now if I’m commuting to work on a mature Citi Bike system.
Mr. De Jong, who grew up in the Netherlands, observes of Amsterdam: “Nobody wears helmets, and bicycling is regarded as a completely normal, safe activity. You never hear that ‘helmet saved my life’ thing.”
Woah, now. While I may joke about the House Rabbit Lobby or the Swedes hitting the comment buttons and deriding me for something I imply on this blog, I don’t usually try to preempt people’s comments by dismissing and belittling their experiences out of hand. That seems condescending. What the heck does the prostate test have to do with hitting your head on a bike? Does it even make sense to imply that in a country where almost nobody wears helmets anyway, most people don’t have a “helmet saved my life” story?
This piece made me, and numerous other folks, angry. The comments section was immediately filled with hundreds of “helmet saved my life” stories. I refrained from posting mine, but you can read it here. I noticed they shut those comments down pretty darn quickly, too.
It seems to me that this is what happens when we start mistaking anything scientific as evidence for our argument, and dismissing all anecdotal evidence as unprovable and therefore, invalid. I’m no scientist, but I’m pretty familiar with how the Scientific Method works. I think we can all agree that during rigorous scientific testing of an actual hypothesis, we don’t want individual’s stories of their experiences to count one fig. But neither of the above examples came from scientists conducting tests. They were, in fact, journalists reporting about stories that affected individual people. I’m not saying science wasn’t involved, but this was hardly Scientific American talking about the latest research in robotics. These were, as old-fashioned news guys used to say, “human interest stories.”
I’m all for really good scientific evidence, even when what it says is hard to hear. I love it when the DNA proves someone couldn’t have done the crime and we must start over to find out who did, and I fully support the idea that we’d better vaccinate our kids, Jenny McCarthy be damned. I get that we are royally screwing the noble polar bear with our desire to drive enormous cars and live in giant houses. However, totally dismissing all eyewitness accounts to something simply because they can’t be empirically tested, when one is reporting on life and not on research, smacks to me of a sort of silly scientific elitism that discounts the reality of human experience.
Human experience can’t be recreated in a lab. Labs are controlled environments, but the world is not a controlled place. Only in 2010, did the National Highway Traffic Safety Administration start doing tests with “female” dummies in the front of the car, much less in the driver’s seat. We’re not very good, even those who are actual scientists, at factoring in all the real world conditions that come with testing something that happens in the real world. We’re even worse at reporting about it.
Which brings me back to those pesky bike helmets. As far as I can tell from reading through most of the studies out there, and endless pages of analysis of those studies, no conclusive test of a helmet’s effectiveness actually exists. There are too few studies, they’re dated, and the methodologies used in those studies and then the way the studies’ findings are interpreted, are often suspect. Certainly, there are many studies examining the social costs of regulating helmet use, and those should be considered when creating helmet legislation. But in terms of individual safety… nope, I’m not convinced by the science, either way.
I’m not saying this gives me carte blanche to ignore legitimate scientific information in favor of my gut. Sometimes, even though we don’t like it, we have to accept that science is correct. But what about when the science leaves us with an open question, and no real answers to the choices we have to make? Until a really good study is done, how are we supposed to make up our minds about whether or not to wear a helmet? I submit that in the absence of conclusive scientific evidence, non-empirical evidence should be taken into account.
When considering non-empirical evidence, you have to apply a certain logic to it, and weed out the parts that make little sense. This requires some critical thinking. I know, I know. We hate doing that. But while individual experiences may not be all that valuable to science, they give each of us the wisdom to make decisions in our own lives. How do we learn anything, if not through experience? If we learned everything through research studies, Americans would be a vastly different, not to mention thinner, people. But along with individual experiences, which can rarely be extrapolated out to anyone other than the person who is telling the story, I believe we should value collective experiences. Until we can conclusively prove that something is, or isn’t, true through real scientific testing, collective experiences provide us with a way of examining broader trends, or illuminating areas of study. They also provide us with a way of making decisions which have to be made, regardless of the current state of scientific research.
When making an argument for or against helmets, citing the science obviously isn’t going to be enough (or we’d no longer be having this argument), nor can people ignore the non-empirical evidence. Folks are going to have to make judgment calls. Putting them down for doing so is just bad journalism, and does nothing to make a point scientifically. That’s what rubbed me the wrong way about the Times piece: if you can’t offer me good science, then don’t put me down for using what evidence I have from my own, and others’, experience.
Do I wear a helmet? Yes, I do: and I think it saved my life once. Did I wear one in Copenhagen or Stockholm? No, I didn’t, because they weren’t readily available. Would I have worn a helmet if I’d had one at that point? You know, I might have. Individual experience has shown me that falling even a relatively short distance can cause significant brain trauma, and I’d like to avoid that if I can. Of course, Dutch and Swedish collective experience would say that helmets aren’t necessary, but then sometimes we have to ignore collective experience, and the lack of scientific evidence, to go with our individual experience. That’s what making truly informed decisions entails: sorting through all the evidence offered: scientific, collective and individual, and making up one’s mind for oneself. We can’t simply dismiss whole portions of that process because there’s no double-blind lab test involved.
And as for Radiolab? Well, if you want to talk about science and all the questions it leads us to, you have to be prepared to run with those questions when they refuse to stick to the science you’ve got. The show was an object lesson in why, though preparing for an interview and a story is a good thing, relying too much on your preconceived notions about where the story is heading is a bad thing. Sometimes we run into places where we can’t answer the fundamental question through the science we have: if “Yellow Rain” isn’t chemical warfare, and yet hundreds of witnesses claim that something coming from the sky was making them suddenly sick or causing people to die, then what the heck WAS happening? I still don’t know the answer to that question, and certainly “bee poop” isn’t going to do it for me, because it doesn’t make any sense. The whole point of the show is to run with those moments so we can get a full perspective, and they didn’t do that. So as a segment, it failed and they should never have aired it. And the interviewers behaved like assholes to an eyewitness, dismissing his account not because it was wrong, but because it didn’t jive with the limited scientific evidence they had taken the time to dig up.
That’s both bad journalism, and bad science.