Archive for January 2011
This term I’m learning some more microeconometrics by way of a health policy class, so I have epidemiology on the brain. I’m about halfway through Siddhartha Mukherjee’s The Emperor of All Maladies: A Biography of Cancer. I am enjoying it, but a little less than I thought I would, from the reviews. Mukherjee has a grating fondness for overwrought metaphors and several unconvincing theories relating attitudes towards cancer to America’s place in the world. But the middle chapters on efforts to compare cancer rates and outcomes over time and how researchers worked out the link between smoking and cancer are fascinating. Here’s one stat that I’ve been thinking about:
By 1953, the average annual consumption of cigarettes had reached thirty-five hundred per person. On average, an adult American smoked ten cigarettes every day, an average Englishman twelve, and a Scotsman nearly twenty. (241)
It reminds me of a similar passage from another recent nonfiction hit, Daniel Okrent’s Last Call: The Rise and Fall of Prohibition.
By 1830 Americans were guzzling, per capita, a staggering seven gallons of pure alcohol a year. […] In modern terms those seven gallons are equivalent of 1.7 bottles of standard 80-proof liquor per person, per week—nearly 90 bottles a year for every adult in the nation, even with abstainers (and there were millions of them) factored in. Once again figuring per capita, multiply the amount Americans drink today by three and you’ll have an idea of what much of the nineteenth century was like. (8)
So this is what I’m wondering: What current habit will seem as absurd to future readers as these do to us now? Salt or saturated fat consumption? Physical inactivity among office workers? Some food additive or packaging material?
Indulge me, and I promise I won’t start calling myself a social media expert.
I’ve noticed that some media improves when you share it. Actually, I’ve noticed this about television. It doesn’t matter how many times I watch the first episode of the West Wing, say, or Friday Night Lights. If I’m watching it with someone who has never seen it before, I enjoy myself. On my own, I’d be bored.
Also: I am a compulsively social reader. By which I mean, I have to read all the best parts out loud to you. This is one of the things that is great about Twitter, or Facebook, or the “email to a friend” buttons on newspaper websites, right? I can pester you even when you’re out of earshot, and that makes reading more fun.
Mr. Ferriss offers advice about so many disparate things — not simply losing weight and building muscle and improving sex and living forever, but learning to hold your breath longer than Houdini (!) and hit baseballs like Babe Ruth (!!) — that paging through “The 4-Hour Body” is like reading the sprawling menu in a dubious diner, quite certain the only thing you’d dare order is the turkey club.
And that’s not even the best part.
I’m really excited about the data scraping guide ProPublica’s Dan Nguyen published a few days ago on their Nerd Blog. On a related note, I will be spending most of my reading week at Investigative Reporters and Editors’ 2011 CAR conference in Raleigh.
And finally, if you’re in Toronto you should know that we now have our very own Hacks and Hackers chapter. I wasn’t able to make either of the first two events because of midterms and then exams, but I will definitely be at the next event, whatever it is.
I see that the New Yorker has ungated Jonah Lehrer’s The Truth Wears Off, on how statistical significance seems to decline over time. Lehrer’s is one of those rare features with a punch line, so if you want to enjoy the piece fully you should read it before this post.
Lots of smart people have weighed in online. The best follow-up I’ve read so far is from Andrew Gelman, who links to his own piece from American Scientist. (H/t to Chris Blattman.) But as plenty of good has been said about the piece already, I’d like to point out the one thing about it that bugged me.
Early in the piece, Lehrer writes at length about Jonathan Schooler, a psych researcher who has found that he has difficulty reproducing his own results.
…while Schooler was publishing these results in highly reputable journals, a secret worry gnawed at him: it was proving difficult to replicate his earlier findings. “I’d often still see an effect, but the effect just wouldn’t be as strong,” he told me. “It was as if verbal overshadowing, my big new idea, was getting weaker.” At first, he assumed that he’d made an error in experimental design or a statistical miscalculation. But he couldn’t find anything wrong with his research.
The central insight of the piece is that declining statistical significance in a field overall is one of the effects of publication bias. When an idea is new, strong positive results are required for publication. As it becomes accepted knowledge, contradictory results become interesting enough to publish. That’s a brilliant, important observation.
The problem is that Schooler’s experience can’t be explained by publication bias. He is just one researcher. If publication bias was the only force in play, individual researchers wouldn’t see their results change over time, only their chances of getting those results published.
Schooler’s problem is probably better described as regression to the mean following a few anomalous results, as the article acknowledges. (Though it is a pretty weird case, and I’d love to hear a better explanation, especially for his tests of the decline effect itself.) That’s a less important issue than publication bias.It’s not a problem with the scientific method if anomalous results are gradually disproven, but publication bias can twist our perception of the world over the long term.
If the piece is really raising questions with the scientific method, why do we read so much about Schooler? And why do transitions throughout the piece seem to relate his research to the decline effect in fields as a whole? I suspect it’s because even though it isn’t illustrative of the articles’ central point, his story is interesting.
This is the sort of compromise that writers and editors make during revision. Some characters seem too compelling to cut, even as inclusion confuses the point, as I believe it does in this case. But we’re supposed to be servants of the truth, not just great stories, and I expect better of the New Yorker. Maybe it’s an unrealistic standard.