When I first heard in the media there was a new study of longevity which had produced a model based on your SNP profile that was “77% accurate” as to whether you’d live to the age of 100 or not, I assumed this was confusion or distortion (perhaps The Daily Mail had broken embargo first and its spin was percolating around the mediasphere). But later I listened to one of the researchers on the radio, and though he seemed to want to tone down the certitude as to that prediction, he did not debunk the claim. Whatever the details, I did not believe that the model was that relevant to most people since very few are going to make it deep into their nineties in any case (I did have a grandfather who made it to 100 [in Bangladesh!], so my chance is presumably greater than the norm). The model would be moving you along the margins. Additionally, over the years it has paid off to be skeptical of the discovery of large effect genes for X, Y and Z. When the X, Y and Z has medical significance I’m even more skeptical, because the non-scientific biases within medical research seem to be really strong. There’s a lot of fame and money to be had. Some of the media were asking the researchers up front whether this might unlock the genetic “Fountain of Youth.” This is entrancing stuff.

So is this post from Dr. Daniel MacArthur, Serious flaws revealed in “longevity genes” study, warrants notice:

If the paper’s claims were true they would be truly remarkable. However, the general feeling from the GWAS community is that the identified associations are likely to be largely or even entirely artefactual, the result of failing to fully control for differences in the genotyping methods used in the cases and controls. The study used a mixture of two different genotyping platforms (albeit both made by Illumina) for their centenarians, while the control data was taken from an online database containing samples examined using multiple platforms. Disturbingly, similar potential genotyping bias also affects their replication cohort.

In the Newsweek piece I mentioned yesterday Kári Stefánsson has this to say about one of the platforms:

Kári Stefánsson, the Icelandic geneticist who founded deCode Genetics, knows something about the 610-Quad—his company has used it too. He says it has a strange and relevant quirk regarding two of the strongest variants linked to aging in the BU study, called rs1036819 and rs1455311. For any given gene, a person will have two “alleles,” or forms of DNA. In the vast majority of people, at the rs1036819 and rs1455311 locations in the genome, these pairs of alleles consist of one “minor” form and one “major” form. But the 610-Quad chip tends to see the wrong thing at those particular locations. It always identifies the “minor” form but not the “major” form, says Stefánsson—even if the latter really is present in the DNA, which it usually is. If you use the error-prone chip in more of your case group than your control group—as the BU researchers did—you’re going to see more errors in those cases. And because what you’re searching for is unusual patterns in your cases, you could very well mistake all those errors (i.e., false positives) for a genetic link that doesn’t actually exist.

Stefánsson says he is “convinced that the reported association between exceptional longevity and most of the 33” variants found in the Science study, including all the variants that other scientists hadn’t already found, “is due to genotyping problems.” He has one more piece of evidence. Given what he knows about the 610-Quad, he says he can reverse-engineer the math in the BU study and estimate what fraction of the centenarians were analyzed with that chip. His estimate is about 8 percent. The actual fraction, which wasn’t initially provided in the Science paper, is 10 percent, the BU researchers tell NEWSWEEK. That’s close, given that Stefánsson’s calculations look at just two of the variants found in the study and there may be similar problems with others.

Stefánsson recognizing one of the 150 SNPs as a problematic one is another red flag. The effect sizes of the SNPs in the study seem really large, so that should make you curious as to what’s going on. Here’s a post from 23andMe suggesting we should be cautious of the results for that reason:

-A large study combining results of four genome-wide association studies of longevity was published in May in the Journals of Gerontology. That study found no associations meeting their pre-specified criteria for genome-wide significance. While they used a more inclusive phenotype (age 90 or older), it is surprising that there could be so many loci associated with survival to age 100 in the new study, some with very large effect sizes, yet none were found in the larger study from earlier this year.

23andMe applied the model (the SNPs) outlined in the paper and attempted to see if it had any utility in to their admittedly small sample within their own database. They found nothing of note:

We took a preliminary look in our customer data to see if the proposed SNP-based model described in Sebastiani et al. is predictive of exceptional longevity. A commonly used measure of test discrimination is to calculate how often, for a randomly selected case and control, a test correctly assigns a higher score to the case. This is known as the “c statistic” or “area under the curve”. The authors of the new study say their model scored a 0.93 for this statistic. But when we compared 134 23andMe customers with age ? 95 to more than 50,000 controls, we obtained a test statistic of 0.532, with a 95% confidence interval from 0.485 to 0.579. Using 27 customers with age ? 100, we get a value of 0.540, with a 95% confidence interval from 0.434 to 0.645. A random predictor of longevity would give a 0.5 on this scale, so based on our data, performance of this model is not significantly better than random. Even with our small sample size, we can also clearly exclude values as high as the published result of 0.93.

If you go back to Dr. MacArthur’s post he has a chart which indicates that even by eyeballing their are indications that the results in the Science paper were artifacts of the methodological limitations. Newsweek ends with this caution:

Still, one has to wonder how the paper wound up in Science, which, along with Nature, is the top basic-science journal in the world. Most laypeople would never catch a possible technical glitch like this—who reads the methods sections of papers this complicated, much less the supplemental material, where a lot of the clues to this mystery were?—but Science’s reviewers should have. It’s clear that the journal—which hasn’t yet responded to the concerns raised here—was excited to publish the paper, because it held a press conference last week and sent a representative to say as much.

This isn’t about the media. They didn’t have to sensationalize too much; the findings themselves if correct are moderately sensational. But if Dr. Daniel MacArthur could spot something indicative of serious problems by scanning the supplements presumably it shouldn’t have made it through the review process without the issue being mooted and addressed. But then again, it’s medical genetics, and there’s a lot of pressure to find the roots of human morbidity and mortality. It’s a field where results like ALH 84001 abound. The heart wants what is wants. That’s why it’s nice to focus on less practical evolutionary genetic questions; no one really cares that much whether we’re descended from Neandertals. Right?

Note: And earlier post from Nature with more quotes from scientists who are skeptical of the findings.