Evilicious

As I have noted in the past, Andrew Gelman, a statistician at Columbia (and someone whose work I both follow and have learned from) has a bee in his bonnet about Marc Hauser (and, it seems, Chomsky) (see here).  In many of his posts he has asserted that Hauser fraudulently published material that he knew to be false, and this is why he takes such a negative view of Hauser (and those like Chomsky who have been circumspect in their criticisms).  Well, Hauser has a new book out Evilicious (see here for a short review). Interestingly, the book has a blurb from Nicholas Wade, the NYTs science writer that covered the original case. More interestingly the post provides links to Wade’s coverage for the NYT of the original “case” and because I have nothing better to do with my time I decided to go back and read some of that coverage.  It makes for very interesting reading.  Here are several points that come out pretty clearly:

1.     The case against Hauser was always quite tenuous (see here and here).  Of the papers for which he as accused of fabrication, two were replicated very quickly to the satisfaction of Science’s referees. The problem for these was not fabrication, but not having the original tapes readily available.  Sloppy? Perhaps. Fraud? No evidence here.

2.     Of the eight charges of misconduct, five involved unpublished material.  This is a very high standard. I would be curious to know how many other scientists would like to be held responsible for what they did not publish.  A Dr. Galef from McMaster University (here) notes incredulously in the NYT (rightly in my view): “How in the world can they get in trouble for data they didn’t publish?”

3.     L’Affaire Hauser then devolves down to one experiment published in Cognition that Galef notes “was very deeply flawed.” However, as I noted (here) the results have since been replicated. That, like the Science replications, suggests that the original paper was not as flawed as supposed. Sloppy? Yes. Flawed? Perhaps (the replication suggests that what was screwed up did not matter much). Fraud? Possible, but the evidence is largely speculative.

4.     The NYT’s pieces provide a pretty good case that (at least one) outside scientists, who reviewed the case against Hauser, thought that “the accusations were unfounded.” Maybe Galef is a dupe, but his creds look ok to me. At any rate, someone with expertise in Hauser’s field reviewed the evidence against him and concluded that there was not enough evidence to conclude fraud, or anything close to it.

5.     Galef further noted that the Harvard investigating committee did not include people familiar with “the culture of an animal behavior laboratory,” which has “a different approach to research and data-keeping” than what one finds in other domains, especially the physical sciences from which the members of the Harvard investigating committee appeared to come from. I’m pretty sure that few behavioral or Social scientists would like to be subject to the standards of experimental hygiene characteristic of the work in the physical sciences. Is it possible that in the investigation that set the tone for what followed, Hauser was not judged by scientists with the right background knowledge?

6.     The final ORI report concentrates on the retracted (subsequently replicated) Cognition article (here). The claim is that “half the data in the graph were fabricated.” Maybe. It would be nice to know what the ORI based this judgment on. All involved admit that the experimental controls were screwed up and that some of the graphs, as reported, did not reflect the experiments conducted. I have no trouble believing that there was considerable sloppiness (though to repeat, it seems not to have been fatal given the subsequent replication), but this would not support ORI’s assertion of fabrication, a term that carries the connotation of intentional deceit.  I suspect that the ORI judgment rests on the prior Harvard findings. This leaves me thinking: why did this particular set of data get through the otherwise pretty reliable vetting process in Hauser’s lab, one that nixed earlier questionable data?  Recall, the data from five of the other investigated papers were vetted before being sent out and as a result the reported data were changed. What happened in this case? Why did this one slip through? I can understand ORI’s conclusion if tons of published data was fabricated. But this is precisely what there is no evidence for. Why then this one case?  Is it so unlikely that some goof up casued the slip? As Altmann, one of Hauser’s early accusers, notes in the NYT: “It is conceivable that the data were not fabricated, but rather that the experiment was set up wrong, and that nobody realized this until after it was published.” In the end, does the whole case against Hauser really devolve to what happened in this one case? With effectively one set of controls? Really?

I suspect that the real judgment against Hauser rests on the fact that he resigned from Harvard and that the Harvard committee initially set up to investigate him (but whose report, so far as I know, has never been publically made available) decided he was guilty. Again, as Altmann notes in the NYT: his earlier accusation was “heavily dependent on the knowledge that Harvard found Professor Hauser guilty of misconduct.” This, coupled with the thought that Hauser would not have quit were he innocent. But, it’s not a stretch to think of many reasons why a non-guilty person might quit, being fed up with one’s treatment perhaps topping the list. I am not saying that this is why Hauser resigned. I don’t know. But to conclude that he must be guilty because he left Harvard (who but a criminal would leave this august place, right? Though on second thought…) is hardly an apodictic conclusion. In fact, given all the evidence to date, it strikes me that the charges of fraud have very little evidential support, and that there are decent alternative explanations for what took place absent the intention to deceive. In fact, I would suggest that given the gravity of the charge, we should set a pretty high evidential bar before confidently declaring fraud. As far as I can tell from reading what’s in the NYT, this bar has not been remotely approached, let alone cleared.

I think that this will be the last post on this topic for me.  I have harped on it because, frankly, I think Hauser got a raw deal, from what I can tell, and that the condemnations he drew down on himself struck me as a tad too smug, uninformed, and self satisfied. As I’ve said before: fraud does not seem to me to be the biggest source of data pollution, nor the hardest one to root out.  However, such charges are serious and should not be leveled carelessly. Careers are at stake. And from what I can tell, in this particular case, Hauser was not given the benefit of the doubt, as he should have been.