More openness would be nice

monkey by Ivan Mlinaric

Hauser inquiry: privacy v reliable evidence:
[Via Butterflies and Wheels]

Harvard is keeping its findings secret, which leaves other researchers unsure which of Hauser’s experiments can be relied on.

[More]

 

While I mentioned this as an example of how science should work, it is also an example of how it does actually work, sometimes not for the best.

For some reason, Harvard is not really releasing the full results. So one paper has been retracted but it is still unclear if any others will be. That is the problem with these sorts of events, especilly ones caused by confirmation bias.

The entire body of work is now put into question. Even though he may be right on many things, without the supporting evidence – and strong evidence at that – even his good work may be down the tubes.

But we do not know for sure. This uncertainty is not really helpful, although I expect it will dissipate as time goes on. Either Harvard will release everything or people will repeat the work.

This is why it is so important for researchers to avoid confirmation bias, to really work hard at making sure the evidence really does strongly support the hypothesis you say it does. Because once you have a reputation for shoddy work like this, then everything you may ever have done comes under scrutiny.

When a scientist creates a Cargo Cult World – in this case one in which lower primates have similar cognitive skills as humans – it can often be a career ending event. If one is lucky, a rapid mea culpa and a lot of hard work may restore some respect.  That is why we work so hard trying to keep confirmation bias and other logical fallacies at bay.

As I found out at CalTech, the difference between a Nobel Prize and fraud can sometimes be very small, especially when working at the bleeding edge of science. But the Nobel Prize goes to those who were right. So you had better do everything you can to be correct.

And that includes having the data for others to examine. Otherwise all there will be is a Cargo Cult World with a single inhabitant.