Two more little provocative pieces

I ran across two more things of interest.

The first is this post on Gelman's blog that reviews a recent paper in PNAS suggesting that the standard stats used to interpret fMRI findings are very very unreliable ("the general message is clear: don't trust FMRI p-values"). Gelman provides what seems to me a reasonable set of comments on all of this, including another discussion of the perverse incentives favoring statistical abuse. However, there is another issue that gets shorter shrift. It appears that even seasoned practitioners have a very hard time applying the techniques correctly (unless we make the silly assumption that most everyone using FMRI over the last 30 years is a fraud). This suggests that we ought to be very skeptical about any stats based report about anything. What the recent replication problems indicate is that even the best labs have a weak grasp of their stats tools.

Coming from a field which is often lectured on the  primitive nature of its data collection techniques,
I admit to experiencing quite a bit of very pleasant schadenfreude reading that the biggest problem in science today seems to be coming from just the techniques that my own field of interest has done without.  IMO, linguistics has done very well despite eschewing statistical sophistication, or indeed statistical crudeness. Of course I know the response: the right use of these stats techniques is what linguistics needs. My reply: first show me that the techniques can be reliably applied correctly! Right now it seems that this is far from obvious.

Indeed, it suggests a counter: maybe the right position is not to to apply the hard to apply technique correctly but to figure out how to get results that don't rely on these techniques at all. Call this Rutherford's dictum: "If your experiment needs statistics, you ought to have done a better experiment." One of the happy facts about most of linguistics is that our experiments, informal as they are, are generally speaking so good as to not require stats to interpret the results. Lucky us!

The second post is an interview with Freeman Dyson. It is short and fun. He says threes things that I found provocative and I'd be interested to hear opinions on them.

The first is his observation that great teachers are ones that can "find the right problem for each student, just difficult enough but not too difficult." I think that this is indeed one important mark of a great graduate mentor, and it is not something that I myself have been very good at. It also focuses on something that we often tend to take for granted. The capacity to generate good solvable problems is as important, maybe more important, that being able to provide solutions to said problems. Getting the problem "right" is more than half the battle, IMO, but I suspect that we tend to identify and value those that do this less than we should.

Second, Dyson rails against the PhD as a useful academic hurdle. He never received one and considers himself lucky never to have been required to. He thinks it antiquated, too arduous, and too intellectually disruptive.

Up to a point I agree. Certainly the classical thesis which develops a single topic over 300 pages with extensive critical review of the literature is more aimed at fields where the book is the primary research vehicle. Many places have long since replaced the book with the "stapled dissertation" where several research papers on possibly diverse topics are a thesis. This does not mean that a long form single topic thesis is a bad idea, only that a paper-oriented dissertation is a legit option. What the long form provides that the stapled essays don't is an opportunity to take a broader view of the discipline in which one is becoming a professional. Once one is a professional then until one attains senior status, thinking big is often (always?) frowned upon. This might be the only chance many get to see forests and not just trees. That said, I'd be curious to know what my younger colleagues think. And what if anything could replace the PhD that would be fair and useful.

Last point Dyson raises is where originality comes from. His vote, ignorance.

First of all, it helps to be ignorant. The time when I did my best work was when I was most ignorant. Knowing too much is a great handicap.

One of the complaints that older people always have about their younger colleagues concerns how little they know. Don't they know that we've done this before? Don't they know about so and sos research? Don't they know anything about the field before [put in very recent date here]? At any rate, what Dyson notes is that knowing too much may well be a problem. In fact, growing knowledge, rather than loss of energy coming with age, maybe what slows down senior scholars.

Syntacticians have had an antidote for this until recently. Chomsky used to change the landscape every decade or so, unnerving past students and emboldening young'uns. When junior you loved this. If senior you grumped. If Dyson is right, what Chomsky did was a great service for the field for he made it possible to be legitimately ignorant: things have changed so being erudite was not that important. Dyson's view is that ignorance is not so much bliss as liberating, allowing one to think about issues in new ways. Is he right? I'm not sure, but then look how old I am.