Comments

Sunday, December 11, 2016

Two more little provocative pieces

I ran across two more things of interest.

The first is this post on Gelman's blog that reviews a recent paper in PNAS suggesting that the standard stats used to interpret fMRI findings are very very unreliable ("the general message is clear: don't trust FMRI p-values"). Gelman provides what seems to me a reasonable set of comments on all of this, including another discussion of the perverse incentives favoring statistical abuse. However, there is another issue that gets shorter shrift. It appears that even seasoned practitioners have a very hard time applying the techniques correctly (unless we make the silly assumption that most everyone using FMRI over the last 30 years is a fraud). This suggests that we ought to be very skeptical about any stats based report about anything. What the recent replication problems indicate is that even the best labs have a weak grasp of their stats tools.

Coming from a field which is often lectured on the  primitive nature of its data collection techniques,
I admit to experiencing quite a bit of very pleasant schadenfreude reading that the biggest problem in science today seems to be coming from just the techniques that my own field of interest has done without.  IMO, linguistics has done very well despite eschewing statistical sophistication, or indeed statistical crudeness. Of course I know the response: the right use of these stats techniques is what linguistics needs. My reply: first show me that the techniques can be reliably applied correctly! Right now it seems that this is far from obvious.

Indeed, it suggests a counter: maybe the right position is not to to apply the hard to apply technique correctly but to figure out how to get results that don't rely on these techniques at all. Call this Rutherford's dictum: "If your experiment needs statistics, you ought to have done a better experiment." One of the happy facts about most of linguistics is that our experiments, informal as they are, are generally speaking so good as to not require stats to interpret the results. Lucky us!

The second post is an interview with Freeman Dyson. It is short and fun. He says threes things that I found provocative and I'd be interested to hear opinions on them.

The first is his observation that great teachers are ones that can "find the right problem for each student, just difficult enough but not too difficult." I think that this is indeed one important mark of a great graduate mentor, and it is not something that I myself have been very good at. It also focuses on something that we often tend to take for granted. The capacity to generate good solvable problems is as important, maybe more important, that being able to provide solutions to said problems. Getting the problem "right" is more than half the battle, IMO, but I suspect that we tend to identify and value those that do this less than we should.

Second, Dyson rails against the PhD as a useful academic hurdle. He never received one and considers himself lucky never to have been required to. He thinks it antiquated, too arduous, and too intellectually disruptive.

Up to a point I agree. Certainly the classical thesis which develops a single topic over 300 pages with extensive critical review of the literature is more aimed at fields where the book is the primary research vehicle. Many places have long since replaced the book with the "stapled dissertation" where several research papers on possibly diverse topics are a thesis. This does not mean that a long form single topic thesis is a bad idea, only that a paper-oriented dissertation is a legit option. What the long form provides that the stapled essays don't is an opportunity to take a broader view of the discipline in which one is becoming a professional. Once one is a professional then until one attains senior status, thinking big is often (always?) frowned upon. This might be the only chance many get to see forests and not just trees. That said, I'd be curious to know what my younger colleagues think. And what if anything could replace the PhD that would be fair and useful.

Last point Dyson raises is where originality comes from. His vote, ignorance.
First of all, it helps to be ignorant. The time when I did my best work was when I was most ignorant. Knowing too much is a great handicap.
One of the complaints that older people always have about their younger colleagues concerns how little they know. Don't they know that we've done this before? Don't they know about so and sos research? Don't they know anything about the field before [put in very recent date here]? At any rate, what Dyson notes is that knowing too much may well be a problem. In fact, growing knowledge, rather than loss of energy coming with age, maybe what slows down senior scholars.

Syntacticians have had an antidote for this until recently. Chomsky used to change the landscape every decade or so, unnerving past students and emboldening young'uns. When junior you loved this. If senior you grumped. If Dyson is right, what Chomsky did was a great service for the field for he made it possible to be legitimately ignorant: things have changed so being erudite was not that important. Dyson's view is that ignorance is not so much bliss as liberating, allowing one to think about issues in new ways. Is he right? I'm not sure, but then look how old I am.
 

50 comments:

  1. With respect to the statistics and fMRI, I agree that the methods are definitely being used wrong. However, I think that the critics are wholly off-base in making statements like "These results question the validity of some 40,000 fMRI studies ". This is because we have massive priors concerning the function of different brain areas, as there are usually dozens of studies with very similar manipulations finding similar results in the same brain areas, and neuropsychological data that adds even more information. This varies depending on the experiment and the brain areas, but I think the far greater problem in neuroimaging is the lack of theory rather than the stats.

    For instance, take sentence processing. If you do a contrast of sentences > word lists, you will undoubtedly activate the anterior temporal lobe. I can easily think of ten studies off the top of my head that have replicated this effect, with very different kinds of materials, languages, modalities (speech vs. sign language), tasks, scanners, etc., and there are probably dozens of such studies. Many (maybe most) of these studies have not applied the correct stats. But who cares - I am not about to doubt that there is something about sentences that activates the anterior temporal lobe relative to word lists. Likewise for sentences with noncanonical sentence structure - a contrast of something like object relative > subject relative will undoubtedly active the left and right inferior frontal gyrus and probably the posterior temporal lobe. I am not about to doubt that there is something about noncanonicity that makes the inferior frontal gyrus activate, regardless of whether the stats were applied perfectly.

    The big problem is theory - nobody knows what is going on with respect to language and the brain, and I think that the hypotheses are only starting to get worse with time. The obsession with stats only drives people further away from theory, which means they are only going to miss the real problem even more. Maybe I am wrong about this with respect to other cognitive neurosciences, but I think that the general point that every fMRI study that is published adds information to our priors about brain areas, and this helps massively with inference to function.

    ReplyDelete
    Replies
    1. " the general point that every fMRI study that is published adds information to our priors about brain areas, and this helps massively with inference to function."
      These aren't priors then but posteriors, and that is exactly the problem.

      Delete
    2. The results of previous studies form the priors for future studies, do they not?

      My point is that there is convergence of results - that greatly strengthens our confidence in these results, does it not?

      Delete
    3. Well no. Not if the statistics in each are weak: that and the file-drawer effect may mean that an entire theory is based on noise.

      In social psychology, people are coming to realize that just having 10 studies that confirm some phenomena is not enough. You do a funnel plot and you can see that there is no real effect.

      It is fine to update your beliefs with each study, but that is done on the assumption that the statistical claims are correct. If they don't do the Bonferroni corrections (or whatever the problem is), then each study provides no reason to update your beliefs at all. So all your "priors" will be wrong.

      Delete
    4. I think that people that have not dealt with neuroimaging data come away with this impression. For instance, Pallier et al. (2011) showed that a set of 6 peri-sylvian brain regions showed enhanced activity for degree of linguistic structure, and that 3 of these regions showed this effect for both jabberwocky and natural stimuli, and the other regions only showed the effect for natural stimuli. We performed a modified replication of this study, showing nearly the exact effects: enhanced activity for more structure in the same network, and the same 3 regions showed the parallel jabberwocky/natural effect while the others showed the effect only for natural. I find it completely ridiculous that this happened by chance.

      This type of thing happens constantly in neuroimaging data for all sorts of paradigms. Maybe this is different for social psychology, but I am not about to believe these findings are all spurious. I AM about to believe that people have very little idea about what it means.

      Delete
    5. That observation does not conflict at all with the general methodological point I was making.

      Delete
  2. Second post containing interview details seems good because I am always fond of knowing new and successful people and knowing all about their entire life. dissertation writing services

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. But I LIKE theses because so many of them function very well as textbooks for professors ... the bibliographies tend to be more extensive and the presentations of the basic ideas slower and more careful than in journal articles.

    ReplyDelete
  5. Re Dyson's third point, consider the conclusion of an interview with molecular biologist Seymour Benzer:

    Aspaturian: I have one more question. The Indian physicist [Subrahmanyan] Chandrasekhar—he and Willy [William A.] Fowler [Institute Professor of Physics, emeritus] shared the Nobel Prize in 1983—is famous for completely changing the direction of his field every ten years or so. He’s said he does it partly for the reasons you enunciated and partly because he feels people tend to
    stagnate in their own fields. Did you consciously think that as well when you moved from one field into another? Was it sort of to rev yourself back up?

    Benzer: I’ll say it again. In every case I switched, it’s because of interest in something different. But I can’t divorce the excitement of that interest from the fact that it also meant getting away from the trappings of another subject that was getting too big. So subconsciously, that’s surely part of the motivation, partly an escape as well as an attraction. When a subject develops very thoroughly, there’s too much you have to know. It gets sort of overwhelming. So that’s why I was saying the big attraction is starting something new and being very stupid about it. Ask stupid questions, and you often get amazing answers.

    (http://resolver.caltech.edu/CaltechOH:OH_Benzer_S)

    ReplyDelete
  6. Very nice blog, Thanks for sharing grate article.
    You are providing wonderful information, it is very useful to us.
    Keep posting like this informative articles.
    Thank you.

    From: Field Engineer
    Deployment Engineer Jobs

    ReplyDelete
  7. Excellent blog information. Very interesting to read everyone. I really love to read such a nice and valuable content. the info you provide on this site has helped me greatly. Thanks for all of your time & work. online custom dissertation rates london -
    help with online economics dissertation help service -
    uk advanced fashion dissertation examples with experts

    ReplyDelete
  8. Hello, I have browsed most of your article posts. This release is probably where I got the most helpful and stunning information for my research on my life. Thanks for posting like this topic, maybe we can see more on this niche information. Thank you so much for this topic buddy. australia assignment -
    auditing assignment -
    university assignment help australia

    ReplyDelete
  9. Very nice blog, Thanks for sharing grate article.
    You are providing wonderful information, it is very useful to us.
    Keep posting like this informative articles.
    Thank you.
    office automation services
    hearing aid Pakistan
    hearing aids in lahore
    custom suits
    ad agency
    chemical industries in lahore
    Architecture Designs in lahore
    mbbs in china
    hearing clinic in lahore

    ReplyDelete
  10. Anchor Green tea is an excellent combination of 10 superb herbs & naturals including Terminalia Arjuna, Cardamom, Green Tea, Cinnamon, Senna Leaves, Aniseed
    green tea

    ReplyDelete
  11. Very useful information here, I really like this blog.

    Best Inventory Management System for small businesses!

    Zoho Inventory API

    ReplyDelete
  12. If you are unsatisfied with the quality of your assignments, you can generally request that we change it for you. We will be eager to assist you as long as you are not totally happy with the substance of your assignment. cheap assignment help , write my assignment

    ReplyDelete
  13. To make your writing more successful, you should go investigate the idea of the PEEL section. It assists you with giving the correct arrangement to your writing and keep on the right track. Critically, PEEL writing is useful when you are dealing with your article. In article writing, you need to introduce your contentions to accentuate a subject.

    ReplyDelete
  14. This comment has been removed by the author.

    ReplyDelete
  15. Hey guys! Nice post! Good info posted here and if you do not mind I want to use something while writing my papers. Just not to forget to say, I am a professional writer and I know how to help students with their academic papers and you can find already written examples of personal statements for graduate school in counseling where you can also find good articles for your future posts.

    ReplyDelete
  16. Thanks for writing this informative based case study blog.

    Small medium enterprise companies are referred to as the individual firms whose employee number varies from one country to the next. Both budget map and investment plan are accurately defined within all these organizations. Moving further towards the categories of small medium enterprise examples depending upon the name of the country and some sort of characteristics.

    ReplyDelete
  17. Sometimes it is better to let the professionals handle your academic assignments rather than sweat and slave on your own, wondering if you will get at least half of the assignment right, work with perfect writing https://perfectwritings.com/

    ReplyDelete
  18. Our essay writers Sydney are completely ready and comprehend the academic issues that you need to look during your business considers since they have gone through a similar as of now subsequently they convey predominant quality Globalization assignment help administration. They know about various types of referring to styles, configurations, designs and sorts of papers requested by your colleges so you can trust our essay writers Sydney. essay writing help
    assignmenthelp

    ReplyDelete
  19. This comment has been removed by the author.

    ReplyDelete
  20. Very informative blog. Keep sharing this wonderful blog.
    If you need someone professional to write your assignments then my assignment help service will provide 100% plagiarism free assignments under your budget.

    ReplyDelete
  21. Hey, I'm Nora when i read your blog this is the very important guideline
    really good information thanks. if you suffering your PC and any internet security so you may go and read this is the very important for you.

    Netgear Extender Setup
    Bitdefender Login
    Avast Login

    ReplyDelete
  22. Very interesting article. It was a real treat to come across your post. I am currently preparing paper writings I would be happy to mention your article in this.

    ReplyDelete
  23. I started using this custom writing service. I often use additional services. This is a very important service for my student life. I can save my time.

    ReplyDelete

  24. Nice Post when I read this I fell glad Thanks for Sharing Here is the my HBO Max Watch party you can go and enjoy this and also another one Technical services Read this AOL Mail
    HBO Max Watch Party

    ReplyDelete
  25. Paypal is one of the leading payment transfer apps in the recent era. Most of us are well aware of PayPal and how it works. In this blog, I am going to answer some of your frequently asked questions and also teach you how you can create an account in PayPal. Paypal has a strong customer base with 286 million active customers and it is increasing day by day. The reason behind this success of PayPal is there exciting features like one-tap pay gives customer relief. The most important thing about PayPal is that you have to worry about your banking credentials.

    primevideo.com/mytv |
    amazon.com/code |
    hbomax.com/tvsignin |
    www.amazon.com/mytv |
    aol mail login |

    ReplyDelete
  26. Dobra web stranica : Terluas
    Dobra web stranica : Resep Masakan
    Dobra web stranica : One Piece
    Dobra web stranica : One Piece
    Dobra web stranica : One Piece
    Dobra web stranica : One Piece
    Dobra web stranica : Biodata
    Dobra web stranica : Resep

    ReplyDelete
  27. In Astrology, 8th house jupiter makes a person a healer, with a strong spiritual mind, long life, and the native's desire to rise.

    ReplyDelete
  28. Good choice. Provocative pieces are good for discussion or open dialog with arguments. I will explore this topic thoroughly and even might consider to write ashort essay sample for my blog http://essay-writer-help.com/

    ReplyDelete
  29. This comment has been removed by the author.

    ReplyDelete
  30. Safe Dryver Dubai service is a professional driving service that provides a safe and reliable driving experience for customers in Dubai. The service provides experienced and licensed drivers who are knowledgeable about the city's roads, traffic patterns, and road safety regulations.

    ReplyDelete
  31. This comment has been removed by the author.

    ReplyDelete
  32. Wow, This blog piqued my interest also a fitness trainer ahsam khan is a professional who designs and implements fitness programs to help clients achieve their health and fitness goals. They may work one-on-one with clients, lead group fitness classes, or design and manage fitness programs for individuals, groups, or organizations. Fitness trainers typically have knowledge of anatomy, exercise science, and nutrition, and may also be certified by a recognized fitness organization.

    ReplyDelete
  33. Thanks for sharing this information, BookMyEssay is also a reliable online platform that offers professional assignment writer in Sydney. Their team of expert writers provides customized and plagiarism-free assignments within the given deadline.

    ReplyDelete
  34. Hello! If you are a student who is still looking for a good film review writing service, visit qualityessay.com because it is an all-round company which can help every student.

    ReplyDelete
  35. ترافل are decadent chocolate confections, often infused with various flavors like nuts, fruits, or liqueurs, offering a luxurious treat for chocolate enthusiasts around the world.

    ReplyDelete
  36. Looking for fly ash Arkansas Texas? Our premium-grade product ensures concrete strength and sustainability for construction projects. Trust our reliable supply and exceptional quality for your building needs.

    ReplyDelete
  37. The findings discussed underscore the importance of critically evaluating statistical methods in fMRI research. For a workspace that encourages critical thinking and innovation, consider collaborating with a leading fit out company in UAE to create an environment conducive to research and discovery.

    ReplyDelete
  38. Two more little provocative pieces" ignite curiosity! Just as these pieces provoke thought, ensure your workforce is fueled and ready with Lunch catering in Houston Texas. Let the provocative discussions be complemented by a robust catering service, providing the energy needed for innovation in the heart of the oil fields. A dynamic blend of intellectual stimulation and nourishment for a thriving workplace.

    ReplyDelete
  39. Two more little provocative pieces" intrigues the mind. Just as these pieces evoke thought, ensure the smooth operation of your processes with hydrocarbon pumps Aberdeen Let the provocative nature of ideas align with the reliability of pump technology, creating a balanced and efficient system. A harmonious blend of intellectual stimulation and industrial precision!

    ReplyDelete
  40. Two more little provocative pieces, a tantalizing glimpse into the world of creativity. Just as art pushes boundaries, let a wood fencing contractor in Edmontonredefine and enhance your property's aesthetic and security. Craft a bold statement both in your space and in your artistic endeavors

    ReplyDelete
  41. Two more little provocative pieces" – an intriguing invitation into creativity! Stimulate your imagination with the richness of ideas, paired perfectly with the aromatic allure ofcoffee beans dubai. A creative blend to ignite inspiration!

    ReplyDelete
  42. It's fascinating how perspectives vary across fields! 🧪📚 Embracing diverse techniques could pave the way for unexpected breakthroughs. Fit out company Abu Dubai

    ReplyDelete
  43. This review on Gelman's blog raises crucial points about the reliability of fMRI findings. It's a reminder to critically assess statistical methods in neuroscience research. Also, if you need emergency demolition contractors in Edmonton, consider reliable professionals for prompt assistance. 🏗️

    ReplyDelete
  44. Intrigued by the provocatively crafted pieces! Dive into these creations and savor the moments with a side of indulgence. الشوكولاته تسليم المنزل can add the perfect touch of sweetness to your contemplative reading session. #ProvocativeReads #ChocolateIndulgence

    ReplyDelete