Revolutions in science; a comment on Gelman

In what follows I am going to wander way beyond my level of expertise (perhaps even rudimentary competence). I am going to discuss statistics and its place in the contemporary “replication crisis” debates. So, reader be warned that you should take what I write with a very large grain of salt. 

Andrew Gelman has a long post (here, AG) where he ruminates about a comparatively small revolution in statistics that he has been a central part of (I know, it is a bit unseemly to toot your own horn, but heh, false modesty is nothing to be proud of either). It is small (or “far more trivial”) when compared to more substantial revolutions in Biology (Darwin) or Physics (Relativity and Quantum mechanics), but AG argues that the “Replication revolution” is an important step in enhancing our “understanding of how we learn about the world.” It may be right. But…

But, I am not sure that it has the narrative quite right. As AG portrays matters, the revolution need not have happened. The same ground could have been covered with “incremental corrections and adjustments.” Why weren’t they? The reactionaries forced a revolutionary change because of their reactions to reasonable criticisms by the likes of Meehl, Mayo, Ioannidis, Gelman, Simonsohn, Dreber, and “various other well-known skeptics.” Their reaction to these reasonable critiques was to charge the critics with bullying or insist that the indicated problems are all part of normal science and will eventually be removed by better training, higher standards etc. This, AG argues, was the wrong reaction and required a revolution, albeit a minor one relatively speaking, to overturn.

Now, I am very sympathetic to a large part of this position. I have long appreciated the work of the critics and have covered their work in FoL. I think that the critics have done a public service in pointing out that stats has served to confuse as often (maybe more often) than it has served to illuminate. And some have made the more important point (AG prominently among them) that this is not some mistake, but serves a need in the disciplines where it is most prominent (see here). What’s the need? Here is AG:[1]

Not understanding statistics is part of it, but another part is that people—applied researchers and also many professional statisticians—want statistics to do things it just can’t do. “Statistical significance” satisfies a real demand for certainty in the face of noise. It’s hard to teach people to accept uncertainty. I agree that we should try, but it’s tough, as so many of the incentives of publication and publicity go in the other direction.

And observe that the need is Janus faced. It faces inwards to relieve the anxiety of uncertainty and it faces outwards in relieving professional publish-or-perish anxiety. Much to AG’s credit he notices that these are different things, though they are mutually supporting. I suspect that the incentive structure is important, but secondary to the desire to “get results” and “find the truth” that animates most academics. Yes, lucre, fame, fortune, status are nice (well, very nice) but I agree that the main motivation for academics is the less tangible one, wanting to get results just for the sake of getting them. Being productive is a huge goal for any academic, and a big part of the lure of stats, IMO, is that it promises to get one there if one just works hard and keeps plugging away. 

So, what AG says about the curative nature of the mini-revolution rings true, but only in part. I think that the post fails to identify the three main causal spurs to stats overreach when combined with the desire to be a good productive scientist.

The first it mentions, but makes less off than perhaps others have. It is that stats are hard and interpreting them and applying them correctly takes a lot of subtlety. So much indeed that even experts often fail (see here). There is clearly something wrong with a tool that seems to insure large scale misuse. AG in fact notes this (here), but it does not play much of a role in the post cited above, though IMO it should have. What is it about stats techniques that make them so hard to get right? That I think is the real question. After all, as AG notes, it is not as if all domain find it hard to get things right. As he notes, psychometricians seem to get their stats right most of the time (as do those looking for the Higgs boson). So what is it about those domains where stats regularly fails to get things right that makes it the case that they so generally fail to get things right?  And this leads me to my second point.

Stats techniques play an outsized role in just those domains where theory is weakest. This is an old hobby horse of mine (see here for one example). Stats, especially fancy stats, induces the illusion that deep significant scientific insights are for the having if one just gets enough data points and learns to massage them correctly (and responsibly, no forking paths for me thank you very much). This conception is uncomfortable with the idea that there is no quick fix for ignorance. No amount of hard work, good ethics, or careful application suffices when we really have no idea what is going on. Why do I mention this? Because, in many of the domains where the replication crisis has been ripest are domains that are very very hard and where we really don’t have much of an understanding of what is happening. Or maybe to put this more gracefully, either the hypotheses of interest are too shallow and vague to be taken seriously (lots of social psych) or the effects of interest are the results of myriad interactions that are too hard to disentangle. In either case, stats will often provide an illusion of rigor while leading one down a forking garden path. Note, if this is right, then we have no problem seeing why psychometricians were in no need of the replication revolution. We really do have some good theory in the domains like sensory perception, and here stats have proven to be reliable and effective tools. The problem is not with stats, but with stats applied where they cannot be guided (and misapplications tamed) by significant theory.

Let me add two more codicils to this point.

First, here I part ways with AG. The post suggests that one source of the replication problem is with people having too great “an attachment to particular scientific theories or hypotheses.” But if I am right this is not the problem, at least not the problem behind the replication crisis. Being theoretically stubborn may make you wrong, but it is not clear why it makes your work shoddy. You get results you do not like and ignore them. That may or may not be bad. But with a modicum of honesty, the most stiff necked theoretician can appreciate that her/his favorite account, the one true theory, appears inconsistent with some data. I know whereof I speak, btw. The problem here, if there is one, is not generating misleading tests and non-replicable results, but if ignoring the (apparent) counter data. And this, though possibly a problem for an individual, may not be a problem for a field of inquiry as a whole. 

Second, there is a second temptation that today needs to be seriously resisted but that severely leads to replication problems: because of the ubiquity and availability of cheap “data” nowadays, the temptation to think that this time it’s different is very alluring. Big Data types often seem to think that get a large enough set of numbers, apply the right stats techniques (rinse and repeat) and out will plop The Truth. But this is wrong. Lars Syll puts it well here in a post entitled correctly “Why data is NOT enough to answer scientific questions”:

The central problem with present ‘machine learning’ and ‘big data’ hype is that so many –falsely- think that they can ge away with analyzing real-world phenomena without any (commitment to) theory. But –data never speaks for itself. Without a prior statistical set-up, there actually are no data at all to process. And – using a machine learning algorithm will only produce what you are looking for.

Clever data mining tricks are never enough to answer important scientific questions. Theory matters.

So, when one combines the fact that in many domains we have, at best, very weak theory, and that nowadays we are flooded with cheap available data the temptation to go hyper statistical can be overwhelming.

Let me put this another way. As AG notes, successful inquiry needs strong theory and careful measurement. Not the ‘and.’ Many read the ‘and’ as an ‘or’ and allow that strong theory can substitute for paucity of data or that tons of statistically curated data can substitute for virtual absence of significant theory. But this is a mistake. But a very tempting one if the alternative is having nothing much of interest or relevance to say at all. And this is what AG underplays: a central problem with stats is that it often tries to sell itself as allowing one to bypass the theory half of the conjunction. Further, because it “looks” technical and impressive (i.e. has a mathematical sheen) it leads to cargo cult science, scientific practice that looks "science" rather than being scientific. 

Note, this is not bad faith or corrupt practice (though there can be this as well). This stems from the desire to be, what AG dubs, a scientific “hero,” a disinterested searcher for the truth. The problem is not with the ambition, but the added supposition that any problem will yield to scientific inquiry if pursued conscientiously. Nope. Sorry. There are times when there is no obvious way to proceed because we have no idea how to proceed. And in these domains no matter how careful we are we are likely to find ourselves getting nowhere.

I think that there is a third source of the problem that resides in the complexity of the problems being studied. In particular, the fact that many phenomena we are interested in arise from the interaction of many causal sub-systems. When this happens there is bound to be a lot of sensitivity to the particular conditions of the experimental set up and so lots of opportunities for forking paths (i.e. p-hacking) stats (unintentional) abuse. 

Now, every domain of inquiry has this problem and needs to manage it. In the physical sciences this is done by (as Diogo once put it to me) “controlling the shit out of the experimental set up.” Physicists control for interaction effects by removing many (most) of the interfering factors. A good experiment requires creating a non-natural artificial environment in which problematic factors are managed via elimination. Diogo convinced me that one of the nice features of linguistic inquiry is that it is possible to “control the shit” out of the stimuli thereby vastly reducing noise generated by an experimental subject. At any rate, one way of getting around interaction effects problem is to manage the noise by simplifying the experimental set up and isolating the relevant causal sub-systems.

But often this cannot be done, among other reasons because we have no idea what the interacting subsystems are or how they function (think, for example, pragmatics).  Then we cannot simplify the set up and we will find that our experiments are often task dependent and very noisy. Stats offers a possible way out. In place of controlling the design of the set up the aim is to statistically manage (partial out) the noise. What seems to have been discovered (IMO, not surprisingly) is that this is very hard to do in the absence of relevant theory. You cannot control for the noise if you have no idea where it comes from or what is causing it. There is no such thing as a theory free lunch (or at least not a nutritious one). The revolution AG discusses, I believe, has rediscovered this bit of wisdom.

Let me end with an observation special to linguistics. There are parts of linguistics (syntax, large parts of phonology and morphology) where we are lucky in that the signal from the underlying mechanisms are remarkably strong in that they withstand all manner of secondary effects. Such data are, relatively speaking, very robust. So, for example, ECP or island or binding violations show few context effects. This does not mean to say that there are no effects at all of context wrt acceptability (Sprouse and Co. have shown that these do exist). But the main effect is usually easy to discern. We are lucky. Other domains of linguistic inquiry are far noisier (I mentioned pragmatics, but even large parts of semantics strike me as similar (maybe because it is hard to know where semantics ends and pragmatics begins)). I suspect that a good part of the success of linguistics can be traced to the fact that FL is largely insulated from the effects of the other cognitive subsystems it interacts with. As Jerry Fodor once observed (in his discussion of modularity), the degree to which a psych system is modular to that degree it is comprehensible. Some linguists have lucked out. But as we more and more study the interaction effects wrt language we will run into the same problems. If we are lucky, linguistic theory will help us avoid many of the pitfalls AG has noted and categorized. But there are no guarantees, sadly.

---

[1]I apologize for not being able to link to the original. It seems that in the post where I discussed it, I failed to link to the original and now cannot find it. It should have appeared in roughly June 2017, but I have not managed to track it down. Sorry.

Some thoughts on the relation between syntax and semantics, through the lens of compositionality: https://omer.lingsite.org/blogpost-meaning-based-syntax-co…/

Language and cognition and evolang

Just back from vacation and here is a starter post on, of all things, Evolang (once again).  Frans de Waal has written a short and useful piece relevant to the continuity thesis (see here). It is useful for it makes two obvious points, and it is important because de Waal is the one making them. The two points are the following:

1.     Humans are the only linguistic species.

2.     Language is not the medium of thought for there are non-verbal organisms that think.

Let me say a few words about each.

de Waal is quite categorical about the each point. He is worth quoting here so that next time you hear someone droning on about how we are just like other animals just a little more so you can whip out this quote and flog the interlocutor with it mercilessly. 

You won’t often hear me say something like this, but I consider humans the only linguistic species. We honestly have no evidence for symbolic communication, equally rich and multifunctional as ours, outside our species. (3)

That’s right: nothing does language like humans do language, not even sorta kinda. This is just a fact and those that know something about apes (and de Waal knows everything about apes) are the first to understand this. And if one is interested in Evolang then this fact must form a boundary condition on whatever speculations are on offer. Or, to put this more crudely: assuming the continuity thesis disqualifies one from participating intelligently in the Evolang discussion. Period. End of story. And, sadly, given the current state of play, this is a point worth emphasizing again and again and again and… So thanks to de Waal for making it so plainly.

This said, De Waal goes on to make a second important point: that even if it is the case that no other animals have our linguistic capacities even sorta kinda, it does not mean that some of the capacities underlying language might not be shared with other animals. In other words, the distinction between a faculty for language in the narrow versus a faculty for language in the broad sense is a very useful one (I cannot recall right now who first proposed such a distinction, but whoever it was thx!). This, of course, cheers a modern minimalist’s heart cockles, and should be another important boundary condition on any Evolang account. 

That said, de Waal’s two specific linguistic examples are of limited use, at least wrt Evolang. The first is bees and monkeys who de Waal claims use “sequences that resemble a rudimentary syntax” (3). The second “most intriguing parallel” is the “referential signaling” of vervet monkey alarm calls. I doubt that these analogous capacities will shed much light on our peculiar linguistic capacities precisely because the properties of natural language words and natural language syntax are where humans are so distinctive. Human syntax is completely unlike bee or monkey syntax and it seems pretty clear that referential signaling, though it is one use to which we put language, is not a particularly deep property of our basic words/atoms (and yes I know that words are not atoms but, well, you know…). In fact, as Chomsky has persuasively argued IMO, Referentialism (the doctrine (see here)) does a piss poor job of describing how words actually function semantically within natural language. If this is right, then the fact that we and monkeys can both engage in referential signaling will not be of much use in understanding how words came to have the basic odd properties they seem to have. 

This, of course, does not detract from de Waal’s correct two observations above. We certainly do share capacities with other animals that contribute to how FL functions and we certainly are unique in our linguistic capacities. The two cases of similarity that de Waal cites, given that they are nothing like what we do, endorses the second point in spades (which, given the ethos of the times, is always worth doing).

Onto point deux. Cognition is possible without a natural language. FoLers are already familiar with Gallistel’s countless discussions of dead reckoning, foraging, and caching behavior in various animals. This is really amazing stuff and demands cognitive powers that dwarf ours (or at least mine: e.g. I can hardly remember where I put my keys, let alone where I might have hidden 500 different delicacies time stamped, location stamped, nutrition stamped and surveillance stamped). And they seem to do this without a natural language. Indeed, the de Waal piece has the nice feature of demonstrating that smart people with strong views can agree even if they have entirely different interests. De Waal cites none other than Jerry Fodor to second his correct observation that cognition is possible without natural language. Here’s Jerry from the Language of Thought:

‘The obvious (and I should have thought sufficient) refutation of the claim that natural languages are the medium of thought is that there are non-verbal organisms that think.’ (3)

Jerry never avoided kicking a stone when doing so was all a philosophical argument needed. At any rate, here Fodor and de Waal agree. 

But I suspect that there would be more fundamental disagreements down the road. Fodor, contra de Waal, was not that enthusiastic of the idea that we can think in pictures, or at least not think in pictures fundamentally. The reason is that pictures have little propositional structure and thinking, especially any degree of fancy thinking, requires propositional structures to get going. The old Kosslyn-Pylyshyn debate over imagery went over all of this, but the main line can be summed up by one of Lila Gleitman’s bon mots: a picture is worth a thousand words, and that is the problem. Pictures may be useful aids to thinking, but only if supplied with captions to guide the thinking. In and of themselves, pictures depict too much and hence are not good vehicles for logical linkage. And if this is so (and it is) then where there is cognition there may not be natural language, but there must be a language of thought (LOT) (i.e. something with propositional structure that licenses the inferences that is characteristic of cognitive expansiveness in a given domain). 

Again this is something that warms a minimalist heart (cockles and all). Recall the problem: find the minimal syntax to link the CI system and the AP system. CI is where language of thought lives. So, like de Waal, minimalists assume that there is quite a lot of cognition independent of natural language, which is why a syntax that links to it is doing something interesting.

Truth be told, we know relatively little about LOT and it properties, a lot less that we know about the properties of natural language syntax IMO. But regardless, de Waal and Fodor are right to insist that we not mix up the two. So don’t.

Ok, that’s enough for an inaugural post vaca post. I hope your last two weeks were as enjoyable as mine and that you are ready for the exciting pedagogical times ahead.

A sort-of BS follow-up: Alternatives to LMS

Norbert's recent post on the BSification of academic life skipped one particular wart that's at the front of my mind now that I'm in the middle of preparing classes for the coming semester: Learning Management Systems (LMS). In my case, it's Blackboard, but Moddle and Canvas are also common. It is a truth universally acknowledged that every LMS sucks. It's not even mail client territory, where the accepted truth is "all mail clients suck, but the one I use sucks less". LMS are slow, clunky, inflexible, do not support common standards, and quite generally feel like they're designed by people who couldn't cut it at a real software company. They try to do a million things, and don't do any of them well. Alright, this is where the rant will stop. Cathartic as it may be, these things have been discussed a million times. Originally I had an entire post here that explores how the abysmal state of LMS is caused by different kinds of academic BS, but I'd like to focus on something more positive instead: alternatives to LMS that work well for me and might be useful for you, too. If somebody really wants to hear me complain about LMS and the academic BS surrounding them, let me know in the comments and I'll post that part as a follow-up.

The BSification of everyday academic life

One of the most useful philosophy tracts written in the last 25 years is Harry Frankfurt’s On Bullshit (OB, here). OB makes an important distinction between lying and bullshitting, the latter being the more insidious as, in contrast to the former that shows a regard for the truth (by deliberately contradicting it), the latter could care less. BS’s insidiousness arise from two features: (i) its actual disregard for the truth and (ii) its great regard to appearto be true. Thus, BS prizes truthiness (h/t to Colbert), but could care less about truth.

This is a powerful insight, and it has been weaponized. Tobacco companies and the large fossil fuel energy companies have understood that the best way to stop rational action is to obfuscate the intellectual terrain (here). The aim is not to persuade so much as to make it impossible to conclude. Ignorance really is bliss for some and BS is a very good way of spreading it. 

The institutional BS business is now widespread enough that there are academics that study it. The problem of how doubt is spread is now an academic discipline with a Greek rooted name, ‘agnotology’ (here), to demarcate it from other kinds of rhetorical studies (BS, MS, PhD indeed).[1]

In this post, I point you to a second useful theoretical treatise on the topic, one that expands the BS descriptor from ideas to occupations. David Graeber (DG) has a new book out on the topic and an interview where he discusses its main thesis (here). It notes that BS decisively shapes the ecology of the workplace so that some jobs are best understood as BS positions. What makes one such? A job is BS (BSJ) when it is “so pointless that even the person doing the job secretly believes that it shouldn’t exist” (1). Anyone in an academic environment can probably point to several such (the hierarchy of assistant deans/provosts (and their assistants) is a good place to look for of BSJs). DG has a nice taxonomy that I commend to your attention. There are at least six categories: flunkies, goons, duct-tapers, box-tickers, task-makers, and bean-counters. I am sure you can figure out their respective skill sets from the evocative titles, but what makes DG’s discussion illuminating is his anthro-socio take on these positions and what forces lead to their proliferation even in enterprises whose aim is to make money. Universities, where lucre is not the obvious organizing principle, act like hothouses and the most exotic versions of these six BSJs are spotted regularly, especially when manured with just a dollop of the latest philosophy from our leading schools of business and management.[2]

So, BS abounds. But, sadly, it is not just confined to specific jobs. It is everywhere. We need another category: BS activities (BSA). BSAs are now part of even the necessary parts of life. Here are some observations concerning how BSAs are now standard features of even the good parts of academic life.

I have ranted before about how the “wider consequences” sections of NSF and NIH funding grants have grown in importance. This is the last section of the grant where you have to say how developing a superior theory of case and agreement will lead to a cure for world hunger, cancer and aphasia. What makes this BS is not merely that it is clearly untrue and unfounded, but that everyone knows that it is, knows that everyone knows that it is, knows that everyone knows that everyone knows that everyone knows that it is…In short, it is BS that everyone recognizes as BS and nonetheless the process demands that everyone take it seriously enough to act as if it is not. In fact, this is critical: what makes BS insidious is not merely that it could care less about the truth of the matter, but when institutionalized it requires that those who deal with it to take it seriously. BS recognized as such can be funny, and even subversive (Colbert has made a career on this). But BS requirements in an NSF/NIH grant cannot be laughed away. They must be taken seriously and all involved are forced to pretend that what is obvious BS is not. 

And this is what makes it so insidious in academic life. Optimists hope that it can be circumscribed to its own little section of the grant (near the end) and limit its affects. But this is a BS hope. Like the camel’s nose under the tent, once in it spreads everywhere and quickly. How quickly? Here’s a conjecture: the prevalence of institutionalized BS is a contributing factor to the replication crisis. As noted, BS prizes truthiness (the appearance of truth) stats magic provides packaged (as in stats packages) ways to manufacture truthiness (recall Twain’s “lies, damn lies and statistics”), so with the rise of BS and the strong incentive to avoid being recognized asBS, we get it everywhere camouflaged in stats. Som,first, the end sections of the grant, then everywhere. There really is a cost to playing along.

Here is a second recent personal example. I was asked to write a “minimalist” chapter for a volume comparing theoretical approaches in linguistics. I agreed.And this was a mistake. First, others would have done a more mainstream job of it. I am quite certain my take on things is quite idiosyncratic. But, moreover, I did not really think that what I had to say really fit in with the spirit of the other contributions (and the reviews made just this point). However, I agreed. And I am lucky I did for it allowed me to experience another small place where BS thrives in academe. Let me relate.

As many of you know, when you contribute something to a non open source publisher you sign away the rights to this work as part of the process of publication. In this case, I got the standard 5 page contract, but this time I read it. It was completely incomprehensible, though from what I could make out it it basically delivered allthe rights to the paper (ideas in it, phrasing of these ideas, everything) to the publisher. It also forbade me from using the paper or a version thereof in the future. As the publisher was European, there were references to various EU laws that underlay the codicils in the contract. I was assured that the contract was pretty standard and that I should sign on the dotted line.

But, I did not like the idea that the paper’s contents no longer belonged to me. And because I am old and am no longer all that focused on padding my CV and can afford to loose the $0.00 royalty check that this chapter was going to generate and am not worried about further academic advancement and…I decided to try to understand what it was that the contract was actually saying and what rights I was actually signing away in return for what services. That was where the complete BSishness of the whole thing became evident.

First the remuneration: The obligation of the publisher was to publish the paper and give me a copy of the tomb in which it would appear. In this day and age, receiving such a door stopper is more like receiving the proverbial white elephant than real remuneration. But, that was it. In return I had to do a whole bunch of things to get the MS in shape, within a certain time frame etc. All in all, the demands were not unreasonable. 

What did I give up? Well, all rights to the paper, or this is what I thought the contract stipulated. I replied that I did not like this idea as I intended to use the material again in a larger project. I also asked about the standard EU laws the contract bandied about and that the publisher insisted needed to be adhered to. And here is when things got fun.

It turns out that nobody I talked to knew what these laws were. Nobody could tell me. Moreover, everyone assured me that regardless of what the contract said, it really didn’t matter because it would not be enforced. Should I decide to use the material again in another project (after a year’s time) the publisher would do nothing about it. Or, more accurately, the publisher rep told me that they knew of no case where anyone who used their own work in future work was called to the mat for doing so. In other words, the contract was a purely formal object whose content was BS and yet we were all obliged to take that content seriously as if it were not BS so that we could get on with ignoring it and get the book published.

Things proceeded from there to the point where we agreed to a two line contract that basically said that they could use the paper, I would not use it for a year and that I could use it after this time as I saw fit. Nobody ever explained the EU laws to me (or to themselves), nobody batted an eye when these EU laws were dropped from the final contract, nobody was concerned to do anything but make this thing go away and have a signed document of little relevance to what was actually happening (or so I was repeatedly assured). It just needed to get signed. And so I did. At least the two codicil version. Pure BS.

The contract that is opaque to all that sign it and all that ask that it be signed is another example of the ritualization of BS in academic life. And like the NSF/NIH version it coarsens intellectual life. Let me say this more strongly, it is especiallycorrosive of academic life. Academics are people whose professional obligations involve taking ideas seriously. In fact, this is the main thing we are trained to do, at least within some small domain. This is the core value: be serious about thoughts! BS is the vice that most challenges this virtue. It insists that you take none if it seriously, because seriousness about ideas is what BS is meant to undermine. BS, BS jobs, BS activities, BS forms, BS sections of forms…, all serve to undermine this seriousness. It grubbies the mind and it keeps on coming. The optimistic view is that it does not really matter. The pessimistic view is that it is too late to change it. The moderately hopeful view is that eternal vigilance is the only possible defense. I swing between the second and the third. 

---

[1]Bull sh*t, more sh*t, piled high and deep!

[2]See herefor an amusing take. But beware: the piece will play to many of your prejudices and so should be read critically. For someone like me, it is all too easy to believe most every judgment passed. Still, a sample quote might whet your appetite (1):

As a principal and founding partner of a consulting firm that eventually grew to 600 employees, I interviewed, hired, and worked alongside hundreds of business-school graduates, and the impression I formed of the M.B.A. experience was that it involved taking two years out of your life and going deeply into debt, all for the sake of learning how to keep a straight face while using phrases like “out-of-the-box thinking,” “win-win situation,” and “core competencies.”

I am sure that it has not escaped your notice that this is high class BS and so now we are in the delightful situation where part of the university manufactures what the other part studies. We may have discovered an intellectual perpetual motion machine.

Slime molds and plants

A while ago I mentioned work done suggesting that Gallistel's conjecture that cognitive computation does not require neural nets is correct. The work discussed "learning" in single cell slime molds and plants. At any rate, this stuff i going mainstream in that Quanta brings this research together in this review (republished in Wired).

The piece focuses on the controversy of whether this can actually be "primitive cognition" noting that for many cognition is only something that brains can do (by brains kogneuro types mean ensembles of neurons). The fear is that this kind of research amounts to "'devaluing' of the specialness of the brain" (12). Others comfort these kogneuro fears by claiming that the "debate is arguably not a war about science, but about words" (13).

Both claims are wrongheaded. These studies are direct challenges to the standard cogneuro paradigm that brain computation is fundamentally inter-neuronal. This is what the Gallistel-King conjecture challenges. The work on slime molds and plants indicates that what fits the behavioral definitions of learning exist in organisms without the requisite neural nets. The conclusion is that neural nets are not necessary for learning. This surely points to the possibility that the standard picture in cog-neuro concerning the centrality of neural nets to cognition needs a fundamental rethink. In fact, it would be biologically amazing if intra-neuronal/cellular cognitive computation was possible and extant in lower organisms but higher organisms didn't use this computational power at all.

Read the review. The content is not news to FoLers. But the reactions to the work and the weird attempts to either discredit, downplay or reinterpret it is fun to look at. The significant thing, IMO, is that this stuff is becoming more and more mainstream. I think we might be on the edge of a big change of mind.

More on the demarcation problem

Here’s another note on the contemporary ubiquitous desire (especially among scientists and “experts”) to demarcate science (and with it “expertise”) from everything else. You know my take: it cannot be done. We currently have no (interesting and principled)[1]way to demarcate scientific inquiry from other kinds and there is little reason to believe that a (non-trivial bright) line will be discovered anytime in the near (or distant) future. FWIW, philosophers have been trying to find this border for a very long time (you can imagine there is a professional interest in being able to distinguish sense from nonsense), and the current wisdom in the philo community is that there is no there there. Here is a recentish short provocative piece on the topic that goes over the familiar ground (henceforth DS). As I read it, it provoked a few questions: Why should we care to demarcate the scientific from the non-scientific? Is this an urgent project for Science (note the big ‘S’) or for individual sciences? And if so, why? And if not, why does it appear to be sprouting everywhere one looks?  Let’s expatiate.

First, we can ask the factual question: what if anything unifies what we collect under the term ‘Science’? The short answer is not much. DS goes over the usual suspects. To the degree that there is a scientific method, it is not refined enough to distinguish things that lie on what those desirous of the demarcation line would put on one side or the other. “Do your best in the circumstances” is probably all that one can milk out as general methodological advice. This is Feyerabend’s familiar (and correct) observation.

If not a single method, what of communal methods? This too is of little help. As DS notes (2):

The methods used to search for subatomic components of the universe have nothing at all in common with field geology methods…Nor is something as apparently obvious as a commitment to empiricism a part of every scientific field. Many areas of theory development, in disciplines as disparate as physics and economics, have little contact with actual facts, while other fields now considered outside of science, history and textual analysis, are inherently empirical.

So, there is no general method and few robust methods that cut across domains of inquiry to be of use. 

Second question: does this matter? Not obviously. An inquiry requires some questions, puzzles, facts, and methods/technology. These are all generally justified in unison. Given a question prompted by an observation, yields a puzzle, that might be explained by deploying a particular method generating a more refined question, leading to a deeper puzzle, …. Of course, one can start someplace else. A puzzle prompts an observation that clashes with an inchoate “theory” that suggests other facts, that enforce/dispel the puzzle etc…. Or an observation suggests a puzzle that provokes an inquiry that leads to a hypothesis that… All of this can be locally monitored and justification can and does take account of the rich circumstantial detail. Engaging in such inquiry requires making the rules up as you go along, including establishing the requisite standards for the clarity of the questions at hand, deciding what counts as a good explanation relative to these questions, an adumbration of the relevant kinds of data, sample examples of what might resolve the puzzles, all leading to refinements of the initial questions and a restart of the process. The aim is not to avoid circularity (it cannot be done) but to progressively widen the circle so that it is not vicious. Anything goes that gets one going, though how one measures whether one is going and in what direction(s) one is moving in is also up for constant negotiation. 

So, within a particular program all the issues relating to method become important for they end up defining the enterprise. There is nothing outside of this process to adjudicate the activity, or at least nothing principled. But this does not mean that within it there are not better and worse arguments or that dispute between conceptions must be irrational. One can, must, and does argue about the interest of the question being asked. One can, must and does argue about the methods being deployed to answer that question. One can, must and does argue about whether proposals actually address the question being asked. And one should do all of this most of the time. However, and this is the main point, none of this requires that we have rich general methodological principles or that what is good in domain A will be of any consequence or relevance in domain B. Of course, looking at other domains to see what they do can be useful and suggestive (IMO, physics envy is an excellent research attitude), but so can banging your head against the wall while reciting the Lord’s Prayer.[2]

Moreover, none of this local wrangling will be useful in evaluating what counts as Science. If justification is local then demarcating the good from the bad in a general manner that applies across domains is likely to be question begging. As any academic knows, all fields have their methods and questions. If these are the measure of Science, then everything is Science. Christian (and Political (and dare I say, Language)) Science included.

So, there is no general Scientific Method and, luckily, as regards individual inquiries it does not matter. So why the endless quest among non-philosophers? Why is it important to demarcate where science ends and non-science begins. As DS notes this is a particularly hot issue for scientists (and “technology and policy oriented intellectuals” [3]).

I can attest to this worry. The whole obsession with STEM and spreading the STEM gospel is testament to this. I get daily appeals from STEM candidates running for congress. There is even an organization that supports getting STEMers elected (314 Action). The idea seems to be that being STEM gives one a leg up on rationality and political insight. In fact, the presupposition seems to be that having STEM endows special authority on those that have it. And where does the authority come from? Well, STEM implies scientific and this implies having expertise of a kind generally applicable to political matters. So demarcating science from non-science is there to separate “those who are granted legitimacy to make claims about what is true in the world from the rest us…” (2). If this is the goal, then the need for global standards becomes apparent and the demarcation problem becomes urgent. Why? Because only then can science be used to protect the enlightened from the unwashed by endowing some with authority and removing it from others. And this needs an objective basis (or at least a perceived objective basis). 

And not only for those on the receiving end.  It is critical that those at the receiving end of authoritative pronouncements believe that these are legit. Grounding them in Science makes them legit. Hence being scientific is critical. Moreover, those that wield authority must also believe that they are doing so legitimately to mitigate cognitive dissonance. This is an important line, and the harder it is to draw the more a blanket justification of some views over others teeters.

Note that none of this is intended to say that all reasons are on a par without being able to demarcate the scientific from all else. Even without a demarcation, there is excellent reason to believe that the planet is getting warmer due to human activity, that evolution operates, that austerity policies during depressed economic times is self-defeating, that FL/UG exists that generates hierarchical Gs exists and that humans have it, etc. These conclusions are not hard to defend. But they are not defended by noting that they are the products of scientific inquiry, but by noting the evidence and the theory for them. That’s what does the work and claims backed by little evidence or theory are of little value regardless of the methods used to generate them. 

Nor does any of this mean that being in a position to adjudicate proposals might not require quite a bit of technical expertise. It might and often does. But, again it is not because the technical expertise is what makes something scientific but because some expertise is grounded in real questions addressed by good theories backed by good data. Technical wizardry can be an indication of cargo cultism rather than insight, as anyone in any mildly technical domain can attest.

So, onereason for the urgency of the demarcation issue today is the challenge to “authority” that is in the air and the hope that cloaking it in “science” will serve to stifle it by lending it legitimacy.

There is another reason as well. Many domains of inquiry are suffering from internal problems. By this I mean problems internal to the domains of inquiry themselves. There is the “replication” crisis in many sciences that is beginning to undermine their status as “sciences” in the public mind (and this has a spillover effect into the public status of (big ‘S’) Science more generally). There is also the fact that some domains seem to have hit an impasse despite their overwhelming success. Fundamental physics seems to be in this position nowadays if the public toing and froing is any indication (see herefor short version of the angst regarding work in this area). So the legitimacy issue is hitting Science form both ends. The replication crises stems from a purported problem with the data. On the other end, fundamental physics is suffering from an unhealthy obsession with beauty (aesthetic benchmarks concerning “simplicity,” “naturalness” and “elegance” (see here)). Both critiques point to an uncomfortable conclusion for many: science as currently practiced is getting away from the “facts” and the results should be treated very skeptically (and what is wrong with a good dose of skepticism anyhow?). But, IMO, this is the wrong conclusion. 

The right one is that we sometimes run into walls where our methods fail us. Or, when we really don’t know what’s going on, then nothing much helps except a good idea that gets us going again. And if a problem is really hard, then good ideas might be very hard to come by. Big surprise! But this idea, it appears, is tough to swallow. Why?

There is a tacit assumption among scientists that there is a scientific way of doing things and if we just do things in this way then insight must follow. Scientists are particularly prone to this point of view. Not only is it self-flattering (thought it is, it really is) but it is also is very hopeful. Given this view, all setbacks are temporary. All mistakes will self-correct. All obstacles will eventually be overcome and all questions will receive deep and insightful answers. No domain is impenetrable. All problems are solvable. There are no limits to knowledge. Ignorance is temporary, even if hard to dispel. This is a very hopeful message as it encourages the idea that there is always something that can be done that if done right will get us moving forward.  

This moral optimism is the decent side of the belief in a scientific method. And this optimism is what these current failures within the sciences challenges. Add to this (i) that nobody likes pessimists (they are such downers), and (ii) that it is never possible to prove that more hard work, more careful experiments and stats etc. won’t get us moving again and the allure and psychic rewards of the hopeful attitude win the day. So, given the positive spin we place on optimism (“Morning in America”) and the negative one we place on pessimism, there is little surprise that when things get tough there is a desire to justify, which in this case means demarcate. This allows us to segregate the rot and justify optimism for the newly refurbished (rot removed) enterprises.

There is, as always, one further ingredient: Money!! Today money is tight. When money is tight you look to defend your share. Science (big ‘S’ again) is a weapon in the funding wars. Sure, lit and history and philosophy and whatever are fluffy and only valuable when we are flush, but Science, well that needs no defense. Of course, this only works if we can tell what is Science and what isn’t, and hence the obsession on demarcation by scientists. 

So what makes the demarcation issue hot again? The trifecta of the perceived decline in the authority of experts, the current failure in some domains of the traditional methods and declining support together provide more than enough reason to motivate the hunt for a methodological grail.

One of the consequences of Rish conceptions of inquiry is the idea that it comes with implied natural limits.[4]Scientific “success” is always a bit of a miracle (for Descartes, only God guaranteed it (Darwin has often been invoked to similar ends, but his powers are decidedly less expansive)). For people like me, this makes cherishing every apparent explanatory breakthrough deserving of the utmost respect. In practical terms, this leads me to firmly hold onto possible explanations even when confronted with a lot of (apparent) counter evidence. Others dump potential explanations (i.e. theories) more quickly. This is partly a matter of scientific taste. However, there are times when tried and true methods fail. Then doing useful work that meets accepted criteria becomes harder. This should not come as a surprise. It’s the flip side of being able to gain non-trivial understandings of anything at all. It’s what any self conscious Rist who does not have faith in divine harmony would expect.

---

[1]There are many uninteresting ways: what the NSF and NIH fund, who the NYT designates an “expert” worth quoting, what Andrew Gelman take to be scientific, etc. It is not excessive, IMO, to observe, that currently, what is scientific sits in the same category as what is prurient: it is at bestknown when seen. And not even then.

[2]The main utility of looking around is to prevent being bullied by methodological sadists and being tripped up by those insisting that asking a question on some particular way or pursuing a program with some particular emphasis falls outside the “scientific.” The best answer to this is to appreciate that there is no obvious way to fall outside the relevant pale as there is no principled border. However, the second bestway is to observe you're your proposals comport with those utilized by other more obviously successful inquiries. The principle goal of physics envy is defensive. It cuts short all sorts of nonsense (e.g. falsifiability, anti-theory hogwash, Eish concerns with idealization, etc.). 

[3]See here.

[4]Though this does not imply that we can know what these limits are. Chomsky has discussed this a lot (scope and limits stuff). It is often derogatorily labeled ‘mysterianism.’ As Chomsky has repeatedly noted, the idea that there are limits to what we can understand is the flip side of noting that we can understand some things deeply.