Monday, July 16, 2018

More on the demarcation problem

Here’s another note on the contemporary ubiquitous desire (especially among scientists and “experts”) to demarcate science (and with it “expertise”) from everything else. You know my take: it cannot be done. We currently have no (interesting and principled)[1]way to demarcate scientific inquiry from other kinds and there is little reason to believe that a (non-trivial bright) line will be discovered anytime in the near (or distant) future. FWIW, philosophers have been trying to find this border for a very long time (you can imagine there is a professional interest in being able to distinguish sense from nonsense), and the current wisdom in the philo community is that there is no there there. Here is a recentish short provocative piece on the topic that goes over the familiar ground (henceforth DS). As I read it, it provoked a few questions: Why should we care to demarcate the scientific from the non-scientific? Is this an urgent project for Science (note the big ‘S’) or for individual sciences? And if so, why? And if not, why does it appear to be sprouting everywhere one looks?  Let’s expatiate.

First, we can ask the factual question: what if anything unifies what we collect under the term ‘Science’? The short answer is not much. DS goes over the usual suspects. To the degree that there is a scientific method, it is not refined enough to distinguish things that lie on what those desirous of the demarcation line would put on one side or the other. “Do your best in the circumstances” is probably all that one can milk out as general methodological advice. This is Feyerabend’s familiar (and correct) observation.

If not a single method, what of communal methods? This too is of little help. As DS notes (2):

The methods used to search for subatomic components of the universe have nothing at all in common with field geology methods…Nor is something as apparently obvious as a commitment to empiricism a part of every scientific field. Many areas of theory development, in disciplines as disparate as physics and economics, have little contact with actual facts, while other fields now considered outside of science, history and textual analysis, are inherently empirical.

So, there is no general method and few robust methods that cut across domains of inquiry to be of use. 

Second question: does this matter? Not obviously. An inquiry requires some questions, puzzles, facts, and methods/technology. These are all generally justified in unison. Given a question prompted by an observation, yields a puzzle, that might be explained by deploying a particular method generating a more refined question, leading to a deeper puzzle, …. Of course, one can start someplace else. A puzzle prompts an observation that clashes with an inchoate “theory” that suggests other facts, that enforce/dispel the puzzle etc…. Or an observation suggests a puzzle that provokes an inquiry that leads to a hypothesis that… All of this can be locally monitored and justification can and does take account of the rich circumstantial detail. Engaging in such inquiry requires making the rules up as you go along, including establishing the requisite standards for the clarity of the questions at hand, deciding what counts as a good explanation relative to these questions, an adumbration of the relevant kinds of data, sample examples of what might resolve the puzzles, all leading to refinements of the initial questions and a restart of the process. The aim is not to avoid circularity (it cannot be done) but to progressively widen the circle so that it is not vicious. Anything goes that gets one going, though how one measures whether one is going and in what direction(s) one is moving in is also up for constant negotiation. 

So, within a particular program all the issues relating to method become important for they end up defining the enterprise. There is nothing outside of this process to adjudicate the activity, or at least nothing principled. But this does not mean that within it there are not better and worse arguments or that dispute between conceptions must be irrational. One can, must, and does argue about the interest of the question being asked. One can, must and does argue about the methods being deployed to answer that question. One can, must and does argue about whether proposals actually address the question being asked. And one should do all of this most of the time. However, and this is the main point, none of this requires that we have rich general methodological principles or that what is good in domain A will be of any consequence or relevance in domain B. Of course, looking at other domains to see what they do can be useful and suggestive (IMO, physics envy is an excellent research attitude), but so can banging your head against the wall while reciting the Lord’s Prayer.[2]

Moreover, none of this local wrangling will be useful in evaluating what counts as Science. If justification is local then demarcating the good from the bad in a general manner that applies across domains is likely to be question begging. As any academic knows, all fields have their methods and questions. If these are the measure of Science, then everything is Science. Christian (and Political (and dare I say, Language)) Science included.

So, there is no general Scientific Method and, luckily, as regards individual inquiries it does not matter. So why the endless quest among non-philosophers? Why is it important to demarcate where science ends and non-science begins. As DS notes this is a particularly hot issue for scientists (and “technology and policy oriented intellectuals” [3]).

I can attest to this worry. The whole obsession with STEM and spreading the STEM gospel is testament to this. I get daily appeals from STEM candidates running for congress. There is even an organization that supports getting STEMers elected (314 Action). The idea seems to be that being STEM gives one a leg up on rationality and political insight. In fact, the presupposition seems to be that having STEM endows special authority on those that have it. And where does the authority come from? Well, STEM implies scientific and this implies having expertise of a kind generally applicable to political matters. So demarcating science from non-science is there to separate “those who are granted legitimacy to make claims about what is true in the world from the rest us…” (2). If this is the goal, then the need for global standards becomes apparent and the demarcation problem becomes urgent. Why? Because only then can science be used to protect the enlightened from the unwashed by endowing some with authority and removing it from others. And this needs an objective basis (or at least a perceived objective basis). 

And not only for those on the receiving end.  It is critical that those at the receiving end of authoritative pronouncements believe that these are legit. Grounding them in Science makes them legit. Hence being scientific is critical. Moreover, those that wield authority must also believe that they are doing so legitimately to mitigate cognitive dissonance. This is an important line, and the harder it is to draw the more a blanket justification of some views over others teeters.

Note that none of this is intended to say that all reasons are on a par without being able to demarcate the scientific from all else. Even without a demarcation, there is excellent reason to believe that the planet is getting warmer due to human activity, that evolution operates, that austerity policies during depressed economic times is self-defeating, that FL/UG exists that generates hierarchical Gs exists and that humans have it, etc. These conclusions are not hard to defend. But they are not defended by noting that they are the products of scientific inquiry, but by noting the evidence and the theory for them. That’s what does the work and claims backed by little evidence or theory are of little value regardless of the methods used to generate them. 

Nor does any of this mean that being in a position to adjudicate proposals might not require quite a bit of technical expertise. It might and often does. But, again it is not because the technical expertise is what makes something scientific but because some expertise is grounded in real questions addressed by good theories backed by good data. Technical wizardry can be an indication of cargo cultism rather than insight, as anyone in any mildly technical domain can attest.

So, onereason for the urgency of the demarcation issue today is the challenge to “authority” that is in the air and the hope that cloaking it in “science” will serve to stifle it by lending it legitimacy.

There is another reason as well. Many domains of inquiry are suffering from internal problems. By this I mean problems internal to the domains of inquiry themselves. There is the “replication” crisis in many sciences that is beginning to undermine their status as “sciences” in the public mind (and this has a spillover effect into the public status of (big ‘S’) Science more generally). There is also the fact that some domains seem to have hit an impasse despite their overwhelming success. Fundamental physics seems to be in this position nowadays if the public toing and froing is any indication (see herefor short version of the angst regarding work in this area). So the legitimacy issue is hitting Science form both ends. The replication crises stems from a purported problem with the data. On the other end, fundamental physics is suffering from an unhealthy obsession with beauty (aesthetic benchmarks concerning “simplicity,” “naturalness” and “elegance” (see here)). Both critiques point to an uncomfortable conclusion for many: science as currently practiced is getting away from the “facts” and the results should be treated very skeptically (and what is wrong with a good dose of skepticism anyhow?). But, IMO, this is the wrong conclusion. 

The right one is that we sometimes run into walls where our methods fail us. Or, when we really don’t know what’s going on, then nothing much helps except a good idea that gets us going again. And if a problem is really hard, then good ideas might be very hard to come by. Big surprise! But this idea, it appears, is tough to swallow. Why?

There is a tacit assumption among scientists that there is a scientific way of doing things and if we just do things in this way then insight must follow. Scientists are particularly prone to this point of view. Not only is it self-flattering (thought it is, it really is) but it is also is very hopeful. Given this view, all setbacks are temporary. All mistakes will self-correct. All obstacles will eventually be overcome and all questions will receive deep and insightful answers. No domain is impenetrable. All problems are solvable. There are no limits to knowledge. Ignorance is temporary, even if hard to dispel. This is a very hopeful message as it encourages the idea that there is always something that can be done that if done right will get us moving forward.  

This moral optimism is the decent side of the belief in a scientific method. And this optimism is what these current failures within the sciences challenges. Add to this (i) that nobody likes pessimists (they are such downers), and (ii) that it is never possible to prove that more hard work, more careful experiments and stats etc. won’t get us moving again and the allure and psychic rewards of the hopeful attitude win the day. So, given the positive spin we place on optimism (“Morning in America”) and the negative one we place on pessimism, there is little surprise that when things get tough there is a desire to justify, which in this case means demarcate. This allows us to segregate the rot and justify optimism for the newly refurbished (rot removed) enterprises.

There is, as always, one further ingredient: Money!! Today money is tight. When money is tight you look to defend your share. Science (big ‘S’ again) is a weapon in the funding wars. Sure, lit and history and philosophy and whatever are fluffy and only valuable when we are flush, but Science, well that needs no defense. Of course, this only works if we can tell what is Science and what isn’t, and hence the obsession on demarcation by scientists

So what makes the demarcation issue hot again? The trifecta of the perceived decline in the authority of experts, the current failure in some domains of the traditional methods and declining support together provide more than enough reason to motivate the hunt for a methodological grail.

One of the consequences of Rish conceptions of inquiry is the idea that it comes with implied natural limits.[4]Scientific “success” is always a bit of a miracle (for Descartes, only God guaranteed it (Darwin has often been invoked to similar ends, but his powers are decidedly less expansive)). For people like me, this makes cherishing every apparent explanatory breakthrough deserving of the utmost respect. In practical terms, this leads me to firmly hold onto possible explanations even when confronted with a lot of (apparent) counter evidence. Others dump potential explanations (i.e. theories) more quickly. This is partly a matter of scientific taste. However, there are times when tried and true methods fail. Then doing useful work that meets accepted criteria becomes harder. This should not come as a surprise. It’s the flip side of being able to gain non-trivial understandings of anything at all. It’s what any self conscious Rist who does not have faith in divine harmony would expect.


[1]There are many uninteresting ways: what the NSF and NIH fund, who the NYT designates an “expert” worth quoting, what Andrew Gelman take to be scientific, etc. It is not excessive, IMO, to observe, that currently, what is scientific sits in the same category as what is prurient: it is at bestknown when seen. And not even then.
[2]The main utility of looking around is to prevent being bullied by methodological sadists and being tripped up by those insisting that asking a question on some particular way or pursuing a program with some particular emphasis falls outside the “scientific.” The best answer to this is to appreciate that there is no obvious way to fall outside the relevant pale as there is no principled border. However, the second bestway is to observe you're your proposals comport with those utilized by other more obviously successful inquiries. The principle goal of physics envy is defensive. It cuts short all sorts of nonsense (e.g. falsifiability, anti-theory hogwash, Eish concerns with idealization, etc.). 
[3]See here.
[4]Though this does not imply that we can know what these limits are. Chomsky has discussed this a lot (scope and limits stuff). It is often derogatorily labeled ‘mysterianism.’ As Chomsky has repeatedly noted, the idea that there are limits to what we can understand is the flip side of noting that we can understand some things deeply. 

11 comments:

  1. "So, there is no general method and few robust methods that cut across domains of inquiry to be of use. . . . If justification is local then demarcating the good from the bad in a general manner that applies across domains is likely to be question begging. . . . So, there is no general Scientific Method and, luckily, as regards individual inquiries it does not matter. . ."

    Reminds me of what your one-time colleague, the late Sidney Morgenbesser said: "To explain why a man slipped on a banana peel, we don't need a general theory of slipping."

    --RC

    ReplyDelete
    Replies
    1. He was a great man. He also noted that to adjudicate between theories one does not need a standard that can adjudicate among all theories. Or, what we need is for between any two theories for there to be a standard not for there to be a standard such that for any two theories it can adjudicate between them. Confusing EA for AE is a real philosophical bummer.

      Delete
  2. My thought is that there is in fact something characteristic of but not necessarily limited to science, which is a kind of 'moral balance' between on the one hand, developing some ideas in a systematic way without collapsing into negativity at every apparent problem (as the generative semanticists tended to do), and, on the other, simpler ignoring evidence of serious, fundamental problems, such as in the 10-15 years that Minimalists were ignoring the evidence from case stacking that agreeement could not depend upon syntactic adjacency. Of course nobody can do this perfectly, all the time.

    ReplyDelete
    Replies
    1. But there is another ingredient missing from the above, which is an intuitive perception of overall progress, in spite of occasional regressions (eg phrenology). This requires a kind of capability of theory wrt data that is messing in many intellectual pursuits. E.g. we clearly don't know anything more than the Alexandrians did 2000+ years ago about why 'Homer' is good.

      Delete
    2. So my conclusion is that although there doesn't seem to be any serious way to describe all the effective instances of the 'scientific method' wrt the environments where they work reasonably well, there do seem to be ways of telling where one the instances is active or has been until recently. Or not, the latter sad case perhaps afforded ATM, to some extent at least, by economics, political science, pain management (where the all treatments that are clearly both safe and effective seem to be based on the placebo effect), and education (all levels).

      Delete
  3. It is worth noting that the demarcation issue in the hands of Popper and the logical empiricists in the 20s/30s was animated by a political/moral urgency, given the then ascendency of, well, evil, supported by all kinds of mystical gibberish and pseudo-science. I agree that, philosophically, demarcating science from non-science is a fool's errand (never say philosophy doesn't make progress:) - BTW, I recommend Larry Laudan and Nancy Cartwright on these issues, who NH has cited previously, I think), but I do wonder how reason might best arm itself politically, as it were. Ultimately, I think there can be no substitute for humility, localism, and truth, as you preach, but I do understand the will for something grander, eye-catching, and more unifying, especially when faced with the high tides of bullshit. Unfortunately, the synoptic prophets of reason around today are a pretty parochial bunch.

    ReplyDelete
    Replies
    1. I pointed to some current work on demarcation in a comment on Norbert's "Methodological Sadism" post, some of which is not strictly philosophy (but rather history or sociology or psychology). I think this work is motivated by a fear of the sort you (rightly) point out motivated earlier demarcationists. Whether the current fear is justified, and whether the responses are good ones, are left as exercises for the interested reader.

      http://facultyoflanguage.blogspot.com/2018/04/methodological-sadism.html#comment-form

      One of the things that Pigliucci does try to do, btw, is answer Laudan's (in)famous dismissal of the demarcation problem. Says so, right in the title of his chapter: 'The demarcation problem: a (belated) response to Laudan'.

      --RC

      Delete
  4. (Apologies if this is double-posted.)

    I stumbled across this blog while searching for commentary on the Weekly Standard article, and I'm baffled. The descriptions of "science," etc., are virtually unrecognizable to me, even though I’ve spent nearly thirty years engaged in the scientific endeavor as a physiologist and a professor of biomedicine. Frankly, it seems like most of the people writing on this issue have no serious experience in science. That’s an unpleasant thing to say, but I think it’s valid, and will try to explain.

    Part of the problem is that I’m not sure what concept of “science” is being critiqued. Is it the popular conception (flawed though it may be)? Is it the philosophy-of-science conception (ditto, but further burdened with characteristic terminological sloppiness)? Is it the one in Plato’s cave, as suggested by “capital-S science”? There are rhetorical flourishes in all directions, but the elementary groundwork has not been done and it’s hard to know how to proceed.

    But not impossible. First some preliminaries: I assume that the arguments are nontrivial--for instance, I’ll concede right up front that the definition of “science” gets fuzzy at the borders; that’s true of every concept and is thus uninteresting. Similarly, I concede that things look different at different levels of examination. Zoom out far enough and everything looks the same as everything else. Zoom in and everything looks different. Again, that’s trivial; Instead we need to know whether we can come up with a general concept that works pretty well.

    We can, of course. Science is a process, a method, a verb—it’s pie-baking, not pie. It’s used to develop an explanation of phenomena in the natural/physical world. We have some foundational assumptions. We come up with some notion of how the world works. Then--and this is the critical part--we test that idea by checking it against the real world through observation or experiment. And we are rigorous about how we check: We consider other possible explanations; we strive to come up with a clear, reliable, unbiased, and systematic way to measure whatever we’re trying to measure. Feynman’s imprecation is worth remembering: "It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong,” and must be modified, constrained, or discarded entirely. Through hard work and luck, we can sometimes come up with a theory (that’s the pie)—a set of principles, broadly supported (but outrunning) empirical data, that provide an account of some phenomenon.

    This definition helps us distinguish between astronomy and astrology, between the scientific way of searching for truth and the legal way. It is hard to believe that this is in any way controversial, and I have rarely known a scientist to be confused about it. What is the issue supposed to be?

    (In a separate post, I’ll deal with some specific problems.)

    ReplyDelete
  5. (Apologies if this is double-posted.)

    I stumbled across this blog while searching for commentary on the Weekly Standard article, and I'm baffled. The descriptions of "science," etc., are virtually unrecognizable to me, even though I’ve spent nearly thirty years engaged in the scientific endeavor as a physiologist and a professor of biomedicine. Frankly, it seems like most of the people writing on this issue have no serious experience in science. That’s an unpleasant thing to say, but I think it’s valid, and will try to explain.

    Part of the problem is that I’m not sure what concept of “science” is being critiqued. Is it the popular conception (flawed though it may be)? Is it the philosophy-of-science conception (ditto, but further burdened with characteristic terminological sloppiness)? Is it the one in Plato’s cave, as suggested by “capital-S science”? There are rhetorical flourishes in all directions, but the elementary groundwork has not been done and it’s hard to know how to proceed.

    But not impossible. First some preliminaries: I assume that the arguments are nontrivial--for instance, I’ll concede right up front that the definition of “science” gets fuzzy at the borders; that’s true of every concept and is thus uninteresting. Similarly, I concede that things look different at different levels of examination. Zoom out far enough and everything looks the same as everything else. Zoom in and everything looks different. Again, that’s trivial; Instead we need to know whether we can come up with a general concept that works pretty well.

    We can, of course. Science is a process, a method, a verb—it’s pie-baking, not pie. It’s used to develop an explanation of phenomena in the natural/physical world. We have some foundational assumptions. We come up with some notion of how the world works. Then--and this is the critical part--we test that idea by checking it against the real world through observation or experiment. And we are rigorous about how we check: We consider other possible explanations; we strive to come up with a clear, reliable, unbiased, and systematic way to measure whatever we’re trying to measure. Feynman’s imprecation is worth remembering: "It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong,” and must be modified, constrained, or discarded entirely. Through hard work and luck, we can sometimes come up with a theory (that’s the pie)—a set of principles, broadly supported (but outrunning) empirical data, that provide an account of some phenomenon.

    This definition helps us distinguish between astronomy and astrology, between the scientific way of searching for truth and the legal way. It is hard to believe that this is in any way controversial, and I have rarely known a scientist to be confused about it. What is the issue supposed to be?

    (In a separate post, I’ll deal with some specific problems.)

    ReplyDelete
  6. It seems to me that, as you suggest, level of detail might be the problem here; somewhat plausibly, the level where there is 'a scientific method' is rather abstract, and in some ways more of an attitude than a method, although it does involve some relatively concrete apsects such as not trying to imprison or execute people who challenge your ideas (not to be taken for granted for most of human history, or in many places even today).

    Many other more concrete things are more problematic, such as the role of experiments ... central in physics, not so much in geology (at least in the past, don't know about contemporary), controversial in linguistics. In the latter case, Chomsky has often said that we don't need to worry too much about doing formal experiments for syntax, since informally asking people 'can you this ...' turns up plenty of good data with very little effort, but it is also becoming clear that sometimes, a more careful approach is needed. e.g. as discussed in http://ling.auf.net/lingbuzz/003663 (there's also work showing that the informal approach works pretty well, a lot of the time).

    But I think we can say that there is no such thing as a 'scientific method' that you could get grant money to apply to arbitrary problems from random areas; and that working out what concrete methods are worthwhile and effective is hard.

    ReplyDelete
  7. I find interesting that some consider history *not* a "science". On what grounds? It's not a natural science, of course.

    ReplyDelete