One of the great essays in modern analytic philosophy is
Harry Frankfurt’s essay On Bullshit
(here). The essay delineates the
conceptual characteristics of BS differentiating it from lying, exaggeration,
hyperbole etc. BS’s main characteristic
is a complete indifference to truth.
Whereas a lie aims to contradict the truth, BS simply disregards
it. Truthiness (a concept not yet
discovered (invented?) when Frankfurt originally wrote his essay) maybe, truth
not at all. Given the ubiquity of BS all around us, the interest of Frankfurt’s
investigation should be self-evident. However, I mention it today for its
relevance to my little part of the world. Let me rant.
Spring comes and a young linguist’s fancy turns to NSF
grants. These have become a larger and
larger part of the linguistic universe as they function to support graduate
education, top up academic salaries, and support travel to conferences in
pleasant venues. I am a big fan of NSF
grants (though I personally think that they have a deep built in conservative
bias, but that’s a topic for another day) and I wish that the linguistics
division had more money to distribute. However, there are two parts of the
process that I believe have helped foster the spread of BS within our small
domain. Indeed, the NSF may be responsible for a recent spike in the science
BS-ometer that pieces like this one in the NYT seem to regularly generate. How
does the NSF foster this? There are two
ways.
First, there is the “look scientific” requirements of the
formats. For example, the NSF has come
to value research agendas that provide detailed schedules specifying when you will
do what work. Example: Year 1- run pilot
studies on X followed in year 2 by wider studies on Y. In year 3 the project
will spend 5 weeks in Z collecting data on W, which will be compiled in the last
two months of year 3. None of this is
crazy to ask for if your project is
primarily empirical. If you are
doing a psycho project then you have planned out pilots that you want to run
and follow up studies depending on how they turn out (actually, as we all know,
these pilots have been run as have
the follow up experiments and we already know the answers before we submit the
grant for if this has not been done there is little chance of getting the
grant. This, of course is another BS accelerator, though I will leave it aside
as it is not of recent vintage). Ditto for a project aiming to study the
properties of X in some languages Y: what to study first and when and where is
pretty easy to outline and it is reasonable for the NSF to ask for your plans.
However, not everything fits nicely into this schema. For
example, some people submit theory grants. By ‘theory’ I don’t mean applying
formal techniques to descriptive ends, I mean trying to figure out how some bit
of formal work relates to another (e.g. a personal favorite, how/whether to
reduce Control to Movement). Now, this
sort of work is not easily timelined. Consider: in the first 6 months I will
come up with some novel/original ideas relating the apparent disparate domains
X and Y. In the next six months, having reduced X and Y to a common core Z, I
will extend the yet to be discovered methods/ideas/principles of Z to X’ and Y’
thereby unifying all of syntax, morphology and phonology. The last six months
of the grant will demonstrate that the principles underlying Z follow from the
Godel numbering of sentences in roman numeral notation. As should be evident,
the problem with theoretical work is that you don’t know what to expect. The
problem is pretty clear (one hopes) and the avenues to be pursued can be
outlined. However, whether the problems
will yield is completely up in the air. Indeed, if a grant could specify a
credible time line it should not be funded for it indicates that the research has
already been done (but see above). So,
for this kind of work, the requirement that the proposal be squeezed into the
indicated rubrics will necessarily generate BS. After all, the proposer knows
that this is silly, the reviewers (if not brain dead) know that this is silly
so the only reason to provide this kind of fiction is that you are required to
provide something. Nobody cares if any
of this is true or reasonable, only that it be there. In other words, asking for this is asking for
BS.[1]
A second place where BS oozes is in the last section where
proposers are asked to indicate “wider impact.”
Now, this section need not be PhD level BS (recall: BS, MS, PhD=
bullshit, more shit, piled high and deep), and asking for wider impact while
possibly asking a researcher to think outside of her/his comfort zone to make
connections with nearby domains of inquiry can actually be a good thing.
However, what the NSF appears to mean by ‘wider impact’ is roughly on the level
of curing cancer, solving world hunger or getting lions and lambs to peacefully
cohabit a king size bed.
The aforementioned NYT’s piece provides some choice
examples. The brain sciences (if funded
lavishly) will explain our “thoughts, desires, agonies and ecstasies”[2]
as these all “emerge from the details of the neural landscape.” Sounds impressive huh? But wait, there’s more!
It will also cure “Alzheimer’s Parkinson’s and Huntington’s” and “brain tumors,
autism, dementia, paralysis and so on.”
I love that ‘and so on.’ Right
now, neuroscience can’t even explain why we see triangles as triangles or can
identify a green blob as a cabbage in a picture. But never let that stop you;
after all that would not be a worthwhile “wider impact” and that’s what we need
to get that spigot open.
Oh, btw, there’s more, much more. It seems that once we cure
all human disease we can move onto “societal health.” Brain science will cure crime, eliminate
mental health problems, win the war on drugs, kick start new industries in
artificial intelligence and bio-engineering. I’m sure I’ve left some stuff out,
or the author has due to space limitations. But you get the gist. Fund me and I will solve ALL your outstanding
problems. This is the kind of BS that the NSF’s insistence on “wider impact”
statements promotes.
Now, you may be thinking that all of this doesn’t really
matter. After all, we all know this is BS so how does it hurt? Precisely because it corrodes the scientific
ethos. Scientists, for obvious reasons,
have all the vices of other humans.
However, culturally, the enterprise favors certain intellectual and
moral virtues (even if they are not easy to live by), e.g. integrity, the
respect for truth, clarity, modesty, care, etc.
BS values none of these and by making the manufacture of BS in large
quantities an integral part of the funding process, we corrode and undermine
these values. And this cannot be a good thing. It makes much of science part of
the advertising industry, just another way to get saps to buy our stuff.
Fooey!!
As I mentioned, this is as true in linguistics as other NSF
funded domains. And it seeps into
everyday linguistics as well. How many papers have you read lately where the
author, after presenting whatever evidence or argument for a certain position
then tells you how the evidence is “surprising” or “strong” or “unexpected”?
Why are these phrases added? Do they signal that the author thinks that all the
other arguments are weak or inconsequential?
Doubt it. Does the author have some measure for identifying the strong
arguments from the weak? Never. In fact, these additions add no cognitive
content. They are just pretty blatant forms of self-advertising aimed at
getting you to ‘yes.’
Linguistics is lucky: we are small, pretty socially marginal
field whose results are not particularly politically charged (compare with
economics or climate physics). This is
what makes it so exciting intellectually. The problems are interesting and can
be explored in relative independence from hot button ideological concerns. However, even here the rising smell of BS is
detectable. Is resistance possible? Damn
if I know.
[1]
There is another possibility, viz. that the NSF linguistics division actually
has no interest in supporting theoretical work and this is just a way of
derailing any such support. I discount this for I have asked if this is so and
have been assured that it is not. Of course, this too may be BS; turtle shit
all the way down?
[2]
You know this is BS just by the conjunction of these two terms. Why not “the
thrill of victory and the agony of defeat”?
Probabaly a movie buff rather than a sports nut.
I agree with much of this column, but not with its negative comments about NSF support for linguistics. The linguistics program at NSF operates within ridiculously tight financial constraints, and is part of the Directorate of Social , Behavioral & Economic Sciences, which is minutely scrutinized by anti-science and anti-social-science members of congress. Have a look at http://www.majorityleader.gov/YouCut/Review.htm -- if you're not acquainted with the mess. In this context, NSF Linguistics tries to optimize for (1) high potential to yield exciting results or lay foundations for future results, and (2) research that clearly could not get done without NSF support. This sounds just right to me, frankly.
ReplyDeleteNow when criterion (1) gets applied to grant proposals, opinions will differ about what counts as exciting or high-potential research. It certainly happens that a particular NSF panel's collective idea of what's exciting or crucial work might not match yours or mine -- but such is life everywhere. I think it's criterion (2), not criterion (1), that skews grants towards support for laboratory experiments and fieldwork -- because those are areas where it's the clearest that the work won't get done without specific financial support above and beyond the salaries and stipends/TAships that faculty and grad students receive from their universities. Within that context, I don't think there is any serious skewing away from theoretical work. Support for fieldwork is often (though not always, of course) simultaneously support for theoretical work that crucially involves findings from an underdescribed language. Likewise, support for experimental work is often (though again not always) simultaneously support for theoretical work that both Norbert and I want others to do.
So though I think the broader context of Norbert's posting is exactly right, I think it's very wrong to put the Linguistics Program of NSF in the crosshairs of this discussion. All of us in linguistics have been on the receiving end of responses to grant proposals that we disagree with (some of them from theoretical or anti-theory perspectives that we find shocking), but I honestly don't know how I would run things better -- especially (but not only) in the actual world in which we all have to live.
I agree with David that the NSF does much of this for a reason; namely congress breathing down their neck. However, whatever the reason, the effect is the same- BS promotion. However, rather than fight over this, let me reiterate what I said in the post: I think that the NSF is great and that it would be nice were the ling division to have more money. I am also NOT complaining about support for my own work, which tends in a theoretical direction. What I am complaining about is the NSF (not ling division exclusively) one size fits all rubrics that necessarily have as a side effect the generation of prose that is necessarily meaningless. We put up with a lot for money, me as much as anyone else. We seldom see that lucre requires unpleasant accommodations. The one that I see as most pernicious is the promulgation of BS at the very center of the research enterprise. Maybe it's what we need to do. I personally am skeptical, but likely that's just me. However, what is worse than doing this, is doing it without acknowledging that this is what is happening. It's when we begin to believe our own BS, something that just creeps up on you as you do it more and more, that the real damage takes place.
ReplyDeleteDoes this happen in particle physics? Do they need to say, well, we need to find this Higgs because it could be that tomorrow we have to build another universe? Backup universe? No BS, please, I'm curious to know the actual answer.
ReplyDeleteI have to disagree with the characterization of NSF's Broader Impacts criterion. It is simply not true that it's an incitement to make outlandish claims about curing disease or creating world peace. Some people do write BS, but reviewers see right through it. There are many ways in which a project can have broader impacts, and these are taken seriously: training benefits for students; development of tools, techniques, or resources that will be useful to other scientists; solving an important problem that is a key enabler for highly valued work; creation of new and sustainable connections between fields, institutions, etc.; engagement with underrepresented populations ... and so on. None of this is BS. Investors want to know that their investments are going to yield the best possible returns.
ReplyDeleteI can't argue that it's easier to map out timelines for some types of work than others, but certainly disagree with the often cited claim that NSF is biased against theoretical work. Many good theoretical proposals have been funded in recent years (and not just of the fieldwork variety, as you know). But what is always needed to float to the top of the pile is (i) innovation, and (ii) persistence. In areas where people depend on funding to do their work, or don't expect to get tenure etc. unless they get funded, they submit, and re-submit, and re-re-submit, twisting themselves in knots to try to satisfy the reviewers. They make it their business to learn how to develop a clear plan, to establish the initial plausibility of the ideas, etc. In some other areas, many simply don't submit, and others may submit once and then react to the initial reviews by saying "NSF is clearly biased against my type of work, it's a waste of my time." This is exacerbated by the fact that in some theoretical fields, external reviewers seem to be more destructive and parochial (I'm looking at you, syntacticians). In other theoretical sub-fields, there seems to be more of a critical-but-constructive review culture, and there is a history of funding for strong theory proposals. When a proposal is BS, it will get noticed.
I agree that at some point funding decisions need to be made on the basis of a clear, thought out proposal that a (hopefully intelligent) layman can understand, and that is a good exercise to do for one's research. But I take Norbert's second point to be that the way we are asked to make the case for our own research often skews us toward BS, at the expense of the actual (perfectly good) reasons we had for pursuing it. To take the NYT article, the argument is that we should do brain research because we will be able to cure a lot of diseases and build better translation programs. The problem with this is that these things appear to be so far down the road that it amounts to BS. I'm not sure that really is true in the case of Alzheimer's research (I just don't know). But I'm certain it's true in the case of MT. That's what Norbert's getting at, these things are so far off that they are really empty promises, and everybody knows it.
ReplyDeleteI suspect the reason is in fact what David is suggesting: if you were allowed to say "it would be inherently interesting to understand how the human brain works", you would not have this problem. But you are constrained to making up "practical" i.e. technological/engineering applications, as opposed to "we will know this" applications, even when there are none that are really foreseeable. And that's not because (a) the only benefits that can be outlined and weighed in terms of their import are technogical; or (b) practical benefits are inherently more important to the general population. It's because you are strongly encouraged to limit yourself to these kinds of benefits.
And if this is right then that is in fact because there is a narrow set of ideologues sitting at the top who are not representative of the general population. That's why I want to know whether physicists have to go through this bottleneck. Is there still this rigamarole when it's so obvious that the best, most convincing reason for doing this research is "we will collectively as a species understand the most basic mechanisms of the universe" - some of the people on the street might be cynical, but I suspect you really don't need to be a physicist to see this as a worthwhile thing to pursue if it's at least in the cards. I think to believe otherwise is just to buy into an anti-science snowjob - there are clear, commonsense intuitions about what theoretical research is interesting.
Which brings me to the fact that you could easily go through the same exercise without this BS-inducing constraint. "Say how this will advance knowledge in a way that makes sense to someone on the street, making reference to questions - but scientific questions, not necessarily engineering questions - that they can appreciate"; "say how this will impact knowledge in other fields"; "say how answering this narrow question fits into the most important questions in your field - even if they're not intuitively sexy for the layman". And you need to make it plausible that this big stuff actually is in the cards, and that it's not just BS. And it's probably *also* a good exercise to try and think up practical applications. But that is not the only kind of impact that research can have, and if we're supposed to sell it that way, then in many cases all we may be left to sell is snake oil. Some things just don't have any practical benefits that aren't BS - not yet anyway.
As for problem 1 (timelines), Norbert, I think you may have hit the nail on a deep and qualitative difference between two types of research, but it's not theoretical versus empirical - it's confirm/prove these hypotheses, versus explore something. A lot of theoretical projects are exploratory. Not exploratory as in "we're going to explore the data and see what we find." As in "we're going to explore an idea and see what we find" in much the same way. "Control as movement - discuss." There are plenty of ways to lay out a plan in theoretical research. You can find data to confirm/refute particular hypotheses; or you can actually prove interesting things if you've got enough laid out formally, and you could lay that out as a goal: "we don't know if all X's are Y's, but we are definitely going to find out - here's why it's interesting, here's a strategy." But you can also do fruitful research without a particular set of "needs to be answered"; when you're exploring the besst you can come up with is a list of things you're going to try. And the same goes for data exploration, empirical but equally hard to plan for because the whole point is that you have no particular set of questions to be answered.
ReplyDeleteStill, I think that not knowing what kind of answer you're expecting can lend itself to a fairly convincing plan, even if you can't say "once we do this, we'll know this, and if this comes out like that we can then do this," and so on. It would satisfy me (I can't speak for the NSF) if you could list ten fairly specific projects you'd like to do in this area, for example: use the theory to study this phenomenon; see what the relation is between this idea and that idea. Lay them out in as much detail as you can, and that sounds like the best one could possibly do in cases like this. It would be good if there were a separate category for this kind of proposal, on the data side too. Suppose I were to get a bunch of data from animal calls and I want to do some analysis on them. It's obviously relevant to some important questions and I can obviously think of about fifty concrete projects I could do with them. But in a lot of cases I can't give you a flowchart or a timeline, because I just have no idea what I'm going to find. If there's enough interest and enough work to be done, I'm sure I can compensate for what sounds like risk by simply documenting what I find, particularly if it's a very new area - but I just think the kind of proposal that makes sense is going to be qualitatively different in the research that you're talking about, and there is a really vague and risky sounding air about it, no matter what you do. It would be good to recognize that the research is qualitatively different from a series of experiments in the way you structure the proposal, and it would be correspondingly nice to feel like there was an understanding of that on the other end.
I've often wondered why there's no Journal of Bullshit Studies.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteIf the proposal is that more, much more, can and should be done to equalize resources world wide, then, personally, I agree. Sadly, were the whole ling section of the NSF given over to this, it would not make a noticeable dent. More to the point: this is not an either/or issue, and as there is no chance that money will be moved from the NSF to feed the hungry, the issue is somewhat moot. But, if you are asking, yes I agree with you that it matters and is important.
ReplyDelete