The Economy of Research

Doing research requires exercising judgment, and doing this means making decisions. Of the decisions one makes, among the most important concern what work to follow and what to (more or less) ignore. Like all decisions, this one carries a certain risk, viz. ignoring that work that one should have followed and following that work that should have been ignored (the research analogue of type I and type II errors).  However, unless you are a certain distinguished MIT University Professor who seems to have the capacity (and tenacity) to read everything, this is the kind of risk you have to run for the simple reason that there are just so may hours in the day (and not that many if, e.g. you are a gym rat who loves novels and blog reading viz. me). So how do you manage your time? Well, you find your favorites and follow them closely and you develop a cadre of friends whose advice you follow and you try to ensconce yourself in a community of diversely interesting people who you respect so that you can pick up the ambient knowledge that is exhaled.  However, even with this, it is important to ask, concerning what you read, what is its value-added. Does it bring interesting data to the discussion, well-grounded generalizations, novel techniques, new ideas, new questions?  By the end of the day (or maybe month) if I have no idea why I looked at something, what it brought to the table, then I reluctantly conclude that I could have spent my time (both research and pleasure time) more profitably elsewhere, AND, here’s the place for the policy statement, I note this and try to avoid this kind of work in the future. In other words, I narrow my mind (aiming for complete closure) so as to escape such time sinks.

Why do I mention this? Well, a recent paper, out on LingBuzz, by Legate, Pesetsky and Yang (LPY) (here) vividly brought it to mind. I should add, before proceeding, that the remarks below are entirely mine. LPY cannot (or more accurately ‘should not,’though some unfair types just might) be held responsible for the rant that follows. This said, here goes.

LPY is a reply to a recent paper in Language, Levinson (2013), on recursion.  Their paper, IMO, is devastating. There’s nothing left of the Levinson (2013) article. And when I say ‘nothing,’ I mean ‘nothing.’ In other words, if they are right, Levinson’s effort has 0 value-added, negative really if you count the time lost in reading it and replying to it. This is the second time I have dipped into this pond (see here), and I am perilously close to slamming my mind shut to any future work from this direction. So before I do so, let me add a word or two about why I am thinking of taking such action.

LPY present three criticisms of Levinson 2013. The first ends up saying that Levinson (2103)’s claims about the absence of recursion in various languages is empirically unfounded and that it consistently incorrectly reports the work of others. In other words, not only are the “facts” cited bogus, but even the reports on other people’s findings are untrustworthy.  I confess to being surprised at this. As my friends (and enemies) will tell you, I am not all that data sensitive much of the time. I am a consumer of other people’s empirical work, which I then mangle for my theoretical ends.  As a result, when I read descriptive papers I tend to take the reported data at face value and ask what this might mean theoretically, were it true.  Consequently, when I read papers by Levinson, Evans, Everett a.o., people who trade on their empirical rectitude, I tend to take their reports as largely accurate, the goal being to winnow the empirical wheat from what I generally regard as theoretical/methodological chaff. What LPY demonstrate is that I have been too naïve (Moi! Naïve!) for it appears that not only is the theoretical/methodological work of little utility, even the descriptive claims must be taken with enough salt to scare the wits out of any mildly competent cardiologist. So, as far as empirical utility goes, Levinson (2013) joins Everett (2005) (see Nevins, Pesetsky and Rodrigues (NPR) for an evisceration) as a paper best left off one’s Must-Read list.

The rest of LPY is no less unforgiving and I recommend it to you. But I want to make two more points before stopping.

First, LPY discuss an argument form that Levinson (2013) employs that I find of dubious value (though I have heard it made several times). The form is as follows: A corpus study is run that notes that some construction occurs with a certain frequency.  This is then taken to imply something problematic about grammars that generate (or don’t generate) these constructions. Here’s LPY’s version of this argument form in Levinson (2013):

Corpus studies have shown that degree-2 center embedding

"occurs vanishingly rarely in spoken language syntax", and degree-3 center embedding is hardly observed at all. These conclusions converge with the well-known psycholinguistic observation that "after degree 2 embedding, performance rapidly degrades to a point where degree 3 embeddings hardly occur".

Levinson concludes from this that natural language (NL) grammars (at least some) do not allow for unbounded recursion (in other words, that the idealization that NLs are effectively infinite should be dropped).  Here are my problems with this form of argument. 

First, what’s the relevance of corpus studies?  Say we concede that speakers in the wild never embed more than two clauses deep. Why is this relevant?  It would be relevant if when strapped to grammatometers, these native speakers flat-lined when presented with sentences like John said that Mary thinks that Sam believes that Fred left, or this is the dog that chased the cat that ate the rat that swallowed the cheese that I made. But they don’t!  Sure they have problems with these sentences, after all long sentences are, well, long. But they don’t go into tilt, like they generally do with word salad like What did you kiss many people who admire or John seems that it was heard that Frank left. If so, who cares whether these sentences occur in the wild?  Why should being in a corpus endow an NL data point with more interest than one manufactured in the ling lab?

Let me be a touch more careful: If theory T says that such and such a sentence is ill formed and one finds instances of such often enough in the wild, then this is good prima facie evidence against T. However, absence of such from a corpus tells us exactly nothing. I would go further, as in all other scientific domains, manufactured data, is often the most revealing.  Physics experiments are highly factitious, and what they create in the lab is imperceptible in the wild. So too with chemistry, large parts of biology and even psychophysics (think Julesz dot displays or Muller-Lyer illusions, or Necker cubes).  This does not make these experiments questionable. All that counts is that the contrived phenomena be stable and replicable.  Pari passu being absent from a corpus is no sign of anything. And being manufactured has its own virtues, for example, being specially designed to address a question at hand. As in suits, bespoke is often very elegant!

I should add, that LPY question Levinson (2013)’s assertion that three levels of embedding are “relatively rare,” noting that this is a vacuous claim unless some baseline is provided (see their discussion). At any rate, what I wish to reiterate is that the relevant issue is not whether something is rare in a corpus but whether the data is stable, and I see no reason to think that judgments concerning multiple embedded clauses manufactured by linguists are unstable, even if they don’t frequently appear in corpora.

Second and final point: Chomsky long ago noted the important distinction “is not the difference between finite and infinite, but the more elusive difference between too large and not too large” (LSLT:150).  And it seems that it doesn’t take much to make grammars that tolerate embedding worthwhile. As LPY notes, a paper by Perfors, Tenenbaum and Regier (2006)

… found that the context-free grammar is favored [over regular grammars,NH] even when one only considers very simple child-directed English, where each utterance averages only 2.6 words, and no utterance contains center embedding or remotely complex structures.

It seems that representational compactness has its own very large rewards. If embedding be a consequence, it seems that this is not too high a price to pay (it may even bring in its train useful expressive rewards!).  The punch line: the central questions in grammar have less to do with unbounded recursion than with projectability; how one generalizes from a sample to a much larger set. And it is here that recursive rules have earned their keep. The assumption that NLs are for all practical purposes infinite simply focuses attention on what kinds of rule systems FL supports. The infinity assumption makes the conclusion that the system is recursive trivial to infer. However, finite but large will also suffice for here too the projection problem will arise, bringing in its wake all the same problems generative grammarians have been working on since the mid 1950s.

I have a policy: those things not worth doing are not worth doing well. It has an obvious corollary: those things not worth reading are not worth reading carefully. Happily, there are some willing to do pro bono work so that the rest of us don’t have to. Read LPY (and NPR) and draw your own conclusions. I’ve already drawn mine.