Friday, February 22, 2013

Acceptability and Grammaticality


Names are important for what you call something can affect how people understand it, no matter how many times you clarify your intentions. Chomsky has been the unfortunate recipient of considerable “criticism” launched against positions that he does not hold. I have discussed this elsewhere but it is worth reiterating for when it comes to criticizing his ideas, it appears, that understanding them is hardly a prerequisite.  The most insistent zombie criticism comes in frantic popular pieces that regularly announce that Universal Grammar has failed because language X doesn’t have property Y (Piraha and recursion fit this template to a T). The fact that Chomsky’s conception of Universal Grammar is not about languages but about the Faculty of Language (viz. humans have a species specific capacity to acquire languages and Universal Grammar is a specification of the mental powers on which this capacity supervenes) and that it does not require that every language exhibit the same properties seems irrelevant to these critics and the secondary market in Chomsky criticism (you know who you are!).  However, satisfying  it is to reiterate this simple but important point (and please feel free to join me in making it at every available opportunity (Tip: it makes a nice part of any wedding or bar mitzvah speech)) let me leave it aside for now and let me instead continue my program of ling-speak reform.  In earlier posts, I (and Paul) suggested that we drop ‘learning’ for the more generic ‘acquisition’ in describing what kids do and reserve the former term for a particular kind of data driven acquisition.  Here I want to consider the use of another term: ‘grammaticality’ as used to describe speakers’ judgments, as in ‘sentence (1a) is grammatical and sentence (1b) is ungrammatical.’

(1)  a. John likes Mary
b. *John like Mary

My proposal is that we swap ‘acceptable’ for ‘grammatical,’ that we use the latter as a predicate of analyses and use the former to describe the data. This terminological proposal amounts to treating ‘grammatical’ as a theoretical term, whereas ‘acceptable’ describes the empirical lay of the land.

This would have several prophylactic advantages.

First, it makes an important distinction between data and what the data is used to probe. We have reliable intuitions of unacceptability but not intuitions of what causes unacceptability. Utterances of sentences can be more or less acceptable.[1] Only sentences can be grammatical and ungrammatical. And, though unacceptability is a prima facie reason for suspecting ungrammaticality, ungrammaticality is neither necessary nor sufficient for the perception of unacceptability.  Grammaticality is a theoretical notion, acceptability an observational one.

So acceptability is a predicate of our linguistic data.  At bottom, a big part of that data is judgments of the acceptability of an utterance of a sentence in a specified context (e.g. we ask “could you say BLAH BLAH BLAH in coxtext C to express M?”). Often, a used sentence token can be judged unacceptable without much specification of context of use (e.g. island violations), but often not (e.g. scope ambiguities). Thus, low acceptability can be traced to a variety of reasons, only one of which concerns the utterance’s grammatical status. Indeed, all the four possible relations between +/- acceptability (+/-A) and +/- Grammaticality (+/-G) exist.  Let’s consider some examples.

An uttered sentence can be unacceptable yet grammatical, –A/+G, the unacceptability attributed to parsing difficulty of some kind. Canonical examples include:

(2)  a. That that that Bill left Mary amused Sam is interesting is sad
b. Dogs dogs dog dog dogs dogs dog
c. The horse raced past the barn fell

(2a) is a case of self embedding (c.f. Chomsky and Miller 1963), (2c) a garden path and (2b), though it has no name is great for parties (other variants include ‘buffalo buffalo buffalo bufalo buffalo buffalo bufalo’ and ‘skunks skunks skunk skunk skunks skunk’).  We have pretty good stories about why these sentences are hard to deal with. For (2a) see Lewis and Vasisth 2005, for (2b) Barton, Berwick and Ristad 1987, and (2c) virtually anybody working on language processing.

Jon Sprouse discusses other examples of the disconnect between grammaticality and acceptability. For example, it is well known that length alone affects acceptability ratings. For example, when uttered, a long sentence like (3a) is judged superior to a short one like (3b) though every theory of grammar will treat them as equally grammatical:

(3)  a. Who did Bill see
b. Who did Frank say that Bill saw

Interestingly, the contrast goes in the other direction as well. There are sentences that sound really very good when uttered, but are nonetheless clearly ungrammatical.  Examples like (4) were first discussed, I believe, by Montalbetti, and recently Alexis Wellwood, Roumi Pancehva and Colin Philips have been investigating them as examples of grammatical illusions.

(4)  More people visited Rome last year than I did

Such sentences when uttered “sound” really good and garner high judgment ratings. In particular, if a native speaker is asked to judge their acceptability, especially if asked to do so quickly, they will rate them very high.  Ask them what the uttered sentence means (or could mean) and they are stumped. Consider (5a). They can be paraphrased as (5b).  On this model (4) should be understood as (6). But (6) is incomprehensible, true word salad.

(5)  a. More people visited Rome last year than visited Venice
b. The number of people who visited Rome last year is greater than the
number of people who visited Venice last year.

(6)  The number of people who visited Rome last year is greater than …the number that I did?/greater than the number of I that did?

You should appreciate that the incomprehensibility of (4) is quite an accomplishment. It is not actually that easy to construct sentences whose utterance are truly meaningless.  Many of our standard “bad” examples are semantically quite transparent. Thus (7a) means (7b) and (8a) has the paraphrase (8b).

(7)  a. *who did you meet a woman who loved
b. Which person is it that you met a woman who loved him

(8)  a. *John seems sleeping
b. John seems to be sleeping


Real semantic mish mash exists, (9a) is word salad and cannot be pretzled into meaning (9b) (it is a well known minimality or PIC (phase impenetrability condition) violation).

(9)  a. *John seems that it was told that Frank left
b. it seems that John was told that Frank left

Note, however, that the semantic incomprehensibility correlates with strong unacceptability. What makes examples like (4) so interesting is that they are judged quite acceptable despite their incoherence.

The other two combinations are the standard cases, +G leading to +A and –G being –A, the ones that make it reasonable to treat (un)acceptability as a leading indicator of (un)grammaticality. But, as the +G/-A and –G/+A cases show, even generally reliable symptoms can mislead.

There is one other upside to adopting this terminological distinction. When theory changes, we often reassess the import of the facts.  This is in and of itself not a bad thing. Facts don’t come marked with their significance and a fact’s import might change as theory does.  However, though what we make of facts might change, the facts themselves do not (at least not by and large).  It is useful to have neutral terminology for describing the explanada and whereas a sentence that is ungrammatical in one theoretical venue might be grammatical in another, an utterable sentence that is unacceptable given one set of theoretical assumptions does not generally become acceptable given another.  Grammaticality shifts. Unacceptability is relatively fixed. This is useful to recall when the excitement of theoretical innovation leads us to set aside heretofore central data points. There is nothing wrong in my view in setting recalcitrant facts to one side, at least temporarily, there is a lot wrong in forgetting that these facts exist.  Not confusing acceptability with grammaticality might help restrain this otherwise enticing move.

However, the real benefit of making this change is that it will remind us that imputations of grammaticality involve dipping one’s toes into the deep seas of theory (they are part of explaining) while noting differences in acceptability belong to the realm of description. Being careful not to confuse description with explanation is a worthwhile precept of methodological hygiene. Incorporating the distinction between acceptability and grammaticality into our ling-speak is a small inoculation against self-confusion.




[1] I use ‘utterance’ here in a wide sense; so reading a token of a sentence counts as an utterance, hearing one does too. Any used token counts.

14 comments:

  1. The trouble with the "learning"/"acquisition" swapout is that "acquisition" has already run its course and been profoundly misunderstood. I only learned this recently, but ask any "language" person at an arm's length from linguistics. Apparently there are hordes of people out there who decided to misread it as "a very particular kind of processs where the majority of the contribution is biologically determined, kind of like the growth of a plant." Now one may think this but the whole point was to strip out the substantive empirical claims from the terminology. Just as "learning" smuggles in unfair weighting favouring the contribution of input, "acquisition" in this reading is basically "growth" or "maturation" and so smuggles in unfair weighting favouring the contribution of non-input factors. I wish the world were not so frustrating, but it is.

    As all of this "debate" is intolerably vague anyway, I think it's better not to operate under the pretense that there is a word that will capture what we want. Try some multiword expressions. "Adaptation of the cognitive system to external stimulus". "Those changes in the state of the mind/brain relevant to language which crucially depend on the information coming in from the senses". We just abbreviate them if we want people to use them. ACSEXS. CHISMSENS. who could turn down the opportunity to say these things? They cannot possibly be understood worse than "acquisition" already has been.

    ReplyDelete
  2. Yes, the swapout I don't like. Because, we're too ignorant about the acquisition of everything to justify trying to introduce new terminology.

    ReplyDelete
  3. NOBODY expects the Language Acquisition!

    ReplyDelete
  4. First apologies for posting an off topic question here - maybe Norbert can move it to some more appropriate place?

    Given that it has been pointed out at a different blog that I might have trouble spotting ambiguities I want to make sure i do not make this mistake with the passage below. In his reply to Margaret Boden [2007] Chomsky writes:

    "In the 1980s, a prominent advocate of Boden's favorite theory, her colleague Gerald Gazdar's GPSG, was "locked out" by being repeatedly invited to visit and teach in our department and offered a permanent position as full professor, has had regular work contacts with "Chomskyans" (sometimes even the demon himself), even co-published with some of them, and is constantly cited by them. And happens to be a personal friend."

    To me it seems clear that Chomsky claims here that Gadzar "was repeatedly invited to visit and teach in our department and offered a permanent position as full professor... And happens to be a personal friend". Is this the claim he makes or is there a different reading of this passage?

    ReplyDelete
  5. @Christina. I think it's clear from the passage that you quote that C could be referring to any advocate of GPSG. There are other people who were associated quite closely with the development of GPSG who come to mind (e.g. Ewan Klein, Ivan Sag, and Geoffrey Pullum). I have no idea if what C says applies to any of these individuals. If he is talking about job offers etc., it may not necessarily be appropriate for anyone who is in the know to comment publicly. (Or maybe it would be — I'm not in a position to make any judgment, just noting the possibility that such individuals might feel that way.)

    ReplyDelete
    Replies
    1. Thank you very much for this. Given that the passage is from a published text [not a private e-mail] in which Chomsky accuses Boden several times to refer to unnamed individuals it would seem hugely appropriate for 'someone who is in the know' to comment and remove any doubt that Chomsky is doing in this passage what he accuses Boden of.

      I only doubted that Chomsky referred to Gazdar because it would have really surprised me to learn Gazdar is 'a personal friend'. I think for the same reason one can rule out Pullum was referred to. But at least for me there is not enough information from the context to decide WHO was referred to. Still, I hope that for someone who is 'in the know' it is quite obvious who was referred to and this is not a case where Chomsky does himself what he accuses Boden of doing - referring to an unnamed individual that his audience has no means of identifying...

      Delete
    2. People aren't necessarily going to feel that it's appropriate to reveal this information just because some guys on the internet are curious to know about it. (Again, just to be clear, I make no judgment — I don't know the facts and it might be entirely appropriate for all I know.) The cases where Chomsky complains that Boden hasn't given names are hardly comparable. Why do you care about any of this gossip anyway? It's pretty far removed from the subject matter of linguistics.

      Delete
    3. Actually in the particular case I care mostly about [potential] ambiguity - which is subject matter of linguistics. If there's an obvious X such that Chomsky must have meant X, there would be no ambiguity. Nor would it be gossip, just stating a fact - which I also want to confirm is what Chomsky did. [Personally, I doubt that a job offer that was made back in the 1980s would be a 'top-secret' matter in 2013, regardless whether X accepted the job. Otherwise I seriously doubt Chomsky would have mentioned it at all. But this really IS speculation I am not interested in.]

      Delete
  6. A few posts ago, in the context of POS arguments, we touched on another often-glossed-over issue in the nature of linguistic data, which I think is also important for keeping things straight: most fundamentally, acceptability is a property of string-meaning pairs. In other words, the general question we are usually asking when we collect judgements (as I see it, at least), is of the form "Can this string s be acceptably associated with the meaning m?", or in somewhat closer to the normal terminology, "Is s acceptable with interpretation/reading/meaning m?" Either way, what we're really asking in the simplest terms is "Is (s,m) acceptable?" Sometimes, we come across a string s such that there is no m such that (s,m) is acceptable, and we might say that this s is unacceptable; but I think confusion often arises if we forget that this acceptability-of-strings is a derivative notion.

    Bearing this in mind doesn't change the significance of the -A/+G examples you mention in (2): in each case, there's a particular meaning m that is the "obviously" intended one to go with the three strings you list, and native speakers will tend to find the relevant (s,m) pairs unacceptable. Similarly with the -A/+G examples in (3).

    But the examples like (4) don't look like obvious cases of +A/-G when we take the relevant objects to be string-meaning pairs. I don't think there is any meaning m such that ("More people have been to Russia than I have", m) is judged acceptable. What is true about these examples is that if you ask whether the string is acceptable without providing any intended interpretation --- roughly, if you ask a question of the form "Is there a meaning m such that (s,m) is acceptable?" --- then people tend to say "yes". This despite the fact that, as everyone points out, if you ask which meaning this is, people are stumped. So in attempting to answer the question "is there a meaning that can go with this string", in a task where they are not required to actually put their finger on any such meaning, people for some reason make a mistake and answer "yes". Why they should make this kind of mistake, I have no idea: presumably the answer might be something like, they start searching for a meaning for the string, and they get close enough to feel confident that a meaning can be found without getting all the way there, so they stop and answer "yes" (since no one is asking for the particular meaning). But whatever the answer is, it seems likely to be something specific to the task of judging a string in isolation as "acceptable"/"unacceptable", and (despite appearances) I think this is a task that is rarely, if ever, used in day-to-day linguistics.

    An example that I think throws a more genuine spanner/wrench in the works, by mismatching +A/-G as properties of string-meaning pairs, might be Chomsky's "No eye-injury is too trivial to ignore". This is a situation where people will typically judge (s,m1) to be acceptable, at least at first, but our best guess is that this pair is in fact ungrammatical and a distinct pair (s,m2) is grammatical. (Are there others of these?)

    ReplyDelete
  7. I agree. Though the standard tests often ignore the pairing. Think acceptability surveys on 7 point scales. This said I agree that the Chomsky case is a great one.

    ReplyDelete
    Replies
    1. Well ... are tests that ignore the pairing really "the standard tests"? Yes, it's common to use something like acceptability surveys that just have a list of strings written on a piece of paper. But my impression is that most of the time, what the survey-writer really wants to know is the acceptability of a particular string-meaning pair, where the string is written on the paper and the intended meaning is left implicit. (For example, when you ask people to judge "What do you think that John likes?" vs. "What do you wonder whether John likes?", you don't bother specifying an intended meaning because we assume it's obvious somehow.) But leaving the intended meaning implicit is quite different from not having any intended meaning at all, even though these two options look the same in terms of what ends up written on the paper survey.

      Of course this is not to say that the "More people have been to Russia" examples can't ever tell us anything interesting. Obviously the string-acceptability judgement task is revealing that something surprising and interesting happens with these strings. But in an (I think) important sense, this string-acceptability judgement task is like a brand new experimental device whose readings we don't know how to interpret yet. It doesn't seem to be the task/device that has provided the data on which our theories of grammar are currently built.

      (This string-meaning pairs thing is perhaps fast becoming my pet point, liable to elicit groans as soon as I bring it up, so I'm somewhat reluctant to keep harping on it ... but I couldn't leave it alone ...)

      Delete
    2. Uncle! You are right. The judgements are about string-meaning pairs and that is what acceptability is about. The new super improved acceptability judgement tests that oour psych friends have been foisting on us abstract away from this and ask for a simple rating. This is not the right way to go and requires some interpretation as you say. So, if by acceptability we mean of the string-meaning pair, then the 'more people...' example fails. I like the neg case you provided as an alternative. FOr my money the important point is that we need not think that acceptability track grammaticality. THey are related, but not identical, concepts.

      Delete
    3. Right, I don't disagree with that main point. And of course the methods that distinguish the "new super improved acceptability judgements" can perfectly well be used in the context of string-meaning pair acceptability too.

      If the eye-injury example is a genuine case of +A/-G, then another one that I just realised might be roughly on a par with the Russia examples are strings like "The cat the rat the dog chased escaped". It's like the -A/+G centre-embedding examples but it's missing one of the verbs, and is therefore nonsense, but sounds relatively good.

      Alright, sorry, I'll stop now.

      Delete