Revisiting Gallistel's conjecture

I recently received two papers that explore Gallistel’s conjecture (see here for one discussion) concerning the locus of neuronal computation. The first (here) is a short paper that summarizes Randy’s arguments and suggests a novel view of synaptic plasticity. The second (here: http://www.nature.com/nature/journal/v538/n7626/full/nature20101.html)[1] accept Randy’s primary criticism of neural nets and couples a neural net architecture with a pretty standard external memory system. Let me say a word about each.

The first paper is by Patrick Trettenbrein (PT) and it appears in Frontiers in Systems Neuroscience. It does three things.

First, it reviews the evidence against the idea that brains store information in their “connectivity profiles” (2). This is the classical assumption that inter-neural connection strengths are the locus of information storage. The neurophysiological mechanisms for this are long term potentiation (LTP) and long term depression (LTD). LTP/D are the technical terms for whatever strengthens or weakens interneuron connections/linkages. I’ve discussed Gallistel and Matzel’s (G&M) critique of the LTP/D mechanisms before (see here). PT reviews these again and emphasizes G&M’s point that there is an intimate connection between this Hebbian “fire together wire together” LTP/D based conception of memory and associationist psychology. As PT puts it: “Crucially, it is only against this background of association learning that LTP and LTD seem to provide a neurobiologically as well as psychologically plausible mechanism for learning and memory” (88). This is why if you reject associationsim and endorse “classical cognitive science” and its “information processing approach to the study of the mind/brain” you will be inclined to find contemporary connectionist conceptions of the brain wanting (3).

Second, there is recent evidence that connection strength cannot be the whole story. PT reviews the main evidence. It revolves around retaining memory traces despite very significant alterations in connectivity profiles. So, for example, “memories appear to persist in cell bodies and can be restored after synapses have been eliminated” (3), which would be odd if memories lived in the synaptic connections. Similarly it has recently been shown that “changes in synaptic strength are not directly related to storage of new information in memory” (3). Finally, and I like this one the best (PT describes it as “the most challenging to the idea that the synapse is the locus of memory in the brain”), PT quotes a 2015 paper by Bizzi and Ajemian which makes the following point:

If we believe that memories are made of patterns of synaptic connections sculpted by experience, and if we know, behaviorally, that motor memories last a lifetime, then how can we explain the fact that individual synaptic spines are constantly turning over and that aggregate synaptic strengths are constantly fluctuating?

Third, PT offers a reconceptualization of the role these neural connections. Here’s an extended quote (5):

…it occurs to me that we should seriously consider the possibility that the observable changes in synaptic weights and connectivity might not so much constitute the very basis of learning as they are the result of learning.

This is to say that once we accept the conjecture of Gallistel and collaborators that the study of learning can and should be separated from the study of memory to a certain extent, we can reinterpret synaptic plasticity as the brain's way of ensuring a connectivity and activity pattern that is efficient and appropriate to environmental and internal requirements within physical and developmental constraints. Consequently, synaptic plasticity might be understood as a means of regulating behavior (i.e., activity and connectivity patterns) only after learning has already occurred. In other words, synaptic weights and connections are altered after relevant information has already been extracted from the environment and stored in memory.

This leaves a place for connectivity, but not as the mechanism of memory but as what allows memories to be efficiently exploited.[2] Memories live within the cell but putting these to good use requires connections to other parts of the brain where other cells store other memories. That’s the basic idea. Or as PT puts it (6):

The role of synaptic plasticity thus changes from providing the fundamental memory mechanism to providing the brain’s way of ensuring that its wiring diagram enables it to operate efficiently…

As PT notes, the Gallistel conjecture and his tentative proposal are speculative as theories of the relevant cell internal mechanisms don’t currently exist. That said, neuroiphsyiological (and computational, see below) evidence against the classical Hebbian view are mounting and the serious problems for storing memories in usable form in connections strengths (the bases of Gallistel’s critique) are becoming more and more well recognized.

This brings us to the second Nature paper noted above. It endorses the Gallistel critique of neural nets and recognizes that neural net architectures are poor ways of encoding memories. It adds a conventional RAM to a neural net and this combination allows the machine to “represent and manipulate complex data structures.”

Artificial neural networks are remarkably adept at sensory processing, sequence learning and reinforcement learning, but are limited in their ability to represent variables and data structures and to store data over long timescales, owing to the lack of an external memory. Here we introduce a machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer. Like a conventional computer, it can use its memory to represent and manipulate complex data structures, but, like a neural network, it can learn to do so from data.

Note that the system is still “associationist” in that learning is largely data driven (and as such will necessarily run into PoS problems when applied to any interesting cognitive domain like language) but it at least recognizes that neural nets are not good for storing information. This latter is Randy’s point. The paper is significant for it comes from Google’s Deep Mind Project and this means that Randy’s general observations are making intellectual inroads with important groups. Good.

However, this said, these models are not cognitively realistic for they still don’t make room for the domain specific knowledge that we know characterizes (and structures) different domains. The main problem remains the associationism that the Google model puts at the center of the system. As we know that associationism is wrong and that real brains characterize knowledge independently of the “input,” we can be sure that this hybrid model will need serious revision if intended as a good cog-neuro model.

Let me put this another way. Classical cog sci rests on the assumption that representations are central to understanding cognition. Fodor and Pylyshyn and Marcus long ago agued convincingly that connectionism did not successfully accommodate representations (and, recall, that connectionist agreed that their theories dumped representations) and that this was a serious problem for connectionist/neural net architectures. Gallistel further argued that neural nets were poor models of the brain (i.e. and not only of the mind) because they embody a wrong concpetion of memory; one that that makes it hard to read/write/retrieve complex information (data structures) in usable form. This, Gallistel noted, starkly contrasts with more classical architectures. The combined Fodor-Pylyshyn-Marcus-Gallistel critique then is that connectionist/neural net theories were a wrong turn because they effectively eschewed representations and that this is a problem both from the cognitive and the neuro perspective. The Google Nature paper effectively concedes this point, recognizes that representations (i.e. “complex data structures) are critical  and resolves the problem by adding a classical RAM to a connectionist front end.

However, there is a second feature of most connectionist approaches that is also wrong. Most such architectures are associationist. They embody the idea that brains are entirely structured by the properties of the inputs to the system. As PT puts it (2):

Associationism has come in different flavors since the days of Skinner, but they all share the fundamental aversion toward internally adding structure to contingencies in the world (Gallistel and Matzel 2013).

Yes! Connectionists are weirdly attracted to associationism as well as rejecting representations. This is probably not that surprising. Once on thinks of representations then it quickly becomes clear that many of their properties are not reducible to statistical properties of the inputs. Representations have formal properties above and beyond what one finds in the input, which, once you look, are found to be causally efficacious. However, strictly speaking associationsim and anti-representationalism are independent dimensions. What makes Behaviorists distinctive among Empiricists is their rejection of representations. What unifies all Empiricists is their endorsement of associationism. Seen form this perspective, Gallistel and Fodor and Pylyshyn and Marcus have been arguing that representations are critical. The Google paper agrees. This still leaves associationism however, and position the Googlers embrace.[3]

So is this a step forward? Yes. It would be a big step forward if the information processing/representational model of the mind/brain became the accepted view of things, especially in the brain sciences. We could then concentrate (yet again) all of our fire on pernicious Empiricism so many Cog-neuro types embrace.[4] But, little steps my friends, little steps. This is a victory of sorts. Better to be arguing against Locke and Hume than Skinner![5]

That’s it. Take a look.

---

[1] Thx to Chris Dyer for bringing the paper to my attention. I put in the URL up rather than link to the paper directly as the linking did not seem to work. Sorry.

[2] Redolent of a competence/performance distinction, isn’t it?  The physiological bases of memory should not be confused with the physical bases for the deployment of memory.

[3] I should add that it is not clear that the Googlers care much about the cog-neuro issues. Their concerns are largely technological, it seems to me. They live in a Big Data world, not one where PoS problems (are thought to) abound. IMO, even in a uuuuuuge data environment, PoS issues will arise, though finding them will take more cleverness. At any rate, my remarks apply to the Google model as if intended as a cog-neuro one.

[4] And remember, as Gallistel notes (and PT emphasizes) much of the connectionism one sees in the brain sciences rests on thinking that the physiology has a natural associationist interpretation psychologically. So, if we knock out one strut, the other may be easier to dislodge as well (I know that this is wishful thinking btw).

[5] As usual, my thinking on these issues was provoked by some comments by Bob Berwick. Thx.

David Poeppel's second provocation

I want to say a couple of words about David Poeppel’s second lecture (see here); the one on which he argues that brains entrain to G like structures in online speech comprehension. In earlier posts (e.g. here), I evinced some skepticism about whether it will be possible for linguistics to make much contact with brain work (BW) until BW starts finding the neural analogues of classical CS notions like register, stack, index, buffer, etc. Here I want to, at once, elaborate, yet also mitigate, this skepticism in light of David’s lectures.

Whence my skepticism? The reason lies with BW methodology. Most BW focuses on what the brain does while being subjected to a particular stimulus. So for example, how does the brain “react” when peppered with sentences that vary in grammaticality (note: I mean ‘grammaticality’ here and not ‘acceptability’).[1] So, while a brain is processing an actual linguistic object, what does it do? By its very nature, this kind of experiment can only indirectly make contact with the basic claims of GG. Why? Because, GG is a theory of competence (i.e. what speakers know), and what BW probes is not knowledge (linguistic or otherwise) per se but how this knowledge is deployed in real time (e.g. how it is used to perform the task of, e.g. analyzing the incoming speech stream). I strongly believe that what you know is implicated in what you do. But what you do (and how you do it) does not rely exclusively on what you know. That’s why it is useful to make the competence/performance distinction. And, back to the main point, BW experiments are all based on online performance tasks where the details of the performance system matter to the empirical outcomes (or so I would suppose).[2] But if these details matter, then in order to even see the contribution of Gs (or FL/UG) it is imperative to have performance systems of the kind that we believe are cognitively viable, and the ones that we think are such are largely combinations of Gs embedded in classical computing devices with Turing-like architectures.[3]

Nor is this mix an accident if people like Randy Gallistel (and Jerry Fodor and Zenon Pylyshyn and Gary Marcus) are to be believed (and I for one make it a cognitive policy to believe most of what Randy (and Jerry) says). Turing architectures are just what cog-neuro (CN) needs to handle representations, and representations are something that any theory hoping to deal with basic cognition will need. As I have gone over these arguments before, I will not further worry this point. But, rest assured, I take this very seriously and because of this I draw the obvious conclusion: until BW finds the neural analogues of basic Turing architecture the possibility of BW and cognition (including linguistics) making vigorous contact will be pretty low.

That said, one of the interesting features of David’s second lecture is that he showed that it is not impossible. In fact, the general program he outlines is very exciting and worth thinking about from a linguistic point of view. This is what I will try to do here (with all the appropriate caveats concerning my low skill set etc.).

The lecture focuses on what David describes as “an interesting alignment between…systems neuroscience….physics…(and) linguistics” (slide 21). More particularly, he argues that the brain have natural “theta rhythms” of 4-8 Hz, that physically speaking the modulation spectrum of speech is 4-5 Hz and that the mean syllable duration cross linguistically is 150-300 ms, which is the right size to fit into these theta bands. So, if speech processing requires chunking into brains sized units, then we might expect the stream to be chopped into theta band sizes for the brain to examine in order to extract further linguistic information necessary to get one to the relevant interpretation. And what we expect, David claims we in fact find. The brain seems to entrain to syllables, phrases and sentences. Or, more exactly, we can find neural measures that seem to correlate with each such linguistic unit (see slides 29-34).

Now showing this is not at all trivial, and therein lies the beauty of the experiments that David so lovingly reported (it was actually a bit embarrassing to see him caressing those findings so intimately in such a public setting (over 150 people watching!!)). Here’s a post to a discussion of (and link to) the paper that David discussed. Suffice it to say that what was required to make this convincing was controlling for the many factors that likely correlate with the syntactic structure. So the Ding et al (David being the last al) paper stripped out prosodic information and statistical transitional probability information so that only linguistic information concerning phrase structure and sentence structure remained. What the paper showed is that even in the absence of these cues in the occurent stimulus the brain entrained to phrases and sentences in addition to syllables. So, the conclusion: brains of native speakers can track G like structures of their native languages on line. In other words, it looks like brains use Gs in online performance.

Now, gorgeous as this is, it is important to understand that the result is very unsurprising from a linguistic point of view. The conventional position is speakers use their knowledge to parse the incoming signal. On the assumption that humans do this in virtue of some feature of their brains (rather than say their ham strings or pituitary glands) then it is not surprising to find a neural correlate of this performance. Nor do David and friends believe otherwise. Oddly, the finding caused somewhat of a buzz in the lecture hall, and this only makes sense if the idea that people use their Gs in performance is considered a way-out-there kind of proposition.

I should add, that nothing about these results tell us where this Gish knowledge comes from. None of the entrainment results implicate piles of innate brain structure or a genetic basis for language or any of the other “sexy” (viz. conceptually anodyne) stuff that gets a psychologist or neuroscientist hopping madly around flailing his/her arms and/or pulling his/her hair. All it implicates is the existence of something like our old recognizable linguistic representations (representations, btw, which are completely uncontroversial linguistically in that they have analogues in virtually every theory of syntax that I know of). They imply that sentences have hierarchical structures of various kinds (syllables, phrases, sentences) and that these are causally efficacious in online processing. How could this conclusion possibly cause a stir.  It shouldn’t, but it did. That we can find a neural correlate of what we know to be happening is interesting and noteworthy, but it is not something that we did not expect to exist, at least absent a commitment to Cartesian Dualism of the most vulgar kind (and yes, there are sophisticated forms of dualism).

What structures do the neural signals correlate with? Many have noted that the relevant level may not be syntactic. In particular, what we are tracking might be phonological phrases, rather than syntactic ones (I said as much at Nijmegen and Ellen Lau noted this in a comment here). However, this does not really matter for the main conclusion. Why? Because the phonological phrases that might be being tracked are products of an underlying syntax from which they are determined. Here’s what I mean.

Say that what is being tracked are phonological phrases of some sort. What the experiment shows is that they are not tracking these objects in virtue of their phonological structure. The relevant phonological information has been weeded out of the stimuli, as has the statistical information. So, if the brain is entraining to phonological phrases, then it is doing so in virtue of tracking syntactic information. Now, again, every theory I know of has a mapping between syntactic and intonational structure so the fact that one is tracking the latter by analyzing structure according to the former is not a surprise and it remains evidence that the brain can use (and does use) G information in parsing the incoming speech string. So the general conclusion that David draws (slide 57) seems to me perfectly correct:

1.     There are cortical circuits in the brain that “generate slow rhythms matching the time scales of larger linguistic structures, even when such rhythms are not present in the speech input” and this “provides a plausible mechanism for online building of large linguistic structures.”

2.     Such tracking is rule/grammar based.

So what is exciting is not the conclusion (that brains use G information in performance) but the fact that we now have a brain measure of it.

There is also an exciting suggestion: that what holds for the syllable, also holds for phrases and sentences. Here’s what I mean. There is ample evidence that David reviews in lecture 2 that the physics of speech results in chunks that have both a nice physical description and that fit into nice neural bands and that find a linguistic analogue; the syllable. So there is a kind of physical/neural basis for this.

The question is whether this analogy extends to larger units?[4] What David shows is that the brain entrains to these units and that it does so using the theta and delta band oscillations to do so. However, is it plausible that phrases and sentences come (at least on average) in certain physical “sizes” that neatly fit into natural brain cyclic bands so that we can argue that there is a physical/neural natural size that phrases and sentences conform to? This seems like a stretch. Is it?

I have no idea. But, maybe we shouldn’t dismiss the analogy that David is pushing too quickly. Say that it is phono phrases that the Ding et al paper is tracking. The question then is what kind of inventory of phono phrases do we find. We know that these don’t perfectly track syntactic phrases (remember: this is the cat, that ate the rat, that stole the cheese, that…). There are “readjustments” that map constituents to their phonological expressions. The question then is whether it is too far fetched to think that something like this can hold for phrases and sentence as well as syllables. Might there be a standard “average” size of a phrase (or a phase that contains a phrase) say? One that is neatly packaged in a brain wave with the right bandwidth? This doesn’t seem too far-fetched to me, but how would I know (remember, I know squat about these matters). At any rate, this is the big idea that David’s second and third lecture hint at. We are looking for natural neural envelopes within which interesting syntactic processing takes place. We are not looking at syntactic processing itself, or at least neither David’s lectures nor the Ding et al paper suggest that we are, but the containers the brain wraps them in when analyzing them online.

That said, the envelope thesis would be interesting were it true, even for a syntactician like me. Why? Because this could plausibly constrain the mapping between (let’s call them) phonological phrase and syntactic phrases and this might in turn tell us something about syntactic phrases. Of course, it might not. But it is a useful speculation worth further investigation, which, I am pretty certain, is what David is going to do.

To wrap up: I am pretty skeptical that current neuro methods can make contact with much of what linguistics does but not because of any problems with linguistics. The problem lies, in the main, with the fact that BW has stopped looking for the structures required to tell any kind of performance story. In a word, it has severed its ties to classical CS as Gallistel has cogently argued.  I believe that once we find classical architectures in the brain (and they are there as the cognitive evidence overwhelmingly shows us) then contact will be rampant and progress will be made understanding how brains do language. This, however, will still mainly tell us about how Gs get used and so only indirectly shine a light on what kinds of representations Gs have. Of course, you can find lost keys even in indirect light so this is nothing to sneeze at, or pooh-pooh, or sneer at or cavalierly dismiss, or…What David’s work has shown is that it might be possible to find interesting linguistically significant stuff even short of cracking the Turing architecture/representation problem. David’s work relies on the idea (truism?) that brains chunk information and the bold hypothesis that this chunking is both based in brain architecture and might (at least indirectly) correlate with significant linguistic units. He has made this conjecture solid when it comes to syllables. It would be a really big deal if he could extend this to phrases and sentences. I hope he is right and he can do it. It would be amazing and wonderful.

One last comment: say that David is wrong. What would it mean for the relation between linguistics and BW? Not much. It would leave us roughly where we are today. The big problem, IMO, is still the one that Gallistel has identified. This is true regardless of whether David’s hope is realizable. Neural envelopes for packaging representations are not themselves representations. They are what you stuff representations into. The big goal of the cog-neuro game should be to understand the neural bases of these representations. If David is right, we will have a narrower place to look, but just as you cannot tell a book by its cover (although, personally I often try to) so you cannot deduce the (full) structure of the representations from the envelopes they are delivered in. So should David fail, it would be really too bad (the idea is a great one) but it would not alter much how we should understand the general contours of the enterprise. In other side, if David wins, we all do. If his conjecture stops at the syllable, this has no implications about the neural reality of phrases and sentences. They are very real, whether or not our current technology assigns them stable neural correlates.

---

[1] A HW problem: what’s the difference?

[2] In fact, that whole point is that these are online tasks that allow you to look into what is being done as it is being done. This contrasts with off line, say, acceptability judgment tasks where all we look at it is the final step of what all concede to be a long and complicated process. Interestingly, it appears that taking a look at this last step allows for a better window into the overall system of knowledge than does looking at how these steps are arrived at. This is not that surprising when one thinks about it, despite the air of paradox. Think forests and trees.

[3] Think Marcus parsers or the Berwick and Weinberg left corner parser (a favorite of mine for personal reasons).

[4] This is what Ellen was skeptical of.

Cakes, Damn Cakes, and Other Baked Goods

As promised in my previous post, here's Omer's reply to my musings on derivations and representations.

---

In a recent post on this blog, Thomas Graf addresses the derivationalism vs. representationalism debate---sparked (Thomas' post, that is) by my remarks in the comments section of this post.

Thomas notes, among other things, that literally any derivational formalism can be recast representationally. For example, one could take the set of licit derivational operations in the former model, and turn them into representational well-formedness conditions on adjacent pairs of syntactic trees in an ordered sequence. (Each tree in this ordered sequence corresponds, informally, to an intermediate derivational step in the former model.) As best I can tell, this translatability of any derivational formalism into representational terms is not even debatable; I certainly wouldn't argue this point.

The Cake is a Lie

In the comments section to Norbert's final remarks on Chomsky's lecture series, Omer Preminger laments the resurrection of representational devices in Minimalism, which started with Chomsky (2004) "Beyond Explanatory Adequacy". I originally took Omer to argue against representational approaches on two levels:

1. They are fundamentally flawed regarding both competence (bad empirical coverage) and performance (no plausible parsing model).
2. Phase-theory is ill-motivated and doesn't get the job done.

After a short email conversation with Omer it actually became clear that this is not an accurate reflection of his views, which he will summarize in a guest post within the next few days summarizes in this follow-up post. Still, now that we've got those two claims laid out in front of us, let's assess their soundness.

I've got no qualms with the second claim. The motivation and empirical use of phases has frequently been criticized in the literature, and phases don't fare any better from a computational perspective. The memory reduction argument for phases is hogwash, phases have no discernible effect on generative capacity (neither weak nor strong), and they do not simplify the learning problem. Norbert captures the state of affairs ever so succinctly: Phases don't walk the walk [...] the rhetoric is often ahead of the results.

The first claim, on the other hand, I can't get behind at all. For one thing, it's a typical case of the cake fallacy: Every cake I've made in my life has been horrible, thus all cakes are horrible (and I'm more of a muffins guy anyways). Even if linguists haven't come up with any good representational models so far --- which I think many syntacticians would emphatically disagree with --- that doesn't mean the approach as such is intrinsically inferior to derivational ones. Now I can already see the flurry of posts about how theory construction is also a probabilistic process where previous failures of specific types of analysis make them less preferable, that the existence of a working representational account is moot if we can't find it, yada yada yada. Step back from your keyboards everyone, I'm actually not interested in arguing this point. My real worry is much more basic. The first claim implies, as is commonly done in linguistics, that there is a meaningful difference between representational and derivational accounts. Well, it turns out that this specific cake is a lie.