More reading for the curious

Deep learning: Here are two papers that investigate the “psychological reality” of  some popular deep learning models. They are particularly important for those wishing to borrow insights from this literature for cognitive ends. What the papers show is that it is possible to construct stimuli that the systems systematically classify (i.e. classify with very high confidence) as objects that no human would mistake them for.  Thus, deep neural networks are easily fooled: see http://arxiv.org/abs/1412.1897

and http://arxiv.org/abs/1312.6199.

These papers do something that linguists commonly do. The papers are about negative data. Negative data describe what humans do not do (e.g. native English speakers do not accept sentences like “*who did you meat a man who saw”). If deep learning models are to be understood as psychological theories, they need to agree both on the good and the bad data (i.e. on what we accept and reject). So far, much of the discussion has been on the positive capacities of such systems. They can be trained to spot a dog in a picture. However, these papers observe that current systems spot dogs that are not there, or, more precisely, categorize some picture as a dog photo that no human would so categorize. Or as the first paper puts it:

A recent study revealed that changing an image (e.g. a lion) in a way imperceptible to humans can cause a DNN [deep neural network NH] to label; the image as something else entirely (e.g. mislabeling a lion a library). Here we show a related result: it is easy to produce images that are completely unrecognizable to humans, but the state of the art DNNs believe to be recognizable objects with 99.99% confidence (e.g. labeling with certainty that white noise state is a lion).

Why do DNNs do this? Right now, it seems that nobody knows. Need I say that these are important results for the psychological “reality” of DNNs? As every linguist knows, explaining negative data is critical in evaluating any proposal aimed at describing our mental powers.

A note on evolution: http://www.newyorker.com/news/daily-comment/evolution-catechism. This is an interesting discussion of the obvious political import that theories of evolution have in the US. The points are mainly obvious and congenial (to me). However, there is one distinction I would have made that Gopnik does not; the difference between the fact of evolution versus the centrality of the mechanism of natural selection (NS) as the prime causal force behind evolution. The fact is completely uncontroversial. Indeed, it was considered commonplace before Darwin, though Darwin did a lot to cement the truth of this fact. What is somewhat controversial today is how large a part NS plays in explaining this fact. All agree that it plays some role, the question is how big.

In many ways the recent Evo-Devo discoveries replay discussions similar to those in the early cog revolution. The Evo-Devo stuff suggests that the range of options that NS has to pick from is quite a bit narrower than earlier believed (viz. there are very few ways of doing anything (e.g. building an eye) and these tend to be strongly conserved over evolutionary time scales. Of course, the fewer the options available, the less one looks to NS to explain the outcome. Why? Because NS relies on the idea that were you are is heavily dependent on the path you took to getting there. But if the number of paths to get anywhere is very small in number then why you got to where you are is less dependent on a long series of linked choices than on the one or two you made at the very start. So, it is not that NS plays no role. Rather the importance of NS’s role depends on how wide the range of possibilities. The question is then not either/or but how much. And these are theoretical/empirical questions.

So, Gopnik is quite right that denying the fact of evolution is a nutty thing to do, sort of like denying that the earth is round or the earth orbits the sun. However, questioning the size of NS effects to evolutionary trajectories is not. NS is a theory. Evolution is the fact. As Gopnik notes, theories evolve and change. One of the changes being currently contemplated is that NS is a less potent factor than heretofore believed. Even a Republican can believe this in scientific good faith.

A neat paper on sources of evolution making a splash:

http://www.cell.com/article/S2211-1247(14)01105-X/abstract

Some scene setting from Bob Berwick:

This is an interesting evolutionary finding because it uncovered a new mechanism by which evolution can work very quickly in a very complex setting.  We don’t know much about how complex organ systems evolve – the “major transitions in evolution” – like our brains. There are so many genes involved – how is it all put together without it getting all tangled up?  But now somebody’s got their foot in the door about one of the biggest transitions of all – how placental mammals evolved pregnancy, and went from laying eggs externally to growing them inside their bodies. Turns out that again (surprise!) this involved a set of regulatory genes, plus – the real surprise –what are called ‘transposons’ – bits of genes that can leap whole genes at a single bound and insert themselves even across chromosomes.[1] (Lynch et al., Ancient Transposable Elements Transformed the Uterine Regulatory Landscape and Transcriptome during the Evolution of Mammalian Pregnancy, Cell Reports (2015.[2] Apparently, the transposons donated regulatory elements to the genes that were recruited to alter the immune system so that the mother wouldn’t reject fetal embryos as foreign (remember a fetus has all those unknown genes from daddy). Lynch et al. demonstrated that this involved thousands of genes in a carefully coordinated orchestration led in part by the transposons, enabling exceptionally rapid evolution.  As Lynch notes, nobody expected to find that evolution could work this way to evolve large complex organ systems. Seems we still have a lot to learn about the basic evolutionary machinery, more than 150 years after Darwin.

More on Minds and Bodies: Those that enjoyed Chomsky’s discussion of the mind-body problem (here) (or should I say the non-existence of the problem given Newton’s excision of body from the equation) and were (rightly) dissatisfied with my discussion (here) might enjoy a real philosophical exposition of the state of the art by John Collins (here). It engages with lots of the philo literature on these matters and is eminently readable, even for linguists. He discusses and defends a position that he calls it “methodological naturalism,” which, if understood and adopted as the standard in the cog-neuro sciences (including linguistics) will remove most of the metaphysical and epistemological underbrush that hinders fruitful collaboration between linguistics and neuro-types. So, take a look and pass it onto your friends (and enemies) in the neurosciences who ignore most of what you have to say.

---

[1] First discovered by Barbara McClintock in corn, in the 1940s.  Nobody believed her at first, because it violated everything people thought they new about Mendelism and genes, but she hammered away at it.  Forty years later she got a Nobel prize.

[2]http://dx.doi.org/10.1016/j.celrep.2014.12.052.