Chris Dyer and Colin Phillips both sent me links (here) to another public display of sagacity by Michael Jordan (the CSer not the ex Bulls phenome). The piece is pretty interesting for someone like me that is a consumer of the sorts of things that he is expert in. What I found most interesting is the care with which hue approaches lots of the recent "successes" that get widely cited in the press: Big Data, Deep learning, Turing Tests, the Singularity, neural nets. He is skeptical about overselling and appreciates (or so it seems) how much of this is old wine in new bottles. Here's a vintage quote or two to give you the flavor of the interview:
"We don't know how neurons learn. Is it actually just a small change in the synaptic weight that's responsible for learning? that's what these artificial neural networks are doing. In the brain, we have precious little idea how learning is actually taking place." (3)
"…it is important to distinguish two areas where the word neural is being used… ONe of them is in deep learning. And there , each "neuron" is really a cartoon…A second area …is aiming to get closer to a simulation of the brain…But the problem I see is that the research is not coupled with any understanding of what algorithmically this system might do. It is not coupled with a learning system that takes in data and solves problems, like in vision. It's really just a piece of architecture with the hope that someday people will discover algorithms that are udful for it. And there is no clear reason why that hope should be borne out…" (3-4)
One interesting thing for readers of FoL is that Jordan seems very high on working on language issues (see how he would spend $1 Billion (7)). He thinks that this is a domain worth exploring and that successful explorations will involve understanding both computational and representational issues. IMO, it would be good for linguistics were the idea generally adopted that understanding a little about how language is put together could be technologically fertile. I thinkJordan thinks that this is so. I hope his views gain wider purchase.
Michael Jordan comments.
ReplyDeleteThx for the comments. I am curious about how you read them. What I found interesting is that he only seems to regret the way his discussion of Big Data was reported. He said nothing about his qualms about neural nets, deep learning, the singularity or turing Tests. I take this to imply that here his comments reflect his views. As for Big Data, he did not step away from what I understood to be his most interesting comment: Big Data is a research problem with lots of loose ends NOT a solution to outstanding problems. In other words, Big Data is a great topic for investigation but it is not now (nor in the very near future) should we see it as the answer to all of our data prayers. It's this last feature about Big Data that has always gotten my goat. THe idea that more data is all by itself the answer to all of our worries. Jordan seems unmoved by this. He thinks we need much more theory to understand where and how it will work. It's not a substitute for the standard form of scientific inquiry, as has been suggested more than once.
DeleteWell, that's what I took away. You?