tag:blogger.com,1999:blog-5275657281509261156.post6043400266827166016..comments2024-06-23T01:13:03.173-07:00Comments on Faculty of Language: Bayesian claims?Norberthttp://www.blogger.com/profile/15701059232144474269noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-5275657281509261156.post-37408856795129850662013-11-19T21:12:39.384-08:002013-11-19T21:12:39.384-08:00I think the methodological Bayes stance fits very ...I think the methodological Bayes stance fits very nicely into Marr's levels. A Bayesian model specifies how a learner analyzes structures into parts. For example, the likelihood function for a dependency tree may compute the probability of the tree in terms of the probabilities of individual arcs between parts of speech, or it may compute the probability of the tree in terms of larger subtrees, or arcs between words and parts of speech, and so on. This is the computational level analysis, and says something like: if this is what our reusable pieces look like (e.g. tree substitution grammar elementary trees), and this is how much we prefer each piece before seeing data (we prefer smaller elementary trees), and this is our dataset that gives us the values of some of those variables (e.g. all the words and sentence boundaries), then here is the probability, for each location, that each piece was used. There is a diverse range of algorithms for actually computing these probabilities, or even just using them (in practice, as Ben keeps reminding me, researchers usually just want to find the most likely composition of pieces, and don't care what the actual probabilities are).<br /><br />Given a model specification, there are many different algorithms for performing inference in the model specification. Some of those algorithms make “soft” choices, combining partial guesses about unobserved structure. Other algorithms will make hard choices, comitting totally to a structure, but make these hard choices in proportion to the posterior probability over the long run of making choices. The choice of algorithm is the algorithmic level analysis. When OJB say that “it is a central characteristic of fully Bayesian models that they represent the full state space,” they must mean that the abstract (computational-level) model represents the full state space, not that the algorithm for performing inference in the model represents the full state space. The whole point of sampling and variational algorithms is that the full state space is too big to represent. Indeed, non-parametric models have an infinite state space that cannot be fully represented in finite memory.John K Patehttps://www.blogger.com/profile/04079185852054818615noreply@blogger.comtag:blogger.com,1999:blog-5275657281509261156.post-37551318984325016732013-11-19T15:36:18.215-08:002013-11-19T15:36:18.215-08:00I think I agreed in note 1. But if this is the app...I think I agreed in note 1. But if this is the appeal we should be told and it's interesting to see there is another point of view. Last, if this is the right conception we should dump all the talk of Marr. Agreed?Norberthttps://www.blogger.com/profile/15701059232144474269noreply@blogger.comtag:blogger.com,1999:blog-5275657281509261156.post-82905835448202390192013-11-19T15:03:04.623-08:002013-11-19T15:03:04.623-08:00One huge attraction for the "methodological B...One huge attraction for the "methodological Bayes" stance is that it provides a principled way of studying how certain assumptions interact with certain inputs. As such, Bayesian modeling is an obvious tool to address questions of what can and cannot, in principle, be acquired from data, by a learner that embodies certain well-specified inductive biases. I don't know whether this is anything like double entry bookkeeping, but it certainly isn't something to sneeze at. <br />I'm happy to agree that there is not as much conceptually clarity as one would hope for. But that's hardly a feature that distinguishes Bayesian computational modelling from theoretical syntax.<br />[corrected typo]benjamin.boerschingerhttps://www.blogger.com/profile/00894608988488218285noreply@blogger.comtag:blogger.com,1999:blog-5275657281509261156.post-1762478109145756422013-11-19T15:01:45.509-08:002013-11-19T15:01:45.509-08:00This comment has been removed by the author.benjamin.boerschingerhttps://www.blogger.com/profile/00894608988488218285noreply@blogger.com