m8ta
You are not authenticated, login. |
|
properties of the brain: 1 .cerebellum in a supervised learner, I agree with the evidence: it learns to predict future outcomes given present states very efficiently. Appears to have a structure that is conducive for learning spatio-temporal structures, with the parallel fibers and purkinje cells. climbing fibers fire on error and cause LTP. Purkinje cells have inhibitor output -> hence error to LTP to less inhibition = movement in the positive direction. Mossy fibers have collaterals to DCN neurons and purkinje cells, i think. this whole structure seems rather strange to me - why the multiple levels of inversion? it is the same as the basal ganglia - striatal output is inhibitory upon the globus pallidus, globus pallidus output is inhibitory on the thalamus. {and, at least in the monkey though probably also in the human, the thalamus is very large and very well organized}. actually, the whole brain seems exceedingly well organized, the problem is that we don't really understand this organization quite yet. E.G the putamen seems to have a somatotopic organization & has units which fire according to motion in the distal joints. (those old papers are great!) . caudate seems to have some sort of cognitive role? blaaa. so, what does the brain do? it learns to live, more or less; it is adaptive. humans seem to be thte most adaptive; we stay in the adaptive phase for the longest part of our life, whereas rhesus seem to grow up rather quickly. learning! as kawato's student explains, learning modifying a function to minimize (or maximize) some evaluative function. In the case the fitness function is some function of the match between desired output and training output, that learning is supervised. We have neural networks to do this, and undoubtably the human mind can do this too. In the case the fitness function is some weighted-sum of a scalar reward, then you have reinforcement learning. Generally, the animal will learn the value of certain states, actions, or state-action pairs, and has to choose which is the best based on either the direct perceived value or the integrated expected future value. Humans think in this way all the time, and use a high-level model of the world, learned basically by example, trial and error, and even book-learning, to 'do the integral' and evaluate which of several paths are best. Once we 'decide', things then become habits. We, and especially monkeys, are exceptionally subject to choosing arbitrarily when the reward is unknown - we explore all of our lives, in order to expand the quality of our models of the world, and improve the reward-evaluation of states and actions. Is this dichotomy between models and evaluations artificial? Is there any reason to believe that they are represented in separate structures/pathways/molecules in the brain? perhaps. take dopamine for example. blocking its reuptake via cocaine is very rewarding, and induces a habit in mammals that are administered the drug. but perhaps it it not so much involved in reward so much as desire. {drug addicts who have their DA1 receptor blocked end up taking /more/ drugs, apparently in the desire to feel something}. DA depletion in parkinsons makes the stick larger in carrot-stick learning: these patients learn worse with reward than controls. {hence, error must not require DA}. _{system function is hard to intuit from such nonspecific effectors like drugs because the system is adaptive; i actually think leasions are better, or at least seem better, due to the precise organization fo the brain. anyway, learning. "the controller learns the inverse model of it's own reflexes" - this is brilliant. only through hebbian learning! I like this a lot. In general, i agree with Kawato (actually, so far everything he has put out seems to be high-quality, well thought out and easy to understand) - the proof is incontrovertible that there are inverse models in the brain, probably at least in the cerebellum. todo: review what is required to make an inverse model. ok time to put the monkey away. |