m8ta
You are not authenticated, login.
text: sort by
tags: modified
type: chronology
[0] Kakade S, Dayan P, Dopamine: generalization and bonuses.Neural Netw 15:4-6, 549-59 (2002 Jun-Jul)

{653}
hide / / print
ref: Kakade-2002.07 tags: dopamine reward reinforcement learning Kakade Dayan date: 12-09-2008 21:27 gmt revision:1 [0] [head]

PMID-12371511[0] Dopamine: generalization and bonuses

  • suggest that some anomalies of dopamine activity is related to generalization and novelty. In terms of novelty, dopamine may be shaping exploration.
  • review results that DA activity signal a global prediction error for summed future reward in conditioning tasks.
    • above, A = pre-training; B = post-training; C = catch trial.
    • this type of model is essentially TD(0); it does not involve 'eligibility traces', but still is capable of learning.
    • remind us that these cells have been found, but there are many other different types of responses of dopmamine cells.
  • storage of these predictions involves the basolateral nuclei of the amygdala and the orbitofrontal cortex. (but how do these structures learn their expectations ... ?)
  • dopamine release is associated with motor effects that are species specific, like approach behaviors, that can be irrelevant or detrimental to the delivery of reward.
  • bonuses, for the authors = fictitious quantities added to rewards or values to ensure appropriate exploration.
  • resolution of DA activity ~ 50ms.
  • Romo & Schultz have found that there are phasic increases in DA activity to both rewarded and non-rewarded events/stimuli - something that they explain as 'generalization'. But - maybe it is something else? like a startle / get ready to move response?
  • They suggest that it is a matter of intermediate states where the monkey is uncertain as to what to do / what will happen. hum, not sure about this.

____References____