m8ta
you are not logged in, login. new entry
text: sort by
tags: modified
type: chronology
{1415}
hide / edit[0] / print
ref: -0 tags: variational free energy inference learning bayes curiosity insight date: 10-31-2018 22:33 gmt revision:0 [head]

Active inference, curiosity and insight.

  • This has been my intuition for a while; you can learn abstract rules via active probing of the environment. This paper supports such intuitions with extensive scholarship.
  • “The basic theme of this article is that one can cast learning, inference, and decision making as processes that resolve uncertanty about the world.
    • References Schmidhuber 1991
  • “A learner should choose a policy that also maximizes the learner’s predictive power. This makes the world both interesting and exploitable.” (Still and Precup 2012)
  • “Our approach rests on the free energy principle, which asserts that any sentient creature must minimize the entropy of its sensory exchanges with the world.” Ok, that might be generalizing things too far..
  • Levels of uncertainty:
    • Perceptual inference, the causes of sensory outcomes under a particular policy
    • Uncertainty about policies or about future states of the world, outcomes, and the probabilistic contingencies that bind them.
  • For the last element (probabilistic contingencies between the world and outcomes), they employ Bayesian model selection / Bayesian model reduction
    • Can occur not only on the data, but exclusively on the initial model itself.
    • “We use simulations of abstract rule learning to show that context-sensitive contingiencies, which are manifest in a high-dimensional space of latent or hidden states, can be learned with straightforward variational principles (ie. minimization of free energy).
  • Assume that initial states and state transitions are known.
  • Perception or inference about hidden states (i.e. state estimation) corresponds to inverting a generative model gievn a sequence of outcomes, while learning involves updating the parameters of the model.
  • The actual task is quite simple: central fixation leads to a color cue. The cue + peripheral color determines either which way to saccade.
  • Gestalt: Good intuitions, but I’m left with the impression that the authors overexplain and / or make the description more complicated that it need be.
    • The actual number of parameters to to be inferred is rather small -- 3 states in 4 (?) dimensions, and these parameters are not hard to learn by minimizing the variational free energy:
    • F=D[Q(x)P(x)]E q[ln(P(o tx)] where D is the Kullback-Leibler divergence.
      • Mean field approximation: Q(x) is fully factored (not here). many more notes

{708}
hide / edit[1] / print
ref: Wagner-2004.01 tags: sleep insight mental restructure integration synthesis consolidation date: 03-20-2009 21:31 gmt revision:1 [0] [head]

PMID-14737168[0] Sleep Inspires Insight.

  • Subjects performed a cognitive task requiring the learning of stimulus–response sequences, in which they improved gradually by increasing response speed across task blocks. However, they could also improve abruptly after gaining insight into a hidden abstract rule underlying all sequences.
    • number reduction task - three numbers 1, 4, 9, in short sequence, with a simple comparison rule to generate a derivative number sequence; task was to determine the last number in sequence; this number was always the same as the second number.
  • This abstract rule was more likely to be learned after 8 hours of sleep as compared to 8 hours of wakefulness.
  • My thoughts: replay during sleep allows synchronous replay of cortical activity seen during the day (presumably from the hippocampus to the neocortex), replay which is critical for linking the second number with the last (response) number. This is a process of integration: merging present memories with existing memories / structure. The difference in time here is not as long as it could be .. presumably it goes back to anything in your cortex that is activated buy the hippocampal memories. In this way we build up semi-consistent integrated maps of the world. Possibly these things occur during dreams, and the weird events/thoughts/sensations are your brain trying to smooth and merge/infer things about the world.

____References____

[0] Wagner U, Gais S, Haider H, Verleger R, Born J, Sleep inspires insight.Nature 427:6972, 352-5 (2004 Jan 22)