you are not logged in, login. new entry
text: sort by
tags: modified
type: chronology
hide / edit[0] / print
ref: -0 tags: variational free energy inference learning bayes curiosity insight date: 10-31-2018 22:33 gmt revision:0 [head]

Active inference, curiosity and insight.

  • This has been my intuition for a while; you can learn abstract rules via active probing of the environment. This paper supports such intuitions with extensive scholarship.
  • “The basic theme of this article is that one can cast learning, inference, and decision making as processes that resolve uncertanty about the world.
    • References Schmidhuber 1991
  • “A learner should choose a policy that also maximizes the learner’s predictive power. This makes the world both interesting and exploitable.” (Still and Precup 2012)
  • “Our approach rests on the free energy principle, which asserts that any sentient creature must minimize the entropy of its sensory exchanges with the world.” Ok, that might be generalizing things too far..
  • Levels of uncertainty:
    • Perceptual inference, the causes of sensory outcomes under a particular policy
    • Uncertainty about policies or about future states of the world, outcomes, and the probabilistic contingencies that bind them.
  • For the last element (probabilistic contingencies between the world and outcomes), they employ Bayesian model selection / Bayesian model reduction
    • Can occur not only on the data, but exclusively on the initial model itself.
    • “We use simulations of abstract rule learning to show that context-sensitive contingiencies, which are manifest in a high-dimensional space of latent or hidden states, can be learned with straightforward variational principles (ie. minimization of free energy).
  • Assume that initial states and state transitions are known.
  • Perception or inference about hidden states (i.e. state estimation) corresponds to inverting a generative model gievn a sequence of outcomes, while learning involves updating the parameters of the model.
  • The actual task is quite simple: central fixation leads to a color cue. The cue + peripheral color determines either which way to saccade.
  • Gestalt: Good intuitions, but I’m left with the impression that the authors overexplain and / or make the description more complicated that it need be.
    • The actual number of parameters to to be inferred is rather small -- 3 states in 4 (?) dimensions, and these parameters are not hard to learn by minimizing the variational free energy:
    • F=D[Q(x)P(x)]E q[ln(P(o tx)] where D is the Kullback-Leibler divergence.
      • Mean field approximation: Q(x) is fully factored (not here). many more notes

hide / edit[1] / print
ref: -0 tags: spike sorting variational bayes PCA Japan date: 04-04-2012 20:16 gmt revision:1 [0] [head]

PMID-22448159 Spike sorting of heterogeneous neuron types by multimodality-weighted PCA and explicit robust variational Bayes.

  • Cutting edge windowing-then-sorting method.
  • projection multimodality-weighted principal component analysis (mPCA, novel).
    • Multimodality of a feature is by checking the informativeness using the KS test of a given feature.
  • Also investigate graph laplacian features (GLF), which projects high-dimensional data onto a low-dimensional space while preserving topological structure.
  • Clustering based on variational Bayes for Student's T mixture model (SVB).
    • Does not rely on MAP inference and works reliably over difficult-to sort data, e.g. bursting neurons and sparsely firing neurons.
  • Wavelet preprocessing improves spike separation.
  • open-source, available at http://etos.sourceforge.net/

hide / edit[3] / print
ref: bookmark-0 tags: machine_learning research_blog parallel_computing bayes active_learning information_theory reinforcement_learning date: 12-31-2011 19:30 gmt revision:3 [2] [1] [0] [head]

hunch.net interesting posts:

  • debugging your brain - how to discover what you don't understand. a very intelligent viewpoint, worth rereading + the comments. look at the data, stupid
    • quote: how to represent the problem is perhaps even more important in research since human brains are not as adept as computers at shifting and using representations. Significant initial thought on how to represent a research problem is helpful. And when it’s not going well, changing representations can make a problem radically simpler.
  • automated labeling - great way to use a human 'oracle' to bootstrap us into good performance, esp. if the predictor can output a certainty value and hence ask the oracle all the 'tricky questions'.
  • The design of an optimal research environment
    • Quote: Machine learning is a victim of it’s common success. It’s hard to develop a learning algorithm which is substantially better than others. This means that anyone wanting to implement spam filtering can do so. Patents are useless here—you can’t patent an entire field (and even if you could it wouldn’t work).
  • More recently: http://hunch.net/?p=2016
    • Problem is that online course only imperfectly emulate the social environment of a college, which IMHO are useflu for cultivating diligence.
  • The unrealized potential of the research lab Quote: Muthu Muthukrishnan says “it’s the incentives”. In particular, people who invent something within a research lab have little personal incentive in seeing it’s potential realized so they fail to pursue it as vigorously as they might in a startup setting.
    • The motivation (money!) is just not there.

hide / edit[4] / print
ref: notes-0 tags: global warming bayes politics plik-l biofuel oil economics date: 11-21-2007 22:20 gmt revision:4 [3] [2] [1] [0] [head]

This was written for the plik-l mailing list, Nov 16 2007

I actually had a bit of an argument yesterday with my dentist, no less, about global warming:
  • Dentist: Hello, how are you today?
  • Tim: Ok.
  • D: Are you still in school?
  • T: <defers complicated explanation for the simplified>
  • D: Oh, so do you believe in global warming?
  • T: <cites scientific study, like http://www.ipcc.ch/ipccreports/assessments-reports.htm>
  • D: Well, i don't believe in it but even if it is happening, nobody is going to stop burning gas.
  • T: Yea, but if gas and electricity were more expensive, then people would make better economic decisions, smaller cars etc.
  • D: That would just prolong the supply. Oil is a great source of energy, and we are not going to stop using it until it become economically infeasible to do it. So, why worry? Oil will be depleted and the C02 will be stuck in the atmosphere, if not by us then by some other country that needs cheap energy to grow its economy, eventually. Economics.
  • T: You seem to be hinting of China, I guess. But, if our leaders decided to let the price of oil float to where it should be, and did not fight wars over it, then there would be greater economic impetus & possibly government funding to develop alternatives to oil. This would give us some energy-independence.
  • D: We are not in iraq for the oil. That's enough now, open up!
  • T: Wait wait! But don't you know what global warming will do to the envirnoment? More storms, droughts, floods, famines, etc - all very expensive, terrible.
  • D: I do not think there is sufficient organization in the world to impose the true costs of burning oil - e.g. the cost, accumulated over the future, of present greedy practices - upon present consumers.
  • T: True, i suppose if we integrated up, the cost would be almost boundless. Hence we should stop burning oil right now!
  • D: A responsibility to the future is not in the nature of man. They eventually die, and are selfish, greedy, and lack foresight during their lives. Besides, abstaining from oil imposes a severe economic disadvantage.
  • T: But what about their - your! children? and the climate then?
  • D: They are going to be rich dentists. See, I'm charging you $50 for 15 minutes of work. It's only going to get better.
  • T: Not if the economy collapses. It seems we have based it on unsustainable growth, fueled by unreasonably cheap energy. This could happen in your lifetime, or mine - and your kids. Present luxury and high wages are based on the efficiency / cheapness of transportation of goods into the US, and the developed world's exploitation of the developing world.
  • D: No. It is based on the labor/economic efficiency of manufacturing and agriculture. Anyway, take Europe for example - the price of oil there is high, and their economy is humming along.
  • T: Yet efficient manufacturing and agriculture is somewhat dependent on cheap transportation. As for Europe, that's because they tax oil to pay for public development, among other things. And Europeans consume half the oil of their American counterparts.
  • D: A gas tax that large would never happen here. People would go nuts, such a law would never pass!
  • T: True. The political system is irrational and irresponsible, but I can't think of an alternative structure. Humans were not designed for this, such responsibility!
  • D: If you keep talking I'm going to have to charge more.
  • T: <opens mouth>

Mostly I'd have to agree with the dentist - the oil is going to be burned eventually, because it is just such a cheap source of energy. We are going to have to deal with the consequences. However, for coal - of which we have a far greater supply, and is considerably more dangerous / expensive to obtain - there is good reason to search for alternatives, and putting a tax on oil/natural gas now fund development of alternatives is probably very future-responsible, and will shift the energy climate so we relinquish coal (and maybe some oil) earlier, resulting in less CO2 in the atmosphere.

There are infinitely many things more worthy/long-range responsible than the war, but our leaders have not touched on that. Correct me if I'm wrong, but there is little evidence that they even measured the worth of all alternatives, and decided rationally, based on integrating (over time and path probability) best-of-present knowledge of benefits and consequences. Or maybe they decided rationally, but with the worth of alternatives measured *personally*. It is this that truly angers me.

Bayes for president 2008!


hide / edit[0] / print
ref: bookmark-0 tags: book information_theory machine_learning bayes probability neural_networks mackay date: 0-0-2007 0:0 revision:0 [head]

http://www.inference.phy.cam.ac.uk/mackay/itila/book.html -- free! (but i liked the book, so I bought it :)

hide / edit[0] / print
ref: bookmark-0 tags: monte_carlo MCMC particle_filter probability bayes filtering biblography date: 0-0-2007 0:0 revision:0 [head]

http://www-sigproc.eng.cam.ac.uk/smc/papers.html -- sequential monte carlo methods. (bibliography)

hide / edit[0] / print
ref: bookmark-0 tags: Numenta Bayesian_networks date: 0-0-2006 0:0 revision:0 [head]


  • shared, hierarchal representation reduces memory requirements, training time, and mirrors the structure of the world.
  • belief propagation techniques force the network into a set of mutually consistent beliefs.
  • a belief is a form of spatio-temporal quantization: ignore the unusual.
  • a cause is a persistent or recurring structure in the world - the root of a spatiotemporal pattern. This is a simple but important concept.
    • HTM marginalize along space and time - they assume time patterns and space patterns, not both at the same time. Temporal parameterization follows spatial parameterization.

hide / edit[0] / print
ref: bookmark-0 tags: Bayes Baysian_networks probability probabalistic_networks Kalman ICA PCA HMM Dynamic_programming inference learning date: 0-0-2006 0:0 revision:0 [head]

http://www.cs.ubc.ca/~murphyk/Bayes/bnintro.html very, very good! many references, well explained too.

hide / edit[0] / print
ref: notes bookmarks-0 tags: spike sorting bayes spectral_analysis date: 0-0-2006 0:0 revision:0 [head]

hide / edit[0] / print
ref: Stapleton-2006.04 tags: Stapleton Lavine poisson prediction gustatory discrimination statistical_model rats bayes BUGS date: 0-0-2006 0:0 revision:0 [head]