m8ta
use https for features. 

{305}  
PMID101388[0] Fine control of operantly conditioned firing patterns of cortical neurons.
____References____  
{255} 
ref: BarGad2003.12
tags: information dimensionality reduction reinforcement learning basal_ganglia RDDR SNR globus pallidus
date: 01162012 19:18 gmt
revision:3
[2] [1] [0] [head]


PMID15013228[] Information processing, dimensionality reduction, and reinforcement learning in the basal ganglia (2003)
____References____  
{806}  
I've recently tried to determine the bitrate of conveyed by one gaussian random process about another in terms of the signaltonoise ratio between the two. Assume $x$ is the known signal to be predicted, and $y$ is the prediction. Let's define $SNR(y) = \frac{Var(x)}{Var(err)}$ where $err = xy$ . Note this is a ratio of powers; for the conventional SNR, $SNR_{dB} = 10*log_{10 } \frac{Var(x)}{Var(err)}$ . $Var(err)$ is also known as the meansquarederror (mse). Now, $Var(err) = \sum{ (x  y  sstrch \bar{err})^2 estrch} = Var(x) + Var(y)  2 Cov(x,y)$ ; assume x and y have unit variance (or scale them so that they do), then $\frac{2  SNR(y)^{1}}{2 } = Cov(x,y)$ We need the covariance because the mutual information between two jointly Gaussian zeromean variables can be defined in terms of their covariance matrix: (see http://www.springerlink.com/content/v026617150753x6q/ ). Here Q is the covariance matrix, $Q = \left[ \array{Var(x) & Cov(x,y) \\ Cov(x,y) & Var(y)} \right]$ $MI = \frac{1 }{2 } log \frac{Var(x) Var(y)}{det(Q)}$ $Det(Q) = 1  Cov(x,y)^2$ Then $MI =  \frac{1 }{2 } log_2 \left[ 1  Cov(x,y)^2 \right]$ or $MI =  \frac{1 }{2 } log_2 \left[ SNR(y)^{1}  \frac{1 }{4 } SNR(y)^{2} \right]$ This agrees with intuition. If we have a SNR of 10db, or 10 (power ratio), then we would expect to be able to break a random variable into about 10 different categories or bins (recall stdev is the sqrt of the variance), with the probability of the variable being in the estimated bin to be 1/2. (This, at least in my mind, is where the 1/2 constant comes from  if there is gaussian noise, you won't be able to determine exactly which bin the random variable is in, hence log_2 is an overestimator.) Here is a table with the respective values, including the amplitude (not power) ratio representations of SNR. "
Now, to get the bitrate, you take the SNR, calculate the mutual information, and multiply it by the bandwidth (not the sampling rate in a discrete time system) of the signals. In our particular application, I think the bandwidth is between 1 and 2 Hz, hence we're getting 1.63.2 bits/second/axis, hence 3.26.4 bits/second for our normal 2D tasks. If you read this blog regularly, you'll notice that others have achieved 4bits/sec with one neuron and 6.5 bits/sec with dozens {271}.  
{5} 
ref: bookmark0
tags: machine_learning research_blog parallel_computing bayes active_learning information_theory reinforcement_learning
date: 12312011 19:30 gmt
revision:3
[2] [1] [0] [head]


hunch.net interesting posts:
 
{252}  
PMID15022843[0] A simulation study of information transmission by multiunit microelectrode recordings key idea:
____References____  
{530}  
 
{229} 
ref: notes0
tags: SNR MSE error multidimensional mutual information
date: 03082007 22:33 gmt
revision:2
[1] [0] [head]


http://ieeexplore.ieee.org/iel5/516/3389/00116771.pdf or http://hardm.ath.cx:88/pdf/MultidimensionalSNR.pdf
 
{7} 
ref: bookmark0
tags: book information_theory machine_learning bayes probability neural_networks mackay
date: 002007 0:0
revision:0
[head]


http://www.inference.phy.cam.ac.uk/mackay/itila/book.html  free! (but i liked the book, so I bought it :)  
{57}  
http://www.cs.rug.nl/~rudy/matlab/
 
{66} 
ref: bookmark0
tags: machine_learning classification entropy information
date: 002006 0:0
revision:0
[head]


http://iridia.ulb.ac.be/~lazy/  Lazy Learning. 