You are not authenticated, login.
text: sort by
tags: modified
type: chronology
[0] Schwartz AB, Cortical neural prosthetics.Annu Rev Neurosci 27no Issue 487-507 (2004)[1] Carmena JM, Lebedev MA, Henriquez CS, Nicolelis MA, Stable ensemble performance with single-neuron variability during reaching movements in primates.J Neurosci 25:46, 10712-6 (2005 Nov 16)

[0] Brockwell AE, Rojas AL, Kass RE, Recursive bayesian decoding of motor cortical signals by particle filtering.J Neurophysiol 91:4, 1899-907 (2004 Apr)

hide / / print
ref: -0 tags: chess evolution machine learning 2004 partial derivative date: 10-26-2009 04:07 gmt revision:2 [1] [0] [head]

A Self-learning Evolutionary Chess Program

  • The evolved program is able to perform at near master level!
  • Used object networks (neural networks that can be moved about according to the symmetries of the problem space). Paul Werbos apparently invented these, too.
  • Approached the problem by assigning values to having pieces at particular places on the board (PVT, positional value tables). The value of a move was the value of the resulting global valuation (sum of value of pieces - value of opponents pieces) + PVT. They used these valuations to look a set number of moves in the future, using an alpha-beta search.
    • Used 4-plys (search depth) while in normal genetic evolution; 6 when pawns would be upgraded.
  • The neural networks looked at the first 2 rows, the last two rows, and a 4x4 square in the middle of the board - areas known to matter in real games. (The main author is a master-level chess player and chess teacher).
  • The outputs of the three neural networks were added to the material and PVT values to assess a hypothetical board position.
  • Genetic selection operated on the PVT values, neural network weights, piece valuation, and biases of the neural networks. These were initialized semi-randomly; PVT values were initialized based on open-source programs.
  • Performed 50 generations of 20 players each. The top 10 players from each generation survived.
  • Gary Kasparov was consulted in this research. Cool!
  • I wonder what would happen if you allowed the program to propose (genetically or otherwise) alternate algorithmic structures. What they describe is purely a search through weight space - what about a genetic search through algorithmic structure space? Too difficult of a search?
  • I mean, that's what humans (the authors) do while they were designing this program/algorithm. The lead author, as mentioned, is already a very good chess player, and hence he could imbue the initial program with a lot of good 'filters' 'kernels' or 'glasses' for looking at the chess board. And how did he arrive at these ideas? Practice (raw data) and communication (other peoples kernels extracted from more raw data, and validated). And how does he play? By using his experience and knowledge to predict probable moves into the future, evaluating their value, and selecting the best. And how does he evaluate his algorithmic? The same way! By using his knowledge of both chess and computer science to simulate hypothetical designs in his head, seeing how he thinks they will perform, and selecting the best one.
  • The problem with present algorithms is that they have no sense of artistic beauty - no love of symmetry, whether it be simple geometric symmetry (beautiful people have symmetric faces) or more fractal (fractional-dimensioned) symmetry, e.g. music, fractals (duh), human art. I think symmetry can enormously cut down the dimension of the search space in learning, hence is frequently worthy of its own search.
    • Algorithms do presently have a good sense of parsimony, at least, through the AIC / regularization / SVD / bayes net's priors / etc. Parsimony can be beauty, too.
  • Another notable discrepancy is that humans can reason in a concrete way - they actively search for the thing that is causing the problem, the thing that is contributing greatly to either good or bad results. They do this by the scientific method, sorta - hold all other things constant, perturb some section of the system, measure the output. This is the same as taking a partial derivative. Such derivative are used heavily/exclusively in training neural networks - weights are changed based on the partial derivative of that weight wrt the output-referenced error. So reasoning is similar to non-parallel backprop? Or a really slow way of taking partial derivatives? Maybe. The goal of both is to assign valuation/causation to a given weight/subsystem.
  • Human reasoning involves dual valuation pathways - internal, based on a model of the world, and external, which of course involves experimentation and memory (and perhaps scholarly journal papers etc). The mammalian cortex-basal ganglia-thalamus loop seems designed for running these sorts of simulations because it is the dual of the problem of selecting appropriate behaviors. (there! I said it!) In internal simulation, you take world state, apply forward transform with perturbation, then evaluate the result - see if your perturbation (partial derivative) yields information. In motor behavior, you take the body state, apply forward transformation with perturbation (muscle contraction), and evaluate the result. Same thing. Of course you don't have to do this too much, as the cortex will remember the input-perturbation-result.
  • Understanding seems to be related to this input-transform-evaluate cycle, too, except here what is changing is the forward transform, and the output is compared to known output - does a given kernel (concept) predict the output/observed data?
  • Now what would happen if you applied this input-transform-evaluate to itself, e.g. you allowed the system to evaluate itself. Nothing? Recursion? (recursion is a very beautiful concept.) Some degree of awareness?
  • Surely someone has thought of this before, and tried to simulate it on a computer. Wasn't AI research all about this in the 70's-80's? People have said that their big problem was that AI was then entirely/mostly symbolic and insufficiently probabilistic or data-intensive; the 90's-21st century seems to have solved that. This field is unfamiliar to me, it'll take some sussing about before I can grok the academic landscape.
    • Even more surely, someone is doing it right now! This is the way the world advances. Same thing happened to me with GPGPU stuff, which I was doing in 2003. Now everyone is up to that shiznit.
  • It seems that machine-learning is transitioning from informing my personal philosophy, to becoming my philosophy. Good/bad? Feel free to edit this entry!
  • It's getting late and I'm tried -> rant ends.

hide / / print
ref: Schwartz-2004.01 tags: Schwartz BMI prosthetics M1 review 2004 date: 04-05-2007 16:12 gmt revision:1 [0] [head]

PMID-15217341[0] Cortical Neuro Prosthetics

  • closed-loop control improves performance. see [1]
    • adaptive learning tech, when coupled to the adaptability of the cortex, suggests that these devices can function as control signals for motor prostheses.


hide / / print
ref: Brockwell-2004.04 tags: particle_filter Brockwell BMI 2004 wiener filter population_vector MCMC date: 02-05-2007 18:54 gmt revision:1 [0] [head]

PMID-15010499[0] Recursive Bayesian Decoding of Motor Cortical Signals by Particle Filtering

  • It seems that particle filtering is 3-5 times more efficient / accurate than optimal linear control, and 7-10 times more efficient than the population vector method.
  • synthetic data: inhomogeneous poisson point process, 400 bins of 30ms width = 12 seconds, random walk model.
  • monkey data: 258 neurons recorded in independent experiments in the ventral premotor cortex. monkey performed a 3D center-out task followed by an ellipse tracing task.
  • Bayesian methods work optimally when their models/assumptions hold for the data being analyzed.
  • Bayesian filters in the past were computationally inefficient; particle filtering was developed as a method to address this problem.
  • tested the particle filter in a simulated study and a single-unit monkey recording ellipse-tracing experiment. (data from Rena and Schwartz 2003)
  • there is a lot of math in the latter half of the paper describing their results. The tracings look really good, and I guess this is from the quality of the single-unit recordings.
  • appendix details the 'innovative methodology ;)


hide / / print
ref: van-2004.11 tags: anterior cingulate cortex error performance monitoring 2004 date: 0-0-2007 0:0 revision:0 [head]

PMID-15518940 Errors without conflict: implications for performance monitoring theories of anterior cingulate cortex.

  • did a event-locked fMRI to study whether the ACC would differentiate between correct and incorrect feedback stimuli in a time estimation task.
  • ACC seems to be not involved in error detection, just conflict.
  • according to one theory, ERN is generated as part of a reinforcement learning process. (Holroyd and Coles 2002): behavior is monitored by an 'adaptive critic' in the basal ganglia.
    • in this theory, the ACC is used to select between mental processes competing to access the motor system.
    • ERN corresponds to a decrease in dopamine.
    • ERN occurs when the stimulus indicates that an error has occured.
  • alternately, the ACC can monitor for the presence of conflict between simultaneously active but incompatible sensory/processing streams.
    • the ACC is active in correct trials in tasks that require conflict resolution. + it makes sense from a modeling strategy: high-energy state is equivalent to a state of conflit: many neurons are active at the same time.
    • that is, it is a stimuli resolver: e.g. the stroop task.
  • some studies localize (and the authors here indicate that the source-analysis that localizes dipole sources is inaccurate) the error potential to the posterior cingulate cortex.
    • fMRI solves this problem.
  • from their figures, it seems that the right putamen + bilateral caudate are involved in their time-estimation task (subjects has to press a button 1 second after a stimulus cue; feedback then guided/misguided them toward/away from 1000ms; subjects, of course, adjusted their behavior)
    • no sign of ACC activation was shown - as hard as they could look - despite identical (more or less) experimental design to the ERN studies.
      • hence, ERN is generated by areas other than the ACC.
  • in contrast, the stroop task fully engaged the anterior cingulate cortex.
  • cool: perhaps, then, error feedback negativity is better conceived as an (absence of) superimposed "correct feedback positivity" 'cause no area was more active in error than correct feedback.
  • of course, one is measuring brain activation through blood flow, and the other is measuring EEG signals.