You are not authenticated, login.
text: sort by
tags: modified
type: chronology
[0] Schmidt EM, McIntosh JS, Durelli L, Bak MJ, Fine control of operantly conditioned firing patterns of cortical neurons.Exp Neurol 61:2, 349-69 (1978 Sep 1)[1] Serruya MD, Hatsopoulos NG, Paninski L, Fellows MR, Donoghue JP, Instant neural control of a movement signal.Nature 416:6877, 141-2 (2002 Mar 14)[2] Fetz EE, Operant conditioning of cortical unit activity.Science 163:870, 955-8 (1969 Feb 28)[3] Fetz EE, Finocchio DV, Operant conditioning of specific patterns of neural and muscular activity.Science 174:7, 431-5 (1971 Oct 22)[4] Fetz EE, Finocchio DV, Operant conditioning of isolated activity in specific muscles and precentral cells.Brain Res 40:1, 19-23 (1972 May 12)[5] Fetz EE, Baker MA, Operantly conditioned patterns on precentral unit activity and correlated responses in adjacent cells and contralateral muscles.J Neurophysiol 36:2, 179-204 (1973 Mar)

[0] Fetz EE, Baker MA, Operantly conditioned patterns on precentral unit activity and correlated responses in adjacent cells and contralateral muscles.J Neurophysiol 36:2, 179-204 (1973 Mar)

[0] Fetz EE, Operant conditioning of cortical unit activity.Science 163:870, 955-8 (1969 Feb 28)[1] Fetz EE, Finocchio DV, Operant conditioning of specific patterns of neural and muscular activity.Science 174:7, 431-5 (1971 Oct 22)[2] Fetz EE, Finocchio DV, Operant conditioning of isolated activity in specific muscles and precentral cells.Brain Res 40:1, 19-23 (1972 May 12)

[0] Loewenstein Y, Seung HS, Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity.Proc Natl Acad Sci U S A 103:41, 15224-9 (2006 Oct 10)

[0] Birbaumer N, Cohen LG, Brain-computer interfaces: communication and restoration of movement in paralysis.J Physiol 579:Pt 3, 621-36 (2007 Mar 15)

hide / / print
ref: Schmidt-1978.09 tags: Schmidt BMI original operant conditioning cortex HOT pyramidal information antidromic date: 03-12-2019 23:35 gmt revision:11 [10] [9] [8] [7] [6] [5] [head]

PMID-101388[0] Fine control of operantly conditioned firing patterns of cortical neurons.

  • Hand-arm area of M1, 11 or 12 chronic recording electrodes, 3 monkeys.
    • But, they only used one unit at a time in the conditioning task.
  • Observed conditioning in 77% of single units and 65% of combined units (multiunits?).
  • Trained to move a handle to a position indicated by 8 annular cursor lights.
    • Cursor was updated at 50hz -- this was just a series of lights! talk about simple feedback...
    • Investigated different smoothing: too fast, FR does not stay in target; too slow, cursor acquires target too slowly.
      • My gamma function is very similar to their lowpass filter used for smoothing the firing rates.
    • 4 or 8 target random tracking task
    • Time-out of 8 seconds
    • Run of 40 trials
      • The conditioning reached a significant level of performance after 2.2 runs of 40 trials (in well-trained monkeys); typically, they did 18 runs/day (720 trials)
  • Recordings:
    • Scalar mapping of unit firing rate to cursor position.
    • Filtered 600-6kHz
    • Each accepted spike triggered a generator that produced a pulse of of constant amplitude and width -> this was fed into a lowpass filter (1.5 to 2.5 & 3.5Hz cutoff), and a gain stage, then a ADC, then (presumably) the PDP.
      • can determine if these units were in the pyramidal tract by measuring antidromic delay.
    • recorded one neuron for 108 days!!
      • Neuronal activity is still being recorded from one monkey 24 months after chronic implantation of the microelectrodes.
    • Average period in which conditioning was attempted was 3.12 days.
  • Successful conditioning was always associated with specific repeatable limb movements
    • "However, what appears to be conditioned in these experiments is a movement, and the neuron under study is correlated with that movement." YES.
    • The monkeys clearly learned to make (increasingly refined) movement to modulate the firing activity of the recorded units.
    • The monkey learned to turn off certain units with specific limb positions; the monkey used exaggerated movements for these purposes.
      • e.g. finger and shoulder movements, isometric contraction in one case.
  • Trained some monkeys or > 15 months; animals got better at the task over time.
  • PDP-12 computer.
  • Information measure: 0 bits for missed targets, 2 for a 4 target task, 3 for 8 target task; information rate = total number of bits / time to acquire targets.
    • 3.85 bits/sec peak with 4 targets, 500ms hold time
    • With this, monkeys were able to exert fine control of firing rate.
    • Damn! compare to Paninski! [1]
  • 4.29 bits/sec when the same task was performed with a manipulandum & wrist movement
  • they were able to condition 77% of individual neurons and 65% of combined units.
  • Implanted a pyramidal tract electrode in one monkey; both cells recorded at that time were pyramidal tract neurons, antidromic latencies of 1.2 - 1.3ms.
    • Failures had no relation to over movements of the monkey.
  • Fetz and Baker [2,3,4,5] found that 65% of precentral neurons could be conditioned for increased or decreased firing rates.
    • and it only took 6.5 minutes, on average, for the units to change firing rates!
  • Summarized in [1].


hide / / print
ref: -0 tags: Kato fear conditioning GABA auditory cortex mice optogenetics SOM PV date: 02-04-2019 19:09 gmt revision:0 [head]

PMID-29375323 Fear learning regulates cortical sensory representation by suppressing habituation

  • Trained mice on CS+ and CS --> lick task.
    • CS+ = auditory tone followed by tailshock
    • CS- = auditory tone (both FM modulated, separated by 0.5 - 1.0 octave).
    • US = licking.
  • VGAT2-ChR2 or PV-ChR2
  • GABA-ergic silencing of auditory cortex through blue light illumination abolished behavior difference following CS+ and CS-.
  • Used intrinsic imaging to locate A1 cortex, then AAV - GCaMP6 imaging to lcoated pyramidal cells.
  • In contrast to reports of enhanced tone responses following simple fear conditioning (Quirk et al., 1997; Weinberger, 2004, 2015), discriminative learning under our conditions caused no change in the average fraction of pyramidal cells responsive to the CS+ tone.
    • Seemed to be an increase in suppression, and reduced cortical responses, which is consistent with habituation.
  • Whereas -- and this is by no means surprising -- cortical responses to CS+ were sustained at end of tone following fear conditioning.
  • ----
  • Then examined this effect relative to the two populations of interneurons, using PV-cre and SOM-cre mice.
    • In PV cells, fear conditioning resulted in a decreased fraction of cells responsive, and a decreased magnitude of responses.
    • In SOM cells, CS- responses were enhanced, while CS+ were less enhanced (the main text seems like an exaggeration c.f. figure 6E)
  • This is possibly the more interesting result of the paper, but even then the result is not super strong.

hide / / print
ref: Teagarden-2007.03 tags: STN striatum operant conditioning behavior rats 2006 DBS date: 02-15-2012 03:36 gmt revision:5 [4] [3] [2] [1] [0] [head]

PMID-17182916[0] Subthalamic and Striatal Neurons Concurrently Process Motor, Limbic, and Associative Information in Rats Performing an Operant Task

  • STN encodes behavioral events (reinforcement, nose poke, correct / incorrect trials). So does the striatum.
  • This study is rather nonspecific, but it makes sense that a conserved and well connected region is active during learning & general behavior.
    • That is, while the subthalamic nucleus is considered an output relay of the basal ganglia, more likely it operates in parallel to facilitate forms of learning; as such, responses are shown to rewards, cues, etc.


[0] Teagarden MA, Rebec GV, Subthalamic and striatal neurons concurrently process motor, limbic, and associative information in rats performing an operant task.J Neurophysiol 97:3, 2042-58 (2007 Mar)

hide / / print
ref: Wiener-2008.08 tags: STN operant conditioning timing rats lesion DBS impulsivity date: 01-26-2012 17:29 gmt revision:3 [2] [1] [0] [head]

PMID-18562098[0] Accurate timing but increased impulsivity following excitotoxic lesions of the subthalamic nucleus.

  • Synopsis: Animals whose STNs were lesioned were able to maintain temporal control and response on a peak interval timing task, but they were unable to inhibit operant responses late into the trial. This suggests that STN may be used in impulse control / behavioral inhibition.


[0] Wiener M, Magaro CM, Matell MS, Accurate timing but increased impulsivity following excitotoxic lesions of the subthalamic nucleus.Neurosci Lett 440:2, 176-80 (2008 Aug 1)

hide / / print
ref: Wyler-1980.08 tags: Wyler operant conditioning fast slow pyramidal tract neurons BMI date: 01-07-2012 22:09 gmt revision:3 [2] [1] [0] [head]

PMID-7409057[0] Operant control of precentral neurons: comparison of fast and slow pyramidal tract neurons.

  • Slow PTN (neurons with antidromic latency > 2ms) are pratically all well controlled in his operant-conditioning task;
  • Fast (< 2ms, mean 1.2ms latency) have a more highly variable firing rates and ISIs.
  • "[I]t appears that the majority of error from fast PT cells was generated by ISIS less than 30 ms, whereas the majority of error for slow PT cells was represented in ISIS greater than 60 ms."
    • Ok, trivial observation, but still interesting.


[0] Wyler AR, Burchiel KJ, Robbins CA, Operant control of precentral neurons: comparison of fast and slow pyramidal tract neurons.Exp Neurol 69:2, 430-3 (1980 Aug)

hide / / print
ref: Fetz-1973.03 tags: operant conditioning Fetz Baker learning BMI date: 01-07-2012 19:34 gmt revision:2 [1] [0] [head]

PMID-4196269[0] Operantly conditioned patterns on precentral unit activity and correlated responses in adjacent cells and contralateral muscles

  • Looked at an operant task through the opposite direction: as a means for looking at reaction time, and muscle responses to trained bursts of activity.
  • recorded from precentral gyrus cells in leg and arm representation.
    • isonel coated tungsten microwires, with great apparent waveform records.
  • also recorded EMG, nylon-insuldated stainless-steel wire, led subcutaneuosly to the head connector.
  • references an even older study concerning the operant conditioning of neural activity in rats by Olds.
  • really simple technology - RC filter to estimate the rate; reward high rate; resets on reward.
    • the evoked operant bursts are undoubtably due to training.
  • looks like it was easy for the monkeys to increase the firing rate of their cortical cells (of course, I'm just skimming the article..)
  • 233 precentral units.
    • which they did some preliminary somatotopic mapping of.
  • neighboring cells mirrored the firing rate changes (logical as they share the local circuitry)
  • in a few sessions the operant bursts were not associated with movements.
  • Could individually condition cells when they happened to record 2 units on the same electrode.


hide / / print
ref: Fetz-1969.02 tags: BMI original Fetz operant conditioning date: 01-07-2012 19:04 gmt revision:7 [6] [5] [4] [3] [2] [1] [head]

PMID-4974291[0] Operant conditioning of cortical unit activity

  • (Abstract) The activity of single neurons in precentral cortex of unanesthetized monkeys (Macaca mulatta) was conditioned by reinforcing high rates of neuronal discharge with delivery of a food pellet. Auditory or visual feedback of unit firing rates was usually provided in addition to food reinforcement. After several training sessions, monkeys could increase the activity of newly isolated cells by 50 to 500 percent above rates before reinforcement.
  • Used 'classical' single unit recording.
  • Trepination 5mm circle over hand area.
  • feedback: click for each AP.
  • reinforced on neuron per day.
  • trained neural activity often bursts, usually involved movement such as flexion of the lebow or rotation of the wrist.
  • controlled for sensory positive-feedback loop by performing extinction trials & looking for PETH response to click.
  • I gotta get one of these pellet feeders. monkeys will likely be more motivated, especially if I titrate how frequently they get the food.
  • images/303_1.pdf

PMID-5000088[1] Operant conditioning of specific patterns of neural and muscular activity.

In awake monkeys we recorded activity of single "motor" cortex cells, four contralateral arm muscles, and elbow position, while operantly reinforcing several patterns of motor activity. With the monkey's arm held semiprone in a cast hinged at the elbow, we reinforced active elbow movements and tested cell responses to passive elbow movements. With the cast immobilized we reinforced isometric contraction of each of the four muscles in isolation, and bursts of cortical cell activity with and without simultaneous suppression of muscle activity. Correlations between a precentral cell and specific arm muscles consistently appeared under several behavioral conditions, but could be dissociated by reinforcing cell activity and muscle suppression.

PMID-4624487[2] Operant conditioning of isolated activity in specific muscles and precentral cells

Recorded precentral units in monkeys, trained to contract 4 arm muscles in isolation, under various conditions: passive movements and cutaneous stimulation, active movements and isometric contractions. Some Ss were also reinforced for activity of cortical cells, with no contingency in muscle activity and with simultaneous suppression of all muscular activity. It is concluded that temporal correlations between activity of precentral cells and some other component of the motor response, e.g., muscle activity, force, or position, may depend as strongly on the specific response pattern which is reinforced as on any underlying physiological connection.


hide / / print
ref: Olds-1967.01 tags: Olds 1967 limbic system operant conditioning recording rats electrophysiology BMI date: 01-06-2012 03:59 gmt revision:2 [1] [0] [head]

PMID-6077726[0] The limbic system and behavioral reinforcement

  • Can't seem to find Olds 1965, as was a conference proceeding .. this will have to do, despite the lack of figures. images/966_1.pdf
  • First reference I can find of chronic (several weeks) (4-9 microelectrodes, single) recording from the rat.
  • Basically modern methods: commutator + solid state preamplifiers mounted to a counterbalanced slack-relieving arm.
    • If unit responses were observed in recordings from a given probe a week after surgery they were usually recordable indefinitely. 44 years later ...
  • Used a primitive but effective analog spike discriminator based on:
    • minimum amplitude
    • maximum amplitude
    • minimum fall time
    • maximum fall time.
  • Also had a head movement artifact detector, which blanked the recordings (stopped the paper roll) for 2 sec.
  • Reinforced on 'bursting', threshold sufficiently high that it only occurred once every 5-15 minutes.
  • Food reinforcement or 1/4 second train of brain stimulation (30ua, 60Hz, sine, in hypothalamus).
  • Reinforcement was conditioned on an 'acquisition' signal, which is visual (?) Bursting is rewarded for 2 minutes, ignored for 8 minutes.
  • Also recorded control neurons.
  • (they were looking at these things as though anew!) "The most striking aspect of the records so formed [on sheets of paper] was that all discriminators at one time or another exhibited rate changes that had the appearance of waves with a period of 10 to 20 minutes. Waves between units in the same animal were to some degree synchronized." Then describes a ramp ..
  • Longer term variations: FR would vary by a factor of 2-5 over a period of several hours.
    • This would make negatively correlated neurons (on a short time scale) appear positively correlated over long time scales (have to fix this in the BMI!)
  • As this was a conditional reinforcement task, they unexpectedly found that the acquisition periods were systematically different than extinction periods
    • More like pavlovian conditioning, esp in the hippocampus, where a conditioned response was also reflected on a control neuron.
    • Even when the light was lit throughout the acquisition period was replaced by a bell at the beginning of the acq. period, there was still a sustained change in FR.
      • Then during the extinction period: it appeared from the record of responses that a definite operant behavior was tried several times and then stopped altogether."
  • In the pontine nucleus (relay from M1 to cerebellum, v. roughly), judging from the control responses, all were conditioned.
    • Pontine responses seem to correspond with movement of the eyes or head that did not set off the movement detector/blanker.
  • Saw brief and very fast bursts during the extinction periods of the kind that Evarts found to characterize pyramical neurons during sleep.
  • When units shifted from food reward to ICS reward, units became undiffarentiated, and within a day they would be reconditioned.
  • Also tried paralyzing the animal to see if it could still generate operant responses; the animal died, results inconclusive.
  • Flood lights made it hard for the rats to produce the operant behavior.


[0] Olds J, The limbic system and behavioral reinforcement.Prog Brain Res 27no Issue 144-64 (1967)

hide / / print
ref: notes-0 tags: clementine operant conditioning 041707 pyramidal tract tlh24 date: 01-06-2012 03:12 gmt revision:4 [3] [2] [1] [0] [head]

It appears that operant/feedback training of one neuron (channel 29, in SMA region) works fine (not great, but fine). In the experiment performed prior to visiting Seattle, on April 10 2007, I was not convinced that the neuron was controlling anything. Now, it is apparent that the monkey has some clue as to what he is doing. Today I made a simple change: I made the filtering function sum (all spikes) 1/12 * x*(x-1)^2, where x = time - time_of_spike. In comparison to a butterworth filter, this has no rebound oscillation & makes the estimation of firing rate much more transparent. It averages over approximately 500ms ~= lowcut of 1.5hz? I see no reason to change this filtering function much, as it works fine. Spikes were binned at 100hz as input to this function, but that should be equivalent to binning at 1khz etc.

Next time, i want to do 2d, where channel 62 controls the Y-axis. really should try to determine the approximate tunings of these cells. I'm somewhat concerned as this channel seems to have a much lower mean firing rate than channel 29. According to the literature, PTNs have high firing rates and strong tuning...

for reference, here is the channel used for the one-neuron BMI, recorded April 10. It has not changed much in the last 7 days.

hide / / print
ref: thesis-0 tags: clementine 051607 operant conditioning tlh24 date: 01-06-2012 03:09 gmt revision:1 [0] [head]

the cells were, basically, as usual for today. did 1-d BMI on channel 29; worked somewhat (nothing dramatic; mk is out of practice?)

hide / / print
ref: thesis-0 tags: clementine 042107 operant conditioning tlh24 date: 01-06-2012 03:08 gmt revision:5 [4] [3] [2] [1] [0] [head]

I tried to train Clem, once again, to do 2d BMI, this time with channel 69 for X and channel 71 for Y. X worked rather well, to a point - he realized that he could control it with left shoulder contractions, and did so (did not get a video of this). I did, however, get a video of the game, which is here:

Y training/performance was abysmal and hence did not try 2D control. Channel 71 would become silent whenever he began to pay attention; I'm not sure why. It would fire vigorously when he turned around and rested; the unit had a high firing rate at rest. I did not get a pic of the sortclient for today, but ch 29 was there as usual (though i did not use it) & channel 71 had the characteristic sharp V shape; perhaps it was an interneuron?? I don't know.

anyway, the data is in SQL on hardm.ath.cx. (the real proof is in the pudding, of course).

we really need to put the BMI game in his home cage, so motivation is not such a large issue

hide / / print
ref: thesis-0 tags: clementine 042007 operant conditioning biofeedback tlh24 date: 01-06-2012 03:08 gmt revision:4 [3] [2] [1] [0] [head]

channel 29 controlled the X direction:

channel 81, the Y direction (this one was very highly modulated, and the monkey could get to a high rate ~60Hz. note that both units are sorted as one -- I ought to do the same on the other channels from now on, as this was rather predictive (this is duplicating Debbie Won's results):

However, when I ran a wiener filter on the binned spike rates (this is not the rates as estimated through the polynomial filter), ch 81 was most predictive for target X position; ch 29, Y target position (?). This is in agreement with population-wide predictions of target position: target X was predicted with low fidelity (1.12; cc = 0.35 or so); target Y was, apparently, unpredicted. I don't understand why this is, as I trained the monkey for 1/2 hour on just the opposite. Actually this is because the targets were not in a random sequence - they were in a CCW sequence, hence the neuronal activity was correlated to the last target, hence ch 81 to target X!

for reference, here is the ouput of bmi_sql:

order of columns: unit,channel, lag, snr, variable

ans =

    1.0000   80.0000    5.0000    1.0909    7.0000
    1.0000   80.0000    4.0000    1.0705    7.0000
    1.0000   80.0000    3.0000    1.0575    7.0000
    1.0000   80.0000    2.0000    1.0485    7.0000
    1.0000   80.0000    1.0000    1.0402    7.0000
    1.0000   28.0000    4.0000    1.0318    8.0000
    1.0000   76.0000    2.0000    1.0238   11.0000
    1.0000   76.0000    5.0000    1.0225   11.0000
    1.0000   17.0000         0    1.0209   11.0000
    1.0000   63.0000    3.0000    1.0202    8.0000

movies of the performance are here:

hide / / print
ref: Wyler-1974.02 tags: Wyler Fetz BMI operant conditioning date: 01-05-2012 00:46 gmt revision:3 [2] [1] [0] [head]

PMID-4207598[0] Behavioral control of firing patterns of normal and abnormal neurons in chronic epileptic cortex.

  • Idea: epilepsy treated through biofeedback.
  • Induced epilepsy in monkeys via alumina.
  • Conditioned 198 cells in epileptiform focus; 107 had normal firing patterns.
  • 91 cells had abnormal patterns:
    • Structured bursts with high, invariant burst indices, and could not be conditioned.
    • Cells did not change burstyness based on behavioral state.
    • Lower and more variable burst indices and were as easily conditioned as normal cells.
      • These cells bursted more when the monkey was not paying attention.
  • Operant control: ref 8, 9.
  • Ach, fascinating:
  • Normal precentral cells rarely exhibited interspike intervals less than 10 msec, except during vigorous movements or sleep.
  • Neurons were deemed 'bursty' if they exhibited spontaneous high-frequency firing with interspike intervals less that 5msec.
  • Monkeys obtained proficiency with high-frequency conditioning more quickly and effectively than with low-freq, even with 40% on high and 60% on low.
  • All conditioned cells corresponded to some movement of the contralateral arm (again).
  • Operant conditioning is interesting in this case, as it indicates if cells are still 'functional' in the ensemble.
  • See also: PMID-809116[1]


[0] Wyler AR, Fetz EE, Behavioral control of firing patterns of normal and abnormal neurons in chronic epileptic cortex.Exp Neurol 42:2, 448-64 (1974 Feb)
[1] Wyler AR, Fetz EE, Ward AA Jr, Firing patterns of epileptic and normal neurons in the chronic alumina focus in undrugged monkeys during different behavioral states.Brain Res 98:1, 1-20 (1975 Nov 7)

hide / / print
ref: Bures-1968 tags: inferior colliculus stimulation classical conditioning plasticity hebb Bures date: 01-03-2012 07:08 gmt revision:5 [4] [3] [2] [1] [0] [head]

bibtex:Bures-1968 Plastic changes of unit activity based on reinforcing properties of extracellular stimulation of single neurons

  • images/972_1.pdf
  • Trained neurons to respond to auditory stimuli throughout the brain (though mostly the IC) to a auditory tone.
    • Hebb's rule, verified.
  • Yoshii & Ogura (22): Reticular units, originally not responding to sciatic nerve US, started to respond to the CS after a few tens of trials, however the conditioned reactions disappeared with continued training.
    • This must be regarded as response to arousal at the initial stages of classical aversive (sciatic nerve pain?) conditioning.
  • Used capilary electrodes 1um in diameter, filled with KCl or sodium glutamate
  • Stimulation current 10-50nA DC, 0.3-1 sec.
  • Were able to record and stimulate at the same time using these glass microelectrodes.
  • The majority of units (cortex, reticular formation, thalamus) showed no response, though some did. These responses tended to fade with overtraining.
  • Quote: "The rather low incidence of positive results int he above experiment might be due to the fact that many examined neurons lack even an indirect acoustic input and cannot, therefore, be activated by acoustic stimuli."
  • Neurons in the IC show the strongest plastic change.
  • Their study is more specific than Loucks (15), Olds and Milner (17) Delgaso (6) Doty(7) which used less specific ICMS.
  • That said, there is no behavior .. so we don't know if the stimuli is being reacted to or attended to (might explain the low # of responses in areas).
  • They also think that the response can be credited to nonspecific phenomena like dominant focus, reflex sensitization, or heterosynaptic facilitation.
    • That said, the IC did show strong responses.

hide / / print
ref: Shinkman-1974.06 tags: Shinkman Bruce Pfingst operant conditioning visual cortex cat ICMS 1974 stimulation date: 12-29-2011 05:13 gmt revision:4 [3] [2] [1] [0] [head]

PMID-4598035[0] Operant conditioning of single-unit response patterns in visual cortex.

  • In cat V1 -- suprising, this is usually considered to be sensory.
  • implanted bilater tripolar stimulating electrodes aimed at the lateral hypothalamus. These were tested for self-stimulation, and preferred locations/currents were selected for optimal ICS reinforcement.
    • 200 bar presses in 8 minute test.
  • Anesthetized, immobilized, head-restrained, contact-lens focused cats.
  • Back projected stimuli onto a screen 50 cm from eye ; dot, bar, or small spot was effective in triggering patterned response, as with many of these studies.
  • For conditioning: set a threshold at the third quartile (1/4 of trials exceeded threshold); comparator circuit counted the number of spikes during stimulus presentation, and if threshold was exceeded, reinforcing ICS was delivered.
    • Reinforcing ICS started 300ms after visual stimulus and lasted 500ms.
  • Conditioning was deemed successful if the mean trial firing rate for the last 50 conditioned trials had a mean firing rate > 30% larger than the first 50 control trials.
    • While recording some cells, ICS reinforcement was delivered at random as control.
  • Conditioning produced changes within stimulus presentation but not outside.
  • They consider the use of an immobilized subject is a pro -- better control, rules out alternative explanations based on motor feedback.


[0] Shinkman PG, Bruce CJ, Pfingst BE, Operant conditioning of single-unit response patterns in visual cortex.Science 184:4142, 1194-6 (1974 Jun 14)

hide / / print
ref: OLDS-1954.12 tags: Olds Milner operant conditioning electrical reinforcement wireheading BMI date: 12-29-2011 05:09 gmt revision:5 [4] [3] [2] [1] [0] [head]

PMID-13233369[0] Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain.

  • The original electrical reinforcement experiment!
  • tested out various areas for reinforcement; septal forebrain area was the best.
  • later work: 1956 Olds, J. Runway and maze behavior controlled by basomedial forebrain stimulation in the rat. J. Comp. Physiol. Psychol. 49:507-12.


[0] OLDS J, MILNER P, Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain.J Comp Physiol Psychol 47:6, 419-27 (1954 Dec)

hide / / print
ref: Loewenstein-2006.1 tags: reinforcement learning operant conditioning neural networks theory date: 12-07-2011 03:36 gmt revision:4 [3] [2] [1] [0] [head]

PMID-17008410[0] Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity

  • The probability of choosing an alternative in a long sequence of repeated choices is proportional to the total reward derived from that alternative, a phenomenon known as Herrnstein's matching law.
  • We hypothesize that there are forms of synaptic plasticity driven by the covariance between reward and neural activity and prove mathematically that matching (alternative to reward) is a generic outcome of such plasticity
    • models for learning that are based on the covariance between reward and choice are common in economics and are used phenomologically to explain human behavior.
  • this model can be tested experimentally by making reward contingent not on the choices, but rather on the activity of neural activity.
  • Maximization is shown to be a generic outcome of synaptic plasticity driven by the sum of the covariances between reward and all past neural activities.


hide / / print
ref: Wyler-1980.08 tags: Wyler Lange Robbins operant conditioning motor neurons contralateral bilateral specificity monkeys motor learning date: 12-06-2011 06:36 gmt revision:1 [0] [head]

PMID-6772272 Operant control of precentral neurons: bilateral single unit conditioning.

  • Used bilateral electrodes.
  • One neuron operantly conditioned, one not.
  • Switched the conditioned / controlled after performance was attained.
  • Evidence: neurons can be individually tuned, and operant control is not the result of spinal-level conditioning or change.
    • It is not the result of increased attention or increased muscle tone.
  • Simple question, simple paper.

hide / / print
ref: Wyler-1985.08 tags: Wyler synchrony operant conditioning BMI date: 12-06-2011 06:36 gmt revision:1 [0] [head]

PMID-4041789 Synchrony between cortical neurons during operant conditioning.

  • Fetz and Baker showed that individual neurons recorded from the same electrode can modulate their firing upon operant conditioning either together, opposite, or independently.
  • Wyler has duplicated this result, and undertakes this further analysis to show that these pairs of neurons recorded from the same electrode show high degrees (67%) of tight 1ms synchrony.
  • This despite the fact that in 80% of cases the firing rates did not covary.
  • This suggests that they must have a common synaptic pathway.
  • Reference (and support) Lemon and Porter's finding that adjacent neurons respond to widely separated peripheral fields.

hide / / print
ref: Burnod-1982.11 tags: operant conditioning motor control learning Burnod Maton Calvet date: 11-26-2011 02:22 gmt revision:0 [head]

PMID-7140894 Short-term changes in cell activity of areas 4 and 5 during operant conditioning.

  • Seems that layers 4 and 5 act differently during operant conditioning of a simple task.
  • Layer 5 neurons become tuned to reward (?)
  • Can't get this article, have to go from the abstract.

hide / / print
ref: Shinkman-1974.06 tags: operant conditioning visual cortex Shinkman date: 11-26-2011 00:40 gmt revision:0 [head]

PMID-4598035 Operant conditioning of single-unit response patterns in visual cortex

  • They successfully conditioned cells in the visual cortex to increase firing response to visual patterns (sensory stimulus).
    • This is conditional response, not conditioning behavior directly.
  • Reinforced using electrical stimulation of the lateral hypothalamus.
    • Optimal reinforcement electrodes were determined via self-stimulation.
  • Immobilized V1 recording appears hardcore. Cats were immobilized but not anesthetized for recording / reinforcement.
  • Delivered fixed ICMS pulse train when threshold number of spikes was exceeded.
  • Data analysis without matlab must have been hard. Actually, the data doesn't look that good, but this may be an artifact of presentation.
  • Controlled for eye movements using a paralytic.

hide / / print
ref: thesis-0 tags: clementine 042607 operant conditioning date: 04-27-2007 16:45 gmt revision:3 [2] [1] [0] [head]

tried 2d again... some success. looked at 29 (still good for x control, but not in BMI mode), channe 71 (still by default silent, correlated to behavior) channel 18 (did not work well) channel 84 (did not work) and channel 54 (like 71, highly correlated to behavior - not sure if the mk learned to control it). have videos etc.

channel 54, new for today and might, might be > 71.. though looking back at the videos, 71 seems pretty good. (it is also a bad idea to keep switching the game..) channels 54 and 71 are different from 29 in that 29 never goes completely silent; 71 goes silent when thew mk is paying attention, 54 when he is not moving. 29 can be modulated + and -, 71 and 54 just + (or so). of course, the monkey is usually in motion so both have high variance and silent periods are short-ish

channel 29, as always

channel 71, as before (very stable!)

channel 54

movies (in the order that they were taken):

hide / / print
ref: thesis-0 tags: clementine 042507 operant conditioning date: 04-25-2007 20:19 gmt revision:2 [1] [0] [head]

OK, today clementine played absolutely abysmally - he did practically nothing, though he did do pole control for a little bit. I think we must stop doing pole control - it is too easy, he must become accustomed to doing brain control from the beginning. Anyway, monkeys never like learning new things (compare to people!); I just have to give him more time. The units are stable (in my agitated state, i forgot to make screenshots). Channel 54 might be very excellent for brain control - however, i did not test it today. If it is still there tomorrow, i will try.

http://m8ta.com/tim/clem042507_trainY.MPG (ignore the first few seconds - he was not trying so hard/was not paying attention)

hide / / print
ref: thesis-0 tags: clementine 042407 operant conditioning date: 04-25-2007 00:21 gmt revision:1 [0] [head]

Today, as yesterday, I tried operantly conditioning primary units on channels 29 (x) and 71 (y) for BMI control. The first few minutes were run in pole control for Miguel's visitors, but i did not save the data. Again as before the monkey was not quite motivated to perform the task. Tomorrow he ought to be thirsty - & I'll try to start him on 2d control after tweaking the gain and offset parameters on the individual axes. During 2d control tomorrow the target size should be expanded also to about 3 to keep the monkey's interest.

There seems to be a bug in the BMI- when two units are sorted, both contribute to the firing rate estimate. I noticed this during X control today, which somewhat decreased the performance. Y performance was slightly better than yesterday, but still not great - he hasn't quite figured it out yet. XY was shitty, i guess.

Among other things, I really need to test the recording system - perhaps make a new file format that is extensible yet compressed? maybe labeled data streams? something like plexon files? Or perhaps just record it to the analog files (that would be easy!) nahh. todo:

  • write some matlab to combine the SQL records.
  • record the unit # in waveform record
  • save the logger output into the SQL db - not just in a file as now.
  • fix the onscreen shapes.

channel 29, at the end of the session:

channel 71. both these channels seem very stable - I hope the mk gets it before the evaporate!

there are no bmisql outputs as I did not run this analysis.


hide / / print
ref: thesis-0 tags: clementine 042307 operant conditioning date: 04-24-2007 01:37 gmt revision:2 [1] [0] [head]

Today, once again, I tried BMI both via pole control and with operant conditioning. The latter worked the best; because the fit/predictions were so shitty i didn't even try brain control with the wiener filter or kalman filter. Here is the output of BMIsql on ~6500 data slices, 18 neurons, 5 taps:

here is the prediction summary... note that target x position is doing rather well (probably because we are training units to respond to this)

output of BMIsql:

order of columns: unit,channel, lag, snr, variable

    2.0000   29.0000         0    1.0872    6.0000
    1.0000   53.0000    3.0000    1.0870    3.0000
    1.0000   53.0000    2.0000    1.0820    3.0000
    1.0000   82.0000    1.0000    1.0801    7.0000
    1.0000   82.0000    5.0000    1.0678    1.0000
    1.0000   82.0000    4.0000    1.0625    1.0000
    1.0000   82.0000    2.0000    1.0563    7.0000
    1.0000   53.0000    1.0000    1.0558    6.0000
    1.0000    8.0000         0    1.0550    8.0000
    1.0000   70.0000    3.0000    1.0549    2.0000
    1.0000   70.0000    2.0000    1.0536    2.0000
    2.0000   82.0000    4.0000    1.0524    1.0000
    2.0000   82.0000    5.0000    1.0516    1.0000
    1.0000   53.0000    4.0000    1.0506    3.0000
    1.0000   70.0000    4.0000    1.0503    2.0000
    2.0000   29.0000    1.0000    1.0497    5.0000
    2.0000   82.0000    3.0000    1.0494    1.0000
    1.0000   82.0000    3.0000    1.0464    7.0000
    1.0000    8.0000    1.0000    1.0454    8.0000
    1.0000   24.0000    1.0000    1.0450    8.0000
    1.0000   24.0000         0    1.0442    8.0000
    1.0000    8.0000    2.0000    1.0415    8.0000
    1.0000   70.0000    5.0000    1.0396    2.0000
    2.0000   82.0000    1.0000    1.0395    7.0000
    1.0000   24.0000    2.0000    1.0392    8.0000
    1.0000   70.0000    1.0000    1.0389    2.0000
    1.0000   81.0000    1.0000    1.0356    8.0000
    1.0000    8.0000    3.0000    1.0355    8.0000
    2.0000   29.0000    2.0000    1.0334    8.0000
    1.0000   81.0000    2.0000    1.0326    8.0000
    1.0000   24.0000    4.0000    1.0318    8.0000
    1.0000    8.0000    4.0000    1.0298    8.0000
    1.0000   24.0000    3.0000    1.0297    8.0000
    1.0000   28.0000    3.0000    1.0293   11.0000
    2.0000   82.0000    2.0000    1.0292    4.0000
    1.0000   28.0000    1.0000    1.0286   11.0000
    1.0000   28.0000    4.0000    1.0262   11.0000
    1.0000   28.0000    2.0000    1.0243   11.0000
    1.0000   28.0000         0    1.0238   11.0000
    2.0000   29.0000    3.0000    1.0221    8.0000
    1.0000   53.0000         0    1.0215    9.0000
    1.0000   81.0000    3.0000    1.0207    8.0000

Operant conditioning worked exceptionally well for the X axis (channel 29, yellow unit 1 - adding both unit's activity together did not work, the monkey would not play). see http://m8ta.com/tim/clem042307_trainX.MPG For a while he tried controlling the cursor position with the joystick, then after a while he realized this was unnecessary and just modulated unit 29.

Initially I tried operant conditioning of channel 82 for the Y axis, but it quickly appeared that he did not care and that it would not work. Hence I switched to channel 71, which was tried on Saturday the 20th. As before, this unit was tonically active while he was asleep, and almost silent while he was paying attention. an attention neuron? possibly. It also showed high firing rate changes when he struggled, suggesting volitional control. He was somewhat able to control it today... see http://m8ta.com/tim/clem042307_trainY.MPG

hide / / print
ref: Birbaumer-2007.03 tags: BMI operant conditioning review BCI date: 04-09-2007 14:25 gmt revision:0 [head]

PMID-17234696[0] Brain-computer interfaces: communication and restoration of movement in paralysis

  • A large gap between the promises of invasive animal and human BCI preparations and the clinical reality characterizes the literature: while intact monkeys learn to execute more or less complex upper limb movements with spike patterns from motor brain regions alone without concomitant peripheral motor activity usually after extensive training, clinical applications in human diseases such as amyotrophic lateral sclerosis and paralysis from stroke or spinal cord lesions show only limited success, with the exception of verbal communication in paralysed and locked-in patients.
  • attempts to train completely locked-in patients with BCI communication after entering the complete locked-in state with no remaining eye movement failed (!)
  • We propose that a lack of contingencies between goal directed thoughts and intentions may be at the heart of this problem. I'm not sure if 'contingencies' (something that can happen, but is generally not anticipated); should there not be a strong causal relationship between brain activity and prosthetic control?
  • still, the focus of this article are non-invasive BMIs.