use https for features.
text: sort by
tags: modified
type: chronology
hide / / print
ref: -2019 tags: Arild Nokland local error signals backprop neural networks mnist cifar VGG date: 02-15-2019 03:15 gmt revision:6 [5] [4] [3] [2] [1] [0] [head]

Training neural networks with local error signals

  • Arild Nokland and Lars H Eidnes
  • Idea is to use one+ supplementary neural networks to measure within-batch matching loss between transformed hidden-layer output and one-hot label data to produce layer-local learning signals (gradients) for improving local representation.
  • Hence, no backprop. Error signals are all local, and inter-layer dependencies are not explicitly accounted for (! I think).
  • L simL_{sim} : given a mini-batch of hidden layer activations H=(h 1,...,h n)H = (h_1, ..., h_n) and a one-hot encoded label matrix Y=(y 1,...,y nY = (y_1, ..., y_n ,
    • L sim=||S(NeuralNet(H))S(Y)|| F 2 L_{sim} = || S(NeuralNet(H)) - S(Y)||^2_F (don't know what F is..)
    • NeuralNet()NeuralNet() is a convolutional neural net (trained how?) 3*3, stride 1, reduces output to 2.
    • S()S() is the cosine similarity matrix, or correlation matrix, of a mini-batch.
  • L pred=CrossEntropy(Y,W TH)L_{pred} = CrossEntropy(Y, W^T H) where W is a weight matrix, dim hidden_size * n_classes.
    • Cross-entropy is H(Y,W TH)=Σ i,jY i,jlog((W TH) i,j)+(1Y i,j)log(1(W TH) i,j) H(Y, W^T H) = \Sigma_{i,j} Y_{i,j} log((W^T H)_{i,j}) + (1-Y_{i,j}) log(1-(W^T H)_{i,j})
  • Sim-bio loss: replace NeuralNet()NeuralNet() with average-pooling and standard-deviation op. Plus one-hot target is replaced with a random transformation of the same target vector.
  • Overall loss 99% L simL_sim , 1% L predL_pred
    • Despite the unequal weighting, both seem to improve test prediction on all examples.
  • VGG like network, with dropout and cutout (blacking out square regions of input space), batch size 128.
  • Tested on all the relevant datasets: MNIST, Fashion-MNIST, Kuzushiji-MNIST, CIFAR-10, CIFAR-100, STL-10, SVHN.
  • Pretty decent review of similarity matching measures at the beginning of the paper; not extensive but puts everything in context.
    • See for example non-negative matrix factorization using Hebbian and anti-Hebbian learning in and Chklovskii 2014.
  • Emphasis put on biologically realistic learning, including the use of feedback alignment {1423}
    • Yet: this was entirely supervised learning, as the labels were propagated back to each layer.
    • More likely that biology is setup to maximize available labels (not a new concept).

hide / / print
ref: -2008 tags: representational similarity analysis fMRI date: 02-15-2019 02:27 gmt revision:1 [0] [head]

PMID-19104670 Representational Similarity Analysis – Connecting the Branches of Systems Neuroscience

  • Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini
  • Alright, there seems to be no math in the article (?), but it seems well cited so best be on the radar.
  • RDM = representational dissimilarity matrices
    • Just a symmetric matrix of dissimilarity, e.g. correlation, euclidean distance, absolute activation distance ( L 1L_1 ?)
  • RSA = representational similarity analysis
    • Comparison of the upper triangle of two RDMs, using the same metrics.
    • Or, alternately, second-order isomorphism.
  • So.. high level:

hide / / print
ref: -0 tags: variational free energy inference learning bayes curiosity insight Karl Friston date: 02-15-2019 02:09 gmt revision:1 [0] [head]

PMID-28777724 Active inference, curiosity and insight. Karl J. Friston, Marco Lin, Christopher D. Frith, Giovanni Pezzulo,

  • This has been my intuition for a while; you can learn abstract rules via active probing of the environment. This paper supports such intuitions with extensive scholarship.
  • “The basic theme of this article is that one can cast learning, inference, and decision making as processes that resolve uncertanty about the world.
    • References Schmidhuber 1991
  • “A learner should choose a policy that also maximizes the learner’s predictive power. This makes the world both interesting and exploitable.” (Still and Precup 2012)
  • “Our approach rests on the free energy principle, which asserts that any sentient creature must minimize the entropy of its sensory exchanges with the world.” Ok, that might be generalizing things too far..
  • Levels of uncertainty:
    • Perceptual inference, the causes of sensory outcomes under a particular policy
    • Uncertainty about policies or about future states of the world, outcomes, and the probabilistic contingencies that bind them.
  • For the last element (probabilistic contingencies between the world and outcomes), they employ Bayesian model selection / Bayesian model reduction
    • Can occur not only on the data, but exclusively on the initial model itself.
    • “We use simulations of abstract rule learning to show that context-sensitive contingiencies, which are manifest in a high-dimensional space of latent or hidden states, can be learned with straightforward variational principles (ie. minimization of free energy).
  • Assume that initial states and state transitions are known.
  • Perception or inference about hidden states (i.e. state estimation) corresponds to inverting a generative model gievn a sequence of outcomes, while learning involves updating the parameters of the model.
  • The actual task is quite simple: central fixation leads to a color cue. The cue + peripheral color determines either which way to saccade.
  • Gestalt: Good intuitions, but I’m left with the impression that the authors overexplain and / or make the description more complicated that it need be.
    • The actual number of parameters to to be inferred is rather small -- 3 states in 4 (?) dimensions, and these parameters are not hard to learn by minimizing the variational free energy:
    • F=D[Q(x)||P(x)]E q[ln(P(o t|x)]F = D[Q(x)||P(x)] - E_q[ln(P(o_t|x)] where D is the Kullback-Leibler divergence.
      • Mean field approximation: Q(x)Q(x) is fully factored (not here). many more notes

hide / / print
ref: -0 tags: feedback alignment Arild Nokland MNIST CIFAR date: 02-14-2019 02:15 gmt revision:0 [head]

Direct Feedback alignment provides learning in deep neural nets

  • from {1423}
  • Feedback alignment is able to provide zero training error even in convolutional networks and very deep networks, completely without error back-propagation.
  • Biologically plausible: error signal is entirely local, no symmetric or reciprocal weights required.
    • Still, it requires supervision.
  • Almost as good as backprop!
  • Clearly written, easy to follow math.
    • Though the proof that feedback-alignment direction is within 90 deg of backprop is a bit impenetrable, needs some reorganization or additional exposition / annotation.
  • 3x400 tanh network tested on MNIST; performs similarly to backprop, if faster.
  • Also able to train very deep networks, on MNIST - CIFAR-10, CIFAR-100, 100 layers (which actually hurts this task).

hide / / print
ref: -2014 tags: Lillicrap Random feedback alignment weights synaptic learning backprop MNIST date: 02-14-2019 01:02 gmt revision:5 [4] [3] [2] [1] [0] [head]

PMID-27824044 Random synaptic feedback weights support error backpropagation for deep learning.

  • "Here we present a surprisingly simple algorithm for deep learning, which assigns blame by multiplying error signals by a random synaptic weights.
  • Backprop multiplies error signals e by the weight matrix W T W^T , the transpose of the forward synaptic weights.
  • But the feedback weights do not need to be exactly W T W^T ; any matrix B will suffice, so long as on average:
  • e TWBe>0 e^T W B e > 0
    • Meaning that the teaching signal Be B e lies within 90deg of the signal used by backprop, W Te W^T e
  • Feedback alignment actually seems to work better than backprop in some cases. This relies on starting the weights very small (can't be zero -- no output)

Our proof says that weights W0 and W
evolve to equilibrium manifolds, but simulations (Fig. 4) and analytic results (Supple-
mentary Proof 2) hint at something more specific: that when the weights begin near
0, feedback alignment encourages W to act like a local pseudoinverse of B around
the error manifold. This fact is important because if B were exactly W + (the Moore-
Penrose pseudoinverse of W ), then the network would be performing Gauss-Newton
optimization (Supplementary Proof 3). We call this update rule for the hidden units
pseudobackprop and denote it by ∆hPBP = W + e. Experiments with the linear net-
work show that the angle, ∆hFA ]∆hPBP quickly becomes smaller than ∆hFA ]∆hBP
(Fig. 4b, c; see Methods). In other words feedback alignment, despite its simplicity,
displays elements of second-order learning.

hide / / print
ref: -0 tags: betzig sparse and composite coherent lattices date: 02-14-2019 00:00 gmt revision:1 [0] [head]

Sparse and composite coherent lattices

  • Focused on the math:
    • Linear algebra to find the wavevectors from the Bravais primitive vectors;
    • Iterative maximization @ lattice points to find the electric field phase and amplitude
    • (Read paper for details)
  • High NA objective naturally converts plane wave to a spherical wave; this can be used to create spherically-constrained lattices at the focal point of objectives.

hide / / print
ref: -0 tags: diffraction terahertz 3d print ucla deep learning optical neural networks date: 02-13-2019 23:16 gmt revision:1 [0] [head]

All-optical machine learning using diffractive deep neural networks

  • Pretty clever: use 3D printed plastic as diffractive media in a 0.4 THz all-optical all-interference (some attenuation) linear convolutional multi-layer 'neural network'.
  • In the arxive publication there are few details on how they calculated or optimized given diffractive layers.
  • Absence of nonlinearity will limit things greatly.
  • Actual observed performance (where thy had to print out the handwritten digits) rather poor, ~ 60%.

hide / / print
ref: -2017 tags: calcium imaging seeded iterative demixing light field microscopy mouse cortex hippocampus date: 02-13-2019 22:44 gmt revision:1 [0] [head]

PMID-28650477 Video rate volumetric Ca2+ imaging across cortex using seeded iterative demixing (SID) microscopy

  • Tobias Nöbauer, Oliver Skocek, Alejandro J Pernía-Andrade, Lukas Weilguny, Francisca Martínez Traub, Maxim I Molodtsov & Alipasha Vaziri
  • Cell-scale imaging at video rates of hundreds of GCaMP6 labeled neurons with light-field imaging followed by computationally-efficient deconvolution and iterative demixing based on non-negative factorization in space and time.
  • Utilized a hybrid light-field and 2p microscope, but didn't use the latter to inform the SID algorithm.
  • Algorithm:
    • Remove motion artifacts
    • Time iteration:
      • Compute the standard deviation versus time (subtract mean over time, measure standard deviance)
      • Deconvolve standard deviation image using Richardson-Lucy algo, with non-negativity, sparsity constraints, and a simulated PSF.
      • Yields hotspots of activity, putative neurons.
      • These neuron lcoations are convolved with the PSF, thereby estimating its ballistic image on the LFM.
      • This is converted to a binary mask of pixels which contribute information to the activity of a given neuron, a 'footprint'
        • Form a matrix of these footprints, p * n, S 0S_0 (p pixels, n neurons)
      • Also get the corresponding image data YY , p * t, (t time)
      • Solve: minimize over T ||YST|| 2|| Y - ST||_2 subject to T0T \geq 0
        • That is, find a non-negative matrix of temporal components TT which predicts data YY from masks SS .
    • Space iteration:
      • Start with the masks again, SS , find all sets O kO^k of spatially overlapping components s is_i (e.g. where footprints overlap)
      • Extract the corresponding data columns t it_i of T (from temporal step above) from O kO^k to yield T kT^k . Each column corresponds to temporal data corresponding to the spatial overlap sets. (additively?)
      • Also get the data matrix Y kY^k that is image data in the overlapping regions in the same way.
      • Minimize over S kS^k ||Y kS kT k|| 2|| Y^k - S^k T^k||_2
      • Subject to S k>=0S^k >= 0
        • That is, solve over the footprints S kS^k to best predict the data from the corresponding temporal components T kT^k .
        • They also impose spatial constraints on this non-negative least squares problem (not explained).
    • This process repeats.
    • allegedly 1000x better than existing deconvolution / blind source segmentation algorithms, such as those used in CaImAn

hide / / print
ref: -0 tags: Hinton google tech talk dropout deep neural networks Boltzmann date: 02-12-2019 08:03 gmt revision:2 [1] [0] [head]

Brains, sex, and machine learning -- Hinton google tech talk.

  • Hinton believes in the the power of crowds -- he thinks that the brain fits many, many different models to the data, then selects afterward.
    • Random forests, as used in predator, is an example of this: they average many simple to fit and simple to run decision trees. (is apparently what Kinect does)
  • Talk focuses on dropout, a clever new form of model averaging where only half of the units in the hidden layers are trained for a given example.
    • He is inspired by biological evolution, where sexual reproduction often spontaneously adds or removes genes, hence individual genes or small linked genes must be self-sufficient. This equates to a 'rugged individualism' of units.
    • Likewise, dropout forces neurons to be robust to the loss of co-workers.
    • This is also great for parallelization: each unit or sub-network can be trained independently, on it's own core, with little need for communication! Later, the units can be combined via genetic algorithms then re-trained.
  • Hinton then observes that sending a real value p (output of logistic function) with probability 0.5 is the same as sending 0.5 with probability p. Hence, it makes sense to try pure binary neurons, like biological neurons in the brain.
    • Indeed, if you replace the backpropagation with single bit propagation, the resulting neural network is trained more slowly and needs to be bigger, but it generalizes better.
    • Neurons (allegedly) do something very similar to this by poisson spiking. Hinton claims this is the right thing to do (rather than sending real numbers via precise spike timing) if you want to robustly fit models to data.
      • Sending stochastic spikes is a very good way to average over the large number of models fit to incoming data.
      • Yes but this really explains little in neuroscience...
  • Paper referred to in intro: Livnat, Papadimitriou and Feldman, PMID-19073912 and later by the same authors PMID-20080594
    • A mixability theory for the role of sex in evolution. -- "We define a measure that represents the ability of alleles to perform well across different combinations and, using numerical iterations within a classical population-genetic framework, show that selection in the presence of sex favors this ability in a highly robust manner"
    • Plus David MacKay's concise illustration of why you need sex, pg 269, __Information theory, inference, and learning algorithms__
      • With rather simple assumptions, asexual reproduction yields 1 bit per generation,
      • Whereas sexual reproduction yields G\sqrt G , where G is the genome size.

hide / / print
ref: -0 tags: VARNUM GEVI genetically encoded voltage indicators FRET Ace date: 02-12-2019 07:35 gmt revision:2 [1] [0] [head]

PMID-30420685 Fast in-vivo voltage imaging using a red fluorescent indicator

  • Other genetically encoded voltage indicators (GEVI):
    • PMID-22958819 ArcLight (Peribone also last author) ; sign of ΔF/F\Delta F / F negative, but large, 35%! Slow tho? improvement in speed
    • ASAP3 ΔF/F\Delta F / F large, τ=3ms.\tau = 3 ms.
    • PMID-26586188 Ace-mNeon FRET based, Acetabularia opsin, fast kinetics + brightness of mNeonGreen.
    • Archon1 -- fast and sensitive, found (like VARNUM) using a robotic directed evolution or direct search strategy.
  • VARNAM is based on Acetabularia (Ace) + mRuby3, also FRET based, found via high-throughput voltage screen.
  • Archaerhodopsin require 1-12 W/mm^2 of illumination, vs. 50 mw/mm^2 for GFP based probes. Lots of light!
  • Systematic optimization of voltage sensor function: both the linker region (288 mutants), which affects FRET efficiency, as well as the opsin fluorophore region (768 mutants), which affects the wavelength of absorption / emission.
  • Some intracellular clumping (which will negatively affect sensitivity), but mostly localized to the membrane.
  • Sensitivity is still imperfect -- 4% in-vivo cortical neurons, though it’s fast enough to resolve 100 Hz spiking.
  • Can resolve post-synaptic EPSCs, but < 1 % ΔF/F\Delta F/F .
  • Tested all-optical ephys using VARNAM + blueshifted channelrhodopsin, CheRiff, both sparsely, and in PV targeted transgenetic model. Both work, but this is a technique paper; no real results.
  • Tested TEMPO fiber-optic recording in freely behaving mice (ish) -- induced ketamine waves, 0.5-4Hz.
  • And odor-induced activity in flies, using split-Gal4 expression tools. So many experiments.

hide / / print
ref: -2019 tags: mosers hippocampus popsci nautilus grid cells date: 02-12-2019 07:32 gmt revision:1 [0] [head]

New Evidence for the Strange Geometry of Thought

  • Wow. Things are organized in 2d structures in the brain. The surprising thing about this article is that only the hiippocampus is mentioned, no discussion of the cortex. Well, it was written by a second year graduate student (though, admittedly, the writing style is perfectly fine.)

hide / / print
ref: -0 tags: superresolution imaging scanning lens nanoscale date: 02-04-2019 20:34 gmt revision:1 [0] [head]

PMID-27934860 Scanning superlens microscopy for non-invasive large field-of-view visible light nanoscale imaging

  • Recently, the diffraction barrier has been surpassed by simply introducing dielectrics with a micro-scale spherical configuration when using conventional optical microscopes by transforming evanescent waves into propagating waves. 18,19,20,21,22,23,24,25,26,27,28,29,30
  • The resolution of this superlens-based microscopy has been decreased to ∼50 nm (ref. 26) from an initial resolution of ∼200 nm (ref. 21).
  • This method can be further enhanced to ∼25 nm when coupled with a scanning laser confocal microscope 31.
  • It has achieved fast development in biological applications, as the sub-diffraction-limited resolution of high-index liquid-immersed microspheres has now been demonstrated23,32, enabling its application in the aqueous environment required to maintain biological activity.
  • Microlens is a 57 um diameter BaTiO3 microsphere, resolution of lambda / 6.3 under partial and inclined illumination
  • Microshpere is in contact with the surface during imaging, by gluing it to the cantilever tip of an AFM.
  • Get an image with the microsphere-lens, which improves imaging performance by ~ 200x. (with a loss in quality, naturally).

hide / / print
ref: -0 tags: Kato fear conditioning GABA auditory cortex mice optogenetics SOM PV date: 02-04-2019 19:09 gmt revision:0 [head]

PMID-29375323 Fear learning regulates cortical sensory representation by suppressing habituation

  • Trained mice on CS+ and CS --> lick task.
    • CS+ = auditory tone followed by tailshock
    • CS- = auditory tone (both FM modulated, separated by 0.5 - 1.0 octave).
    • US = licking.
  • VGAT2-ChR2 or PV-ChR2
  • GABA-ergic silencing of auditory cortex through blue light illumination abolished behavior difference following CS+ and CS-.
  • Used intrinsic imaging to locate A1 cortex, then AAV - GCaMP6 imaging to lcoated pyramidal cells.
  • In contrast to reports of enhanced tone responses following simple fear conditioning (Quirk et al., 1997; Weinberger, 2004, 2015), discriminative learning under our conditions caused no change in the average fraction of pyramidal cells responsive to the CS+ tone.
    • Seemed to be an increase in suppression, and reduced cortical responses, which is consistent with habituation.
  • Whereas -- and this is by no means surprising -- cortical responses to CS+ were sustained at end of tone following fear conditioning.
  • ----
  • Then examined this effect relative to the two populations of interneurons, using PV-cre and SOM-cre mice.
    • In PV cells, fear conditioning resulted in a decreased fraction of cells responsive, and a decreased magnitude of responses.
    • In SOM cells, CS- responses were enhanced, while CS+ were less enhanced (the main text seems like an exaggeration c.f. figure 6E)
  • This is possibly the more interesting result of the paper, but even then the result is not super strong.

hide / / print
ref: -0 tags: curiosity exploration forward inverse models trevor darrell date: 02-01-2019 03:42 gmt revision:1 [0] [head]

Curiosity-driven exploration by Self-supervised prediction

  • Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, Trevor Darrell
  • Key insight: “we only predict the changes in the environment that could possibly be due to actions of our agent or affect the agent, and ignore the rest”.
    • Instead of making predictions in the sensory space (e.g. pixels), we transform the sensory input into a feature space where only the information relevant to the agent is represented.
    • We learn this feature space using self-supervision -- training a neural network via a proxy inverse dynamics task -- predicting the agent’s action from the past and future sensory states.
  • We then use this inverse model to train a forward dynamics model to predict feature representation of the next state from present feature representation and action.
      • The difference between expected and actual representation serves as a reward signal for the agent.
  • Quasi actor-critic / adversarial agent design, again.
  • Used the asynchronous advantage actor critic policy gradient method (Mnih et al 2016 Asynchronous Methods for Deep Reinforcement Learning).
  • Compare with variational information maximization (VIME) trained with TRPO (Trust region policy optimization) which is “more sample efficient than A3C but takes more wall time”.
  • References / concurrent work: Several methods propose improving data efficiency of RL algorithms using self-supervised prediction based auxiliary tasks (Jaderberg et al., 2017; Shelhamer et al., 2017).
  • An interesting direction for future research is to use the learned exploration behavior / skill as a motor primitive / low level policy in a more complex, hierarchical system. For example, the skill of walking along corridors could be used as part of a navigation system.

hide / / print
ref: -0 tags: lillicrap segregated dendrites deep learning backprop date: 01-31-2019 19:24 gmt revision:2 [1] [0] [head]

PMID-29205151 Towards deep learning with segregated dendrites https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5716677/

  • Much emphasis on the problem of credit assignment in biological neural networks.
    • That is: given complex behavior, how do upstream neurons change to improve the task of downstream neurons?
    • Or: given downstream neurons, how do upstream neurons receive ‘credit’ for informing behavior?
      • I find this a very limiting framework, and is one of my chief beefs with the work.
      • Spatiotemporal Bayesian structure seems like a much better axis (axes) to cast function against.
      • Or, it could be segregation into ‘signal’ and ‘error’ or ‘figure/ground’ based on hierarchical spatio-temporal statistical properties that matters ...
      • ... with proper integration of non-stochastic spike timing + neoSTDP.
        • This still requires some solution of the credit-assignment problem, i know i know.
  • Outline a spiking neuron model with zero one or two hidden layers, and a segregated apical (feedback) and basal (feedforward) dendrites, as per a layer 5 pyramidal neuron.
  • The apical dendrites have plateau potentials, which are stimulated through (random) feedback weights from the output neurons.
  • Output neurons are forced to one-hot activation at maximum firing rate during training.
    • In order to assign credit, feedforward information must be integrated separately from any feedback signals used to calculate error for synaptic updates (the error is indicated here with δ). (B) Illustration of the segregated dendrites proposal. Rather than using a separate pathway to calculate error based on feedback, segregated dendritic compartments could receive feedback and calculate the error signals locally.
  • Uses the MNIST database, naturally.
  • Poisson spiking input neurons, 784, again natch.
  • Derive local loss function learning rules to make the plateau potential (from the feedback weights) match the feedforward potential
    • This encourages the hidden layer -> output layer to approximate the inverse of the random feedback weight network -- which it does! (At least, the jacobians are inverses of each other).
    • The matching is performed in two phases -- feedforward and feedback. This itself is not biologically implausible, just unlikely.
  • Achieved moderate performance on MNIST, ~ 4%, which improved with 2 hidden layers.
  • Very good, interesting scholarship on the relevant latest findings ‘’in vivo’’.
  • While the model seems workable though ad-hoc or just-so, the scholarship points to something better: use of multiple neuron subtypes to accomplish different elements (variables) in the random-feedback credit assignment algorithm.
    • These small models can be tuned to do this somewhat simple task through enough fiddling & manual (e.g. in the algorithmic space, not weight space) backpropagation of errors.
  • They suggest that the early phases of learning may entail learning the feedback weights -- fascinating.
  • ‘’Things are definitely moving forward’’.

hide / / print
ref: -0 tags: STDP dopamine hippocampus date: 01-16-2019 21:56 gmt revision:1 [0] [head]

PMID-26516682 Retroactive modulation of spike timing-dependent plasticity by dopamine.

  • Here we show that dopamine, a positive reinforcement signal, can retroactively convert hippocampal timing-dependent synaptic depression into potentiation.
  • This effect requires functional NMDA receptors and is mediated in part through the activation of the cAMP/PKA cascade.
  • Mouse horizontal slices.
  • Plasticity induced by 100 pairings of a single EPSP followed by a postsynaptic spike (heavy-handed?)
  • Pre-before-post @ 10ms -> LTP
  • Post-before-pre @ -20ms -> LTD
  • Post-before-pre @ -10ms -> LTP (?!)
    • Addition of Dopamine antagonist (D2: sulpiride, D1/D5: SCH23390) prevented LTP and resulted in LTD.
  • Post-before-pre @ -20ms -> LTP in the presence of 20 uM DA.
    • The presence of DA during coordinated spiking activity widense the timing interval for induction of LTP.
  • What about if it's applied afterward?
  • 20 uM DA applied 1 minute (for 10-12 minutes) after LTD induction @ -20 mS converted LTD into LTP.
    • This was corrected by addition of the DA agonists.
    • Did not work if DA was applied 10 or 30 minutes after the LTD induction.
  • Others have shown that this requires functional NMDA receptors.
    • Application of NMDA agonist D-AP5 after post-before-pre -20ms did not affect LTD.
    • Application of D-AP5 before DA partially blocked conversion of LTD to LTP.
    • Application of D-AP5 alone before induction did not affect LTD.
  • This is dependent on the cAMP/PKA signaling cascade:
    • Application of forskolin (andenylyl cyclase AC activator) converts LTD -> LTP.
    • Dependent on NMDA.
  • PKA inhibitor H-89 alsoblocked LTD -> P.

hide / / print
ref: -2012 tags: octopamine STDP locust LTP LTD date: 01-05-2019 01:50 gmt revision:3 [2] [1] [0] [head]

PMID-22278062 Conditional modulation of spike-timing-dependent plasticity for olfactory learning.

  • Looked at the synapes from the Muschroom body (Kenyon cells, sparse code) to the beta-lobe (bLN) in locusts.
  • Used in-vivo dendrite patch, sharp micropipette.
  • Found that, with a controlled mushroom body extracellular stim for plasticity induction protocol at the KC-> bLN synapese, were able to get potentiation and depression in accord with STDP.
  • This STDP became pure depression in the presence of octopamine

hide / / print
ref: -0 tags: nanophotonics interferometry neural network mach zehnder interferometer date: 12-24-2018 22:23 gmt revision:2 [1] [0] [head]

Deep Learning with Coherent Nanophotonic Circuits

  • Used a series of Mach-Zehnder interferometers with thermoelectric phase-shift elements to realize the unitary component of individual layer weight-matrix computation.
    • Weight matrix was decomposed via SVD into UV*, which formed the unitary matrix (4x4, Special unitary 4 group, SU(4)), as well as Σ\Sigma diagonal matrix via amplitude modulators. See figure above / original paper.
    • Note that interfereometric matrix multiplication can (theoretically) be zero energy with an optical system (modulo loss).
      • In practice, you need to run the phase-moduator heaters.
  • Nonlinearity was implemented electronically after the photodetector (e.g. they had only one photonic circuit; to get multiple layers, fed activations repeatedly through it. This was a demonstration!)
  • Fed network FFT'd / banded recordings of consonants through the network to get near-simulated vowel recognition.
    • Claim that noise was from imperfect phase setting in the MZI + lower resolution photodiode read-out.
  • They note that the network can more easily (??) be trained via the finite difference algorithm (e.g. test out an incremental change per weight / parameter) since running the network forward is so (relatively) low-energy and fast.
    • Well, that's not totally true -- you need to update multiple weights at once in a large / deep network to descend any high-dimensional valleys.

hide / / print
ref: -0 tags: synaptic plasticitiny 2-photon imaging inhibition excitation spines dendrites date: 12-13-2018 05:03 gmt revision:0 [head]

PMID-22542188 Clustered dynamics of inhibitory synapses and dendritic spines in the adult neocortex.

  • Cre-recombinase-dependent labeling of postsynapitc scaffolding via Gephryn-Teal fluorophore fusion.
  • Also added Cre-eYFP to lavel the neurons
  • Electroporated in utero e16 mice.
    • Low concentration of Cre, high concentrations of Gephryn-Teal and Cre-eYFP constructs to attain sparse labeling.
  • Located the same dendrite imaged in-vivo in fixed tissue - !! - using serial-section electron microscopy.
  • 2230 dendritic spines and 1211 inhibitory synapses from 83 dendritic segments in 14 cells of 6 animals.
  • Some spines had inhibitory synapses on them -- 0.7 / 10um, vs 4.4 / 10um dendrite for excitatory spines. ~ 1.7 inhibitory
  • Suggest that the data support the idea that inhibitory inputs maybe gating excitation.
  • Furthermore, co-inervated spines are stable, both during mormal experience and during monocular deprivation.
  • Monocular deprivation induces a pronounced loss of inhibitory synapses in binocular cortex.

hide / / print
ref: -0 tags: US employment top 100 bar chart date: 11-12-2018 00:02 gmt revision:1 [0] [head]

After briefly searching the web, I could not find a chart of the top 100 occupations in the US. After downloading the data from the US Bureau of Labor Statistics, made this chart:

Click for full-size.

Surprising how very service heavy our economy is.