m8ta
use https for features.
text: sort by
tags: modified
type: chronology
{1522}
hide / / print
ref: -2017 tags: schema networks reinforcement learning atari breakout vicarious date: 09-29-2020 02:32 gmt revision:2 [1] [0] [head]

Schema networks: zero-shot transfer with a generative causal model of intuitive physics

  • Like a lot of papers, the title has more flash than the actual results.
  • Results which would be state of the art (as of 2017) in playing Atari breakout, then transferring performance to modifications of the game (paddle moved up a bit, wall added in the middle of the bricks, brick respawning, juggling).
  • Schema network is based on 'entities' (objects) which have binary 'attributes'. These attributes can include continuous-valued signals, in which case each binary variable is like a place fields (i think).
    • This is clever an interesting -- rather than just low-level features pointing to high-level features, this means that high-level entities can have records of low-level features -- an arrow pointing in the opposite direction, one which can (also) be learned.
    • The same idea is present in other Vicarious work, including the CAPTCHA paper and more-recent (and less good) Bio-RNN paper.
  • Entities and attributes are propagated forward in time based on 'ungrounded schemas' -- basically free-floating transition matrices. The grounded schemas are entities and action groups that have evidence in observation.
    • There doesn't seem to be much math describing exactly how this works; only exposition. Or maybe it's all hand-waving over the actual, much simpler math.
      • Get the impression that the authors are reaching to a level of formalism when in fact they just made something that works for the breakout task... I infer Dileep prefers the empirical for the formal, so this is likely primarily the first author.
  • There are no perceptual modules here -- game state is fed to the network directly as entities and attributes (and, to be fair, to the A3C model).
  • Entity-attributes vectors are concatenated into a column vector length NTNT , where NN are the number of entities, and TT are time slices.
    • For each entity of N over time T, a row-vector is made of length MRMR , where MM are the number of attributes (fixed per task) and R1R-1 are the number of neighbors in a fixed radius. That is, each entity is related to its neighbors attributes over time.
    • This is a (large, sparse) binary matrix, XX .
  • yy is the vector of actions; task is to predict actions from XX .
    • How is X learned?? Very unclear in the paper vs. figure 2.
  • The solution is approximated as y=XW1¯y = X W \bar{1 } where WW is a binary weight matrix.
    • Minimize the solution based on an objective function on the error and the complexity of ww .
    • This is found via linear programming relaxation. "This procedure monotonically decreases the prediction error of the overall schema network, while increasing its complexity".
      • As it's a issue of binary conjunctions, this seems like a SAT problem!
    • Note that it's not probabilistic: "For this algorithm to work, no contradictions can exist in the input data" -- they instead remove them!
  • Actual behavior includes maximum-product belief propagation, to look for series of transitions that set the reward variable without setting the fail variable.
    • Because the network is loopy, this has to occur several times to set entity variables eg & includes backtracking.

  • Have there been any further papers exploring schema networks? What happened to this?
  • The later paper from Vicarious on zero-shot task transfer are rather less interesting (to me) than this.

{496}
hide / / print
ref: bookmarks-0 tags: EMG schematic amplifier prosthetic myopen date: 01-03-2012 23:08 gmt revision:3 [2] [1] [0] [head]

{365}
hide / / print
ref: Akin-1995.06 tags: Najafi neural recording technology micromachined digital TETS 1995 PNS schematics date: 01-01-2012 20:23 gmt revision:8 [7] [6] [5] [4] [3] [2] [head]

IEEE-717081 (pdf) An Implantable Multichannel Digital neural recording system for a micromachined sieve electrode

  • Later pub: IEEE-654942 (pdf) -- apparently putting on-chip isolated diodes is a difficult task.
  • 90mw of power @ 5V, 4x4mm of area (!!)
  • targeted for regenerated peripheral neurons grown through a micromachined silicon sieve electrode.
    • PNS nerves are deliberately severed and allowed to regrow through the sieve.
  • 8bit low-power current-mode ADC. seems like a clever design to me - though I can't really follow the operation from the description written there.
  • class e transmitter amplifier.
  • 3um BiCMOS process. (you get vertical BJTs and Zener diodes)
  • has excellent schematics. - including the voltage regulator, envelop detector & ADC.
  • most of the power is dissipated in the voltage regulator (!!) - 80mW of 90mW.
  • tiny!
  • rather than using pseudoresistors, they use diode-capacitor input filter which avoids the need for chopping or off-chip hybrid components.
  • can record from any two of 32 input channels. I think the multiplexer is after the preamp - right?

____References____

Akin, T. and Najafi, K. and Bradley, R.M. Solid-State Sensors and Actuators, 1995 and Eurosensors IX.. Transducers '95. The 8th International Conference on 1 51 -54 (1995)

{250}
hide / / print
ref: engineering-0 tags: schematic capture layout PCB design engineering date: 03-17-2007 23:44 gmt revision:0 [head]

http://archives.seul.org/geda/user/Aug-2005/msg00027.html