m8ta
you are not logged in, login. new entry 

{1341} 
ref: 0
tags: image registration optimization camera calibration sewing machine
date: 07152016 05:04 gmt
revision:20
[19] [18] [17] [16] [15] [14] [head]


Recently I was tasked with converting from image coordinates to real world coordinates from stereoscopic cameras mounted to the endeffector of a robot. The end goal was to let the user (me!) click on points in the image, and have the robot record that position & ultimately move to it. The overall strategy is to get a set of points in both image and RW coordinates, then fit some sort of model to the measured data. I began by printing out a grid of (hopefully evenlyspaced and perpendicular) lines via a laserprinter; spacing was ~1.1 mm. This grid was manually aligned to the axes of robot motion by moving the robot along one axis & checking that the lines did not jog. The images were modeled as a grating with quadratic phase in $u,v$ texture coordinates: ${p}_{h}(u,v)=\mathrm{sin}(({a}_{h}u/1000+{b}_{h}v/1000+{c}_{h})v+{d}_{h}u+{e}_{h}v+{f}_{h})+0.97$ (1) ${p}_{v}(u,v)=\mathrm{sin}(({a}_{v}u/1000+{b}_{v}v/1000+{c}_{v})u+{d}_{v}u+{e}_{v}v+{f}_{v})+0.97$ (2) $I(u,v)=16{p}_{h}{p}_{v}/(\sqrt{2+16{p}_{h}^{2}+16{p}_{v}^{2}})$ (3) The 1000 was used to make the parameter search distribution more spherical; ${c}_{h},{c}_{v}$ were bias terms to seed the solver; 0.97 was a dutycycle term fit by inspection to the image data; (3) is a modified sigmoid. $I$ was then optimized over the parameters using a GPUaccelerated (CUDA) nonlinear stochastic optimization: $({a}_{h},{b}_{h},{d}_{h},{e}_{h},{f}_{h}\mid {a}_{v},{b}_{v},{d}_{v},{e}_{v},{f}_{v})=\mathrm{Argmin}{\sum}_{u}{\sum}_{v}(I(u,v)\mathrm{Img}(u,v){)}^{2}$ (4) Optimization was carried out by drawing parameters from a normal distribution with a diagonal covariance matrix, set by inspection, and mean iteratively set to the best solution; horizontal and vertical optimization steps were separable and carried out independently. The equation (4) was sampled 18k times, and equation (3) 34 billion times per frame. Hence the need for GPU acceleration. This yielded a set of 10 parameters (again, ${c}_{h}$ and ${c}_{v}$ were bias terms and kept constant) which modeled the data (e.g. grid lines) for each of the two cameras. This process was repeated every 0.1 mm from 0  20 mm height (z) from the target grid, resulting in a sampled function for each of the parameters, e.g. ${a}_{h}(z)$ . This required 13 trillion evaluations of equation (3). Now, the task was to use this model to generate the forward and reverse transform from image to world coordinates; I approached this by generating a data set of the grid intersections in both image and world coordinates. To start this process, the known image origin ${u}_{\mathrm{origin}}{\mid}_{z=0},{v}_{\mathrm{origin}}{\mid}_{z=0}$ was used to find the corresponding roots of the periodic axillary functions ${p}_{h},{p}_{v}$ : $\frac{3\pi}{2}+2\pi {n}_{h}={a}_{h}uv/1000+{b}_{h}{v}^{2}/1000+({c}_{h}+{e}_{h})v+{d}_{h}u+{f}_{h}$ (5) $\frac{3\pi}{2}+2\pi {n}_{h}={a}_{v}{u}^{2}/1000+{b}_{v}uv/1000+({c}_{v}+{d}_{v})u+{e}_{v}v+{f}_{v}$ (6) Or .. ${n}_{h}=\mathrm{round}(({a}_{h}uv/1000+{b}_{h}{v}^{2}/1000+({c}_{h}+{e}_{h})v+{d}_{h}u+{f}_{h}\frac{3\pi}{2})/(2\pi )$ (7) ${n}_{v}=\mathrm{round}(({a}_{v}{u}^{2}/1000+{b}_{v}uv/1000+({c}_{v}+{d}_{v})u+{e}_{v}v+{f}_{v}\frac{3\pi}{2})/(2\pi )$ (8) From this, we get variables ${n}_{h,\mathrm{origin}}{\mid}_{z=0}\mathrm{and}{n}_{v,\mathrm{origin}}{\mid}_{z=0}$ which are the offsets to align the sine functions ${p}_{h},{p}_{v}$ with the physical origin. Now, the reverse (world to image) transform was needed, for which a twostage newton scheme was used to solve equations (7) and (8) for $u,v$ . Note that this is an equation of phase, not image intensity  otherwise this direct method would not work! First, the equations were linearized with three steps of (911) to get in the right ballpark: ${u}_{0}=640,{v}_{0}=360$ ${n}_{h}={n}_{h,\mathrm{origin}}{\mid}_{z}+[30..30],{n}_{v}={n}_{v,\mathrm{origin}}{\mid}_{z}+[20..20]$ (9) ${B}_{i}=\left[\begin{array}{c}\frac{3\pi}{2}+2\pi {n}_{h}{a}_{h}{u}_{i}{v}_{i}/1000{b}_{h}{v}_{i}^{2}{f}_{h}\\ \frac{3\pi}{2}+2\pi {n}_{v}{a}_{v}{u}_{i}^{2}/1000{b}_{v}{u}_{i}{v}_{i}{f}_{v}\end{array}\right]$ (10) ${A}_{i}=\left[\begin{array}{ccc}{d}_{h}& & {c}_{h}+{e}_{h}\\ {c}_{v}+{d}_{v}& & {e}_{v}\end{array}\right]$ and $\left[\begin{array}{c}{u}_{i+1}\\ {v}_{i+1}\end{array}\right]=\mathrm{mldivide}({A}_{i},{B}_{i})$ (11) where mldivide is the Matlab operator. Then three steps with the full Jacobian were made to attain accuracy: ${J}_{i}=\left[\begin{array}{ccc}{a}_{h}{v}_{i}/1000+{d}_{h}& & {a}_{h}{u}_{i}/1000+2{b}_{h}{v}_{i}/1000+{c}_{h}+{e}_{h}\\ 2{a}_{v}{u}_{i}/1000+{b}_{v}{v}_{i}/1000+{c}_{v}+{d}_{v}& & {b}_{v}{u}_{i}/1000+{e}_{v}\end{array}\right]$ (12) ${K}_{i}=\left[\begin{array}{c}{a}_{h}{u}_{i}{v}_{i}/1000+{b}_{h}{v}_{i}^{2}/1000+({c}_{h}+{e}_{h}){v}_{i}+{d}_{h}{u}_{i}+{f}_{h}\frac{3\pi}{2}2\pi {n}_{h}\\ {a}_{v}{u}_{i}^{2}/1000+{b}_{v}{u}_{i}{v}_{i}/1000+({c}_{v}+{d}_{v}){u}_{i}+{e}_{v}v+{f}_{v}\frac{3\pi}{2}2\pi {n}_{v}\end{array}\right]$ (13) $\left[\begin{array}{c}{u}_{i+1}\\ {v}_{i+1}\end{array}\right]=\left[\begin{array}{c}{u}_{i}\\ {v}_{i}\end{array}\right]{J}_{i}^{1}{K}_{i}$ (14) Solutions $(u,v)$ were verified by plugging back into equations (7) and (8) & verifying ${n}_{h},{n}_{v}$ were the same. Inconsistent solutions were discarded; solutions outside the image space $[0,1280),[0,720)$ were also discarded. The process (10)  (14) was repeated to tile the image space with gird intersections, as indicated in (9), and this was repeated for all $z$ in $(0..0.1..20)$ , resulting in a large (74k points) dataset of $(u,v,{n}_{h},{n}_{v},z)$ , which was converted to full realworld coordinates based on the measured spacing of the grid lines, $(u,v,x,y,z)$ . Between individual z steps, ${n}_{h,\mathrm{origin}}{n}_{v,\mathrm{origin}}$ was reestimated to minimize (for a current $z\prime $ ): $({u}_{\mathrm{origin}}{\mid}_{z\prime +0.1}{u}_{\mathrm{origin}}{\mid}_{z\prime +0.1}{)}^{2}+({v}_{\mathrm{origin}}{\mid}_{z\prime +0.1}+{v}_{\mathrm{origin}}{\mid}_{z\prime}{)}^{2}$ (15) with gridsearch, and the method of equations (914). This was required as the stochastic method used to find original image model parameters was agnostic to phase, and so phase (via parameter ${f}_{}$ ) could jump between individual $z$ measurements (the origin did not move much between successive measurements, hence (15) fixed the jumps.) To this dataset, a model was fit: $\left[\begin{array}{c}u\\ v\end{array}\right]=A\left[\begin{array}{ccccccccccccccccccccccccccc}1& & x& & y& & z& & x{\prime}^{2}& & y{\prime}^{2}& & {\textstyle \prime}z{\prime}^{2}& & {w}^{2}& & x\prime y\prime & & x\prime z\prime & & y\prime z\prime & & x\prime w& & y\prime w& & z\prime w\end{array}\right]$ (16) Where $x\prime =\frac{x}{10}$ , $y\prime =\frac{y}{10}$ , $z\prime =\frac{z}{10}$ , and $w=\frac{20}{20z}$ . $w$ was introduced as an axillary variable to assist in perspective mapping, ala computer graphics. Likewise, $x,y,z$ were scaled so the quadratic nonlinearity better matched the data. The model (16) was fit using regular linear regression over all rows of the validated dataset. This resulted in a second set of coefficients $A$ for a model of world coordinates to image coordinates; again, the model was inverted using Newton's method (Jacobian omitted here!). These coefficients, one set per camera, were then integrated into the C++ program for displaying video, and the inverse mapping (using closedform matrix inversion) was used to convert mouse clicks to realworld coordinates for robot motor control. Even with the relatively poor wideFOV cameras employed, the method is accurate to $\pm 50\mu m$ , and precise to $\pm 120\mu m$ .  
{711}  
PMID19299587[0] Optical Deconstruction of Parkinsonian Neural Circuitry.
____References____
 
{1340}  
PMID26867734 A biophysical model of the cortexbasal gangliathalamus network in the 6OHDA lesioned rat model of Parkinson’s disease
Overall, not a bad paper. Not very well organized, which is not assisted by the large amount of information presented, but having slogged through the figures, I'm somewhat convinced that the model is good. This despite my general reservations of these models: the true validation would be to have it generate actual behavior (and learning)! Lacking this, the approximations employed seem like a step forward in understanding how PD and DBS work. The results and discussion are consistent with {1255}, but not {711}, which found that STN projections from M1 (not the modulation of M1 projections to GPi, via efferents from STN) truly matter.
 
{1339}  
* Watch the [http://homes.cs.washington.edu/~todorov/index.php?video=MordatchSIGGRAPH12&paper=Mordatch,%20SIGGRAPH%202012 movies! Discovery of complex behaviors through contactinvariant optimization]
 
{1338}  
ZeroMQ  much better sockets framework than native TCP/UDP sockets.
 
{1337}  
Useful numbers for estimating molecular meanfreepath in vacuum systems: "
 
{1336}  
A contact lens with embedded sensor for monitoring tear glucose level
 
{1335}  
What are the concentrations of the monoamines in the brain? (Purpose: estimate the required electrochemical sensing area & efficiency)
 
{1334}  
PMID26627311 Monolithically Integrated μLEDs on Silicon Neural Probes for HighResolution Optogenetic Studies in Behaving Animals.
 
{1333}  
 
{1330}  
META II: Digital Vellum in the Digital Scriptorium: Revisiting Schorre's 1962 compilercompiler
 
{1204}  
PMID21867803[0] Can histology solve the riddle of the nonfunctioning electrode? Factors influencing the biocompatibility of brain machine interfaces.
____References____
 
{1327}  
PMID26436341 Threedimensional macroporous nanoelectronic networks as minimally invasive brain probes.
 
{1328}  
Utah/blackrock group has been working on improving the longevity of their parlyene encapsulation with the addition of ~50nm Al2O3.
 
{1326}  
PMID25627426 Rapid evaluation of the durability of cortical neural implants using accelerated aging with reactive oxygen species.
 
{1325}  
https://mitpress.mit.edu/sites/default/files/titles/free_download/9780262526548_Art_of_Insight.pdf  
{1324}  
Problem: have a Qswitched Nd:YAG laser, (flashlamp pumped, passively Qswitched) from ebay (see this album). Allegedly it outputs 1J pulses of 8ns duration; in practice, it may put several 100mJ pulses ~ 16ns long while the flashlamp is firing. It was sold as a tattoo removal machine. However, I'm employing it to drill microvias in fine polyimide films. When focused through a 10x objective via the camera mount of an Leica microscope, 532nm (KTP doubled, second harmonic generation (SHG)) laser pulses both ablates the material, but does not leave a clean, sharp hole: it looks more like 'blasting': the hole is ragged, more like a crater. This may be from excessive 1064nm heating (partial KTP conversion), or plasma/flame heating & expansion due to absorption of the 532nm / 1064nm light. It may also be due to excessive pulse duration (should the laser not actually be qswitched... photodiode testing suggests otherwise, but I'd like to verify that), excessive pulse power, insufficient pulse intensity, or insufficient polyimide absorption at 532nm. The solution to excessive plasma and insufficient polyimide absorption is to shift the wavelength to 355nm (NUV) via third harmonic generation, 1064 + 532 = 355nm. This requires sum frequency generation (SFG), for which LBO (lithium triborate) or BBO (betabarium borate) seem the commonly accepted nonlinear optical materials. To get SHG or THG, phase and polarization matching of the incoming light is critical. The output of the Nd:YAG laser is, I assume, nonpolarized (or randomly polarized), as the KTP crystal simply screws on the front, and so should be rotationally agnostic (and there are no polarizing elements in the simple laser head  unless the (presumed) Cr:YAG passive Qswitch induces some polarization.) Output polarization of the KTP crystal will be perpendicular to the incoming beam; if the resulting THG / SFG crystal needs Type1 phase matching (both in phase and parallel polarization), will need a halfwave plate for 1064nm; for TypeII phase matching, no plate is needed. For noncritical phase matching in LBO (which I just bought), an oven is required to heat the crystal to the correct temperature. This suggests 73C for THG, while this suggests 150C (for SHG?). Third harmonic frequency generation by typeI critically phasematched LiB3O5 crystal by means of optically active quartz crystal Suggests most lasers operate in Type1 SHG, and TypeII THG, but this is less efficient than dual Type1; the quartz crystal is employed to rotate the polarizations to alignment. Both SHG and THG crystals are heated for optimum power output. Finally, Short pulse duration of an extracavity sumfrequency mixing with an LiB3O5 (LBO) crystal suggests that no polarization change is required, nor oven control LBO temperature. Tight focus and high energy density is required, of course (at the expense of reduced crystal lifetime). Likely this is the Type1,TypeII scheme alluded to in the paper above. I'll try this first before engaging further complexity (efficiency is not very important, as the holes are very small & material removal may be slow.)  
{1266}  
Thin films and microelectrode arrays for neuroprosthetics
