m8ta
You are not authenticated, login. |
|
{760} |
ref: -0
tags: LDA myopen linear discriminant analysis classification
date: 01-03-2012 02:36 gmt
revision:2
[1] [0] [head]
|
|
How does LDA (Linear discriminant analysis) work? It works by projecting data points onto a series of planes, one per class of output, and then deciding based which projection plane is the largest. Below, to the left is a top-view of this projection with 9 different classes of 2D data each in a different color. Right is a size 3D view of the projection - note the surfaces seem to form a parabola. Here is the matlab code that computes the LDA (from myopen's ceven % TrainData and TrainClass are inputs, column major here. % (observations on columns) N = size(TrainData,1); Ptrain = size(TrainData,2); Ptest = size(TestData,2); % add a bit of interpolating noise to the data. sc = std(TrainData(:)); TrainData = TrainData + sc./1000.*randn(size(TrainData)); K = max(TrainClass); % number of classes. %%-- Compute the means and the pooled covariance matrix --%% C = zeros(N,N); for l = 1:K; idx = find(TrainClass==l); % measure the mean per class Mi(:,l) = mean(TrainData(:,idx)')'; % sum all covariance matrices per class C = C + cov((TrainData(:,idx)-Mi(:,l)*ones(1,length(idx)))'); end C = C./K; % turn sum into average covariance matrix Pphi = 1/K; Cinv = inv(C); %%-- Compute the LDA weights --%% for i = 1:K Wg(:,i) = Cinv*Mi(:,i); % this is the slope of the plane Cg(:,i) = -1/2*Mi(:,i)'*Cinv*Mi(:,i) + log(Pphi)'; % and this, the origin-intersect. end %%-- Compute the decision functions --%% Atr = TrainData'*Wg + ones(Ptrain,1)*Cg; % see - just a simple linear function! Ate = TestData'*Wg + ones(Ptest,1)*Cg; errtr = 0; AAtr = compet(Atr'); % this compet function returns a sparse matrix with a 1 % in the position of the largest element per row. % convert to indices with vec2ind, below. TrainPredict = vec2ind(AAtr); errtr = errtr + sum(sum(abs(AAtr-ind2vec(TrainClass))))/2; netr = errtr/Ptrain; PeTrain = 1-netr; | ||
{588} | ||
images/588_1.pdf -- Good lecture on LDA. Below, simple LDA implementation in matlab based on the same: % data matrix in this case is 36 x 16, % with 4 examples of each of 9 classes along the rows, % and the axes of the measurement (here the AR coef) % along the columns. Sw = zeros(16, 16); % within-class scatter covariance matrix. means = zeros(9,16); for k = 0:8 m = data(1+k*4:4+k*4, :); % change for different counts / class Sw = Sw + cov( m ); % sum the means(k+1, :) = mean( m ); %means of the individual classes end % compute the class-independent transform, % e.g. one transform applied to all points % to project them into one plane. Sw = Sw ./ 9; % 9 classes criterion = inv(Sw) * cov(means); [eigvec2, eigval2] = eig(criterion); See {587} for results on EMG data. | ||
{587} | ||
Below, emg classification by computing the autoregressive coefficients and feeding them into linear discriminant analysis (LDA). LDA code from here; data in myopen svn. Nine classes of movement in the data, 4 repetitions of each. The input data is 16-dimensional: 4 AR coefficients per 4 channels. This is consistent with Blair Lock's thesis. For reference, here is an imagesc() of the raw coefficients (the 4 different color bands correspond to the 4 different channels): |