Online, pixel-wise image processing with a Scanbox plugin

Sometimes you need your results right away. One example is when subsequent experiments are tailored to the properties of neurons in the population at hand.

Here is a simple Scanbox plugin that computes an orientation map preference on a pixel-by-pixel basis as the data are collected.  The variable mmfile.Data.header(4) indicates if a stimulus is being presented or not during the acquisition of a frame. If a stimulus is present, mmfile.Data.header(13) contains the direction of motion of the stimulus.

% Example of real-time, pixelwise image processing in Scanbox

% Open memory mapped file -- define just the header first

mmfile = memmapfile('scanbox.mmap','Writable',true, ...
'Format', { 'int16' [1 16] 'header' } , 'Repeat', 1);
flag = 1;

% Process all incoming frames until Scanbox stops


while(mmfile.Data.header(1)0 % a stim present?

idx = (mmfile.Data.header(13) / 30) + 1;
acc(:,:,idx) = acc(:,:,idx) + double(intmax('uint16')-mmfile.Data.chA);
N(idx) = N(idx)+1;
update = true;

elseif update && (sum(N)>0) % stim not present and needs update?

z = zeros(size(mchA)); % orientation map
n = zeros(size(mchA));

for k = 1:12
if N(k)>0
z = z + acc(:,:,k) .* exp(1i*(k-1)*30*2*pi/180) / N(k);
n = n + acc(:,:,k)/N(k);
z = z./n;
h.CData = angle(z);
q = abs(z);
q = q/max(q(:)); % normalize
h.AlphaData = q;
update = false;



mmfile.Data.header(1) = -1; % signal Scanbox that frame has been consumed!


clear(mmfile); % close the memory mapped file
close all; % close all figures

The code simply averages the frames for each orientation and then computes the vector resultant.  The angle of the resultant is color-coded to indicate the preferred direction of the neurons and the magnitude of the resultant is used as the alpha layer of the figure.

Here is a typical result obtained with 3 repeats of each direction (about 4 min data collection):

Annotation 2019-07-25 163019

Here is one movie taken with my phone showing the process in action:

And a screen captured movie:

Such online analysis can also be useful to stop data collection once one can reliable estimate the desired properties.  Thus, in the example above, one could stop data collection automatically as soon as the map becomes stable.