Month: February 2016

Memory-mapped data-sharing for real-time processing

To enhance the real-time processing capabilities of Scanbox we now provide a general mechanism to share frames being collected with another application running concurrently on the same computer via a memory mapped file.

This general and flexible mechanism allows any user to extend the real-time processing of Scanbox beyond what is already offered.

As an example, this interface allows one to perform pixel-by-pixel image processing in ways that do not require prior definitions of ROIs. One such case would be the calculation of temporally encoded cortical maps by on-the-fly calculations of the magnitude/phase of each pixel.

All you have to do to enable this feature is to set the “mmap” variable to 1 in the configuration file and, in those imaging sessions you want to stream the data, enable the “Memory map” checkbox in the real-time processing panel:

realtime

The memory mapped file contains a 16 word header (int16s) that allows communication between Scanbox and another process, which is then followed by the imagery data.  Only the first 4 words of the header are used at the moment (the other are reserved for future use).

The first word is used as a semaphore between the applications to control access to the memory mapped file.  The value of this variable is -1 if the file can be used by Scanbox to stream data, it is -2 if Scanbox has stopped imaging, and it is a non-negative number that corresponds to a frame if Scanbox has copied the data to the memory mapped file.

The second and third variables represent the size of the frame, used by the application processing the data to interpret the organization of the file as an image.

The fourth variable is a copy of the TTL1, which is high during a stimulus presentation and low otherwise.

As an example of a Matlab process that consumes data from Scanbox and all it does is compute the average frame is listed below.


% Simple example of memory mapped processing in Scanbox

% Open memory mapped file -- just header first
mmfile = memmapfile('scanbox.mmap','Writable',true, ... 
 'Format', { 'int16' [1 16] 'header' } , 'Repeat', 1);
flag = 1;

% Process all incoming frames until Scanbox stops

while(true)

   while(mmfile.Data.header(1)<0) % wait for a new frame...
     if(mmfile.Data.header(1) == -2) % exit if Scanbox stopped
     return;
     end
   end
 
 display(sprintf('Frame %06d',mmfile.Data.header(1))) % print frame# being processed
 
 if(flag) % first time? Format chA according to lines/columns in data
   mmfile.Format = {'int16' [1 16] 'header' ; ... 
   'uint16' double([mmfile.Data.header(2) mmfile.Data.header(3)]) 'chA'};
   mchA = double(mmfile.Data.chA);
   flag = 0;
 end
 
 mchA = mchA + double(mmfile.Data.chA);
 
 % Here you can do whatever you want with
 % your data.... I am just accumulating.
 
 mmfile.Data.header(1) = -1; % signal Scanbox that frame has been consumed!
end

clear(mmfile)

imagesc(mchA); % display mean image
truesize

The interface allows you to use any language that provides you access to memory mapped files, such as Python.

So, if you happen to write some neat real-time image processing module for Scanbox, feel free to share!

A set of more readable functions that encapsulate the basic functionality shown above will be provided soon to make the writing of real-time processing applications easier, and we may also adopt a double-buffering or ring-buffer mechanisms.

Line scan and continuous resonant modes

The latest version of Scanbox includes two new features requested by users.  These appear are small toggle buttons on the left within the Scanner panel.

scanner

The button on the bottom right of the panel is already known to you — it allows you to toggle between Unidirectional and Bidirectional scanning modes.  The letter shown in the button indicates the current mode.

The middle button is new and allows you to toggle between Normal and Continuous Resonant modes.  The normal mode is the one you have been using so far.  Scanbox stops the resonant mirror between imaging sessions and requires 2-3 seconds at the beginning of each session to “warm up” before scanning can start. The new continuous version allows you to turn the resonant mirror and leave it on.  One advantage is that you can start your scanning right away and there is no warm-up period.  A second advantage is that there is no auditory cue triggered by the resonant mirror that may affect behavior at the beginning of the trial.  One disadvantage is that you can expect the lifetime of the mirror be reduced by the increase usage.

The top button at the top toggles between Area and Line Scan modes.  Area mode is what you have been using up to now.  In line-scan mode scanbox fixes the position of the galvo to be that of the middle of the image (y-axis).  To perform a line scan simply position the line of interest in the middle of the frame and then switch to line scan mode.  The image that will be displayed will now consist of space on the horizontal (resonant) axis and time in the vertical axis. The number of lines/frame is not really relevant any more (as all lines are the same) but for reasons that I will explain later please use a value > 256.

(Real estate in the Scanbox panels is slowly becoming tight — we will likely switch to tabbed panels soon to be able to use more meaningful labels across the various panels.)

Measuring the resonant frequency

The actual time a line takes to scan depends on the resonant frequency of the mirror.  This is typically around 8kHz but it can vary from one device to another.  One easy way to measure it is to split the signal that goes from TRIG OUT to the TRIG IN of the Alazartech board and feed it to an oscilloscope.  The pulses you will see mark the beginning of each line, and you can have the oscilloscope measure the  frequency.  In my case, I end up with 7924Hz (see below).

If your frames have a total of nlines per frame, then the time at which the n-th frame and k-th line began scanning is T = (n*nlines+k)/res_freq (in unidirectional mode).

IMG_0610

 

 

Simultaneous imaging and electrophysiology

Scanbox now allows the possibility of doing simultaneous imaging and electrophysiology.  An external DAQ device, supported by Matlab’s data acquisition toolbox, is need to do this.  In our case, we had an old NI-USB-6221 lying around that we decided to use.

In the present case, we set things up to do patch-clamp recordings from a single channel.  The NI box needs access to the the analog signal (which goes into AI.1) as well as input from the CAM0 signal from Scanbox, which signals the beginning of each microscope frame.  This second signal is required for the alignment of the electrophysiology and imaging data.

ephys.001

All you need to do to use this new feature of Scanbox is edit two lines in the scanbox_config.m file:

sbconfig.ephys = 1; % enable ephys data acquisition
sbconfig.ephysRate = 32000; % sampling rate (samples/sec)

The first variable enables the acquisition of electrophysiology data (set to zero otherwise), and the second variable sets the sampling rate that will be used, in this case 32kHz.

That’s all you need to do…

Now, at the end of your imaging session, you will find an additional file with the suffix “.ephys” containing binary data.  The file consists of a sequence of single-precision floating point number pairs. The first number in the pair is the frame of the microscope at which the sample was taken. The second number is the value of the signal.

The postdoc who requested these features, and who is brave enough to venture into doing patch-clamp and imaging, has promised to get some data soon. An example of the first successful recording will be posted below:

postdoc

 

Meet Knobby

Knobby is finally here!

Knobby provides an alternative to the 3D mouse interface for controlling the position of the microscope.

Knobby offers simultaneous control of all axes of the microscope by independent knobs (X, Y, Z and objective angle, A) and is very responsive to the user movements producing nice, smooth motion.

The position control interface appears now on a touch-screen that replaces the panel within Scanbox (which will be grayed out if you use Knobby instead of the 3D mouse).

As the layout and functionality of the touch-screen is identical to that of the Scanbox panel, users should find it very easy to make the transition.

knobbyscreen

The position of the microscope is displayed in real-time on Knobby’s screen as the knobs are turned (compared to updates at the end of each movement with the 3D mouse), so now one can easily adjust the position with sub micrometer precision in a smooth fashion.

A lock/unlock feature prevents accidentally moving the microscope by touching the knobs.

Knobby is uses a USB port to allow for communication with Scanbox and firmware upgrades.

Knobby also makes Scanbox faster by offloading position control to an external device, instead of having to handle it within the Scanbox main loop.

As mentioned earlier, Scanbox will continue to support the 3D mouse interface.

If interested in Knobby, please contact Josh.