Automatic Optotune Calibration

We previously explained how to calibrate the optotune manually.  With the introduction of Knobby 2, we are able to make this process automatic.  You will now find a ‘Calibration’ button in the Optotune panel.  To use it, do the following:

  1. Set the optotune slider to its lowest value (slide all the way down)
  2. Bring some pollen grains into focus
  3. Stop focusing
  4. Make sure the data directory has a directory named xx0
  5. Click the ‘Calibrate’ button
  6. Sit back and relax.  Wait for the process to complete.

Knobby will run some z-stack acquisitions for different values of the optotune current setting.  The volumetric data will be used to calculate the shift in z at various values of the current. A panel will display the progress in processing the images (it takes about 2 min). Scanbox will then plot the raw data and a fit by a quadratic polynomial, which may look something like this:

opto_calib

Scanbox will write a calibration file which will take effect next time you start Scanbox.

After restarting Scanbox, you can check the calibration as follows:

  1. Set the optotune slider to its lowest value (slide all the way down)
  2. Set Knobby to super-fine mode
  3. Focus on some pollen grains
  4. Zero Knobby (XYZ)
  5. Move out-of-focus by moving the optotune slider up to some value
  6. Now bring sample back into focus using the z-axis knob
  7. Compare the reading of the z-axis in Knobby’s screen with the depth noted in the optotune panel. These two numbers should match very closely.

 

Click-and-Center with Knobby Mouse Control

If you have been making use of Knobby’s spatial calibration button, you can now move on to interacting with Knobby remotely.

The new Knobby scheduler panel looks a bit expanded.  The speed (coarse/fine/super-fine), the mode (normal/rotate), and zero (XYZ/XYZA) buttons should be self explanatory. They do exactly the same as if you were to be using them on Knobby.  The action you select on the panel will be reflected on Knobby’s screen as well.

knobbymouse

The range, steps and frames entries provide a quick way to edit the scheduler’s table if you intend to do a simple z-stack.  Range is the extent of the distance in z you want to span, steps is the the size of the step (in um), and frames is the number of frames you want to scan at each position.

Finally, there is new checkbox labeled “Mouse control”. When this is active and you are scanning you can easily move around the sample by clicking the cell you want to bring to the center of the screen, as shown by the video example above.  Once you click on a cell, Knobby does the rest. We call this feature click-and-center. This works at all magnifications (even if you change it on the fly). At the same time, scrolling the mouse wheel allows you to focus up and down.  The speed of the movement will be controlled by the selected speed. You may want to start at the slowest speed (Super-fine).

Spatial Calibration and return to origin

We added two new features to the knobby scheduler panel, as shown below by the arrows.

knobby2nf

One (left arrow) allows knobby to perform an automatic spatial calibration of the system, measuring the (x,y) size of pixels at all magnifications.  To perform this calculation do the following:

  1. Focus on some pollen grains.  Make sure one of the pollen grains is well centered on the screen at the highest magnification (x8).
  2. Make you there is an empty folder named xx0 in the data directory you selected.
  3. Hit the calibration button (pointed by the left arrow).
  4. The system will ask you if you want to proceed.
  5. Scanbox will then loop over all magnifications, collecting data for 8 sec and moving the sample by a known distance.
  6. When it finishes, Scanbox will display the optimal value of a resonant gain magnification variable that will make your pixels square.  Note that value.

Now, when you restart Scanbox the system will read the new spatial calibration in the calibration button.  The format is [xsize ysize] [xfov yfov].  The left pair shows the width and height of a pixel (in micrometers), and the right pair shows the width and height of the field-of-view at the current magnification setting (also in micrometers).  In the example above, the (x,y) size is [0.72 0.70] um, meaning the aspect ratio is close to one.

If the aspect ratio is far from one, and you prefer square pixels, do the following:

  1. Change the value of the “gain_resonant_mult” variable in the configuration file to the one suggested by the Scanbox spatial calibration, restart Scanbox.
  2. Re-run the spatial calibration.  Thi will create a new calibration file.
  3. Restart Scanbox to read the newly created calibration file.  Now the aspect ratio should be close to one.

The second feature is one is a checkbox labeled Return. When you run a knobby schedule you have click this box to make knobby return to the initial position it started from. So, for example, upon completion of a z-stack the system will go back to where it started. Note: this feature uses the Store/Recall C function, so if you have anything stored there it will be erased.

 

 

A Plug-in Server

Scanbox has a memory mapped mechanism to share the incoming data stream with other processes that intend to consume the data in real-time.

Examples of simple uses of such data sharing include the calculation of rolling averages, displaying a real-time histogram, or generating a montage display for volumetric data.

Of course, different experiments may need to be processed by different plug-ins.  To simplify this process we now provide a single mechanism that allows the selection of a plug-in via a pull-down menu in the Scanbox GUI.

The configuration variable “plugin” can be used to list a number of different options to call.  In the example below, two possible options are listed.

sbconfig.plugin = {'rolling','montage'}; %plugin options

These must be names of Matlab scripts that will be called to process the data.

When Scanbox starts it will list these options in a pull down menu within the real-time processing panel, along with two check-boxes, one labeled “plugin” and another labeled “focus”.

plugin

The pull down menu allows the user to select one plugin among the available selections.

The plug-in checkbox indicates you want the data to be shared via the memory mapped mechanism, and the “focus” checkbox indicates whether you want the data to be shared not only during a grabbing operation, but also when you are focusing.

For example, adjusting the laser intensity by analyzing the histogram of values in the incoming data may be something you want to do ahead of an actual experiment, and something you will want to run while focusing.

Now, when you start Scanbox a new Matlab instance is opened and calls the sbxplugin_server.m script (assuming mmap=true in the config file).  This is a simple script that waits for new data to arrive and calls the selected plug-in.

% Scanbox plug in server script
% This code will run upon startup of Scanbox on a separate matlab instance if mmap=true
% in the scanbox_config file

% Open memory mapped file -- define just the header first

scanbox_config; % read the scanbox config structure

mmfile = memmapfile('scanbox.mmap','Writable',true, ...
 'Format', { 'int16' [1 16] 'header' } , 'Repeat', 1);
flag = 1;

disp('Plugin server ready');

% Process all incoming frames until Scanbox stops

running = false;

while(true)

 disp('Waiting for imaging data stream');

 while ~running
 running = mmfile.Data.header(1)>=0;
 end

 plugin_id = mmfile.Data.header(7); % get the plug in to be used

 fprintf('Plugin id=%d\n',plugin_id);
 fprintf('ROI=%d Expt=%d\n',mmfile.Data.header(8),mmfile.Data.header(9));

 while running

 running = (mmfile.Data.header(1) ~= -2); % no stop signal present

 if running && mmfile.Data.header(1)>=0 % if not stopped and frame present

 switch plugin_id

 case 1
 % replace the line below with your own script for
 % plugin with id #1
 fprintf('Frame: %05d Plugin: %d\n',mmfile.Data.header(1),plugin_id);

 case 2
 % replace the line below with your own script for
 % plugin with id #2
 fprintf('Frame: %05d Plugin: %d\n',mmfile.Data.header(1),plugin_id)

 otherwise
 disp('Invalid plugin id number');
 end

 mmfile.Data.header(1) = -1; % signal Scanbox that frame has been consumed!

 end
 end
end

clear(mmfile); % close the memory mapped file
close all; % close all figures

The default server code simply lists number of of the incoming frame and the selected plug-in.  Just add an eval() call to the script name at the indicated locations.

The output of the script, as is, will look like:


Waiting for imaging data stream
Plugin id=2
ROI=0 Expt=0
Frame: 00000 Plugin: montage
Frame: 00001 Plugin: montage
Frame: 00002 Plugin: montage
Frame: 00003 Plugin: montage
Frame: 00004 Plugin: montage
Frame: 00005 Plugin: montage
Frame: 00006 Plugin: montage
Frame: 00007 Plugin: montage
Frame: 00008 Plugin: montage
Frame: 00009 Plugin: montage
Frame: 00010 Plugin: montage
Waiting for imaging data stream

The user can now easily switch between plugins from the Scanbox window by changing the selection in the pull down menu.

 

New alignment and segmentation tools

Improved alignment and segmentation tools have now been released in the latest version of Scanbox, while retaining much of the functionality of the last version.

sbxaligntool. The new alignment tool, shown below, adds batch processing of files, including the processing of eye and ball motion if those data are present.  A region-of-interest (ROI) can optionally be selected manually or automatically.  For file entries where manual selection was specified, the program will stop and present a rectangle on the screen for the user to specify the ROI.  Typically, automatic ROI works fine, and it does not require the user to stand by the computer to specify the ROI each time a new file starts to process.

aligntool

As the files are aligned, the Status column and Status message will display the progress. The alignment procedure can also be visualized by clicking the Live update checkbox, which will display the mean of the entire image stack as the process moves along.  Pan and Zoom buttons allow the user to inspect details in the live image, such as fine branches, as the system is carrying out the alignment. This tool performs rigid alignment and the result is stored in a *_rigid.sbx file.  The original data is left untouched. The tool can align images relatively fast (about 65 frames/sec in my computer), but it will take a few minutes to compute the reference image if the sequence is 15 min or more (please be patient). Alignment improves with the number of passes requested.  Usually one pass is very good, but you can try two or more passes by changing the appropriate entry in the column. The alignment algorithm has been improved.

sbxsegmenttool. The segmentation tool works in a similar way as before. After loading the aligned *_rigid.sbx file, it will display the correlation map.  Segmentation then proceeds as in the previous version.

segmenttool

Once a number of cells are selected, you must save the segmentation and then extract the signals by pressing the corresponding buttons. After the signals are extracted you can select a cell with the pull down menu on the bottom left and the traces corresponding to that cell (now highlighted in green) will be displayed.  The blue trace represents the average signal within the cell, the gray trace is the neuropil, and the trace is the estimated spike rate using the Vanilla algorithm with parameters optimized for GCaMP6f.

Improvements include an Undo button, which will remove the last cell segmented. The ability to load a previous segmentation (it will load automatically after you select the *_rigid.sbx file), to continue adding cells to it.  The ability to define a ROI in the correlation map to automatically increase the contrast of the correlation map as the most salient cells are selected. A zoomed version of the mean image on the right to go along with the correlation map.  And the tool now saves the neuropil and deconvolved signal as well.

Give these tools a try. Report back any suggestions for improvements or problems you encounter.

Spikefinder: the Vanilla algorithm

Over the last few months, the Spikefinder challenge has provided a playing ground for colleagues to offer ideas about how to estimate the spiking of individual neurons based on the measured activity of fluorescent calcium indicators.

The challenge was for people to come up with strategies that beat the performance of state-of-the-art algorithms, STM & oopsi. A good number of algorithms were able to achieve this in short time, including one I submitted, termed Vanilla.

The best performing algorithms seem to have relied on modern machine learning methods. Vanilla is nothing more than a linear filter followed by a static-nonlinearity y(t) = \phi ( h(t) * x(t) )  — thus the name.

The filter h(t) is a linear combination of an even filter, estimating the mean of the signal at time t, and an odd filter, estimating the derivative of the signal at time t.

The even filter is a Gaussian, h_{even} = A \exp ( -t^2 / 2 \sigma^2 ), and the odd filter is the derivative of a Gaussian h_{odd} = B t \exp (-t^2 / 2 \sigma^2).  The constants A and B are such that the norm of the filters is normalized to one, \| A \| = \| B \| = 1.  These two filters are linearly combined while keeping the norm of resulting filter equal to one, h(t) = \cos \alpha \: h_{even}(t) + \sin \alpha \: h_{odd}(t).

The output nonlinearity is a rectifier to a power, \phi ( x ) = (x- \theta)^\beta if x>\theta, and zero otherwise.

The model has only 4 parameters, \{\sigma, \alpha, \theta, \beta \}. The amount of smoothing of the signal is controlled by \sigma, the shape of the filter is controlled by \alpha, and the threshold \theta and power \beta determine the shape of the nonlinearity.

The model is fit by finding the optimal values of \{\sigma, \alpha, \theta, \beta \} that maximize the correlation between its output y(t) and the recorded spiking of the neuron.  I used Matlab’s fminsearch() to perform this optimization, which was typically finished in about 60 sec or less for most datasets.

The only pre-processing done was a z-scoring of the raw signals.  In one dataset (dataset #5, GCaMP6s in V1), we allowed for an extra-delay parameter between the signal and the prediction.

I was surprised this entry did relatively well, as the algorithm it is basically a version of STM. I think the particular shape of the output nonlinearity (rectifier+power vs exponential), the constrain imposed on the shape of the filters, and the resulting small number of parameters, paid a role in Vanilla doing better overall.

The top algorithms reached an absolute performance of about 0.47 and it seems unlikely this performance can be improved by a lot. This seems to highlight the limitations of the current families of calcium indicators in yielding precise spiking information — so there is plenty of opportunity to improve them.

It is interesting that despite its simplicity, the relative performance of Vanilla, with a correlation coefficient of 0.428 was not dramatically inferior to that of the top performing, deep network models, with all its bells and whistles, which landed at 0.464.  So, one must pay due respects to deep networks, but I was honestly expecting Vanilla to be completely blown out of the water in terms of performance, and I don’t think it was.

Finally, Vanilla is a non-casual algorithm, as both past and future samples are used to predict the response at time t. In some situations, however, when we are trying close the loop as fast as possible by controlling a stimulus based on neural activity itself, we need algorithms that are casual and can provide a fast, online estimate of spiking activity. I wonder if any of the submissions are causal algorithms and, if not, what would be the best performance the methods can attain if we allow them to provide estimates of spiking based only on past samples.

External TTL trigger

To automatically start /stop acquisition by means of an external TTL signal follow the following instructions.

First, connect an appropriate TTL control signal to the P1.6 (pin #21) of the extension header of the Scanbox board.  The view below shows a top view of the Scanbox control board.  The pin in question is located on the back row of connectors when viewing the board from the front.  As a ground pin, you can use pin #3.  To make it easier to make the appropriate connections it helps to get the this cable and route it outside the box.

Capture

StartCapture the Scanbox software and operate as usual by focusing and selecting the area you want to record.  When ready to switch to external trigger control, simply click the “External TTL Trigger” checkbox, which is located in the middle of the Scanner control panel.

After enabling the TTL trigger, the manual Focus/Grab buttons will be grayed out and blocked from usage.  If you want to go back to manual control simply deselect the TTL Trigger checkbox.

The rising edge of the TTL control signal is used to start/stop the microscope.  Minimum pulse width is 1 ms.

While controlling the microscope using an external TTL signal it is useful to run it in continuous resonant mode (so you avoid waiting for the resonant mirror to warm up) , and set the “autoinc” configuration variable to “true”, so file numbers increment automatically after the completion of each session.

To use this feature you have to update to the latest version of the firmware/software.

Update [6/23/17]:

If you are willing to give use of TTL1 as an event line, a new trig_sel configuration variable allows you to use it instead of the signal from the header to start and stop the acquisition.  You will have to upgrade to the latest version of the software.  Follow the instructions here.  You can skip the steps asking you to update knobby, the motor box, and run vc_redist.x64.exe.  After the update, simple set the trig_sel configuration variable to true if you want to trigger with TTL1 (or false if you want to trigger using the header signal).

Knobby scheduler

CaptureA new Scanbox panel allows users to define arbitrary changes in (x,y,z) position over time (frames) which are then executed by Knobby (version 2 only) while imaging.

Each entry define changes in x, y and z (in micrometers) relative to the present position and the frame number at which they will take place.

The “mem” column allows one to specify one of the stored absolute coordinates instead (memory locations are coded A=1, B=2, C=3).  If a memory location is defined the other entries are ignored and the position in the referenced memory is used instead.

This mechanism extends the z-stack functionality to include the ability to tile a sample and brings back the control window to one of the panels in Scanbox (as opposed to being controlled in Knobby’s screen).  The Knobby table is also saved in the info.knobby_table variable.

Paths can be computed offline and stored in a Matlab file that can be loaded.  The example below shows knobby moving the sample along a circular path.

Update [ 7/3/17]: You will now note an additional checkbox in the Knobby Scheduler panel called “Return”. When you arm knobby scheduler and check this box the microscope will automatically return to the initial position after imaging is done.  This feature uses the Store/Recall C function for functionality.  Anything stored previously in C is going to be erased if you use this feature.