alignment

New alignment and segmentation tools

Improved alignment and segmentation tools have now been released in the latest version of Scanbox, while retaining much of the functionality of the last version.

sbxaligntool. The new alignment tool, shown below, adds batch processing of files, including the processing of eye and ball motion if those data are present.  A region-of-interest (ROI) can optionally be selected manually or automatically.  For file entries where manual selection was specified, the program will stop and present a rectangle on the screen for the user to specify the ROI.  Typically, automatic ROI works fine, and it does not require the user to stand by the computer to specify the ROI each time a new file starts to process.

aligntool

As the files are aligned, the Status column and Status message will display the progress. The alignment procedure can also be visualized by clicking the Live update checkbox, which will display the mean of the entire image stack as the process moves along.  Pan and Zoom buttons allow the user to inspect details in the live image, such as fine branches, as the system is carrying out the alignment. This tool performs rigid alignment and the result is stored in a *_rigid.sbx file.  The original data is left untouched. The tool can align images relatively fast (about 65 frames/sec in my computer), but it will take a few minutes to compute the reference image if the sequence is 15 min or more (please be patient). Alignment improves with the number of passes requested.  Usually one pass is very good, but you can try two or more passes by changing the appropriate entry in the column. The alignment algorithm has been improved.

sbxsegmenttool. The segmentation tool works in a similar way as before. After loading the aligned *_rigid.sbx file, it will display the correlation map.  Segmentation then proceeds as in the previous version.

segmenttool

Once a number of cells are selected, you must save the segmentation and then extract the signals by pressing the corresponding buttons. After the signals are extracted you can select a cell with the pull down menu on the bottom left and the traces corresponding to that cell (now highlighted in green) will be displayed.  The blue trace represents the average signal within the cell, the gray trace is the neuropil, and the trace is the estimated spike rate using the Vanilla algorithm with parameters optimized for GCaMP6f.

Improvements include an Undo button, which will remove the last cell segmented. The ability to load a previous segmentation (it will load automatically after you select the *_rigid.sbx file), to continue adding cells to it.  The ability to define a ROI in the correlation map to automatically increase the contrast of the correlation map as the most salient cells are selected. A zoomed version of the mean image on the right to go along with the correlation map.  And the tool now saves the neuropil and deconvolved signal as well.

Give these tools a try. Report back any suggestions for improvements or problems you encounter.

New Features in Scanbox 3.4

The latest release adds a few new features and fixes.

A new panel called “Deadband & Alignment” offers the option to change some of the configuration variables on the fly.

dbalign

The deadband sliders allow to dynamically change the size of the left and right margins respectively, where the laser is blanked out.  You can optionally click the save button to save the settings at the end of the configuration file for future use.

Within this panel there is also the option of adjusting the alignment of even/odd lines during bidirectional scanning on the fly.  To use this first make sure the resfreq and lasfreq configuration variables are set to the correct values of the resonant and laser frequencies.

Typically the resonant frequency will be around 7930Hz, while the laser frequency, measured at 920nm wavelength, will be slightly over 80MHz.  Try to measure these values are accurately as possible.

The ncolbi configuration variable is now gone from this and any future versions. Only bishift is used at each magnification to adjust the line alignment, which can now be positive or negative.

To calibrate, pick a magnification, set the scope in continuous resonant mode for a few minutes, and start scanning.  You can now dynamically adjust the alignment by clicking on the << and >> buttons.  Using pollen grains is ideal for this step.

To have a closer look at the alignment you can digitally zoom into some features of the image by clicking Zoom in the navigation panel and then zooming into the region of interest.

The adjustment will look something like this:

Once satisfied with the adjustment you can stop scanning, move to another zoom setting, and repeat the process.  At any point you can hit the Save button to the right.  This will append a line with the new bishift settings to the end of the configuration file (effectively overriding the previous setting).

In the previous release a rounding bug caused some pixels to appear displaced in the image during bidirectional scanning.  This has now been fixed. The new version should allow you to get a very good results. Here is an example of a 40 LPMM Ronchi at a magnification of 1 and digitally zooming in to see details:

ub0

Zooming in…

ub1

And zooming in…

ub2

And zooming in again (you can click on the images to get them in full size):

u3

If you have trouble with bidirectional alignment let me know.

Make sure the HSYNC signal is symmetric and approximately in-phase before starting the calibration process as described in the first section here.

Note: you should not change magnifications on the fly during bidirectional scanning. You need to stop the scope, change magnification, and start scanning again.  The next version will allow you to do this.

Another feature in the new version is the ability to change the warm up time for the resonant mirror.  This is the time the resonant is turned on before the scan begins.  The default is 500 ms but can be changed with the wdelay configuration variable, which dictates the delay in units of tens of msec (so 50 = 500 ms).

Finally, you will see some new controls under the Laser panel.  These are additional controls to control the Coherent Discovery laser.  They provide control of the shutter for the fixed laser and the GDD slider for the pre-compensator.  To enable these controls the laser_type should be set to ‘DISCOVERY’ — otherwise they will be grayed out.

Non-rigid image alignment in twenty lines of Matlab

We have previously discussed rigid and non-rigid image alignment algorithms for calcium imaging data. If you have the image processing toolbox, a particularly compact solution for non-rigid image registration can be written in ~20 lines of code or so.  The code below returns a displacement field for each frame in the image sequence (output variable ‘disp’) that needs top be used to align the image.  The mean image after alignment is returned as well (output variable ‘m’).  The strategy follows the same recursive approach we used previously for rigid alignment.  The main Matlab function doing the calculation is imregdemons.


function [m,disp] = sbxalign_nonrigid(fname,idx)

if(length(idx)==1)
   A = squeeze(sbxread(fname,idx(1),1)); % just one frame... easy!
   m = A;
   disp = {zeros([size(A) 2])};
elseif (length(idx)==2) % align two frames
   A = squeeze(sbxread(fname,idx(1),1)); % read the frames
   B = squeeze(sbxread(fname,idx(2),1));
   [D,Ar] = imregdemons(A,B,[32 16 8 4],'AccumulatedFieldSmoothing',2.5,'PyramidLevels',4,'DisplayWaitBar',false);
   m = (Ar/2+B/2);
   disp = {D zeros([size(A) 2])};
else
   idx0 = idx(1:floor(end/2)); % split dataset in two
   idx1 = idx(floor(end/2)+1 : end); % recursive alignment
   [A,D0] = sbxalign_nonrigid(fname,idx0);
   [B,D1] = sbxalign_nonrigid(fname,idx1);
   [D,Ar] = imregdemons(A,B,[32 16 8 4],'AccumulatedFieldSmoothing',2.5,'PyramidLevels',4,'DisplayWaitBar',false);
   m = (Ar/2+B/2);
   D0 = cellfun(@(x) (x+D),D0,'UniformOutput',false); % concatenate distortions
   disp = [D0 D1];
end

Once the displacement field is computed one can obtain the aligned images by applying the distortion to each frame individually using imwarp.

The example below shows the original (top) and stabilized sequences in one case where imaging at high magnification clearly leads to non-rigid deformations that need correction.

Bidirectional scanning in Scanbox

We have implemented bidirectional scanning to Scanbox. To switch between bidirectional and unidirectional scanning all you have to do is hit the toggle button within the “Scanner” panel.

The defaubidirectionallt setting is “U”, standing for unidirectional scanning; hitting the button will switch it to “B”, indicating bidirectional scanning.  You can only switch between the two modes while the microscope is not scanning.

In bidirectional mode scanning samples are obtained while the laser scans in both directions of motion of the resonant mirror, and lines are advanced at both edges of the scan. On the plus side, this means that you can scan the same area twice as fast. You will notice the frame rate box doubling its value when you switch from unidirectional scanning to bidirectional scanning.

However, bidirectional scanning has some drawbacks as well.  First, you should be aware that that the Pockels’ cell will be active during both phases of the scan, while in unidirectional scanning it was active only in one direction. The result is that you will be delivering twice the amount of power onto the tissue, which may limit the length of your experiments due to potential photo-damage. Thus, use bidirectional scanning only if your indicators are fast enough and the scientific question you need to answer requires doubling the scanning speed.

Second, in bidirectional scanning even and odd lines are sampled in different directions, so they should be aligned properly to yield an image without spatial discontinuities. This alignment has to be fine tuned by the user in each setup, as there is variability in the resonant frequency of mirrors, the laser pulse frequency, and the phase of the sync pulse from the resonant mirror controller.

Scanbox allows for this fine tuning by changing a single variable in the configuration file named ‘ncolbi’ — a vector that allows alignment to be fined tuned separately for the three magnification factors.  This is necessary as well because the resonant frequency changes slightly with amplitude.

Once properly tuned, Scanbox performs compensation for non-uniform sampling and the alignment of even/odd lines during bidirectional scanning in real time.  Below is the mean image of 100 frames collected under bidirectional scanning.  You can see the alignment is very good by looking at the fine vertical processes in the image.

Alignment of even/odd lines in bidirectional scanning t 31 frames per second.

A final drawback of bidirectional scanning is that your file size will roughly double. This means that all other processing will also take longer — from aligning the images to extracting the signals.

Having said this, the faster rates achieved in bidirectional scanning will be helpful when coupled with the ability to do volumetric imaging by means of the optotune lens.

The bidirectional mode of Scanbox is now in beta testing.  I have not fully evaluated the extent to which other features of Scanbox, such as real time processing and image stabilization work in bidirectional mode. I welcome feedback from brave users who are willing to test bidirectional mode in their setups.

If you plan on giving bidirectional scanning a try let me know how it goes!  The updated code will be in Scanbox Github shortly.