gpu processing

New alignment and segmentation tools

Improved alignment and segmentation tools have now been released in the latest version of Scanbox, while retaining much of the functionality of the last version.

sbxaligntool. The new alignment tool, shown below, adds batch processing of files, including the processing of eye and ball motion if those data are present.  A region-of-interest (ROI) can optionally be selected manually or automatically.  For file entries where manual selection was specified, the program will stop and present a rectangle on the screen for the user to specify the ROI.  Typically, automatic ROI works fine, and it does not require the user to stand by the computer to specify the ROI each time a new file starts to process.

aligntool

As the files are aligned, the Status column and Status message will display the progress. The alignment procedure can also be visualized by clicking the Live update checkbox, which will display the mean of the entire image stack as the process moves along.  Pan and Zoom buttons allow the user to inspect details in the live image, such as fine branches, as the system is carrying out the alignment. This tool performs rigid alignment and the result is stored in a *_rigid.sbx file.  The original data is left untouched. The tool can align images relatively fast (about 65 frames/sec in my computer), but it will take a few minutes to compute the reference image if the sequence is 15 min or more (please be patient). Alignment improves with the number of passes requested.  Usually one pass is very good, but you can try two or more passes by changing the appropriate entry in the column. The alignment algorithm has been improved.

sbxsegmenttool. The segmentation tool works in a similar way as before. After loading the aligned *_rigid.sbx file, it will display the correlation map.  Segmentation then proceeds as in the previous version.

segmenttool

Once a number of cells are selected, you must save the segmentation and then extract the signals by pressing the corresponding buttons. After the signals are extracted you can select a cell with the pull down menu on the bottom left and the traces corresponding to that cell (now highlighted in green) will be displayed.  The blue trace represents the average signal within the cell, the gray trace is the neuropil, and the trace is the estimated spike rate using the Vanilla algorithm with parameters optimized for GCaMP6f.

Improvements include an Undo button, which will remove the last cell segmented. The ability to load a previous segmentation (it will load automatically after you select the *_rigid.sbx file), to continue adding cells to it.  The ability to define a ROI in the correlation map to automatically increase the contrast of the correlation map as the most salient cells are selected. A zoomed version of the mean image on the right to go along with the correlation map.  And the tool now saves the neuropil and deconvolved signal as well.

Give these tools a try. Report back any suggestions for improvements or problems you encounter.

Sbxsegmenttool: A simple GUI for off-line segmentation

Sbxsegmenttool is a simple GUI that replicates the same mechanism used in Scanbox’s online segmentation to assist in the segmentation of cells and/or processes in data that has already been collected.

Sbxsegmenttool expects the images to have already been rigidly aligned.  After loading an *.align file, the mean mean aligned image will be shown momentarily and then replaced by a computed correlation map.

Then, as the cursor is moved over this image the computer will display which pixels, across the entire image, are correlated with the one under the cursor above a certain threshold. Those pixels that exceed a given threshold are shown by a green pixel.  The value of the threshold can be changed up or down by moving the wheel on the mouse.  The higher the threshold, the fewer green pixels that will show up in the image.

Finally, all the green pixels that form a connected component and include the pixel under the cursor are shown in blue. These group pixels are potential candidates for defining a ROI. Once you have a connected component defined which you find acceptable, click the left mouse button.

Once a ROI is defined the image in the red channel will be re-scaled automatically to expose areas of lower correlation in other parts of the image while ignoring those in the already defined ROIs.

The fact that the correlations are computed for all pixels in the image allows one to also highlight extended processes where signals are correlated, as shown in the image below.

segtool1

Once you are satisfied with the ROIs selected click Save to store them in a corresponding *.segment file, which can then be used to extract signals via sbxpullsignals().

Note that the correlation map will highlight cells whose activity change over time.  Many cells which are labeled but not responding are not going to be picked up.

This is a beta version of the tool and we will be adding more features as users provide feedback. Play with it and let us know what features you would like to see added.

 

Online cell segmentation in Scanbox

There is one hidden feature of Scanbox that has been around for some time, but I have not yet described. This feature allows for computer-assisted segmentation during an experiment.

As you must already know, one way in which region of interests (ROIs) can be defined is manually.  This process can be initiated by clicking on the Add button within the real-time processing panel.  This allows you to trace the boundary of cells manually and define a ROI to calculate mean real-time signal during an experiment.

A faster, more accurate, computer-assisted method also exists.  To use this method you first image the tissue for some minimum amount time.  Scanbox will log some of the incoming frames to the GPU for subsequent processing.  The maximum number of frames logged and the interval between them are determined by two variables in the configuration file:

sbconfig.gpu_pages = 250;
sbconfig.gpu_interval = 10;

The default values are to collect 250 frames total spaced 10 apart. During imaging, the number of GPU pages logged by the system is shown in real time on the bottom-left counter within the window. When the system reaches the maximum number of frames to be logged it will stop streaming them to the GPU (but it will keep collecting data). The subset of data streamed to the GPU serves as the dataset used to provide the type of assisted segmentation described next.

Once scanning is stopped you can click on the Segment button within the real-time panel.  This puts Scanbox into segmentation mode. The first thing it will do is to show the correlation map image in as varying intensity in the red channel.  This image corresponds to the average temporal correlation of each pixel with those within a 3×3 neighborhood, and it provides a quick way to see where potential cells for segmentation are located.

As the computer mouse is moved over this image Scanbox will display which pixels, in the entire image, are correlated with the one under the cursor above a certain threshold.  Those pixels that exceed a given threshold are shown by a green pixel.  The value of the threshold is shown on the bottom left corner of the image, and it can be readily changed up or down by moving the wheel on the mouse.  The higher the threshold, the fewer green pixels that will show up in the image.

Finally, all the green pixels that form a connected component and include the pixel under the cursor are shown in blue. These group pixels are potential candidates for defining a ROI.  As you move the cursor around in the image, all these calculations take place in real-time on the GPU and the display is updated. Once you have a connected component defined which you find acceptable, click the left mouse button.  The region will be highlighted with a red outline and it will be added as an ROI with the real-time list panel.  You can continue with this process until you defined as many ROIs as you need.  The process ends by clicking the Segment button again, which exists segmentation mode.

The ROIs defined by this computer assisted method behave exactly the same way as the ones defined manually.  In fact, you can have a mixture of the two within an experiment.

[vimeo 134891586 w=500&h=264]

The video above shows a quick example of how the process takes place.  Of course, if you plan on using this technique it helps to have already asked Scanbox to stabilize the images.  The GPU stack would then consist of the stabilized dataset, which provides a better start point for the definition of ROIs.  Moreover, you will want to keep stabilizing the images during actual data collection, as you don’t want the ROIs to become misaligned with the incoming stream.