optotune

Sampling on a Surface

Not long ago I mentioned Scanbox’s new ability to sample on a surface. Now you can access this new feature in the GUI by navigating to the Surface Sampling panel.

Surface sampling allows you to link lines of the resonant scan to depths determined by the optotune setting.  In other words, it allows you to sample on a surface along the galvo axis (the vertical axis in Scanbox).  Of course, limits are imposed by the range of the optotune and its dynamical response.

Below is an example of how the process works.  Here, I am imaging a slide of pollen grains that is tilted along the vertical axis.

Because the slide is tilted, different settings of the optotune bring the grains in different lines into focus, as shown at the beginning of the video.

To compensate for the tilt we can establish a link between lines in the scan and depth. To do this, change the optotune setting while focusing, then hit the Link button, and then click on the grains that are in focus.  In this example, I repeat this a handful (3) of times.

Now, when the Enable button is clicked, Scanbox interpolates a value of the optotune for each line given the established links and uploads the resulting values to the Scanbox firmware.  When we image the slide with the link active we see all the grains in focus. In other words, Scanbox is now sampling on a slanted surface and compensating for the tilt of the slide.

This is a useful feature to use when compensating for the curvature of a structure that is being imaged or tilting the imaging plane without physically tilting the objective.

Try it and let me know if you run into any problems.

The use of this feature requires an update of Scanbox and the firmware to version 4.0.

 

Processing Volumetric Data

After collecting volumetric data with the Optotune you can now process your data as follows.

First, use sbxsplit() to generate separate data files for each “slice” of the optotune.

For example, if I have a data file gn7_008_001.sbx collected some data with an optotune waveform having a period of 5 the command

>> sbxsplit('gn7_008_001')

will generate a set of 5 files named gn7_008_001_ot_NNN, where NNN.sbx will range from 000 to 004, corresponding to each separate optical plane.

Second, the resulting files can then be aligned and segmented by treating each plane individually.  This will generate the corresponding *.signals and *.segment files.

Finally, you can call:

>> sbxmerge('gn7_008_001')

This will generate gn7_008_001_merged.signals and gn7_008_001_merged.segment.

The signals matrix will have as many rows as frames were present during acquisition while interpolating the missing samples for each plane (that is, when the optotune was sampling from other planes).

The interpolated signals are then deconvolved as usual to generate an estimate of spiking in the spks matrix.  From here on you can process the data as if it came from a typical experiment where only one plane was sampled.

The mask variable in the segmented file will have a size of [ny nx plane], where [ny, nx] is the size of each frame and plane is the period of the optotune waveform.  Each cell has a unique ID value corresponding to its column in the signals matrix.

sbxmerge.001

Note that each setting of the optotune waveform is treated independently, even though thay may potentially represent the same plane (as it may happen using sinusoidal or triangular z-scanning).

Visualizing Individual Slices during Volumetic Imaging

During volumetric imaging, Scanbox displays all images as they are acquired. This can be inconvenient if we are trying to visually assess the activity of neurons within any one optical slice.  One solution is to display the images separately in a montage by means of a Scanbox plug-in.  However, it would be better to have such an option integrated into Scanbox.  A new feature offers this possibility.

Turning on a  “Slice” checkbox within the Optotune panel will make a pull-down menu within the display appear.  This menu allows selection of the optical slice you want to visualize. Unchecking the slice checkbox allows Scanbox to go back to its normal operation of showing all images in the incoming stream.

Here is a brief demo showing this feature.  Enjoy…

 

 

 

Automatic Optotune Calibration

We previously explained how to calibrate the optotune manually.  With the introduction of Knobby 2, we are able to make this process automatic.  You will now find a ‘Calibration’ button in the Optotune panel.  To use it, do the following:

  1. Set the optotune slider to its lowest value (slide all the way down)
  2. Bring some pollen grains into focus
  3. Stop focusing
  4. Make sure the data directory has a directory named xx0
  5. Click the ‘Calibrate’ button
  6. Sit back and relax.  Wait for the process to complete.

Knobby will run some z-stack acquisitions for different values of the optotune current setting.  The volumetric data will be used to calculate the shift in z at various values of the current. A panel will display the progress in processing the images (it takes about 2 min). Scanbox will then plot the raw data and a fit by a quadratic polynomial, which may look something like this:

opto_calib

Scanbox will write a calibration file which will take effect next time you start Scanbox.

After restarting Scanbox, you can check the calibration as follows:

  1. Set the optotune slider to its lowest value (slide all the way down)
  2. Set Knobby to super-fine mode
  3. Focus on some pollen grains
  4. Zero Knobby (XYZ)
  5. Move out-of-focus by moving the optotune slider up to some value
  6. Now bring sample back into focus using the z-axis knob
  7. Compare the reading of the z-axis in Knobby’s screen with the depth noted in the optotune panel. These two numbers should match very closely.

 

Surface sampling in Scanbox

Answering a request from colleagues in London, the upcoming version of the Scanbox firmware will allow users to change the depth of imaging on a line-by-line basis.

Up to now, during volumetric imaging, users were allowed to change the setting of the electronically tunable lens (ETL) once per frame (during the transition from one frame to another).

The new version offers the possibility of changing the depth (z) as a function of the line number (y).  For example, the image of a pollen grain below, was obtained using a sinusoidal modulation in depth.

capture-1

There are at least two potential uses for this new feature.

First, one can increase the yield of imaged cells by first measuring a z-stack and then designing a (smooth) sampling surface z = f(y) that maximizes the number of cells that can be imaged.

For example, assume the projection of cell bodies within the volume is given by the scatter-plot below. Then sampling with the solid curve line will yield many more cells (red circles) than sampling with any horizontal plane (such as the dashed horizontal lines).

sampling

I will explain how to calculate an optimal (smooth) function of depth that maximizes the number of cells sampled in a separate post (the algorithm is too large to fit in the margin here.) [Actually, I added it below].

The surface sampling method offers a compromise between random access and volumetric imaging across planes, many of which may not contain many cells, thereby reducing the temporal resolution unnecessarily.  Have any groups done something like this before?

The second use of this new feature is that it can allow us to correct for the ringing in ETL focal plane that results from a step change.

If you have been doing volumetric imaging, where N frames are sampled at each depth, you might have realized that a few of the lines at the top of the frame on the first frame after a depth transition is screwy.  This is due to the ringing of the ETL. One can potentially use the ability to change ETL command on the fly to compensate for this ringing.

I have not yet implemented this.  Are there any Scanbox users doing volumetric imaging that are willing to help?

Algorithm: 

Given a set of cell body projections (y_i, z_i) we want to find a smooth function z=f(y) that intersects as many cell bodies as  possible.

We do this by restricting z(y) to be sum of the first N harmonics, z(y) =a_0+\sum_{k=1}^N a_k \cos ( 2 \pi k y + \phi_k ), so it is both smooth and periodic as well, which will prevent ringing in the ETL during fly back to the first line. The function has 2N+1 parameters.  For simplicity, here we normalize the total number of lines in the frame is normalized to be in the range [0,2 \pi].

Given a set of parameters, denote by d_i the minimum distance between the location of cell i and any of the points on the curve z(y). Our objective function is J= \sum_{k=1}^M \tanh ( (d_i - r_0)  \beta ).  Here, r_0 represents the average radius of a cell and \beta controls the sharpness of the error function near that boundary, M is the total number of cells in the volume. For a give sent of points, we used Matlab’s fminsearch to find the optimal parameters for the curve.

How much improvement in yield can we expect using surface sampling versus just one plane?

We ran a few simulations where the number of points is uniformly distributed within the volume and calculated the fraction of cells we can intersect as a function of the number of harmonics used. Zero harmonics means just the horizontal plane that intersects the maximum number of cells.yield  The graph on the left shows the results.

Even in the simple case of a uniform distribution one can more than double the yield expected from a single plane by having a few (~5-6) harmonics. In other words, using surface sampling we can double the number of cells with respect to a horizontal plane without any penalty in temporal resolution.  Not bad at all.

We will re-do this analysis with some actual volumetric data from our Lab soon. I suspect this estimate represents a lower bound on what can actually be achieved.

 

A montage display for real-time display of volumetric data

 

By default, Scanbox displays the incoming image stream on its main window. Thus, during volumetric scanning, one sees the incoming images as depth is changing over time.  If one is imaging only a handful of optical planes, it is difficult to see what is really going on.

A different way to visualize the data in such recordings is to have a montage showing the different optical sections separately, each being updated as new data becomes available. Here, we offer a plug-in for Scanbox that implements this mode of visualization.

As discussed in previous examples, Scanbox shares date with other processing by means of memory-mapped files. A header at the beginning of the file provides a mechanism for exclusive access to the data via a semaphore and exposes basic information about the data, such as the size of the images, the frame # being shared at any one time, and the period (in frames) of the volumetric scanning waveform, among other information.

Such data, along with a readily available montage function from Matlab, allows one to easily display the data as separate optical planes during acquisition.

Here is the code:


% Plug-in Demo: Display optical sections separately

close all; % Close all open figs

% Open memory mapped file -- define just the header first

mmfile = memmapfile('scanbox.mmap','Writable',true, ...
    'Format', { 'int16' [1 16] 'header' } , 'Repeat', 1);
flag = 1;

% Process all incoming frames until Scanbox stops

while(true)
    
    while(mmfile.Data.header(1)<0) % wait for a new frame...
        if(mmfile.Data.header(1) == -2) % exit if Scanbox stopped
            return;
        end
    end
        
    if(flag) % first time? Format chA according to lines/columns in data
        mmfile.Format = {'int16' [1 16] 'header' ; ...
            'uint16' double([mmfile.Data.header(2) mmfile.Data.header(3)]) 'chA'};
        mchA = double(intmax('uint16')-mmfile.Data.chA);
        flag = 0;
        nplanes = mmfile.Data.header(6);
        I = zeros([size(mchA) 1 nplanes]);
        I(:,:,1,1) = mchA;
        imaqmontage(I);
        axis off;           % remove axis
        colormap gray;      % use gray colormap
    else
        I(:,:,1,mod(mmfile.Data.header(1),nplanes)+1) = double(intmax('uint16')-mmfile.Data.chA);
        imaqmontage(I);
    end
    
    mmfile.Data.header(1) = -1; % signal Scanbox that frame has been consumed!
    drawnow limitrate;
    
end

clear(mmfile); % close the memory mapped file
close all;     % close all figures

And here are, side-by-side, the result of viewing the incoming stream in the main Scanbox window and in the volumetric plug-in.

Calibrating the Optotune

If you have been using the Optotune, you must have noticed that the values displayed within the optotune panel for the setting of the slider and the parameters of the z-scanning waveform are unit-less and not very useful.

The latest update to Scanbox allows you to calibrate the Optotune and have all the units within the Optotune panel in micrometers.

A new configuration variable “optocal” in scanbox_config.m can be either empty or contain calibration data.  If optocal is empty, Scanbox behaves as it has done up to now — showing the unit-less values.

Start by setting optocal=[].

To calibrate simply take a green fluorescent slide and make a small marking on its surface.  Small dots with a fine, black sharpie should do.  Focus the microscope near the very top of the slide and pick one feature with the optotune slider set at zero.  Now, zero the position counters.  Our first point, corresponding to pairs of (values, depth) is (0,0).

Move the optotune slider up to a value near 200.  Then, compensate by lowering the objective (z-position) to bring back the feature you selected into focus. Once this is done, write down your next pair (value, depth), as the value of the slider and the depth you read in the position panel (or in Knobby). Continue this process, increasing the value of the optotune slider in steps of ~200, up to a value of ~2000.

Here is the set of measurements I obtained in my setup:

optocal

The curve saturates at a value of ~1760. This is because the current source has reached its maximum output voltage. The range of depths spanned by the optotune is ~340um (vertical axis). The smooth rising part of the curve, before saturation, is well approximated by a second-order polynomial (red curve).  To obtain the values of the coefficients use Matlab’s polyfit function by calling “polyfit(vals,depth,2)” (don’t include the saturating part of the curve!).  The coefficients of these polynomial are what the optocal variable should be set to.  In my case, I get the optimal coefficients to be [0.0001  0.0732 1.3162] and, therefore, I modified the config file so that optocal = [0.0001  0.0732  1.3162].

optotuneum

Once the optocal variable is set, you can restart Scanbox.  The result should be that all the values within the Optotune panel will read in microns (as shown on the left). This includes the slider and the parameters to the waveforms.  Moreover, as the function is slightly nonlinear, the actual waveforms will be linearized accordingly — so a linear ramp should very closely approximate a linear change in depth.

 

Quick start guide to the Optotune and volumetric scanning

Scanbox uses an on-board current source to control the Optotune lens, so that changes in depth are synchronized to the beginning of each microscope frame.

Once the lens is installed, the easiest way to verify its function is to start imaging a sample and move the slider in the Optotune panel to change the plane of focus.  The value of the slider can be read at the bottom, ranging from 0 to 4095.  In the example below, the slider is set at 1879. (*)

ot_panel

To image a volume at approximately constant intensity Scanbox allows linking the power of the laser to different settings of the focal distance.

Start by moving the slider to the top of your imaging volume and adjust the laser power to the desired level.  By pressing the Link button on the right of the slider you establish a link between the current depth and the power level.

Repeat the process by moving the slider to focus at lower depths, which will probably require you to increase the laser power to achieve the same intensity as before.  It is typically sufficient to define 3-5 points bracketing the range of depths you will image.  Scanbox linearly interpolates among these points to cover the entire range.

Finally, to activate the link between focal plane and laser power click the Enable checkbox right below the “Link” button.  If at any point you want to define a new link between depth and power you must first clear the existing table by clicking on Clear.

Once a link between focal point and laser power is activated it will be automatically used by the system.  Try imaging a volume and change the depth using the optotune slider.  You should see the power of the laser change accordingly on the fly while keeping an approximately uniform illumination. If this is the case, you are in a position to start your volumetric imaging.

Finally, you can use the controls on the right of the panel to choose a z-scanning method.  The three boxes define the minimum value of the current, the maximum value, and its period in microscope frames. The pull-down menu allows you to switch among different waveforms: square, triangular, sinusoidal and saw-tooth.  Once you finished defining the parameters of the waveform click the upload button to send it to the Scanbox card then check Enabled in the check-box below to make the waveform active.

Now you are ready to go!

If you start scanning you should see the system scanning the desired volume, at approximate intensity throughout, and with the waveform selection you used.

You can use the Optotune in both unidirectional and bidirectional scanning.

All the optotune parameters used to collect data are stored in the info structure saved by Scanbox.

The video below illustrates the process described above:

(*) Presently the slider saturates at around 3000 due to the maximum current the present version of the board can generate (this will be corrected in future version of the board.)

 

 

Simultaneous control of an electrically tunable lens and a pockels cell

We are preparing Scanbox for fast, volumetric scanning, by providing simultaneous control of an electrically tunable lens (to focus on different planes) and a Pockels cell (to control laser power).

The Scanbox card has an integrated, 12-bit current source used to control the focus tunable lens. Similarly, an integrated DAC on the Scanbox card controls laser power via a Pockels cell. The mechanism implemented still allows for user-defined blanking of the margins.  Arbitrary tables can be uploaded from the Scanbox GUI to the card to allow for simultaneous changes in these variables that are precisely synchronized to the frames of the microscope.

Below is a demonstration showing a sinusoidal change in focusing along with matching changes in laser power.  Fast scanning with synchronized depth/power control is coming to Yeti soon!

[vimeo 119605807 w=500&h=280]

New features in latest Scanbox firmware

To take advantage of the new features in the latest release of the Yeti firmware you should start by getting rid of the half-wave plate control of the laser power. To do so start by removing the motor and half-wave plate from the optical path feeding the laser beam directly into the Pockels cell as shown in the picture below.  Make use of a detector card and/or power meter to check for alignment.
pockels

 

Make sure the Pockels cell is aligned so that there is minimum transmission when the reading in the front panel is zero.  The power of the laser in this new arrangement is controlled by a slider in the Chameleon panel, which provides control from 0% to 100% of the power.

powercontrol

The software takes into account the fact that at 920nm the relationship between power and control voltage is non-linear, so you should get an approximate linear control of the power using the slider.  This new feature provides for an absolute control of the power across experiments (as compared to the the relative position of the wave-plate in the original design).  You should also get increased power, as the new setup avoid loses via multiple reflections.

Using the new firmware you also have control over the size of the dead-band at the edges of the scanning.  A new variable in  the configuration file, sbconfig.deadband, can be changed by the user.  The two numbers of the vector reflect the size of the dead-band at the left and right margins of the scan in percent of the total number of samples before spatial correction.

The reason for this changes is that this new setup also allows the software to rapidly change the power, a feature that will be needed when the Optotune lens is incorporated for fast z-scanning in the near future.  If you plan to use the optotune, you should be upgrading to this new firmware as soon as possible.

Updating the firmware

The lastest firmware is located in the Scanbox/drivers/ subdirectory.

To update the driver start the Bootloader Host application (look under the Cypress Programs Folder).

Select the new Firmware file you want to upload (the one in the drivers/ directory), power cycle Scanbox to get it into boot loader mode.  Once you turn the box on you will have 20 sec to upload the new firmware (once the time elapses Scanbox will start running the existing firmware).

Select the USB Human Interface Device that will appear in the ports panel, and click the download button (the one highlighted in blue).

If everything works you will get a message that the firmware was successfully uploaded (it should take 3-4 seconds to complete).  Once the new software is uploaded Scanbox should be ready to use.

bootloader

If you have trouble with the  firmware updating process just let me know…

Note: You can download the PSoC suite from here but make sure you select “Custom” installation (instead of “Typical”) and select  the “bootloader host” only for installation (otherwise you will be installing the whole suite and it will take a very large amount of time/space).