# Virtual Knobby

Happy new year! We have plenty of exciting Scanbox developments happening this year, so stay tuned to the blog.  You don’t want to miss anything!

We recently introduced a wireless version of knobby that runs on Android tablets. The same software is now available to run on Windows, side-by-side your Scanbox application.

The controls and behavior are identical to the tablet version.  To use virtual knobby simply set the tri_knob configuration variable to “127.0.0.1”.

After launching Scanbox from Matlab go to the yeti/knoby_virtual/ directory and launch the knobby_virtual.exe application.  That’s all…  Go ahead, give it a try!

So you now have three options for position control: classic knobby, knobby tablet and virtual knobby.

Note also that if you are a user of classic knobby and run into some issues (like a rotary encoder going bad) you can always use virtual knobby as an emergency replacement while the hardware version gets fixed.  So no more downtime for a broken knobby.

# Surface sampling in Scanbox

Answering a request from colleagues in London, the upcoming version of the Scanbox firmware will allow users to change the depth of imaging on a line-by-line basis.

Up to now, during volumetric imaging, users were allowed to change the setting of the electronically tunable lens (ETL) once per frame (during the transition from one frame to another).

The new version offers the possibility of changing the depth ($z$) as a function of the line number ($y$).  For example, the image of a pollen grain below, was obtained using a sinusoidal modulation in depth.

There are at least two potential uses for this new feature.

First, one can increase the yield of imaged cells by first measuring a z-stack and then designing a (smooth) sampling surface $z = f(y)$ that maximizes the number of cells that can be imaged.

For example, assume the projection of cell bodies within the volume is given by the scatter-plot below. Then sampling with the solid curve line will yield many more cells (red circles) than sampling with any horizontal plane (such as the dashed horizontal lines).

I will explain how to calculate an optimal (smooth) function of depth that maximizes the number of cells sampled in a separate post (the algorithm is too large to fit in the margin here.) [Actually, I added it below].

The surface sampling method offers a compromise between random access and volumetric imaging across planes, many of which may not contain many cells, thereby reducing the temporal resolution unnecessarily.  Have any groups done something like this before?

The second use of this new feature is that it can allow us to correct for the ringing in ETL focal plane that results from a step change.

If you have been doing volumetric imaging, where N frames are sampled at each depth, you might have realized that a few of the lines at the top of the frame on the first frame after a depth transition is screwy.  This is due to the ringing of the ETL. One can potentially use the ability to change ETL command on the fly to compensate for this ringing.

I have not yet implemented this.  Are there any Scanbox users doing volumetric imaging that are willing to help?

Algorithm:

Given a set of cell body projections $(y_i, z_i)$ we want to find a smooth function $z=f(y)$ that intersects as many cell bodies as  possible.

We do this by restricting $z(y)$ to be sum of the first $N$ harmonics, $z(y) =a_0+\sum_{k=1}^N a_k \cos ( 2 \pi k y + \phi_k )$, so it is both smooth and periodic as well, which will prevent ringing in the ETL during fly back to the first line. The function has $2N+1$ parameters.  For simplicity, here we normalize the total number of lines in the frame is normalized to be in the range $[0,2 \pi]$.

Given a set of parameters, denote by $d_i$ the minimum distance between the location of cell $i$ and any of the points on the curve $z(y)$. Our objective function is $J= \sum_{k=1}^M \tanh ( (d_i - r_0) \beta )$.  Here, $r_0$ represents the average radius of a cell and $\beta$ controls the sharpness of the error function near that boundary, $M$ is the total number of cells in the volume. For a give sent of points, we used Matlab’s fminsearch to find the optimal parameters for the curve.

How much improvement in yield can we expect using surface sampling versus just one plane?

We ran a few simulations where the number of points is uniformly distributed within the volume and calculated the fraction of cells we can intersect as a function of the number of harmonics used. Zero harmonics means just the horizontal plane that intersects the maximum number of cells.  The graph on the left shows the results.

Even in the simple case of a uniform distribution one can more than double the yield expected from a single plane by having a few (~5-6) harmonics. In other words, using surface sampling we can double the number of cells with respect to a horizontal plane without any penalty in temporal resolution.  Not bad at all.

We will re-do this analysis with some actual volumetric data from our Lab soon. I suspect this estimate represents a lower bound on what can actually be achieved.

# Disabling Knobby’s automatic reset

[Note: please do not use this feature yet — there are some bugs that need to be worked out. I will keep you posted.]

Up to now starting a new Scanbox session from Matlab causes knobby will auto-reset, bringing all the position counters back to zero and clearing all its position memories. This is done because there is no way for Scanbox to know if the microscope stages were moved manually from its last position (there are no absolute encoders), or if the motor box was powered off, since its last session.

Some users have asked for the option of not resetting Knobby between Scanbox sessions under the assumption that neither the stages nor the motor box have experienced  any changes between runs.  This would preserve the position counters and memories.

You can now obtain this behavior by using the knobbyreset configuration variable in the scanbox_config.m file.  Setting the variable to one will force automatic reset when starting a new Scanbox session (the default behavior up to now); a value of zero will disable the automatic reset.

The feature is now in beta testing so users should make sure everything works as expected before running new experiments with this feature turned on.  Try storing and retrieving memories across sessions, zeroing, and so on.  If you see inconsistent behavior across sessions let me know.

You can use this feature only with the wired version of knobby.  Knobby tablet does not yet support this feature.

# New Features in Scanbox 3.4

The latest release adds a few new features and fixes.

A new panel called “Deadband & Alignment” offers the option to change some of the configuration variables on the fly.

The deadband sliders allow to dynamically change the size of the left and right margins respectively, where the laser is blanked out.  You can optionally click the save button to save the settings at the end of the configuration file for future use.

Within this panel there is also the option of adjusting the alignment of even/odd lines during bidirectional scanning on the fly.  To use this first make sure the resfreq and lasfreq configuration variables are set to the correct values of the resonant and laser frequencies.

Typically the resonant frequency will be around 7930Hz, while the laser frequency, measured at 920nm wavelength, will be slightly over 80MHz.  Try to measure these values are accurately as possible.

The ncolbi configuration variable is now gone from this and any future versions. Only bishift is used at each magnification to adjust the line alignment, which can now be positive or negative.

To calibrate, pick a magnification, set the scope in continuous resonant mode for a few minutes, and start scanning.  You can now dynamically adjust the alignment by clicking on the << and >> buttons.  Using pollen grains is ideal for this step.

To have a closer look at the alignment you can digitally zoom into some features of the image by clicking Zoom in the navigation panel and then zooming into the region of interest.

The adjustment will look something like this:

Once satisfied with the adjustment you can stop scanning, move to another zoom setting, and repeat the process.  At any point you can hit the Save button to the right.  This will append a line with the new bishift settings to the end of the configuration file (effectively overriding the previous setting).

In the previous release a rounding bug caused some pixels to appear displaced in the image during bidirectional scanning.  This has now been fixed. The new version should allow you to get a very good results. Here is an example of a 40 LPMM Ronchi at a magnification of 1 and digitally zooming in to see details:

Zooming in…

And zooming in…

And zooming in again (you can click on the images to get them in full size):

If you have trouble with bidirectional alignment let me know.

Make sure the HSYNC signal is symmetric and approximately in-phase before starting the calibration process as described in the first section here.

Note: you should not change magnifications on the fly during bidirectional scanning. You need to stop the scope, change magnification, and start scanning again.  The next version will allow you to do this.

Another feature in the new version is the ability to change the warm up time for the resonant mirror.  This is the time the resonant is turned on before the scan begins.  The default is 500 ms but can be changed with the wdelay configuration variable, which dictates the delay in units of tens of msec (so 50 = 500 ms).

Finally, you will see some new controls under the Laser panel.  These are additional controls to control the Coherent Discovery laser.  They provide control of the shutter for the fixed laser and the GDD slider for the pre-compensator.  To enable these controls the laser_type should be set to ‘DISCOVERY’ — otherwise they will be grayed out.

# Installing knobby tablet

Before you start you will need an android tablet and a wireless router in the Lab.

I have a Samsung Galaxy Tab E and an Airport Express, but other tablets and routers should work as well.

To install knobby tablet follow the following steps:

3. Download and install PyOSC.  To install, open a terminal, go to the directory where you uncompressed PyOSC, and type “python setup.py install”.
4. Copy over the settings in the scanbox_config.m file.
5. Connect the computer running Scanbox to your local wireless router
6. Open a terminal window and check the IP assigned to the computer by typing ifconfig.
7. Set the tri_knob configuration variable in scanbox_config.m to a string that contains the IP address, for example, ‘164.34.123.31’.
8. Start Scanbox.  You should also get a command window with the knobby tablet console.  We will check if commands from the tablet are received there.
9. Open knobby tablet on your Android.  You will likely get a message saying that the default IP is invalid.  Click Ok.
10. Click on the IP number button.  A keyboard will show up.  Enter the IP number you found above.  However, make sure that every section has 3 digits (use zeros if necessary).  Thus, in the example above, one would type 164.034.123.031
11. If the tablet cannot connect, you will see an “Invalid IP” message.  Otherwise, you should be ready to go.  The IP gets stored in the tablet, so launching knobby tablet later will retrieve it.
12. Remove the objective from the microscope (just in case).
13. Make sure the scope is roughly in the middle of its travel for all axes.
14. Click the “Normal” button on the tablet.  Do you see a message printed on the knobby console?  If so, communication is working Ok.  If not, something is wrong.  Contact me.
15. Stand close to the motor box so you can quickly power it off if something does not work as expected.
16.  Now touch the button corresponding to the Z axis on the red region.  The z-axis will begin to move so long as you keep touching it.  The speed of movement will be faster the farther away you are from the center.  Slide your finger left and right, the speed will change and it should reverse if you move over to the blue side.
17. Test the other axes.
18. Touch “Velocity” to switch to velocity mode.
19. Warning: the microscope will move fast now…  Touch the red or blue areas of each button.  The microscope will move at a fast/fixed speed so long as you keep touching the area.  It will stop when you release.
20. Store/Recall buttons work as before
21. Go back and try the Fine and Super-Fine modes.
22. Play with it for a while and install the objective back after you familiarize yourself with the tablet interface and everything seems to be working Ok.
23. You can minimize the knobby console once testing is done (I will do this automatically in future versions).
24. The green buttons on the side lock the button area (bottom) or knob area (top).  So if you are acquiring data and don’t want to have a situation where people touch the tablet by mistake and move the microscope you can use the buttons to lock them.  Touching them again will unlock.
25. Make sure you open knobby on your tablet  after you start Scanbox.

If you run into problems don’t hesitate to write.

# Knobby tablet is on the Google Play Store

You can download it on any Android tablet and give it a try. I will explain how to set things up in detail after I return from SfN.

In the Lab, your tablet will need to be on the same wireless network as the desktop computer running Scanbox.  If that computer does not have a wireless card, I recommend getting this USB adapter.  That’s all you need.

I have had no problem running on Samsung Galaxy Tab or the Acer Iconia, but have not tested others.

# Sub-pixel alignment in bidirectional scanning mode

When imaging at low magnification in bidirectional mode cells can be small, with a diameter of ~6 pixels or so.  In this and other situations the mechanism offer by the ncolbi variable to align even and odd lines may prove too coarse for optimal alignment.

We added a new variable to the configuration file, named bishift, that allows for sub-pixel alignment.  Like ncolbi, this array contains one entry per magnification factor.  At the beginning of your calibration set all the entries in bishift to zero and follow the same procedure as described earlier to find the best ncolbi possible.

The bishift variable allows for sub-pixel shifts only in one direction.  Thus, to use it, we need to begin with the images slightly misaligned in the opposite direction.

As an example, assume that you determined that for magnification factor of 1, the best ncolbi(1) value was 1264. Select a new value by reducing this number by 2, in this case it will result will be 1262.  Now, when you look at the images they will be misaligned, but you can start increasing the value of bishift (which is a non-negative, integer number), to bring them into alignment.

The example below shows zoomed images of a single pollen grain after this misalignment (top left) and increasing values of bishift(1), from 0 to 15. A setting of bishift equal to 7 brings the lines into optimal alignment with sub-pixel accuracy. You can repeat the procedure for each magnification factor.

A change of 4 in bishift is equivalent to a change of 1 unit in ncolbi.  So, you can see that if we are caught in a situation similar to the panel 5 below, where the images are only slightly misaligned, a change in ncolbi by 1 unit will bring it to the situation in panel 9, which is also slightly misaligned but in the opposite direction.  The addition of bishift now allows for finer control.

Perform your calibration of ncolbi/bishift while the mirror is in continuous resonant mode and has been running for at least 5-10 min, to allow the frequency to stabilize.

There is no need to restart Scanbox to change the ncolbi and bishift variables.  To have access to these variables while Scanbox is running simply define a global sbconfig variable in Matlab’s command window and you can now change their values between acquisition runs.

We will soon be modifying the sbautocalbd function to make use of both ncolbi and bishift.

Here is one more example at higher magnification (x4) of a pollen grain: