September 7, 2021
Article by Artur Speiser

Machine Learning Improves
Super-resolution Microscopy

Single-molecule localization microscopy is a powerful method to image cellular structures with nanometer resolution. We developed DECODE, a deep learning based analysis algorithm that makes this technique faster and more precise.

Scientists use super-resolution microscopy to study previously undiscovered cellular worlds. It allows them, for example, to examine subcellular architecture and molecular interactions at the nanoscale. These methods revolutionised light microscopy and earned its inventors the 2014 Nobel Prize in Chemistry.

To understand the impact of this technology, let us first consider the limits of previously existing techniques. Fluorescence microscopy measures the light emitted by so-called fluorophores that are attached to biological structures of interest. Like in other forms of light microscopy, the resolution that can be achieved this way is limited by the physical properties of light itself. When looking at tiny objects, for example a fluorescently labeled molecule, what we observe is a bright spot which is much bigger than the molecule itself. If two such light sources are too close to each other, their images blur into each other and they can no longer be distinguished from each other.

Super-resolution microscopy methods overcome this fundamental limit. One of these methods is single-molecule localization microscopy which circumvents the resolution limit with a simple trick: Instead of activating all fluorophores in the specimen at the same time, only a few fluorophores are randomly activated at each time point, resulting in a series of images with isolated spots that can be localized with high precision. Subsequently, localizations from many such images are combined to reconstruct a super-resolved image.

However, this method has one major drawback: typically, achieving a high localization precision requires the spots to be well separated. Therefore, prior to our development, one needed to make sure that only a few spots were visible in each image, which in turn resulted in long imaging times.

The DECODE algorithm uses deep learning to enable precise localisation under difficult conditions

In a collaborative effort between the labs of Jakob Macke (University of Tübingen), Jonas Ries (EMBL Heidelberg) and Srinivas Turaga (Janelia Research Campus, Virginia, USA), led by Artur Speiser and Lucas-Raphael Müller, we developed an algorithm called DECODE that overcomes this limitation. It accurately detects and localizes fluorophores at high activation densities, which enables up to ten times higher imaging speeds with minimal loss of resolution. Our results have been published in Nature Methods.

FIGURE 1 / DECODE is a novel algorithm that increases the performance in single-molecule localization microscopy experiments. Figure: Jonas Ries

Our algorithm is based on deep learning: We use an artificial neural network which takes camera images as input, and produces predictions of the underlying fluorophore locations (See Figure 1). To make this work, we had to solve several challenges.

Training neural networks requires pairs of inputs and reference labels, i.e. the desired network outputs, in our case the locations of the fluorophores. However, in our application, we do not know where exactly the fluorophores are. We therefore used a method that has recently become popular for training neural networks in such situations, and which the group has also successfully used in other contexts. Rather than training the network on real images, we used synthetic data that we generated by numerical simulation.

By carefully incorporating what we know about the microscope, the camera, and the molecules’ patterns of fluorescence, we achieve a close similarity between real data and simulations. As a result, networks trained to localize fluorophores on simulated data are then also able to do it on real images!

DECODE allows for ten times higher imaging speeds

Furthermore, we developed a network architecture and loss function that enables our network to localize fluorophores with a precision that is close to the theoretical limit. At the same time it produces uncertainty estimates, i.e. it tells us about the quality of each localization. Removing poor localizations can further enhance the final resolution of the image.

As Figure 2 illustrates, we were able to show that DECODE maintains a high resolution, even when working on emitter densities that are around 10 times higher than those usually employed in single-molecule localization microscopy.

FIGURE 2 / We imaged the same biological structure under different conditions, resulting in higher activation densities and in turn in lower imaging times. Even when imaging with 12 times higher speed the resolution achieved by DECODE only slightly deteriorates, while CSpline, another popular single-molecule localization microscopy algorithm optimized for high-density data, produces a far less detailed reconstruction. Figure: Jonas Ries

Remarkably, on a public challenge that uses authentic simulated datasets to benchmark single-molecule localization microscopy software, DECODE outperformed all 39 algorithms in all conditions, often by a substantial margin. This means that DECODE is always the algorithm of choice, not only for particularly dense data.

With DECODE many labs can improve their images

Our work shows how a novel approach to an ostensibly simple problem like the localization of spots can result in considerable advantages. We built a software package which implements the DECODE algorithm and which is available online. We believe that just by upgrading their analysis software to DECODE many labs performing single-molecule localization microscopy can improve their images, even on their existing datasets. More generally, DECODE shows the power of using numerical simulations to train neural networks to solve challenging analysis problems in the natural sciences.

Original publication: Speiser A., Muller L.-R. et al. Deep learning enables fast and dense single-molecule localization with high accuracy. Nat. Methods. (2021).

Text: Artur Speiser
Cover image: COS-7 cell imaged with single-molecule localization microscopy. Reconstruction by Artur Speiser et. al., data provided by Wesley Legant et. al.


I have taken note of the privacy policy.
Latest Research
July 19, 2021 Michael Deistler, Jonathan Oesterle

Identifying Models in
Neuroscience with Machine Learning

Computer models are a great tool to analyze neuronal mechanisms in the brain, but tuning these models to match brain activity has long been a daunting task for scientists. We developed a new machine learning tool that automates this process and used it to develop a simulation environment for a retinal implant.
Latest Research
July 19, 2021 Theresa Authaler

Responsibility Cannot Be Delegated to an Algorithm

Computer Science Professor Ulrike von Luxburg speaks in an interview about the opportunities and challenges of trimming machine learning systems to fairness. Prof. von Luxburg also explains why she is convinced that people, rather than machines should resolve certain questions.