View as Slides

Image Registration Workshop

Ella Bahry
Nov 4, 2026

Helmholtz Imaging

Helmholtz Imaging is here for you with support units at 4 centers

What is Image Registration?

Registration

Spatial alignment of two or more images.

  • It’s an essential step for comparing or integrating data in many scientific fields.

Common needs in research

Image Registration Aim

Image Transformation Types

Link: example_notebooks/2_transformation_examples.ipynb

Local Deformations - Common Approaches

Image Interpolation - Reason

  • When we transform an image, we need to estimate pixel values at the new coordinates.
  • If for example, you transform an image by up-scaling it:
  • Interpolation is the process of estimating pixel values from non-integer coordinates.

Image Interpolation - Common Types


Image by Cmglee, license: CC BY-SA 4.0
Link to interpolation weights and examples notebook: example_notebooks/3_interpolation.ipynb

Image Interpolation Example

Image Interpolation - Anti-Aliasing in Down-Sampling

Image Registration Techniques - Categories

Integrated Image Registration Techniques



  • Intensity-Based Registration
    • Iterative process that optimizes aligned pixel intensity similarities (e.g. correlation coefficient or MSE).

  • Mutual Information-Based Registration
    • Iteratively aligns multimodal images by maximizing the statistical relationship between them.

  • Frequency Domain Methods
    • Transforms images into the Fourier space to compute alignment transformations.

Technique: Intensity Based (Correlation Coefficient)

Technique: Mutual Information

Mutual Information iterations
  • Looking at the equation, the more structure we have in the joint histogram, the lower its entropy, and thus the mutual information is higher.

ITK / ITKElastix - Common tool

  • Elastix / ITKElastix is a powerful open-source tool for intensity-based image registration, widely used in medical imaging and other fields.
  • ANTs is another popular tool (ITK based) for intensity-based registration, known for its advanced algorithms and versatility in handling different image modalities.
    • It’s especially useful for 3D and elastic deformation registration tasks.
    • Originally developed for neuroimaging applications but applicable to other domains as well.
    • Python / R / command line interface.





Technique: Phase Correlation (Frequency Domain)

  • Checks every shift at once (via the Fourier transform): fast, no optimizer, no starting point.
  • Works for similar or overlapping images, e.g. stitching tiles (BigStitcher). Fails on repetitive or low-texture regions.
  • Rotation and scale: possible too, via log-polar remapping (Fourier-Mellin).

2 Step Techniques - Feature Detection & Transformation

two main steps:

  1. Detecting and matching similarities: identifying corresponding regions or features
    • Feature-Based Registration (SIFT, SURT, ORB, BRISK, FAST)
    • Segmentation-Based Registration
    • Model Fitting
    • Graph-Based Methods
  2. Estimating and applying transformations: Finding and applying the optimal transformation

Technique: Feature-Based Registration (SIFT)

  1. Detecting Similarities:
    • Feature Detection: Detect keypoints and their descriptors (e.g. using SIFT)
    • Feature Matching: Match features between images to select keypoints to use.
  2. Estimating and Applying Transformations: one image is transformed in space to match the other
    • Transformation Estimation: Compute transformation matrix (e.g. affine) using matched keypoints.
    • Warping: Apply transformation to align images.
sift keypoints and matches

Technique: Model Based (Pose Estimation)

  1. Predefined Feature Detection: e.g. pose estimation.
    • Manual selection of features
    • Annotation of training data
    • Deep learning landmark detection
      • Model selection
      • training
      • prediction of landmark locations on all images of the dataset
  2. Estimating and Applying Transformations using the detected landmarks.

wing landmarks modelwing landmarks model

Deep Learning Based Image Registration

  • Unlike many vision tasks, deep learning has not replaced classical registration, but it is fast and learns complex deformations from training data.

VoxelMorph

  • CNN that predicts the transformation (2D/3D, affine or elastic)
  • Loss: image similarity + smoothness. Unsupervised (optional: anatomical labels)
  • Needs a large training dataset
  • Very fast, good interpretability
  • Intermediate user level

Not only medical, the same idea is used on seismic surveys (modified VoxelMorph), satellite images and X-ray/neutron images (similar approach).

Choosing an Approach

  • Start global (rigid / affine). Add a local (elastic) step only if deformations remain.
  • Mostly a shift, similar images with overlap (e.g. tile stitching) → phase correlation. Fast, but fails on low-texture or repetitive images.
  • Easier cases: moderate transformation, similar-looking images → intensity-based. Simple metric (MSE, correlation) for the same modality, mutual information also handles multimodal.
  • Harder cases: large transformation, or very different appearance → feature-based. Might fail with no distinct features, or with repetitive ones.
  • Many similar pairs, GPU, need for speed, or a pretrained model exists → deep learning. Needs training data.

More Considerations

  • Preprocess:
    • Denoise, correct intensity, rescale, segment, apply a filter (e.g., edge detector)
    • Hard cases: add extrinsic information while imaging (i.e., physical landmarks)
  • Transformation type:
    • Use a local transformation only if local deformations exist. Each extra step can add error.
  • Apply to additional data:
    • Whichever approach you used, the same transformation can be applied to any additional data in the same space (e.g., the original, un-preprocessed image, other channels, masks, points)

Summary of Software Tools for Image Registration

  • Fiji/ImageJ
    • Popular plugins: Feature Extraction, Warpy (QPath), TrakEM2, Register Virtual Stack Slices
  • Python Libraries
    • OpenCV (C++), scikit-image
  • ITKElastix (C++) is a powerful open-source tool (standalone or as a python package) for intensity-based registration.
  • ANTs Advanced normalization tools (C++) for intensity-based registration - great for local deformations and 3D.
  • VoxelMorph Deep Learning based image registration framework (Python, TensorFlow/Pytorch).
  • DeepLabCut Open-source deep learning based pose estimation and model based feature detection (and tracking).

Registration Validation

  • Visual inspection first: overlay the two images with different colormaps and partial opacity, or flip between the fixed and the registered image. Zoom in on small structures.
  • The optimized metric is not a verdict: validating with the same metric used for the alignment is misleading, since the method already optimized it. Absolute values say little, but comparing runs or steps (before vs. after, affine vs. elastic) is informative.
  • Red flags without ground truth: the image collapsed or blew up (its area changed massively), almost no overlap is left between registered and target, or (feature-based) only a few matched points, all clustered in one region.
  • Create your own ground truth: click matching points in both images by hand. Measure their distance after registration. Don’t use the same points for the registration itself.
  • Elastic transformations: look at the deformation (grid or field). It should be smooth, without folds or extreme stretching.
  • Is the error acceptable for your use case? Decide what accuracy your analysis needs.

Thank You!

Thanks for participating. Please feel free to reach out with any questions.

Contact:     ella.bahry@mdc-berlin.de    support@helmholtz-imaging.de

Workshop available on: github.com/ida-mdc/registration-workshop

License: CC BY 4.0

Thanks to the HIDA team for offering this workshop!
https://www.helmholtz-hida.de/en/

Thanks to Deborah Schmidt for the template!