Image Registration Workshop

Ella Bahry

Sep 24, 2024

What is Image Registration?

It’s an essential step for comparing or integrating data in many scientific fields.

Multimodal:

Stitching:

Stack:

Viewpoint:

Temporal:

Microscopy: Aligning slices in a 3D stack, channels, runs, time points, tiles (stitching), and modalities.
Medical Imaging: Viewpoints, stacks, normalization to an atlas, co-registering images from different modalities (e.g., MRI, CT).
Earth Science: Georeferencing, integration from different sensors, aligning satellite images for change detection.
Material Science: Comparing material properties under varying conditions.

When we transform an image, we need to estimate pixel values at the new coordinates.
If for example, you transform an image by up-scaling it:

Interpolation is the process of estimating pixel values at non-integer coordinates.

When down-sampling an image, aliasing artifacts can occur, thus applying anti-alising filters can help to reduce these artifacts.
But, anti-aliasing filters can also blur the image, so it’s a trade-off between sharpness and aliasing artifacts.

Intensity-Based Registration
- Iterative process that optimizes aligned pixel intensity similarities (e.g. correlation coefficient or MSE).
Mutual Information-Based Registration
- Iteratively aligns multimodal images by maximizing the statistical relationship between them.
Frequency Domain Methods
- Transforms images into the Fourier space to compute alignment transformations.
Deep Learning-Based Registration
- Uses neural networks to predict transformations from image data, learning complex patterns.

Looking at the equation, the more structure we have in the joint histogram, the lower its entropy, and thus the mutual information is higher.

two main steps:

Detecting and matching similarities: identifying corresponding regions or features
- Feature-Based Registration (SIFT, SURT, ORB, BRISK, FAST)
- Segmentation-Based Registration
- Model Fitting
- Graph-Based Methods
Estimating and applying transformations: Finding and applying the optimal transformation

Detecting Similarities:
- Feature Detection: Detect keypoints and their descriptors (e.g. using SIFT)
- Feature Matching: Match features between images to select keypoints to use.
Estimating and Applying Transformations: one image is transformed in space to match the other
- Transformation Estimation: Compute transformation matrix (e.g. affine) using matched keypoints.
- Warping: Apply transformation to align images.

Predefined Feature Detection: e.g. pose estimation.
- Manual selection of features
- Annotation of training data
- Deep learning landmark detection
  - Model selection
  - training
  - prediction of landmark locations on all images of the dataset
Estimating and Applying Transformations using the detected landmarks.

wing landmarks model

Method Selection:
- Match image type (e.g., multimodal) to appropriate method
Transformation Type:
- Fit transformation to deformation (e.g., rigid vs. non-rigid)
Preprocessing:
- Denoising, intensity correction, rescaling, applying filters
- In hard cases - Use extrinsic information (e.g., physical landmarks)

Fiji/ImageJ
- Popular plugins: Feature Extraction, Warpy (QPath), TrakEM2, Register Virtual Stack Slices
Python Libraries
- OpenCV (C++), scikit-image
Elastix
- ITKElastix (C++) is a powerful open-source tool (standalone or as a python package) for intensity-based registration.
DeepLabCut
- Open-source deep learning based pose estimation and model based feature detection (and tracking).

VoltRon: R package that includes an interactive GUI for image registration.
https://bioinformatics.mdc-berlin.de/VoltRon/index.html
SIFT based image registration Python package:
https://gitlab.com/ida-mdc/image-registration-tool

Presentation template: Deborah Schmidt - 3d Data Visualization Workshop