3D Data Visualization Workshop
Deborah Schmidt
Head of Helmholtz Imaging Support Unit, MDC Berlin
Sep 25, 2024
Introduction
Hi, I’m Deborah, head of the Helmholtz Imaging Support Unit at MDC. Members of all Helmholtz Imaging Support Units.Helmholtz Imaging is here for you with Support Units at 3 centers, working in close collaboration with the Helmholtz Imaging Research Units.
Introduction
Consulting along the entire pipeline and across all research domains
Automation of rendering tasks
Album & the Image Challenges catalog
- Automate visualization tasks: Use Album to manage multiple tools from a single launcher.
3D Dataset types
- Voxel-Based Datasets (Euclidean-structured)
- Meshes (Non-Euclidean-structured)
- Point Clouds (Non-Euclidean-structured)
Examples of different 3D data type representations. Credit: Gezawa, A. et al. (2020), CC BY 4.0.
3D Dataset types
Voxel-Based Images
- Represent the entire volume of an object in a structured grid.
- Each voxel holds a scalar value, often representing intensity in medical scans or simulation data.
- Best for representing interior details of a structure, e.g., in CT/MRI scans or simulations.
- Volumetric rendering and slice-based views are common visualization techniques.
Voxel based data representation. Credit: Hasanov, S. et al. (2021), CC BY-SA 4.0.
Visualizing volumetric datasets
- Slice-Based Visualization: This involves rendering 2D cross-sections or “slices” of the 3D dataset, often used in medical imaging.
- Volume raycasting (max intensity, emission absorbtion)
Slicing, Max. Intensity, Emission Absorbtion
Visualizing volumetric datasets
Transfer functions
Figure by Stefan Bruckner from the following publication:
Visualizing volumetric datasets
Volume rendering with Fiji: BigDataViewer and related tools
- Supports large data formats: BDV and Fiji can handle massive 3D datasets and allow arbitrary slicing.
- Ecosystem of tools: BDV serves as a foundation for other Fiji plugins that support multi-scale rendering and slicing.
Visualizing volumetric datasets
Volume rendering with Fiji: Animation with 3DScript
- Simple scripting: Create animations by writing basic scripts in natural language.
- Automated rendering: Generate complex 3D animations for presentations or publications.
- Project website
Visualizing volumetric datasets
Python based tools
Visualizing volumetric datasets
Web based rendering with Neuroglancer
- Collaboration-friendly: Share URLs with collaborators to provide access to the 3D visualization.
Converting volumetric datasets into meshes
Annotations
- Transfer functions: Used for visualizing unannotated datasets, adjusting colors and opacities based on intensity values.
- Fixed thresholds: Used to generate meshes by separating foreground from background using a set intensity threshold.
- Content-based annotations: Create precise meshes by using annotated regions to define boundaries.
Converting volumetric datasets into meshes
Converting volumetric datasets into meshes
Marching Cubes
Marching cubes algorithm. Credit: Ryoshoru, Jmtrivial on Wikimedia, CC BY-SA 4.0
Converting volumetric datasets into meshes
Optimization
- Binary masks vs. Probability maps
Converting volumetric datasets into meshes
Reducing mesh complexity
- Decimation: A process to reduce the number of polygons in a mesh while maintaining the overall shape and detail.
- Remeshing: Tools like MeshLab and Blender offer remeshing techniques that can optimize mesh topology for better performance.
- LOD (Level of Detail): Use LOD techniques to switch between different levels of mesh complexity based on the viewer’s distance.
Converting volumetric datasets into meshes
Conversion scripts
Mesh Processing
MeshLab: A powerful tool for cleaning, decimating, and refining 3D meshes. It supports:
- Smoothing: Remove sharp edges or rough areas in the mesh.
- Decimation: Reduce the number of polygons while maintaining the overall shape.
- Repair: Fix holes or non-manifold geometry in the mesh for better usability.
Other tools: Blender and VTK also offer additional mesh processing capabilities.
Rendering meshes
Rendering meshes
Rendering pipeline
- Vertex Shader: Transforms 3D coordinates and applies basic vertex processing.
- Geometry Shader (optional): Generates new geometry (e.g., additional triangles) from existing primitives.
- Fragment Shader: Computes the final color of each pixel.
- Blending and Depth Testing: Determines how pixels are blended and which ones are visible.
Credit: Joey de Vries,https://learnopengl.com/, CC BY 4.0
Rendering meshes
Rendering meshes with VTK
- VTK rendering features: Customize surface properties like color, opacity, and lighting. VTK can also handle interactive rendering, where users can rotate and zoom in on the rendered mesh.
Rendering meshes
Rendering meshes with Blender
Rendering meshes
Cutting volumes in Blender
Choosing colors
- Don’t underestimate the impact of choosing colors matching your story!
Point Clouds - Project BESSY2 Reconstruction
Ongoing Helmholtz Imaging Collaboration of the DKFZ Support Unit and HZB Researchers
Ongoing challenges and opportunities
- Web based viewers
- High Throughput
- Automated workflows
- Dimensionality reduction
Would you be interested in more specific tutorials?
- Animation
- Different approaches to transparency in Blender
- Point cloud handling
- Time series
Thank you!
Helmholtz Imaging Support Units at DESY, DKFZ, and MDC.