Plugins
NiiVue is modular. While the core module provides many functions, the primary module is designed to be lean with minimal dependencies. The plugin design allows you to solve more complex tasks by combining building blocks. Below are examples of existing modules, with minimal examples for their deployment. The diagram below illustrates the NiiVue architecture: the core and its dependencies (shown in gray) are written in JavaScript, with WebGL shaders authored in GLSL. In contrast, the optional plugins (shown in green) are often implemented in WebAssembly (WASM).
Loader Plugins
The core NiiVue module supports many voxel and mesh formats. These natively supported formats are generally popular and simple, which helps minimize dependencies. For more complex or less common formats, NiiVue offers a plugin-based loader system that allows developers to add optional format support. This mechanism is also well-suited for prototyping and testing support for new formats.
Custom loaders must map incoming data to one of NiiVue’s core internal formats: NIfTI for voxel data and MZ3 for mesh data. These core formats impose certain constraints that loaders must accommodate. For example, the MZ3 format supports only triangulated meshes, so loaders handling polygonal formats must convert all faces to triangles to meet the expectations of modern low-level graphics APIs.
- cbor-loader meshes and voxels support ITK-Wasm.
- dicom-loader uses a WebAssembly copy of dcm2niix to handle DICOM voxel-based images.
- minc-loader handles voxel-based MINC images.
- tiff-loader handles some Tagged Image File Format images, though note that a single TIFF file can store images with different resolutions (unlike NIfTI).
- vox-loader handles the very simple MagicaVoxel .vox format, so this is an excellent skeleton for creating a new voxel-based loaded.
WebAssembly plugins
WebAssembly provides a way to use mature, high performance code written in languages like C on web browsers. Below are minimal examples illustrating integrating several popular tools with NiiVue, providing drag and drop solutions for many problems.
- niivue-niimath general image processing.
- niivue-itk-wasm interfaces with ITK-Wasm packages.
- niivue-flirt provides image coregistration.
- niivue-bet provides brain extraction.
AI plugins
AI models are revolutionizing medical imaging. While models are computationally expensive to generate, clever models can be simple to deploy using web technologies (WebGL and WebGPU). We provide minimal examples showcasing deployment of the brainchop models.
- T2lesion uses TensorFlowJS to identify brain injury on T2-weighted MRI scans.
- niivue-onnx uses ONNX for segmenting issues.
- niivue-tinygrad uses tinygrad.
Combining plugins
You can mix and match plugins to solve complex problems.
- ct2print.org combines niimath and ITK-Wasm to convert voxels to meshes.
- brainprint.org combines brainchop AI models with niimath and ITK-Wasm to segment T1-weighted MRI scans and subsequently convert these voxels to meshes.