nv-ext-image-processing
@niivue/nv-ext-image-processing adds four operations to NiiVue. Each one
takes a volume and gives you back a new one, which you can show on top of
the original or use in its place:
- Otsu thresholding splits an image into 2 to 4 classes by brightness.
- Haze removal blanks out the dim noise in the air around a head.
- Conform reslices a volume to a 256 x 256 x 256 grid of 1 mm voxels, the way FreeSurfer does.
- Connected labels gives each separate blob in a mask its own number.
The work runs in a Web Worker, so the page stays responsive while it computes. The worker ships inside the package, so there is nothing to set up in your bundler.
Install
npm install @niivue/nv-ext-image-processing@next @niivue/niivue@nextpnpm add @niivue/nv-ext-image-processing@next @niivue/niivue@nextyarn add @niivue/nv-ext-image-processing@next @niivue/niivue@nextbun add @niivue/nv-ext-image-processing@next @niivue/niivue@nextYou install NiiVue alongside the extension.
Your first segmentation
import NiiVue from '@niivue/niivue'import { otsu } from '@niivue/nv-ext-image-processing'
const nv = new NiiVue()const ctx = nv.createExtensionContext()ctx.registerVolumeTransform(otsu)
await nv.attachToCanvas(canvas)await nv.loadVolumes([{ url: '/volumes/mni152.nii.gz' }])
const classes = await ctx.applyVolumeTransform('otsu', ctx.volumes[0])await ctx.addVolume(classes)You register an operation once, then apply it by name to any volume. The
original stays as it was: you get a new volume, and addVolume draws it on
top.
Changing the settings
Pass settings as the third argument. Anything you leave out keeps its default:
const classes = await ctx.applyVolumeTransform('otsu', ctx.volumes[0], { levels: 4,})Each operation’s settings are described in its own section below.
Showing the result
Set a colormap and an opacity before you add the result, so the image underneath stays visible:
classes.colormap = 'actc'classes.opacity = 0.5await ctx.addVolume(classes)Those are the values Otsu thresholding suggests. Connected labels suggests
the random colormap at 0.8 opacity, which gives each blob its own color.
You find the suggestions in otsu.resultDefaults and
connectedLabel.resultDefaults; NiiVue does not apply them for you. See
Overlays for more on colormaps and opacity.
To use the result instead of the original, as you would after removing haze, swap them:
const clean = await ctx.applyVolumeTransform('removeHaze', ctx.volumes[0])await ctx.removeAllVolumes()await ctx.addVolume(clean)Splitting an image by brightness
otsu finds the brightness cutoffs that best separate the voxels into
groups (Otsu’s method). Each voxel gets a label: 0 for the darkest group,
then 1, 2 and so on up to the brightest. The result is an 8-bit label
volume.
The levels setting picks how many groups: 2, 3 (the default) or 4. Use 2
to split foreground from background, and 3 or 4 to separate tissues of
different brightness.
Removing the haze around a head
removeHaze finds a brightness cutoff the same way and sets every voxel
below it to the image’s lowest value. Brighter voxels keep their values.
The result is stored as 32-bit floats.
The level setting goes from 1, which removes the most, to 5, which
removes the least. The default is 5, the most careful setting.
Reslicing to a standard grid
conform reslices a volume to 256 x 256 x 256 voxels of 1 mm, centered on
the original, like FreeSurfer’s conform step. By default the result matches
a FreeSurfer conformed image: LIA orientation, smooth (linear)
interpolation, and brightness rescaled to 0 to 255 and stored as 8-bit.
Three settings change that:
toRAS: truegives RAS orientation instead of LIA.isLinear: falseuses nearest neighbor instead of linear interpolation.asFloat32: truestores 32-bit floats from 0 to 1 instead of 8-bit.
const resliced = await ctx.applyVolumeTransform('conform', ctx.volumes[0], { toRAS: true,})Labeling separate blobs
connectedLabel finds groups of touching voxels and numbers them 1, 2, 3
and so on. Zero stays background. Voxels only join a group when they have
the same value, so in a label volume each class is split into its own
blobs. Its settings:
conn: which neighbors count as touching. 6 means voxels that share a face, 18 adds shared edges, and 26, the default, adds shared corners.binarize: truetreats every nonzero voxel as one class, as in a mask.onlyLargestClusterPerClass: truekeeps only the biggest blob of each class, with its original value, and clears the rest.
That last setting tidies up a segmentation, such as the result of Otsu thresholding, by dropping stray specks. Register both operations first:
const classes = await ctx.applyVolumeTransform('otsu', ctx.volumes[0])const cleaned = await ctx.applyVolumeTransform('connectedLabel', classes, { onlyLargestClusterPerClass: true,})await ctx.addVolume(cleaned)Building a settings form
Each operation describes its own settings, so your app can build a form for it instead of hard-coding one:
const info = nv.getVolumeTransformInfo('otsu')for (const field of info?.options ?? []) { console.log(field.name, field.label, field.type, field.default)}A field’s type is checkbox or select, and a select lists its choices
in options. info.description is a one-line summary for the form’s
title, and nv.volumeTransforms lists every registered name, which is
handy for a menu.
When something does not work
Transform "otsu" already registered: every viewer on the page shares one list of operations, so register each one only once, for example when the page starts rather than in code that runs for each viewer.Unknown volume transform: otsu: the name was misspelled or never registered. The names areotsu,removeHaze,conformandconnectedLabel.
Try it
The live demo runs all four operations on a sample brain, with a settings form built as shown above. Its source is in apps/demo-ext-image-processing, and the package source is in packages/nv-ext-image-processing. For other add-ons, see all packages.