Skip to content

Memory and bandwidth

View as Markdown

A whole-slide image or a streamed OME-Zarr store is far bigger than the memory a browser tab can spare, so NiiVue only ever holds part of it. Each part has its own limit. Slides (NVSlide) and streamed volumes (loadChunkedVolume, streamOmeZarr) are separate systems with separate settings.

The demo streams a 53783 x 49534 pixel slide, with separate CPU and GPU caches (maxCacheBytes, maxTileTextureBytes). This hosted copy leaves out the full-resolution level, so its levels are L1 to L3 (4, 16 and 32 times smaller). Try a 12 MiB cache, then zoom in: the line under the canvas shows NiiVue drawing a coarser level than it wanted, because the finer level’s visible tiles do not fit; a 16 MiB GPU cache does the same, reported as limited by texture. Fewer parallel loads fill the view more slowly. A finer detail setting fetches more tiles for the same view.

At a glance

Question Slides Streamed volumes
GPU memory maxTileTextureBytes (128 MiB) of tile textures per renderer, shared by the slides it draws; maxRawPoolBytes (32 MiB) of it holds raw channels for fast windowing maxChunkResidencyBytes (1.5 GB) caps the resident bricks; budgetBytes (1.5 GB) and maxBricks (240) cap one plan
CPU memory maxCacheBytes (96 MiB) of decoded tiles cacheBytes (256 MiB) of raw store bytes; persist adds a disk cache (512 MiB, off by default)
Detail versus coverage targetScreenPixelsPerTilePixel (0.75) and levelChoice ('auto'); steps down a level automatically when the view will not fit budgetPlan ('focus'), or its parts focus, radius, detail; minLevel caps detail; swapPolicy ('replace')
Prefetch Coarse coverage of the view, at the lowest priority (fixed) Predicts the next chunks along a scroll or pan (fixed)
Network maxConcurrentLoads (12) maxConcurrentLoads (6) and workers (up to 4)

The defaults are in brackets. Everything not marked fixed can be set.

Slides

import { createSlideView, NVSlide } from '@niivue/niivue'
const slide = NVSlide.fromSource(source, {
maxCacheBytes: 192 * 1024 * 1024, // decoded tiles kept in memory
maxConcurrentLoads: 6, // tile fetches and decodes at once
targetScreenPixelsPerTilePixel: 1, // raise for coarser, fewer tiles
})
const view = await createSlideView(canvas, {
slide,
maxTileTextureBytes: 256 * 1024 * 1024, // GPU tile textures
maxRawPoolBytes: 0, // none of it kept for raw channels
})
  • maxCacheBytes bounds the decoded tiles: 4 bytes a pixel for a colour tile, or 1, 2 or 4 bytes a pixel for each channel of a scalar tile. When it is full, the least recently used tile goes. It is set when the slide is made.
  • targetScreenPixelsPerTilePixel chooses the level in 'auto' mode: the finest level with at least this many screen pixels per tile pixel. Screen pixels are device pixels, so a Retina display asks for up to four times the tiles. Can be changed at any time.
  • levelChoice (or setLevelChoice) pins a level by index, or 'auto'.
  • maxTileTextureBytes bounds the GPU tile textures of one view, shared evenly by the slides it draws. Every tile in view needs a texture, so this also limits the level (see below). It is set when the view is made.
  • maxRawPoolBytes (up to half of maxTileTextureBytes) keeps raw channel samples of multi-channel tiles on the GPU, so a window or colormap change does not upload them again. 0 keeps them on the CPU only.

When the visible tiles of the level asked for would not fit in three quarters of maxCacheBytes (or of the GPU tile textures), NiiVue draws the finest coarser level that does fit, rather than loading and evicting tiles forever. The slide’s lastVisibleTiles then carries requestedLevel and limitedBy (cache, texture or tiles), and the slide fires levelchange. If even the coarsest level does not fit, only the block at the centre is drawn (capped) until the user zooms in. cacheBytes and pendingCount report what is held and what is still loading.

Streamed volumes

import { streamOmeZarr } from '@niivue/nv-ext-tile/zarr'
const nv = new NiiVue({ maxChunkResidencyBytes: 1_000_000_000 })
await streamOmeZarr(
nv,
url,
{ cacheBytes: 512 * 1024 * 1024, persist: true }, // the store
{ budgetPlan: 'interactive', swapPolicy: 'keepUntilCovered' }, // the plan
)

The volume is a tree of bricks: fine ones near the point of interest, coarser ones further away. The plan decides which bricks to load:

budgetPlan Finest bricks Up to For
'focus' (default) around the crosshair, sized to the view 240 bricks reading around the crosshair
'uniform' everywhere 240 bricks one picture of the whole volume
'interactive' around the crosshair 128 bricks smooth rotation and zoom
  • budgetBytes is the GPU memory one plan may use. Set it once for the device, since every preset shares it; a value you pass wins over the plan. setBudget changes it later.
  • maxChunkResidencyBytes (a NiiVue option) caps the bricks that stay on the GPU, from this plan and earlier ones. The least recently seen go first, and come back when needed.
  • minLevel caps detail: 0 allows the finest level. setMaxDetail changes it later.
  • focus, radius and detail set where the finest bricks go, how far they reach and how fast detail falls away, overriding the plan.
  • coarseFloor (on by default) keeps the coarsest level of the whole volume on the GPU, so a region never shows as empty while finer bricks arrive.
  • swapPolicy: when the focus moves, 'replace' frees the old bricks at once, using the least memory, and their regions show the coarse floor until the new bricks arrive. 'keepUntilCovered' keeps drawing each old brick until the new ones over its region have arrived, so the view never gets coarser while it refines. It needs the old and new bricks to fit maxChunkResidencyBytes together, and replaces as usual when they do not. It applies to the 3D render only.
  • cacheBytes and persist belong to the store (see OME-Zarr). With workers running, both budgets are split across them, so the total stays the same.

setBudgetPlan and setSwapPolicy switch the plan while the volume is shown.

Fixed policies

These are not settings today:

  • Visible tiles and chunks always load before anything speculative. Slides keep a quarter of maxCacheBytes for the other users of the cache (the 3D slide plane, thumbnails and coarse coverage).
  • A slide fetches coarse coverage of the view only when the view needs more than one round of loads.
  • A streamed volume looks 16 chunks ahead and predicts where a scroll or pan is heading. At most 16 fetched chunks wait to be uploaded, and some of those slots are kept free for chunks that are actually visible.
  • A slide drawn on a plane in a 3D view has its own 128 MiB of tile textures.
  • targetFrameMs in a plan is recorded but not yet acted on: 'interactive' gets its smoothness from a lower brick count.