BIAP7 - Loading multiple images

Author:

Matthew Brett

Status:

Draft

Type:

Standards

Created:

2015-07-18

Background

Some formats store images with different shapes in the same file

The ECAT file format can contain more than one type of image in a single image file.

ECAT can store many frames in a single image file. Each frame has its own subheader. The subheader specifies the 3D image size; each frame can therefore have a different image size.

We currently raise an error if you try and load an ECAT file where the frames do not have the same 3D dimensions.

It would be better if we could allow loading multiple images with different image dimensions, from a single ECAT file.

Vista data format and Lipsia format are other formats that allow saving multiple images with different image dimensions in the same file. We don’t currently support Lipsia or Vista formats and it is not clear how we would do that with the current load API.

We have had some discussion about saving multiple images into a single HDF5 file - see https://github.com/nipy/nibabel/pull/215#issuecomment-122357444

It can be useful to load 4D images as multiple 3D images

We sometimes want to load a 4D image as multiple 3D images.

When we are doing motion correction, we often want to split up a 4D image into separate 3D images.

Motion estimation results in different affines for each volume in the 4D time series. At the moment we have no API for returning these affines with a 4D image. One way of doing that is to load the 4D image and affines as a sequence of 3D images, each with their own affine.

We currently have a proposal open for a JSON header extension that can store these 4D affines for a 4D NIfTI file.

SPM saves the affines in an associated .mat file, with one affine per volume in the 4D image.

Options

Return an image sequence from load for some file formats

We don’t currently load ECAT files from the top-level nibabel.load function.

We do have nibabel.ecat.load, which raises an error for an ECAT file having frames with different image dimensions.

We could therefore choose to return a sequence of images from nibabel.load on an ECAT file, with one element per frame in the ECAT file.

Most ECAT images are 4D images, in the sense that the frames in the file do all have the same image dimensions and data type, so this might be cumbersome as a default.

We would have to work out how to deal with nibabel.ecat.load.

The same principles apply to the Lipsia / Vista formats, except we have no backward-compatibility problems, and it seems to be more common for these formats to mix image types in a single file.

Add a load_multi top-level function

nibabel.load_multi always returns an image sequence.

nibabel.load always returns a single image.

nibabel.load on ECAT (etc) files could first do load_multi, then check the resulting image dimensions, raising an error if incompatible, concatenating otherwise.

load_multi on current formats like NIfTI could return one image per volume, where each volume might have its own affine, as loaded from the JSON header extension or the SPM .mat file.

Next steps:

  • Make sure there are use-cases where you could wish to call load vs. load_multi on the same image (perhaps a Nifti image with different affines for each volume)

  • Investigate AFNI file formats as a use-case for this.

  • Check the nilearn codebase, see if iter_img and slice_img functions might offer a post-load alternative. Also check if those functions could be deprecated in favor of slicing / iterating on dataobj

  • Create a new issue to implement getting an iterator on dataobj?