G-RIPS Research Musing #2

Of late, have read much more about statistical shape models: their origins and the various ways people have implemented them, their connections to long-running computer vision research, and state of the art work with neural networks.

The computer vision connections are rather deep. The first step in any SSM modeling is to perform image segmentation, whereby you extract the anatomy of interest from the 3d image data. This is to say, image segmentation is how you get from a block of CT scan data to an interactable 3d model of a bone or liver or etc. Image segmentation is generally done by the marching cubes algorithm, which is pretty simple to explain luckily but seems like a bear in practice. Marching cubes fits a coordinate grid onto the whole image and then essentially performs edge detection along the segment in question. This is in fact pretty easy in 2d, but in 3d the number of possible ambiguities explodes. All the same, take it to be the case that we can perform segmentation from a 3d image.

Once we have the segment we need to make it comparable to our existing dataset. First, we need to register the image data, which is done by patchification, the process of manually assigning the same landmarks. This allows us to put any point in the new segment in correspondence with its sister point on every segment in the known database. We also need to rescale it, which is done by generalized Procrustes analysis, which is an optimization algorithm attempting to minimize differences to a reference shape in rotation, scale, and translation.

So now we have a new anatomical segment and can perform statistics on this segment against the world of known segments belonging to the same class. However, we want to deal with the extremely high dimensionality of this image data, which does not get better upon segmentation. To that end, we perform PCA on the world of known segments and truncate the basis of eigenvectors attained via PCA to be a number around 100. That means that we should now be able to describe any segment in the dataset as a convex combination of these eigenvectors.

We now seek to project the new segment into this reduced-dimension known-segment space. In this way, we smooth out the noise particular to the individual who generated the segment and obtain a new, conforming model of their segment.

One question involved here has to do with interpolations. Interpolation of 3d shapes is in general not easy - see here computer graphics approaches like Poisson interpolation, which should do better than linear but are still not without their own problems. One open question we have is how FlowSSM, which does not require landmarking, handles interpolation – can we characterize the differences between convex combinations of segments obtained via FlowSSM and those obtained via traditional methods?

Otherwise recently was working on another approach, which essentially does Gaussian process regression on the shape space to get a final segment. This is implemented in software by the people at Scalismo, and I’m attempting to follow their tutorial to understand both 3d modeling better and how Gaussian processes inform generation of new segments. This is closer to the traditional SSM approach, which we’ll thereafter put into comparison with the FlowSSM approach.

More to come!