PixlMap: A generalisable pixel classifier for cellular phenotyping in multiplex immunofluorescence images

Multiplexed methods for the detection of protein expression generate extremely data-rich images of intact tissue sections. These images are invaluable for the quantification and analysis of complex biology and biomarker development. However, their interpretation presents a considerable analytical challenge.

Cell segmentation from images is a key bottleneck and a major focus of research activity in artificial intelligence. Most current methods depend initially on the use of a nuclear counterstain to identify nuclear boundaries, which is a relatively straightforward task. The cellular boundary is then assigned either by expansion of the nuclear outline, or by the use of membrane or cytoplasm-specific stains to delineate cell boundaries, or by some combination of the two. The task is critical, as inaccurate segmentation leads to information loss and data contamination from neighbouring cells. Increasingly sophisticated methods are being developed to address these issues, but each has its own shortcomings. We present an alternative method which is inspired by the fact that the assignation of a cellular phenotype ‘by eye’ does not depend upon the accurate identification of cell boundaries. We present an easy-to-use deep learning-based cellular phenotyping method which leverages this human capacity to assign phenotypes without segmenting the entire cell, and which can accurately phenotype cells based on nuclear segmentation alone.

Using human ground truth annotations of entire cellular regions, we developed a classifier leveraging the U-Net architecture within a commercially available deep learning image analysis platform, but the principle is transferrable to any deep-learning framework. Crucially, training requires only a single example of each compartmental stain (nuclear/cytoplasmic/membranous). The resulting algorithm assigns class identities to cells with nuclear labelling alone, without the need for whole cell expansion. The method is highly novel, broadly generalisable, and comparable in accuracy to intensity-based phenotyping methods, bridging the gap between inaccurate cellular segmentation and accurate phenotype generation.