ImmuNet: A Segmentation-Free Machine Learning Pipeline for Immune Landscape Phenotyping in Tumors by Muliplex Imaging

  1. Shabaz Sultan
  2. Mark A. J. Gorris
  3. Evgenia Martynova
  4. Lieke L. van der Woude
  5. Franka Buytenhuijs
  6. Sandra van Wilpe
  7. Kiek Verrijp
  8. Carl G. Figdor
  9. I. Jolanda M. de Vries
  10. Johannes Textor

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Tissue specimens taken from primary tumors or metastases contain important information for diagnosis and treat-ment of cancer patients. Multiplex imaging allows in situ visualization of heterogeneous cell populations, such as immune cells, in tissue samples. Most image processing pipelines first segment cell boundaries and then measure marker expression to assign cell phenotypes. In dense tissue environments, this segmentation-first approach can be inaccurate due to segmentation errors or overlapping cells. Here we introduce the machine learning pipeline “ImmuNet” that identifies positions and phenotypes of cells without segmenting them. ImmuNet is easy to train: human annotators only need to click on an immune cell and score its expression of each marker. This approach al-lowed us to annotate 34,458 cells. We show that ImmuNet consistently outperforms a state-of-the-art segmentation-based pipeline for multiplex immunohistochemistry analysis across tissue types, cell types and tissue densities, achieving error rates below 5-10% on challenging detection and phenotyping tasks. We externally validate Im-muNet results by comparing them to flow cytometric measurements from the same tissue. In summary, ImmuNet is an effective, simpler alternative to segmentation-based approaches when only cell positions and phenotypes, but not their shapes, are required for downstream analyses. Thus, ImmuNet helps researchers to analyze multiplex tissue images more easily and accurately.

Article activity feed

  1. Arcadia Science

    The authors have made significant contributions by developing a segmentation-free machine learning pipeline, ImmuNet, which efficiently identifies and phenotypes immune cells across diverse human tissues. The ability to bypass computational cell segmentation, a challenging task in many instances, underscores the profound utility of this technique.

    However, while the potential application of ImmuNet beyond the domain of immunology and human studies is intriguing, a comprehensive understanding of the technique's limitations is paramount for its broader applicability. In this context, the authors could consider addressing the following points:

    1. Marker Specificity: ImmuNet seems to heavily rely on well-defined markers, along with antibodies that can reproducibly and specifically detect these molecules. Could the authors elucidate any potential limitations or challenges this reliance might present, particularly when expanding the use of ImmuNet beyond its current scope?

    2. Novel Cell Type Detection: The manuscript illustrates ImmuNet's efficacy in identifying well-characterized cell types. However, could the technique also shed light on cells that remain undetected with the current samples studied? Would it be possible to use ImmuNet to potentially identify novel cell types? Could the authors discuss any future directions or enhancements to ImmuNet's pipeline that might allow for such advancements?

    These discussions could add to the depth of the current study, paving the way for a broader understanding and application of ImmuNet across different fields.

    I used ChatGPT to help craft the language in this feedback, but verify that the content is accurate. -Galo

  2. Version published to 10.1101/2021.10.22.464548v2 on bioRxiv
  3. Version published to 10.1101/2021.10.22.464548v1 on bioRxiv