Photonic embedding learning with high energy efficiency exceeding 100 GOPS/W/mm2

Read the full article See related articles

Listed in

This article is not in any list yet, why not save it to one of your lists.
Log in to save this article

Abstract

Photonic neuromorphic computing offers substantial enhancements in machine vision processing by providing ultrahigh operation bandwidth and reduced energy consumption, thereby outperforming conventional electronic systems based on Von Neumann architectures. However, scalability challenges persist in implementing chip-scale photonic computing --- particularly when accommodating high-dimensional tensor inputs --- due to inherent physical constraints and the complexity of control engineering. In this paper, we introduce a compact photonic embedding unit (PEU) that integrates with on-chip diffractive multi-kernel optical convolution to enable parallel, high-complexity vision perception. By leveraging amplitude-phase co-modulatio{}n within the PEU, high-dimensional tensors can be simultaneously loaded and processed with energy efficiency exceeding 100 giga-operations per watt per square milimeter (GOPS/W/mm2). We experimentally validate the feasibility of PEU by demonstrating diverse machine vision applications including Megapixel color image compression with a maximum 4.5:1 compression ratio, image classification over CIFAR-4 dataset with a 70.3% accuracy, and video-based multi-frame fusion for human action recognition with a 97.5% accuracy and a 66.7% reduction in computation load. Our work provides a pathway for future large-scale and high-dimensional vision embedding with photonic integrated circuits, enabling the practical application of chip-scale photonic computing in increasingly sophisticated scenarios --- such as autonomous driving, astronautics, and telecommunications --- at picosecond latency and femtojoule-level energy consumption per operation.

Article activity feed