Multimodal Feature Extraction and Fusion for Determining RGP Lens Specification Base-Curve through Pentacam Images

Patients diagnosed with irregular astigmatism require certain means of vision correction. In this regard, the use of a Rigid Gas Permeable (RGP) lens is among the most effective treatment methods. However, RGP lens base-curve detection is among the challenging issues. Current techniques have faced drawbacks in providing accuracy in detection. In this paper, a new method is defined based on multi-modal feature fusion on Pentacam images for automatic RGP lens base-curve detection using image processing and machine learning techniques. To this end, four types of features have been extracted from Pentacam images followed by a serial feature fusion mechanism. The fusion technique provides all possible combinatory views of these feature types to a Multi-Layered Perceptron (MLP) network to determine the base-curve. The first type of feature is obtained from the middle layer after passing the RGB combination of maps through a Convolutional Autoencoder (CAE) neural network. The second set is obtained by calculating the ratio of the area of the colored areas of the front cornea map. A feature vector is derived from the Cornea Front parameters as the third modality and the fourth feature vector is the radius of the reference sphere/ellipse of the front elevation map. Our evaluations on a manually labeled dataset show that the proposed technique provides an accurate detection rate with a 0.005 means square error (MSE) and a coefficient of determination of 0.79, superior to previous methods. This can be considered an effective step towards automatic base-curve determination, minimizing manual intervention in lens fitting.