An Empirical Comparative Study of Classical Dimensionality Reduction Methods: MDS, Isomap, and LLE

Classical manifold learning methods such as Multidimensional Scaling (MDS), Isometric Mapping (Isomap), and Locally Linear Embedding (LLE) have played a pivotal role in nonlinear dimensionality reduction. While these approaches are well-established, a systematic empirical comparison under diverse data characteristics remains valuable for both research and practical applications. In this work, we present a comprehensive experimental evaluation of MDS, Isomap, and LLE, along with notable variants, across synthetic and real-world datasets of varying dimensionalities and structures. We assess their performance in terms of reconstruction error, runtime efficiency, and structural preservation, and analyze their behavior in different manifold settings. Our results reveal distinct strengths and limitations: LLE variants consistently excel on datasets with strong local geometric properties, while Isomap provides a favorable balance between runtime and structure preservation for high-dimensional image data. We also highlight the computational trade-offs of each method, providing practical guidelines for method selection in contemporary machine learning pipelines.