Quantum-Enhanced Hybrid-Model Compressionusing Knowledge Distillation

Quantum computing has emerged as a promising paradigm capable of addressing computational tasks that are intractable for classical systems, leveraging quantum mechanical principles such as superposition and entanglement to efficiently explore vast solution spaces. In recent years, hybrid quantum-classical approaches have gained increasing attention as a means to exploit the representational power of quantum systems while maintaining the practicality of classical machine learning frameworks. This paper presents a study on quantum-enhanced model compression via knowledge distillation with feature space alignment, in which a large classical neural network (the teacher) transfers knowledge to a substantially smaller student model. We introduce a hybrid quantum-classical student architecture that incorporates a parameterized quantum circuit (PQC) implemented in Qiskit, using a multi-Pauli feature map followed by an EfficientSU2 ansatz to embed and process classical features in a high-dimensional Hilbert space. The hybrid student is directly compared to a purely classical student of comparable size to evaluate the representational benefits of the quantum component. We describe the knowledge-distillation framework and examine alternative gradient-estimation techniques for optimizing the quantum circuit, analyzing their influence on performance and convergence. The results indicate that the hybrid quantum student achieves superior parameter efficiency and expressive capacity compared to a classical counterpart of similar scale. While the hybrid model demonstrates improved compression capabilities and competitive accuracy, it incurs higher computational overhead and exhibits increased training instability relative to purely classical baselines. We discuss practical limitations related to noise, scalability, and current hardware constraints, and highlight directions for future work aimed at reducing computational costs and improving robustness. Overall, our findings suggest that parameterized quantum circuits can serve as compact and expressive components within neural compression schemes, providing a promising avenue for further exploration as quantum hardware and optimization techniques mature.