Structured Supervision from Quantum Models: Distilling Robustness into Classical Networks

Abstract

Quantum machine learning offers a practical opportunity to leverage near-term quantum devices without requiring quantum inference at deployment. We propose a quantum-classical hybrid knowledge distillation framework in which a variational quantum circuit, equipped with a Quantum Fourier Transform-inspired positional encoding, acts as a soft-label teacher for a compact classical student. Rather than serving as a standalone classifier, the quantum model is used offline to generate structured probability distributions that encode global spectral information and uncertainty. These quantum-derived soft labels are distilled into a lightweight classical network using a hybrid loss that combines hard and soft supervision. Experiments on MNIST demonstrate that students trained with quantum soft labels exhibit consistent and statistically meaningful robustness improvements to Gaussian noise and in-plane rotations compared to classical distillation baselines, while maintaining strong clean accuracy and calibration. These results highlight a distinct role for NISQ-era quantum models as supervisory signal generators, enabling quantum-informed learning within fully classical deployment pipelines.