Li, Jingru; Zhou, Sheng; Li, Liangcheng; Wang, Haishuai; … - 2023
In scenarios like privacy protection or large-scale data transmission, Data-free knowledge distillation (DFKD) methods are proposed to learn Knowledge Distillation (KD) when the data is not accessible. They generate pseudo samples that distill the knowledge from a pretrained teacher model and...