Object Detection Network Using Feature and Prediction Distillation with Convolutional Block Attention Module
Jaehong Yoon Department of Image, Chung-Ang University,Image Processing Intelligent Systems Lab
Jaehong Yoon
| Department of Image, Chung-Ang University,Image Processing Intelligent Systems Lab
Traditional knowledge distillation methods in object detection face challenges due to feature discrepancies between Teacher and Student networks. Many current approaches rely exclusively on response-based techniques, where the Student network emulates the Teachers detection predictions. However, this can inadvertently transfer erroneous predictions from Teacher to Student. To address this, we present a knowledge distillation network that employs the CBAM technique alongside the Teacher networks features to transfer knowledge to the Student network. Unlike conventional methods that focus solely on the Teacher models output predictions, our approach utilizes CBAM to emphasize what and where to focus of the Teacher and Student network, eventually enhancing the Students performance. Experiments on the COCO 2017 dataset demonstrate that our method achieves superior results compared to existing techniques.