Knowledge Distillation from Vision Transformers to Convolutional Neural Networks

Toshiharu Maeba, Makoto Kawano, Yutaka Matsuo

Abstract

This paper studies knowledge distillation from vision transformers to convolutional neural networks.

Type
Conference paper
Publication
IPSJ SIG Ubiquitous Computing Systems
Knowledge Distillation Vision Transformers