Knowledge Distillation

Knowledge Distillation from Vision Transformers to Convolutional Neural Networks

Knowledge distillation from vision transformers to CNNs.