Deep Model Compression: Distilling Knowledge from Noisy Teachers

Bharat Bhusan Sau Vineeth N Balasubramanian
Abstract: The remarkable successes of deep learning models across various applications have resulted in the design of deeper networks that can solve complex problems. However, the increasing depth of such models also results in a higher storage and runtime complexity, which restricts the deployability of such very deep models on mobile and portable devices, which have limited storage and battery capacity. While many methods have been proposed for deep model compression in recent years, almost all of them have focused on reducing ...