抄録
C-007
A Comprehensive Analysis of Low-Impact Computations in Deep Learning Applications
A Comprehensive Analysis of Low-Impact Computations in Deep Learning Applications
Recently, deep learning techniques have been widely used in many important applications. However, the cost behind this is the significant computations required to complete the training and inference tasks. This paper conducts a comprehensive analysis of several popular deep learning models. Our analysis reveals an interesting finding. That is, many computations in these popular models are actually low impact. This is because the output of a network layer is typically processed by the following ReLU layer, which updates all negative results generated by the previous layer to zeroes. This finding can inspire many potential optimizations for existing deep learning models.