Title:
Rethinking Common Practices in Deep Learning
Abstract:
We will examine and shed new light on several common practices and beliefs in Deep Learning: the effect of batch size on generalization, the use of early-stopping and the role of the final classifier in convolutional networks. Both theoretical and empirical arguments will be used to show that current methods and understanding may prove misguided.