Theory of the Frequency Principle for General Deep Neural Networks

arXiv:1906. 09235v3 Announce Type: replace-cross Abstract: Along with fruitful applications of Deep Neural Networks (DNNs) to realistic problems, recently, some empirical studies of DNNs reported a universal phenomenon of Frequency Principle (F-Principle): a DNN tends to learn a target function from low to high frequencies during the training.