Dear All, You are kindly invited to attend the next Stochastics and Finance seminar. On Tuesday April 4 at 2pm, Kevin Qu will give a talk at EA LT 315 (and on zoom). Location: Eastern Avenue Lecture Theatre 315 Zoom link: https://uni-sydney.zoom.us/j/84240252220 Speaker: Kevin Qu (School of Physics, University of Sydney) Title: Dynamical and computational properties of heavy-tailed deep neural networks Abstract: In deep neural networks (DNNs), the assumption of Gaussian statistics in theoretical studies has been pervasive. However, empirical data has shown that heavy- tailed connectivity is prevalent in commonly used pretrained DNNs. To elucidate the fundamental impact of such heavy-tailed connectivity on the dynamical and computational properties of DNNs, we develop a novel mean field theory that integrates theories of heavy-tailed random matrices and non-equilibrium statistical physics. Our theoretical framework demonstrates that heavy-tailed weights enable the emergence of the extended criticality (edge-of-chaos) in a broad parameter region, leading to improved and faster learning of real-world tasks without the need to fine-tune the weight statistics, compared to networks with Gaussian weights. Notably, we find that heavy-tailed DNNs exhibit multifractal eigenvectors and internal neural representations of input data, which are associated with profound computational advantages such as enhanced robustness and increased signal-to-noise ratio in the context of few-shot learning. Overall, our findings challenge the commonly held assumption of Gaussian statistics in DNNs and offer new insights into the underlying mechanisms of deep learning. This is a joint project with Asem Wardak and Pulin Gong under the Complex Systems, School of Physics. https://www.maths.usyd.edu.au/u/SemConf/Stochastics_Finance/seminar.html Please feel free to forward this message to anyone who might be interested in this talk. Best wishes, Anna