Our understanding of GAN (generative adversarial net) training is still very limited since it is a non-convex-non-concave min-max optimization. As a result, most recent studies focused on local analysis. In this talk, we discuss how to perform a global analysis of GANs. We prove that the original JS-GAN has exponentially many bad strict local minima which are perceived as mode-collapse. We show that a 2-line modification to JS-GAN called relativistic standard GAN (RS-GAN) eliminates all bad basins. We extend the two results to a large class of losses as well: for separable GANs (including JS-GAN, hinge-GAN, LS-GAN) exponentially many bad basins exist, while for R-GANs no bad basins exist. The effectiveness of R-GANs were verified by a few empirical works before (e.g. ESR-GAN in super resolution). Based on theory, we predict that R-GANs has a bigger advantage for narrower neural nets, and our experiments verify that R-GANs (e.g. RS-GAN) can beat their separable counter-parts (e.g. JS-GAN) by 5-10 FID scores in narrower nets. Our theory also implies that the advantage is larger for higher-dimensional images; we show that for high-resolution images like LSUN, while JS-GAN only generates noise, RS-GAN can generate quite good images.
8月21日
10:30am - 11:30am
地點
https://hkust.zoom.us/j/5616960008
講者/表演者
Prof. Ruoyu Sun
University of Illinois at Urbana-Champaign
主辦單位
Department of Mathematics
聯絡方法
mathseminar@ust.hk
付款詳情
對象
Alumni, Faculty and Staff, PG Students, UG Students
語言
英語
其他活動
6月21日
研討會, 演講, 講座
IAS / School of Science Joint Lecture - Alzheimer’s Disease is Likely a Lipid-disorder Complication: an Example of Functional Lipidomics for Biomedical and Biological Research
Abstract Functional lipidomics is a frontier in lipidomics research, which identifies changes of cellular lipidomes in disease by lipidomics, uncovers the molecular mechanism(s) leading to the chan...
5月24日
研討會, 演講, 講座
IAS / School of Science Joint Lecture - Confinement Controlled Electrochemistry: Nanopore beyond Sequencing
Abstract Nanopore electrochemistry refers to the promising measurement science based on elaborate pore structures, which offers a well-defined geometric confined space to adopt and characterize sin...