Neural network learns the Mandelbrot [Part 2]

Hey all! This massive improvement in approximation was by the suggestion of reddit user u/jnbrrn, who suggested the use of fourier features as an input for the network. Essentially this means rather than giving the network an (x, y) coordinate, I am now giving it an an encoding of [cos(x), sin(x), cos(2x), sin(2x), cos(3x), ...] up to order 32 of the fourier series for both x and y coordinates. This improved the approximation dramatically and significantly reduced training time. This network was trained for 150 epochs, where it's loss finally plateaued regardless of the learning rate. The rest of the network is the same architecture as the previous attempt, though slightly smaller to fit on my gpu. Here is the video and paper that explain the theoretical basis behind this work: https://youtu.be/nVA6K6Sn2S4 https://arxiv.org/abs/2006.10739 I believe r/jnbrrn is actually one of the co authors of this paper, so huge thanks to them for the reference! The Fourier series is another robust universal function approximator, and it's fascinating to see it work so well on this problem. Hooray for calculus! Code: https://github.com/MaxRobinsonTheGreat/mandelbrotnn Music: https://soundcloud.com/user-82623269