Why Neural Networks can learn (almost) anything
Emergent Garden
Mar 12, 2022 14:00
Create an account to hide videos, mark them watched, or save them to Don't Miss.
Create an account to hide videos, mark them watched, or save them to Don't Miss.
Create an account to hide videos, mark them watched, or save them to Don't Miss.
Chapters
A video about neural networks, how they work, and why they're useful.
My twitter: https://twitter.com/max_romana
SOURCES
Neural network playground: https://playground.tensorflow.org/
Universal Function Approximation:
Proof: https://cognitivemedium.com/magic_paper/assets/Hornik.pdf
Covering ReLUs: https://proceedings.neurips.cc/paper/2017/hash/32cbf687880eb1674a07bf717761dd3a-Abstract.html
Covering discontinuous functions: https://arxiv.org/pdf/2012.03016.pdf
Turing Completeness:
Networks of infinite size are turing complete: Neural Computability I & II (behind a paywall unfourtunately, but is cited in following paper)
RNNs are turing complete: https://binds.cs.umass.edu/papers/1992_Siegelmann_COLT.pdf
Transformers are turing complete: https://arxiv.org/abs/2103.05247
More on backpropagation:
https://www.youtube.com/watch?v=Ilg3gGewQ5U
More on the mandelbrot set:
https://www.youtube.com/watch?v=NGMRB4O922I
Additional Sources:
Neat explanation of universal function approximation proof: https://www.youtube.com/watch?v=Ijqkc7OLenI
Where I got the hard coded parameters: https://towardsdatascience.com/can-neural-networks-really-learn-any-function-65e106617fc6
Reviewers:
Andrew Carr https://twitter.com/andrew_n_carr
Connor Christopherson
TIMESTAMPS
MUSIC
https://www.youtube.com/watch?v=SmkUY_B9fGg
Science & Technology