I am trying to make list for AI/Deep Learning in terms of Big Picture/Trends, Lectures/Course. I am creating another list mained focused on Application of this technology listed here.
Followings are topics on which I am compiling videos in this note.
Big Picture/History/Trends
- Andrew Ng - The State of Artificial Intelligence (2017)
- Andrew Ng: Artificial Intelligence is the New Electricity (2017)
- Introduction to Convolutional Neural Networks for Visual Recognition (2017)
- Geoffrey Hinton: What is wrong with convolutional neural nets? (2017)
- Deep Learning in Medical Imaging - Ben Glocker (2017)
- Heroes of Deep Learning: Andrew Ng interviews Geoffrey Hinton (2017)
- Scaling of Machine Learning (2018)
- Scaling of Machine Learning (2019)
- MIT Deep Learning Basics: Introduction and Overview(2019)
- Exponential Progress of AI: Moore's Law, Bitter Lesson, and the Future of Computation (2020)
- Artificial Intelligence | Machine Learning | Documentary | Canadian Economy | Robots | Robotics | AI (2020)
- The 2021 State of Enterprise Machine Learning (Dec 2020)
- AI: A Double Edged Sword // Documentary (2021)
- Artificial Intelligence: The Good, the Bad, and the Ugly - Yaser Abu-Mostafa- caltech (2023)
- MIT 6.S191: Deep Learning New Frontiers - Alexander Amini/MIT (2023)
- The Godfather in Conversation: Why Geoffrey Hinton is worried about the future of AI - University of Toronto (2023) ***
- What we see and what we value: AI with a human perspective—Fei-Fei Li (Stanford University) - Paul G. Allen School (Jan 2024)
- Geoffrey Hinton in conversation with Fei-Fei Li — Responsible AI development - Arts & Science - University of Toronto (Feb 2024)
- Tutorial | LLMs in 5 Formulas (Standard Format) - Harvard Data Science Initiative (Feb 2024)
- Visualizing transformers and attention | Talk for TNG Big Tech Day '24 - Grant Sanderson (2024)
- Andrej Karpathy - A YouTube Channel
Tutorials
- Andrej Karpathy - A YouTube Channel
- LLM Visualization
- Tiktokenizer
Lecture/Courses
At first I intended to put only the first lecture of each course/lectures and then let you follow through subsequent lectures, but I myself had difficulties in following through all the lectures in sequence. I was partly because in most case the unversity didn't post the video in a well packaged play list and partly because the video suggested by YouTube AI is not always recommending the lectures in the course that I wanted. So I decided to list all of the lectures for each course here.
- Geoffrey Hinton: "Introduction to Deep Learning & Deep Belief Nets" (2012)
- Neural Networks for Machine Learning - Geoffrey Hinton, Nitish Srivastava, Kevin Swersky (2016)
- Lecture 1.1 — Why do we need machine learning
- Lecture 1.2 — What are neural networks
- Lecture 1.3 — Some simple models of neurons
- Lecture 1.4 — A simple example of learning
- Lecture 1.5 — Three types of learning
- Lecture 2.1 — Types of neural network architectures
- Lecture 2.2 — Perceptrons: first-generation neural networks
- Lecture 2.3 — A geometrical view of perceptrons
- Lecture 2.4 — Why the learning works
- Lecture 2.5 — What perceptrons can't do
- Lecture 3.1 — Learning the weights of a linear neuron
- Lecture 3.2 — The error surface for a linear neuron
- Lecture 3.3 — Learning weights of logistic output neuron
- Lecture 3.4 — The backpropagation algorithm
- Lecture 3.5 — Using the derivatives from backpropagation
- Lecture 4.1 — Learning to predict the next word
- Lecture 4.2 — A brief diversion into cognitive science
- Lecture 4.3 — The softmax output function
- Lecture 4.4 — Neuro-probabilistic language models
- Lecture 4.5 — Dealing with many possible outputs
- Lecture 5.1 — Why object recognition is difficult
- Lecture 5.2 — Achieving viewpoint invariance
- Lecture 5.3 — Convolutional nets for digit recognition
- Lecture 5.4 — Convolutional nets for object recognition
- Lecture 6.1 — Overview of mini batch gradient descent
- Lecture 6.2 — A bag of tricks for mini batch gradient descent
- Lecture 6.3 — The momentum method
- Lecture 6.4 — Adaptive learning rates for each connection
- Lecture 6.5 — Rmsprop: normalize the gradient
- Lecture 7.1 — Modeling sequences: a brief overview
- Lecture 7.2 — Training RNNs with back propagation
- Lecture 7.3 — A toy example of training an RNN
- Lecture 7.5 — Long term Short term memory
- Lecture 8.1 — A brief overview of Hessian-free optimization
- Lecture 8.2 — Modeling character strings
- Lecture 8.3 — Predicting the next character using HF
- Lecture 8.4 — Echo State Networks
- Lecture 9.1 — Overview of ways to improve generalization
- Lecture 9.2 — Limiting the size of the weights
- Lecture 9.3 — Using noise as a regularizer
- Lecture 9.4 — Introduction to the full Bayesian approach
- Lecture 9.5 — The Bayesian interpretation of weight decay
- Lecture 9.6 — MacKay 's quick and dirty method
- Lecture 10.1 — Why it helps to combine models
- Lecture 10.2 — Mixtures of Experts
- Lecture 10.3 — The idea of full Bayesian learning
- Lecture 10.4 — Making full Bayesian learning practical
- Lecture 10.5 — Dropout
- Lecture 11.1 — Hopfield Nets
- Lecture 11.2 — Dealing with spurious minima
- Lecture 11.3 — Hopfield nets with hidden units
- Lecture 11.4 — Using stochastic units to improve search
- Lecture 11.5 — How a Boltzmann machine models data
- Lecture 12.1 — Boltzmann machine learning
- Lecture 12.2 — More efficient ways to get the statistics
- Lecture 12.3 — Restricted Boltzmann Machines
- Lecture 12.4 — An example of RBM learning
- Lecture 12.5 — RBMs for collaborative filtering
- Lecture 13.1 — The ups and downs of backpropagation
- Lecture 13.2 — Belief Nets
- Lecture 13.3 — Learning sigmoid belief nets
- Lecture 13.4 — The wake sleep algorithm
- Lecture 14.1 — Learning layers of features by stacking RBMs
- Lecture 14.2 — Discriminative learning for DBNs
- Lecture 14.3 — Discriminative fine tuning
- Lecture 14.4 — Modeling real valued data with an RBM
- Lecture 14.5 — RBMs are infinite sigmoid belief nets
- Lecture 15.1 — From PCA to autoencoders
- Lecture 15.2 — Deep autoencoders
- Lecture 15.3 — Deep autoencoders for document retrieval
- Lecture 15.4 — Semantic Hashing
- Lecture 15.5 — Learning binary codes for image retrieval
- Lecture 15.6 — Shallow autoencoders for pre-training
- Lecture 16.1 — Learning a joint model of images and captions
- Lecture 16.2 — Hierarchical Coordinate Frames
- Lecture 16.3 — Bayesian optimization of hyper-parameters
- Lecture 16.4 — The fog of progress
- MIT 6.S094 - Lex Fridman (2017)
- Introduction to Deep Learning and Self-Driving Cars
- Deep Reinforcement Learning for Motion Planning
- Convolutional Neural Networks for End-to-End Learning of the Driving Task
- Deep Learning for Human-Centered Semi-Autonomous Vehicles
- Recurrent Neural Networks for Steering Through Time
- MIT Sloan: Intro to Machine Learning (in 360/VR)
- Standford University : CS231 - Fei Fei Li, Justine Johnson, Serena Yeung (2017)
- Lecture 1 | Introduction to Convolutional Neural Networks for Visual Recognition
- Lecture 2 | Image Classification
- Lecture 3 | Loss Functions and Optimization
- Lecture 4 | Introduction to Neural Networks
- Lecture 5 | Convolutional Neural Networks
- Lecture 6 | Training Neural Networks I
- Lecture 7 | Training Neural Networks II
- Lecture 8 | Deep Learning Software
- Lecture 9 | CNN Architectures
- Lecture 10 | Recurrent Neural Networks
- Lecture 11 | Detection and Segmentation
- Lecture 12 | Visualizing and Understanding
- Lecture 13 | Generative Models
- Lecture 14 | Deep Reinforcement Learning
- Lecture 15 | Efficient Methods and Hardware for Deep Learning
- Lecture 16 | Adversarial Examples and Adversarial Training
- MIT 6.S094 - Lex Fridman (2018)
- Stanford CS230: Deep Learning - Andrew Ng (2018)
- Lecture 1 - Class Introduction and Logistics
- Lecture 2 - Deep Learning Intuition
- Lecture 3 - Full-Cycle Deep Learning Projects
- Lecture 4 - Adversarial Attacks / GANs
- Lecture 5 - AI + Healthcare
- Lecture 6 - Deep Learning Project Strategy
- Lecture 7 - Interpretability of Neural Network
- Lecture 8 - Career Advice / Reading Research Papers
- Lecture 9 - Deep Reinforcement Learning
- Lecture 10 - Chatbots / Closing Remarks
- Stanford CS221 - Percy Liang (2019)
- Lecture 1: Overview
- Lecture 2: Machine Learning 1 - Linear Classifiers, SGD
- Lecture 3: Machine Learning 2 - Features, Neural Networks
- Lecture 4: Machine Learning 3 - Generalization, K-means
- Lecture 5: Search 1 - Dynamic Programming, Uniform Cost Search
- Lecture 6: Search 2 - A*
- Lecture 7: Markov Decision Processes - Value Iteration
- Lecture 8: Markov Decision Processes - Reinforcement Learning
- Lecture 9: Game Playing 1 - Minimax, Alpha-beta Pruning
- Lecture 10: Game Playing 2 - TD Learning, Game Theory
- MIT 6.S191 - Alexander Amini et al (2019)
- MIT Deep Learning Basics - Lex Fridman (2019)
- Deep Learning State of the Art
- Introduction and Overview
- Introduction to Human-Centered Artificial Intelligence (AI)
- Introduction to Deep Reinforcement Learning (Deep RL)
- University of Toronto - Introduction to Machine Learning Course -(2019)
- Lecture 1 - Introduction to Machine Learning Course
- Lecture 2- Linear Model
- Lecture 3- PLA
- Lecture 4- Generalization - Loss Function
- Lecture 5- Linear Regression (LR)
- Lecture 6- Least Squares
- Lecture 7- Logistic Regression
- Lecture 8- Logistic Regression - II
- Lecture 9- Gradient Descent
- Lecture 10- Stochastic Gradient Descent
- Lecture 11- Momentum
- Lecture 12- Multiclass Logistic Regression / Softmax
- Lecture 13- Neural Networks - Part I
- Lecture 14- Neural Networks - Part II
- Lecture 15- Forward Propagation
- Lecture 16- Backward Propagation
- Lecture 17- Implementation of Neural Nets - Part I
- Lecture 18- Implementation of Neural Nets - Part II
- Lecture 19- Convolutional Neural Networks (CNNs)
- Lecture 20- Clustering
- Lecture 21 - Density Estimation - Part I
- Lecture 22 - Density Estimation - Part II
- Lecture 23 - EM Algorithm (1/2) - Density Estimation - Part III
- Lecture 24 - EM Algorithm (2/2) - Density Estimation - Part IV
- Lecture 25 - P.A.C Learning - Part I
- Lecture 26 - P.A.C Learning - Part II
- Lecture 27 - P.A.C Learning - Part III
- Lecture 28 - Dichotomy - VC Dimension
- Lecture 29 - Dichotomy - VC Dimension - Part II
- Lecture 30 -Bias - Variance - Part I
- Lecture 31 -Bias - Variance - Part II
- Lecture 32 - Validation - Part I
- Lecture 33 - Validation - Part II
- Lecture 34 - Validation - Part III
- Lecture 35 - SVM - Part I
- Lecture 36 - SVM - Part II
- Lecture 37 - SVM - Part III
- Lecture 38 - SVM - Part IV
- Deep Learning - Jeramy Howard (2019)
- Lesson 1: Deep Learning 2019 - Image classification
- Lesson 2: Deep Learning 2019 - Data cleaning and production; SGD from scratch
- Lesson 3: Deep Learning 2019 - Data blocks; Multi-label classification; Segmentation
- Lesson 4: Deep Learning 2019 - NLP; Tabular data; Collaborative filtering; Embeddings
- Lesson 5: Deep Learning 2019 - Back propagation; Accelerated SGD; Neural net from scratch
- Lesson 6: Deep Learning 2019 - Regularization; Convolutions; Data ethics
- Lesson 8 - Deep Learning from the Foundations
- Lesson 9 - How to train your model
- Lesson 10 - Looking inside the model
- Lesson 11 - Data Block API, and generic optimizer
- Lesson 12 - Advanced training techniques; ULMFiT from scratch
- Lesson 13 - Basics of Swift for Deep Learning
- Lesson 14 - Swift: C interop; Protocols; Putting it all together
- Linear Algebra and Learning from Data (2019)
- Deep Learning State of the Art (2020) | MIT Deep Learning Series (2020) - Lex Fridman
- MIT Introduction to Deep Learning | 6.S191 (2020) - Alexander Amini
- Meta Code (Korean)
Reinforcement Learning
- Lecture 10: Reinforcement Learning (2013)
- RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning (2015)
- Reinforcement Learning - A Simple Python Example and A Step Closer to AI with Assisted Q-Learning (2017)
- How reinforcement learning works || Practical Example (2018)
- An introduction to Reinforcement Learning (2018)
- Q-Learning Explained - A Reinforcement Learning Technique (2018)
- Deep Q-Learning - Combining Neural Networks and Reinforcement Learning (2018)
- Training a Deep Q-Network - Reinforcement Learning (2018)
- Replay Memory Explained - Experience for Deep Q-Network Training (2018)
- Build Deep Q-Network - Reinforcement Learning Code Project (2019)
- CS885 Lecture 1a: Course Introduction - Reinforcement Learning (2018)
Other Online Courses
- Deep Neural Networks Easily Explained || What is deep Neural Network and how it works (2018)
- Learnable Parameters in a Convolutional Neural Network (CNN) explained (2018)
- Practical Deep Learning for Coders - Full Course from fast.ai and Jeremy Howard (2020)
Blogs/Sites
- The Complete Guide to Artificial Neural Networks: Concepts and Models
- 7 Types of Neural Network Activation Functions: How to Choose?