These are notes I took while studying deep learning during my semester in Budapest, adapted from lectures given by my delightful instructor Pál Zsámboki. My original, handwritten notes can be found here.
Still under construction; check back soon.
Deep Learning
Notes from my deep learning course.
Notes 8
1
Introduction
What deep learning is, why it works, and the basic building blocks
2
Linear Regression
Linear regression, mean squared error, least squares solution, and the train-test split.
3
Classification and Logistic Regression
Classification, the probability simplex, softmax, and multinomial logistic regression.
4
Negative Log Likelihood
The NLL loss and its probabilistic motivation.
5
Binary Logistic Regression
Binary classification as a special case of multinomial logistic regression, the sigmoid function, and binary cross-entropy.
6
Gradient Descent
Full-batch, stochastic, and mini-batch gradient descent, with hyperparameter tuning and the train-validation-test split.
7
Confidence Intervals and Ensembles
Confidence intervals, deriving the Student t-distribution, and ensemble training (briefly).
8
Hyperparameter Search and Early Stopping
Best-checkpoint tracking across an ensemble, early stopping, masking, and grid search over hyperparameters.