In this course, I will assume that you already have some basic knowledge about what machine learning is, and roughly what deep learning is. I will also assume that you are aware of the differences between supervised learning and unsupervised learning. If not, here’s a good resource explaining the difference.
For this course, we will primarily look at supervised learning.
Given a model, we will feed it multiple paired samples of input and output data and hope that the model will learn the relationship between them. More formally, we have two sets $\mathscr X, \mathscr Y$ and a sequence of pairs
$$ \mathscr D=((x_i, y_i)\in\mathscr X\times\mathscr Y: i\in I)\text{.} $$
We want to have the model predict $y_i$ from $x_i$. Here, $\mathscr D$ can be referred to as the dataset 1.
$\mathscr X$ is referred to as the input feature space. The $X \in \mathscr X$ are feature vectors, and the components $x_i \in X$ are the features (or observations).
On the other hand, $\mathscr Y$ is referred to as the target space. $y_i \in \mathscr Y$ are targets. This $\mathscr Y$ may look different depending on the task (you will see more concrete examples of this later).
Our goal is to predict $\mathscr Y$ meaningfully from $\mathscr X$. There are two ways to do this:
-
Given a particular sample $x_i$, we can try to directly approximate it’s corresponding $y_i$. In other words, we need to find (or “learn”) some function $\mathscr X\xrightarrow f\mathscr Y$ such that $f(x_i)$ is close to $y_i$ for all $i\in I$. This is called regression.
-
Alternatively, we can predict a target distribution. Given $x_i$, we can approximate the $Y\|X=x_i$. This is called classification. For each $x\in\mathscr X$ and $y\in\mathscr Y$, we can attempt to approximate the probability $\mathbf P(Y=y\|X=x)$. In other words, we want to find a function $\mathscr X\xrightarrow f\mathscr P(\mathscr Y)$ where $\mathscr P(\mathscr Y)$ denotes the set of probability distributions on the set $\mathscr Y$.
We will explore both of the above approaches later.
You will later see that the point of deep learning is to iteratively refine feature vectors into some internal/hidden representation which will make it easier to predict the target.
-
Just to remark, $\mathscr D$ is not actually a subset of $\mathscr X\times\mathscr Y$, since you can have $(x_i,y_i)=(x_{i'},y_{i'})$ for some $i\ne i'$. ↩