N1H111SM's Miniverse

# Bias-Variance Tradeoff of the Learning Algorithms

2020/05/13 Share

Materials

First recall the variance definition.

Substitute random variable $X$ with $\hat \theta - \theta$, where $\hat \theta$ represents the estimate of the parameters given a certain architecture of the learning algorithm, and $\theta$ stands for the oracle optimal parameter setting.

Since $\theta$ is constant,

The second term is the definition of the mean squared error (MSE),

The third term is the definition of the squared bias,

Thus by moving the terms, finally we have,

CATALOG