N1H111SM's Miniverse

Bias-Variance Tradeoff of the Learning Algorithms

字数统计: 148阅读时长: 1 min
2020/05/13 Share


Bias-Variance Tradeoff

First recall the variance definition.

Substitute random variable $X$ with $\hat \theta - \theta$, where $\hat \theta$ represents the estimate of the parameters given a certain architecture of the learning algorithm, and $\theta$ stands for the oracle optimal parameter setting.

Since $\theta$ is constant,

The second term is the definition of the mean squared error (MSE),

The third term is the definition of the squared bias,

Thus by moving the terms, finally we have,

  1. 1. Bias-Variance Tradeoff