1) Draw bias, variance, training error, test error, and irreducible
error curves, on a single plot, as we go from less flexible statistical
learning methods towards more flexible approaches. The x-axis should
represent the amount of flexibility in the method, and the y-axis should
represent the values for each curve.
a)Training MSE:
The training MSE declines monotonically as flexibility increases, this is because as flexibility increases thef
curve fits the observed data more closely.
b) Test MSE:
The test MSE initially declines as flexibility increases but at some point it levels off and then starts to increase again (U-shape), this is because when af
curve yields a small training MSE but a large test MSE we are actually
overfitting the data (our procedure tries too hard to find patterns in
the training data that are maybe only caused by chance rather than by
true properties of the unknown f ).
c) Variance:
The squared bias decreases monotonically and the variance increases monotonically; as a general rule, as we use more flexible methods, the variance will increase and the bias will decrease. Variance refers to the amount by whichf^
would change if we estimated it using a different training data set, so
if the curve fits the observations very closely, changing any point may
cause f^
to change considerably, and so will result in some variance.
d)Bias:
Bias refers to the error that is introduced by approximating a real-life problem by a much simpler model, so if we use a very simple model (linear regression) it is unlikely that any real-life problem has such a simple linear relationship, and so performing linear regression will result in some bias in the estimate off .
e) Irreducible error
Because the irreducible error is a noise the method can't fit it.So it will be there.The irreducible error is a constant so it is a parallel line, this curve lies below the test MSE curve because the expected test MSE will always be greater theVar(ε)
Explanation:
a)Training MSE:
The training MSE declines monotonically as flexibility increases, this is because as flexibility increases the
b) Test MSE:
The test MSE initially declines as flexibility increases but at some point it levels off and then starts to increase again (U-shape), this is because when a
c) Variance:
The squared bias decreases monotonically and the variance increases monotonically; as a general rule, as we use more flexible methods, the variance will increase and the bias will decrease. Variance refers to the amount by which
d)Bias:
Bias refers to the error that is introduced by approximating a real-life problem by a much simpler model, so if we use a very simple model (linear regression) it is unlikely that any real-life problem has such a simple linear relationship, and so performing linear regression will result in some bias in the estimate of
e) Irreducible error
Because the irreducible error is a noise the method can't fit it.So it will be there.The irreducible error is a constant so it is a parallel line, this curve lies below the test MSE curve because the expected test MSE will always be greater the
I loved your post.Much thanks again. Fantastic.
ReplyDeleteMachine Learning Online Course