Mathematics
The K-Nearest Neighbours Algorithm and Its Bias-Variance Tradeoff
Quick fact
k-NN has no training phase; it memorizes the entire dataset and makes predictions at query time by averaging the labels of its k nearest examples. This makes it a 'lazy' learner, and the choice of k directly controls the bias-variance tradeoff: a single neighbor (k=1) perfectly fits training data but often overfits noise, while a huge k smoothes everything and can underfit.
Why this is interesting
You want to predict an unknown value—like the price of a house. The simplest approach might be to ask your closest neighbors how much they paid. But how many neighbors should you ask? The answer reveals a deep tension in all of machine learning.