Capacity and the Interpolation Threshold
A capacity-controlled threshold separates approximation from interpolation: once a network has enough parameters to fit every training point exactly, the optimization problem shifts from finding shared patterns to finding any solution, and the learned function changes character accordingly.